8.10. Initializing Consumers
Once a replication agreement is created, the consumer must be initialized; that is, the data must be physically copied from the supplier server to the consumer servers. This section first describes consumer initialization in detail and then provides instructions on the two different methods for initializing consumers.
Replication will not begin until the consumer is initialized.
Consumer initialization involves copying data from the supplier server to the consumer server. Once the subtree has been physically placed on the consumer, the supplier server can begin replaying update operations to the consumer server.
Under normal operations, the consumer should not ever have to be reinitialized. However, any time there is a chance that there is a big discrepancy between the supplier's data and the consumer's, reinitialize the consumer. For example, if the data on the supplier server is restored from backup, then all consumers supplied by that server should be reinitialize. As another example, if the supplier has not been able to contact the consumer for a long time, like a week, the supplier may determine that the consumer is too far out of date to be updated, and must be reinitialized.
The consumer can either be initialized online using the Console or manually using the command-line. Online consumer initialization using the Console is an effective method of initializing a small number of consumers. However, since each replica is initialized in sequence, this method is not suited to initializing a large number of replicas. Online consumer initialization is the method to use when the consumer is initialized as part of configuring the replication agreement on the supplier server.
Manual consumer initialization using the command-line is a more effective method of initializing a large number of consumers from a single LDIF file.
Online consumer initialization using the Console is the easiest way to initialize or reinitialize a consumer. However, for replicating across a slow link, this process can be very time-consuming, and manual consumer initialization using the command-line may be a more efficient approach. This is described in more detail Section 8.10.4, “Manual Consumer Initialization Using the Command Line”.
When a consumer server is being initialized using the online consumer creation method, all operations (including searches) on the replica are referred to the supplier server until the initialization process is completed.
To initialize or reinitialize a consumer online, do the following:
Create a replication agreement.
On the supplier server, on the Directory Server Console, select the Configuration tab.
Expand the Replication folder, then expand the replicated database. Right-click the replication agreement, and choose Initialize Consumer from the pop-up menu.
A message opens warning that any information already stored in the replica on the consumer will be removed.
Click Yes in the confirmation box.
Online consumer initialization begins immediately. To check the status of the online consumer initialization, open the Summary tab in the Status box. If online consumer initialization is in progress, the status shows that a replica is being initialized.
To update this window, right-click the replicated database icon in the navigation tree, and choose Refresh Replication Agreements. When online consumer initialization finishes, the status changes to reflect this.
For more information about monitoring replication and initialization status, see Section 8.17, “Monitoring Replication Status”.
Online consumer initialization can be performed through the command line by adding the nsds5BeginReplicaRefresh attribute to the replication agreement entry. This attribute is absent by default, and it will be automatically deleted once the consumer initialization is complete.
Find the DN of the replication agreement on the supplier server that is for the consumer to be initialized. For example:
ldapsearch -h supplier1.example.com -p 389 -D "cn=directory manager" -w password -s sub
-b cn=config "(objectclass=nsds5ReplicationAgreement)"
This command returns all of the replication agreements configured on the supplier in LDIF format. Get the DN of the replication agreement with the consumer to be initialized. This is the replication agreement which will be edited.
Edit the replication agreement, and add the nsds5BeginReplicaRefresh attribute:
ldapmodify -h supplier1.example.com -p 389 -D "cn=directory manager" -w password dn: cn=ExampleAgreement,cn=replica,cn="dc=example,dc=com",cn=mapping tree,cn=config changetype: modify replace: nsds5beginreplicarefresh nsds5beginreplicarefresh: start
ldapmodify does not prompt for input; simply type in the LDIF statement, and then hit enter twice when the LDIF statement is complete. Close the ldapmodify utility by hitting Ctrl+C.
To check the initialization status, do an ldapsearch for the replication agreement entry.
ldapsearch -h hostname -p port -D "cn=directory manager" -w password -s base
-b 'cn=ExampleAgreement,cn="dc=example,dc=com", cn=mapping tree, cn=config' '(objectclass=*)'
If the nsds5BeginReplicaRefresh attribute is present, the initialization is still in progress. If the initialization is complete, then the attribute nsds5ReplicaLastInitStatus shows the status. If the initialization was successful, the value of nsds5ReplicaLastInitStatus is Total update succeeded. If the initialization was not successful, this attribute shows information about the error; check the error logs for both the supplier and consumer for additional information.
The replication monitoring attributes are described in more detail in the Directory Server Configuration, Command, and File Reference.
Manual consumer initialization using the command-line is the fastest method of consumer initialization for sites that are replicating very large numbers of entries. However, the manual consumer initialization process is more complex than the online consumer initialization process. Red Hat suggests using the manual process whenever the online process is inappropriate due to performance concerns.
Initializing or reinitializing a server manually has three steps:
Create a replication agreement.
Export the replica on the supplier server to an LDIF file.
Import the LDIF file with the supplier replica contents to the consumer server.
See Section 8.10.4.2, “Importing the LDIF File to the Consumer Server”.
There are three ways to convert a replica database to LDIF:
When creating a replication agreement, by selecting Create consumer initialization file in the Initialize Consumer dialog box of the Replication Agreement Wizard.
From the Directory Server Console, by right-clicking the replication agreement under the Replication folder and choosing Create LDIF File from the pop-up menu.
From the command-line by using the export command, as described in Section 4.2.3, “Exporting to LDIF from the Command-Line”.
Import the LDIF file which contains the supplier replica contents to the consumer server by using the import features in the Directory Server Console or by using either the ldif2db script or ldif2db.pl script. Both import methods are described in Section 4.1.3, “Importing from the Command-Line”.
With the ldif2db.pl script, the LDIF file import operation does not require a server restart. For more information on command-line scripts, see the Directory Server Configuration, Command, and File Reference.
A very large database, such as one with several million entries, can take an hour or more to initialize a consumer from the Console or even with manual initialization. To save time, use filesystem replica initialization.
Directory Server has the capability to initialize a replica using the database files from the supplier server. This avoids the need to rebuild the consumer database and can be done at essentially the speed of the network between the two servers by transferring the files with FTP or NFS, for example. Instead of sending entries via LDAP to replica servers, filesystem replica initialization populates the new database on the destination server by backing up the supplier database on one server and restoring the database on the destination server.
This method of initializing consumers is especially useful in replication over wide-area networks or over networks with slow or unstable connections.
For smaller databases, Red Hat recommends using manual initialization or initialize consumers from the Console.
The destination server must have the same architecture and the same bit size as the supplier server for the initialization to succeed. For example, Red Hat Enterprise Linux 32-bit to Red Hat Enterprise Linux 32-bit.
Create a new database on the destination server to match the database from the source server.
Before initializing the consumer from the backup files, be certain that the appropriate database has been created on the destination server so that the database exists to be restored and initialized.
Enable replication on the backend as a dedicated consumer.
If there is already a replication agreement to that host and port, then replication should resume immediately after running the restore script. Otherwise, create the replication agreement on the source server (or whatever server is the supplier), and select the Do not initialize consumers at this time radio button.
Stop the source Directory Server if it is running. For example:
service dirsrv stop slapd-example
From the command-line, run the db2bak utility, and archive the current directory installation.
/usr/lib/dirsrv/slapd-instance_name/db2bak
In addition, a new backup can be created by hitting the Back Up Directory Server button in the Tasks tab of the Directory Server Console and then putting the name of the archive directory in the Directory:... field. Alternatively, select any previous back-up to initialize the consumer.
This backup information is recovered in the destination replica.
Restart the source Directory Server. For example:
service dirsrv start slapd-example
Copy the archived files to a directory on the destination server, such as archiveDirectory.
Stop the destination Directory Server if it is running.
service dirsrv stop slapd-example2
On the destination server, restore the archives with the bak2db script, using the optional -n parameter to specify the backend instance name. This -n parameter is similar to the -n used with ldif2db and db2ldif. For example:
/usrlib/dirsrv/slapd-example2/bak2db /tmp/archiveDirectory -n userRoot
Restart the destination Directory Server. For example:
service dirsrv start slapd-example2
Replication will begin on schedule as soon as the destination server is restarted.
For more information on using these scripts, see the Directory Server Configuration, Command, and File Reference.