Database Replication

DataCaster supports database replication to remote systems using transaction logs. This is useful for keeping a warm standby of the database in case of failure of the primary database. It may also be used in read-centric databases with fewer writes, for offloading some read activity to one or more replicas, which requires some application changes not discussed here.

Note DB Replication does not curently support schema changes. Schema chanes will need to be performed and synchronized before replication (re)commences.

The database replication uses log publishing over a TCP connection and enables a remote database to replicate all the transactions being generated by the oriiginal. Replication can be performed using either the WebAdmin console, or the command-line tool, which can be used only for an idle standby.

Two modes of replication are supported.

  • Replication of complete databases. In this case either the databases must be be in identical starting state to begin, or the replica has to begin in the same state as the original database was just before generating the first log message being applied.
  • Replication of tables. In this case one or more tables can be listed, for which the changes from the original are replicated. This optioon is currently not documented.

You can pick the starting point for applying log changes if you know the timestamp of the log message. And timestamps are used to pickup any backlog of changes in case the replica is shutdown, or otherwise ceases to get log changes.

Replication requires the original database to be running on a DataCaster server. The replica may be running on a DataCaster server or client installation.

Replication Setup

The following steps will be needed to setup replication.

  • Configure remote clients on the server with the original database. Setup user or server-level access for remote clients, so that the setup is used. Use WebAdmin or the command line tools to setup the remote clients on the server.
  • Configure access passwords on the replicate client to allow access to the remote server. If using DataCaster server you can use WebAdmin or the command line tools. On DataCaster clients, only the command line tools are available.
  • Initialize the databases to the same state, or use an archive or other backup of the original database and find out the timestamp of the first log entry you want to apply.
  • Start the source database by starting the DataCaster server. The database being replicated must be in the server startup databases list in serverprefs.xml, otherwise no remote client can connect to that database.
  • Start the replication tool, or start the server and use the WebAdmin console to start replication.

Replication Using WebAdmin Console

The following steps are needed to start database replication using WebAdmin Console.
  • Open the page Database Replicator under Server Utilities group, provide the required inputs, like the local database name, the remote url to be used and the start time and then Click Initialize
  • You will now be presented the Manage Applications page, you can now start the Database Replicator application using the Start link.
  • When the Database Replicator application is already initialized, you will not be able to initialize it again, you can do that by removing the application.
  • You can stop the database replication using the Stop link. The timestamp of the last log event applied will be shown as the Status Message on the Manage Applications page.

Command-line Replication Tool

The command line replication tool can be used to replicate a source database. The options for the command-line replication tool are: 

java com.applibase.db.tools.ReplicateRemoteDB local_database remote_url startTime [ adminuser adminpwd ]

It is recommended that you run the tool using nohup (on Unix systems) to ensure the process remains active even after you logout or are diconnected.

Remote URL

The remote URL for the source database takes the form dcp://source_hostname:9999/TxLog/db_name

The source_hostname is the name of the host with the original database. The db_name is the name of the source database. The default port used by the log publisher is 9999. Change the port number in the URL if a non-default port is used.

Using Start Times (timestamp)

  • A timestamp (startTime) of 0 or any negative number except -10 starts applying the log from the current time (when a connection is made to the original database).

  • A timestamp (startTime) of -10 starts applying the entire history of the log available on the original database server. This is useful to set a starting point for replication, without synchronizing startup of both databaes. This is possible if the logs are deleted for the original database before making a copy and restarting.

  • The command line replication tools prints the timestamp of the last log event applied. This can be used to resume replication if the replication process is terminated for any reason.

Stopping Replication

To stop replication, kill the command line process, ReplicateRemoteDB (do not use kill -9). The program prints the timestamp of the last log entry processed from the source database (this will be in nohup.out if run with nohup). Use the printed timestamp to resume replication as needed.