Server: Ongoing Tasks: External Replication


  • Schedule an External Replication Task in order to have a live replica of your data in another database:

    • In a separate RavenDB cluster on local machines or a cloud instance, which can be used as a failover if the source cluster is down.
      Note: External Replication task does not create a backup of your data and indexes.
      See more in Backup -vs- Replication
    • In the same cluster if you want a live copy that won't be a client failover target.
  • "Live" means that the replica is up to date at all times. Any changes in the source database will be reflected in the replica once they occur.

  • This ongoing task replicates one-way - from the source to the destination.

  • For additional functionality such as filtration and two-way replication consider Hub/Sink Replication.

  • To replicate between two separate, secure RavenDB servers, you need to pass a client certificate from the source server to the destination.

In this page:

General Information about External Replication Task

What is being replicated:

What is not being replicated:

Why are cluster-level features not replicated?

To provide for architecture that prevents conflicts between clusters, especially when ACID transactions are important, RavenDB is designed so that data ownership is at the cluster level.
To learn more, see Data Ownership in a Distributed System.

Conflicts:

  • Two databases that have an External Replication task defined between them will detect and resolve document conflicts according to each database conflict resolution policy.
  • It is recommended to have the same policy configuration on both the source and the target databases.

Code Sample

The required elements of an External Replication task are:

  • The UpdateExternalReplicationOperation() method.
  • The destination server needs the certificate from the source server so that it will trust the source.
  • The connection string with destination server URL and any other details needed to access the destination server.
  • The following properties in the ExternalReplication object:
    • ConnectionStringName
      The connection string name.
    • Name
      The target database name.

Properties
        public class ExternalReplication
        {
            public TimeSpan DelayReplicationFor { get; set; }
            public string Name { get; set; }
            public string ConnectionStringName { get; set; }
            public string MentorNode { get; set; }
        }

Optional elements include the following properties in the ExternalReplication object:

  • MentorNode
    Preferred responsible node in source server.
  • DelayReplicationFor
    Amount of time to delay replication.
    The following sample shows a 30-minute delay. Delay can also be set by days, hours, and seconds.

//define external replication with mentor node and delay timespan
await sourceStore.Maintenance.SendAsync(
    new UpdateExternalReplicationOperation(new ExternalReplication
{
    ConnectionStringName = connectionStrName,
    Name = "targetDatabaseName",
    MentorNode = "B",
    DelayReplicationFor = TimeSpan.FromMinutes(30)
}));
           

ExternalReplication properties:

public class ExternalReplication
{
    public TimeSpan DelayReplicationFor { get; set; }
    public string Name { get; set; }
    public string ConnectionStringName { get; set; }
    public string MentorNode { get; set; }
}

Step-by-Step Guide

To create an external replication task via the RavenDB Studio, see the Step-by-Step Guide

Definition

To learn how to define an external replication task via code, see code sample.

You can also configure external eplication tasks via RavenDB Studio.

Offline Behavior

  • When the source cluster is down (and there is no leader):

    • Creating a new Ongoing Task is a Cluster-Wide operation,
      thus, a new Ongoing External Replication Task cannot be scheduled.

    • If an External Replication Task was already defined and active when the cluster went down,
      then the task will not be active, no replication will take place.

  • When the node responsible for the external replication task is down

    • If the responsible node for the External Replication Task is down,
      then another node from the Database Group will take ownership of the task so that the external replica is up to date.
  • When the destination node is down:

    • The external replication will wait until the destination is reachable again and proceed from where it left off.

    • If there is a cluster on the other side, and the URL addresses of the destination database group nodes are listed in the connection string, then when the destination node is down, the replication task will simply start transferring data to one of the other nodes specified.

Delayed Replication

In RavenDB we introduced a new kind of replication, delayed replication. It replicates data that is delayed by X amount of time. Delayed replication works just like normal replication but instead of sending data immediately, it waits X amount of time. Having a delayed instance of a database allows you to "go back in time" and undo contamination to your data due to a faulty patch script or other human errors.
While you can and should always use backup for those cases, having a live database makes it super fast to failover to prevent business losses while you repair the faulty databases.