Cluster: How a Client Integrates with Replication and the Cluster

• In RavenDB, the replication is not a bundle and is always enabled if there are two nodes or more in the cluster. This means that the failover mechanism is always turned on by default.

• The client contains a list of cluster nodes per database group.
  Each time the client needs to do a request to a database, it will choose a node that contains this database from this list to send the request to. If the node is down and the request fails, it will select another node from this list.

• The choice of which node to select depends on the value of ReadBalanceBehavior, which is taken from the current conventions. For more information about the different values and the node selection process, see Load balance & Failover Conventions.

When the client is initialized, it fetches the topologies and populates the nodes list for the load-balancing and failover functionality. During the lifetime of a RavenDB Client object, it periodically receives the cluster and the databases topologies from the server.

The topology is updated with the following logic:

• Each topology has an etag, which is a number
• Each time the topology has changed, the etag is incremented
• For each request, the client adds the latest topology etag it has to the header
• If the current topology etag at the server is higher than the one in the client, the server adds "Refresh-Topology:true" to the response header
• If a client detects the "Refresh-Topology:true" header in the response, the client will fetch the updated topology from the server Note: if ReadBalanceBehavior.FastestNode is selected, the client will schedule a speed test to determine the fastest node

The client configuration is handled in a similar way:

• Each client configuration has an etag attached
• Each time the configuration has changed at the server-side, the server adds "Refresh-Client-Configuration" to the response
• When the client detects the aforementioned header in the response, it schedules fetching the new configuration

How and When the Topology is Updated

• The first time any request is sent to RavenDB server, the client fetches cluster topology
• Each subsequent request happens with a fetched topology etag in the HTTP headers, under the key 'Topology-Etag'
• If the response contains the 'Refresh-Topology: true' header, then a thread responsible for updating the topology will be spawned

Listing any node in the initialization of the cluster in the client is enough to be able to properly connect to the specified database. Each node in the cluster contains the full topology of all databases and all nodes that are in the cluster. Nevertheless, it is possible to specify multiple node urls at the initialization. But why list multiple the nodes in the cluster, if url of any cluster node will do?

By listing multiple the nodes in the cluster, we can ensure that if a single node is down and we bring a new client up, we'll still be able to get the initial topology. If the cluster sizes are small (three to five nodes), we'll typically list all the nodes in the cluster. But for larger clusters, we'll usually just list enough nodes that having them all go down at once will mean that you have more pressing concerns then a new client coming up.

using (var store = new DocumentStore
{
    Database = "TestDB",
    Urls = new [] { 
                    "http://[node A url]",
                    "http://[node B url]",
                    "http://[node C url]"
                  }
})
{
    store.Initialize();
    
    // the rest of ClientAPI code
}

In RavenDB clusters, the databases are hosted in database groups. Since there is a master-master replication configured between database group members, a write to one of the nodes will be replicated to all other instances of the group. If there are some writes that are important, it is possible to make the client wait until the transaction data gets replicated to multiple nodes. It is called a 'write assurance', and it is available with the WaitForReplicationAfterSaveChanges() method.

using (var session = store.OpenSession())
{
    var user = new User
    {
        Name = "John Dow"
    };

    session.Store(user);

    //make sure that the comitted data is replicated to 2 nodes
    //before returning from the SaveChanges() call.
    session.Advanced
        .WaitForReplicationAfterSaveChanges(replicas: 2);

    session.SaveChanges();
}