Sharding Overview

As a database grows very large, storing and managing it may become too demanding for any single node.
System performance may suffer as resources like RAM, CPU, and storage are exhausted, routine chores like indexing and backup become massive tasks, responsiveness to client requests and queries slows down, and the system's throughput spreads thin serving an ever-growing number of clients.

With sharding, as the volume of stored data grows, the database can be scaled out by splitting it into shards.
This allows the database to be managed by multiple nodes and effectively removes most limits on its growth.
In this manner, the size of the overall database, comprised of all shards, can reach dozens of terabytes and more,
while keeping the resources of each shard in check and maintaining high performance and throughput.

Licensing

• On a Developer license, the replication factor is restricted to 1.
• On Community and Professional licenses, all shards must be on the same node.
• Learn more about licensing here.

Client compatibility

Sharding is managed by the RavenDB server;
clients require no special adaptation when accessing a sharded database:

• The client API remains unchanged when using a sharded database.
• Clients using RavenDB versions older than 6.0, which lack sharding support,
  can seamlessly connect to a sharded database without any adaptations or even realizing it is sharded.

Specific modifications to RavenDB features in a sharded environment are documented in detail
in feature-specific articles.

Client-Server communication

When a client connects to a sharded database, it is appointed a RavenDB server that functions as an orchestrator,
mediating all the communication between the client and the database shards.
The client remains unaware of this process and uses the same API used by non-sharded databases to load documents, query, and perform other operations.

Note that this additional communication between the client and the orchestrator, as well as between the orchestrator and the shards, introduces some overhead compared to using a non-sharded database.

When should sharding be used?

While sharding solves many issues related to the storage and management of high-volume databases,
the overhead it introduces can outweigh its benefits when the database size still poses no problem.

You can postpone the transit to a sharded database when, for example, the database size is 100 GB,
the server is well-equipped and can comfortably handle a much larger volume,
and no dramatic increase in the number of potential users is expected any time soon.

We recommend that you plan ahead for a transition to a sharded database when your database size
is in the vicinity of 250 GB, so the transition is already well established when it reaches 500 GB.

While each cluster node of a non-sharded database handles a full replica of the entire database,
each shard is assigned a subset of the entire database content.

As far as clients are concerned, a sharded database is still a single entity:
clients are not required to detect whether the database is sharded or not, and clients of RavenDB versions prior to 6.0, which had no sharding support, can access a sharded database without any alterations.

That said, shard-specific operations are also available:
a client can, for example, track the shard where a document is stored and query that shard.
The Studio can be used to relocate (reshard) documents from one shard to another.

Studio Document View

Shard replication

Similar to non-sharded databases, shards can be replicated across cluster nodes to ensure the continuous availability of all shards in case of a node failure, provide multiple access points, and load-balance the traffic between shard replicas.

The number of nodes a shard is replicated to is determined by the Shard Replication Factor.

Shard Replication

• In the image above, a 3-shard database is hosted by a 5-node cluster
  (where two of the nodes, D and E, are not used by this database).

• The Shard Replication Factor is set to 2, maintaining two replicas of each shard.

Buckets

Documents in a sharded database are stored within virtual containers called buckets.
The number of documents and the amount of data stored in each bucket may vary.

Buckets allocation

Upon creating a sharded database, the cluster reserves 1,048,576 (1024 x 1024) buckets for the entire database.
Each shard is assigned a range of buckets from this overall set, where documents can be stored.
(Note: This default reservation method differs when using prefixed sharding. Learn more in Bucket management).

A range of buckets is assigned to each shard

Buckets population

The cluster automatically populates the buckets with documents in the following way:

A hashing algorithm is applied to each document ID, generating a number between 0 and 1,048,575.
The resulting number determines the bucket number where the document is stored.
(Note: This default hashing method differs when using prefixed sharding. Learn more in Bucket management.)

Since the buckets are pre-assigned to the shards,
the bucket number assigned to a document also determines which shard the document will reside on.

Buckets Population

Document extensions storage

Document extensions (i.e. Attachments, Time series, Counters, and Revisions) are stored in the same bucket as the document they belong to. To achieve this, the bucket number (hash code) for these extensions is calculated using the ID of the document that owns them.

Resharding is the relocation of data from one shard to another to maintain a balanced database,
where all shards handle approximately the same volume of data.

The resharding process moves all data related to a certain bucket, including documents, document extensions,
tombstones, etc., to a different shard and then associates the bucket with the new shard.

From the client's perspective, paging is conducted similarly in both sharded and non-sharded databases,
using the same API.

However, paging in a sharded database is more costly because the orchestrator must load data from each shard
and sort the retrieved results before handing the selected page to the client.

Read more about paging here.

The local IP address of a cluster node can be exposed, allowing other cluster nodes to prioritize it over the public IP address when accessing the node. Using a node's local IP address for inter-cluster communications can speed up the service and offer substantial cost savings over time.

Using this method can be particularly helpful in a sharded cluster, where each client request is handled by an orchestrator that may need to communicate with all other shards to process the request and its results.

Use this configuration option to expose a node's local IP address to other nodes.

• When a database is created, the user can choose whether it will be sharded or not.
  RavenDB (version 6.0 and later) provides this option by default, with no further steps required to enable the feature.

• A sharded database can be created via the Studio or the Client API.

• A RavenDB cluster can run both sharded and non-sharded databases in parallel.