Max Batch Size
When heavy-duty indexing is done on a large dataset, there are ways to prevent exhausting memory and other system resources.
Indexing is done in batches so that we can calibrate the number of documents that are processed per batch
in accordance with available system resources.
Additionally, batching creates pauses which free resources so that RavenDB can quickly serve its clients
before it continues building the index.
- e.g. Processing 1,000,000 documents might exhaust memory and other system resources
if the configuration is set to process them all in one batch.
Instead, the index can be set to process the documents in batches of 10,000 to save system resources for other tasks.
- If RavenDB is indexing documents in multiple batches, it will continue processing each new batch where the previous batch stopped.
- To calibrate your indexing batch size, you can adjust the
Indexing.MapBatchSize configuration setting.
The factors to consider when adjusting the max indexing batch size:
Cloud Indexing Batch Size
RavenDB Cloud sets batch sizes with the following default formula:
Indexing.MapBatchSize = max(PowerOf2(iops * 5), 1024);
Explanation of the configuration value:
In an instance with IOPS of 500
max() = returns the larger of two arguments:
PowerOf2(iops * 5) = The power of two that's larger than the argument passed
500 * 5 = 2,500. The next power of two after 2,500 is 2 ^ 12, which is 4,096 documents.
1024 = minimum number of documents
In this example, 4,096 is larger than 1024, so the maximum batch size will be 4,096 documents.
Overriding the Default Setting
Change cloud configurations in Studio.
Value: Set your preferred maximum number of documents per batch.
Changing the cloud instance IOPS number
(IOPS - Input/Output Operations Per Second - can be adjusted according to your needs.)
Find IOPS Number
In the Cloud Portal, click the Products tab.
Your current IOPS number is written here.
It can be adjusted by clicking "Change Storage".