Indexing can be a taxing operation on CPU resources.
There are a number of configurations that efficiently use
batch stops to break up huge batch processes into smaller batches to prevent exhausting resources.
If a configuration is specific to an index, it can be set in the Studio.
If it is a server-wide only configuration, it must be set in the server's settings.json.
While they prevent system exhaustion, batch stops also point to potential opportunities to optimize your indexes.
Batch stops break up processes into smaller batches when
Some indexes are responsible for a huge dataset and/or have very complex, demanding definitions.
To prevent resource exhaustion, RavenDB can break up large batches into smaller ones.
You can configure batch stops with the following methods:
Low Memory resources can slow down your system and result in batch stops.
On a local machine
- You can upgrade your hardware, divide the work onto more machines in a cluster, and/or optimize your indexes.
- Until then, there are a number of indexing configurations that you can configure
to break up processes into smaller batches.
Your indexing process will continue until it is finished, but will be broken up into smaller batches and continue when enough CPU credits accumulate.
This can happen on basic-level cloud instances.
Concurrent Processing of Too Many Indexes
- Limit concurrent index processes -
RavenDB can handle multiple index processes at the same time, but if there are too many, it will exhaust the system resources and cause a
noticeable slow-down. The
Indexing.MaxNumberOfConcurrentlyRunningIndexes method enables you to have many indexes without exhausting resources by allowing you to set
the number of concurrent index processes.
Indexing referenced/related data
can be useful (even in a NoSql database) when developers need to pull information from different documents into an indexing process. The
creates a relationship between the two documents and ensures that whenever the referenced document is updated, the referencing documents will be re-indexed to
stay current with the new details.
LoadDocument is a useful feature, but problems arise if a large number of documents reference a single document (or a small set of them) that is frequently changed.
If frequent changes are made to this document, all the documents referencing it will also need to be reindexed.
In other words, the amount of work that an index has to do because
of a single document frequently changing can be extremely large and may cause delays in indexing.
- The high IO demands in this situation can then cause further problems such as longer request duration and cluster instability.
LoadDocument misuse is caused by trying to apply relational modeling approaches to document-based databases.
If you're accustomed to relational data modeling, you can learn about effective document modeling in the "Inside RavenDB" book.