Indexes: Index Throttling

Indexing a rapidly modified group of items (e.g. a time series, a group of counters, or a patched collection) can preoccupy RavenDB with lengthy consecutive indexing of small batches of items.

Throttles Index Performance View

The Indexing Performance view shown above demonstrates the procession of a throttled index along the timeline, with a 1500 Milliseconds gap between consecutive batches.
Throttling offers indexing-free time gaps that the database can use for other tasks and the procession of larger batches when indexing does take place.

The delay in index procession increases the size of Item batches that are processed during indexing intervals.

In special cases, throttled indexes' batches are processed without delay.

• The batch following a batch that cannot be processed for known reasons, will be processed without delay.
• Accumulated data (collected due to batch size configuration, for example) whose procession requires multiple batches, will be processed with no delay between batches.

Server-Wide Index Throttling

Index Throttling can be set server-wide using a designated configuration option.
Setting the server-wide configuration option will apply to all databases on a given node.

• Set the index throttling time interval in Milliseconds using the Indexing.Throttling.TimeIntervalInMs configuration option, e.g.:
  "Indexing.Throttling.TimeIntervalInMs": "5000"

Index Throttling Per-Database

Enable or disable index throttling for a specific database using the designated database configuration key.
Setting this property overrides the Server-Wide default.

• From Studio:

  

  Database Configuration Keys

  1. Open Settings > Database Settings view.
  2. Filter Keys - Enter a search string to locate the configuration key.
  3. Edit - Click to edit values (see next image for details).
  4. Configuration Key -
     Indexing.Throttling.TimeIntervalInMs - Index Throttling configuration key.
  5. Effective Value - The current configuration.
  6. Origin - The origin of the current configuration.
     Can be - Default | Database
  

  Edit Values

  1. Override - Toggle to override the server-wide configuration.
  2. Edit Value - Enter a new time in Milliseconds, or leave empty for null (no throttling).
  3. Set Default - Click 'Set Default' to select the server-wide default value.
  4. Save - Apply changes.
     

     An edited configuration key's value will become effective only after the database is reloaded.
     

Index Throttling in an Index Definition

Setting throttling in an index definition overrides server-wide and database settings configuration.

• From Code:
  Set throttling for a specific index using the Indexing.Throttling.TimeIntervalInMs property.
  

  var index = new IndexDefinition
  {
      Name = "ByOrderedAt",
      Maps = {"from o in orders select new {o.OrderedAt}"},
      Configuration = new IndexConfiguration
      {
          { "Indexing.Throttling.TimeIntervalInMs", "2000" }
      },
  };

  The batch size can be left for RavenDB to decide, or you can set it yourself using the Indexing.MapBatchSize property.
  

  Configuration = new IndexConfiguration
  {
      { "Indexing.Throttling.TimeIntervalInMs", "2000" },
      { "Indexing.MapBatchSize", "50" }
  },

• From Studio:
  Use the Configuration tab of an index definition to set the index's Indexing.Throttling.TimeIntervalInMs configuration key value in Milliseconds.
  

  1. Configuration Tab - Open to set index configuration keys.
  2. Add Customized Indexing Configuration - Click to add a configuration key.
  3. Indexing Configuration Key -
     Type Indexing.Throttling.TimeIntervalInMs or select this key from the droplist to set index throttling.
  4. Value - Type the throttling time interval in Milliseconds.