Creating and deploying indexes

Indexes are used by server to satisfy queries, whenever a user issues a query RavenDB will use an existing index if it matches query or create a new one if no match is found.

Remember

Indexes created by issuing a query are called dynamic or Auto indexes and can be easily identified, due to their name that starts with Auto/ prefix.

Indexes created explicitly by user are called static.

Static indexes

There are a couple of ways to create static index and send it to server, we can use low-level commands or create a custom class. There is also a possibility to scan an assembly and deploy all found indexes or even deploy indexes side by side.


using AbstractIndexCreationTask

If you are interested in having a strongly-typed syntax during index creation, have an ability to deploy indexes using assembly scanner, avoid hard-coding index names in every query and do not worry about setting sorting for numerical fields, then AbstractIndexCreationTask should be your choice.

Note

We recommend creating and using indexes in this form due to its simplicity, many benefits and few disadvantages.

Safe By Default

When index is deployed using AbstractIndexCreationTask then all numerical fields in generated index definition will contain appropriate sorting options set.

Naming conventions

Actually there is only one naming conventions: each _ in class name will be translated to / in index name.

e.g.

In Northwind samples there is a index called Orders/Totals. To get such a index name, we need to create class called Orders_Totals.

public class Orders_Totals : AbstractIndexCreationTask<Order>
{
	// ...
}

Sending to server

There is not much use from an index if it is not deployed to server. In order to do so, we need to create instance of our class that inherits from AbstractIndexCreationTask and use one of the deployment methods: Execute or ExecuteAsync for asynchronous call.

// deploy index to `DefaultDatabase` for given `DocumentStore`
// using store `Conventions`
new Orders_Totals().Execute(store);

// deploy asynchronously index to `DefaultDatabase` for given `DocumentStore`
// using store `Conventions`
await new Orders_Totals().ExecuteAsync(store.AsyncDatabaseCommands, store.Conventions);

// deploy index to `Northwind` database
// using store `Conventions`
new Orders_Totals().Execute(store.DatabaseCommands.ForDatabase("Northwind"), store.Conventions);

Safe By Default

If index exists on server and stored definition is the same as the one that was send, then it will not be overwritten, which implies that indexed data will not be deleted and indexation will not start from scratch.

Using assembly scanner

All classes that inherit from AbstractIndexCreationTask can be deployed at once using one of IndexCreation.CreateIndexes method overloads.

// deploy all indexes (and transformers) 
// from assembly where `Orders_Totals` is found
// to `DefaultDatabase` for given `DocumentStore`
IndexCreation.CreateIndexes(typeof(Orders_Totals).Assembly, store);

Underneath, the IndexCreation will attempt to create all indexes in a single request. If it fails, then it will repeat the execution by calling Execute method one-by-one for each of found indexes in separate requests.

Warning

IndexCreation.CreateIndexes will also deploy all classes that inherit from AbstractTransformerCreationTask (more about it here).

Example

public class Orders_Totals : AbstractIndexCreationTask<Order>
{
	public class Result
	{
		public string Employee { get; set; }

		public string Company { get; set; }

		public decimal Total { get; set; }
	}

	public Orders_Totals()
	{
		Map = orders => from order in orders
				select new
				{
					order.Employee,
					order.Company,
					Total = order.Lines.Sum(l => (l.Quantity * l.PricePerUnit) * (1 - l.Discount))
				};
	}
}

public static void Main(string[] args)
{
	using (DocumentStore store = new DocumentStore
	{
		Url = "http://localhost:8080",
		DefaultDatabase = "Northwind"
	})
	{
		store.Initialize();

		new Orders_Totals().Execute(store);

		using (IDocumentSession session = store.OpenSession())
		{
			IList<Order> orders = session
				.Query<Orders_Totals.Result, Orders_Totals>()
				.Where(x => x.Total > 100)
				.OfType<Order>()
				.ToList();
		}
	}
}

using Commands

Low-level PutIndex command from DatabaseCommands (which API reference can be found here) can be used also to send index to server.

The benefit of this approach is that you can choose the name as you feel fit and change various settings available in IndexDefinition, but loose the ability to deploy using assembly scanner. Also you will have to use string-based names of indexes when querying.

store
	.DatabaseCommands
	.PutIndex("Orders/Totals", new IndexDefinition
	{
		Map = @"from order in docs.Orders
			select new { order.Employee,  order.Company, Total = order.Lines.Sum(l => (l.Quantity * l.PricePerUnit) * (1 - l.Discount)) }"
	});

IndexDefinitionBuilder

IndexDefinitionBuilder is a very useful class that enables you to create IndexDefinitions using strongly-typed syntax with an access to low-level settings not available when AbstractIndexCreationTask approach is used.

IndexDefinitionBuilder<Order> builder = new IndexDefinitionBuilder<Order>();
builder.Map = orders => from order in orders
						select new
						{
							order.Employee,
							order.Company,
							Total = order.Lines.Sum(l => (l.Quantity * l.PricePerUnit) * (1 - l.Discount))
						};

store
	.DatabaseCommands
	.PutIndex("Orders/Totals", builder.ToIndexDefinition(store.Conventions));

Remarks

Information

Commands or IndexDefinitionBuilder approaches are not recommended and should be used only if you can't do that by inheriting from AbstractIndexCreationTask.

Auto indexes

Auto-indexes are created when queries that do not specify an index name are executed and (after in-depth query analysis) no matching index is found on server-side.

Naming convention

As mentioned earlier auto-indexes can be recognized by Auto/ prefix in name. Their name also contains name of a collection that was queried and list of fields that were required to find valid query results.

For instance, issuing query like this

List<Employee> employees = session
	.Query<Employee>()
	.Where(x => x.FirstName == "Robert" && x.LastName == "King")
	.ToList();

will result in a creation of a index named Auto/Employees/ByFirstNameAndLastName.

Disabling creation of Auto indexes

By default, Raven/CreateAutoIndexesForAdHocQueriesIfNeeded configuration option is set to true, which allows auto-index creation if needed. To disable it server or database-wide please refer to this article.

Auto indexes and indexing prioritization

To reduce the server load, if auto-indexes are not queried for a certain amount of time defined in Raven/TimeToWaitBeforeMarkingAutoIndexAsIdle setting (1 hour by default), then they will be marked as Idle. You can read more about implications of marking index as Idle here.

If this is disabled, you'll have to manually ensure that all queries have covering indexes. This is not a recommended configuration.

Remarks

In-memory indexes

By default, new indexes are created in-memory (can be disabled by changing Raven/DisableInMemoryIndexing to false) and persisted to disk when one of the following conditions are met:

  • index is not stale after write,
  • when in-memory size of an index exceeds 64MB (can be altered by changing Raven/NewIndexInMemoryMaxMB),
  • when difference between current write time and index creation time exceeds 15 minutes (can be altered by changing Raven/NewIndexInMemoryMaxTime),
  • when backup is created,
  • when database is closed,
  • on user request. More info here.