RavenDB version 2.0. Other versions:

Stale indexes

RavenDB performs data indexing in a background thread, which is executed whenever new data comes in or existing data is updated. Running this as a background thread allows the server to respond quickly even when large amounts of data have changed, however in that case you may query stale indexes.

The notion of stale indexes comes from an observation deep in RavenDB's design, assuming that the user should never suffer from assigning the server big tasks. As far as RavenDB is concerned, it is better to be stale than offline, and as such it will return results to queries even if it knows they may not be as up-to-date as possible.

And indeed, RavenDB returns quickly for every client request, even if involves re-indexing hundreds of thousands of documents. And since the previous request has returned so quickly, the next query can be made a millisecond after that and results will be returned, but they will be marked as Stale.

Checking for stale results

As part of the response when an index is queried, a property is attached indicating whether the results are stale - that is, whether there are currently any tasks outstanding against that index. It can be fetched using the RavenQueryStatistics object:

    RavenQueryStatistics stats;
    var results = session.Query<Product>()
    	.Statistics(out stats)
    	.Where(x => x.Price > 10)
    	.ToArray();
    
    if (stats.IsStale)
    {
    	// Results are known to be stale
    }
    
    

When IsStale is true, that means someone probably added or changed a Product, and the indexes haven't had time to fully update before we queried.

For most cases you don't really care about that, however there are scenarios where you cannot work with data that could be stale.

Explicitly waiting for non-stale results

When it is a requirement to get non-stale results back from a query, it is possible to specify this while querying:

    RavenQueryStatistics stats;
    var results = session.Query<Product>()
    	.Statistics(out stats)
    	.Where(x => x.Price > 10)
    	.Customize(x => x.WaitForNonStaleResults(TimeSpan.FromSeconds(5)))
    	.ToArray();
    
    

Note that in the sample above a time-out of 5 seconds was specified. While you can ask RavenDB to wait indefinitely until there are non-stale results, this should only be used in unit-testing, and never in a real-world application, unless you are 100% sure you understand the implications, and that is what you want to have.

Setting cut-off point

Even when using WaitForNonStaleResults with a time-out like shown above, it is still possible to get back stale results - for example when a very lengthy indexing task is executing. A better approach to make sure you are working with non-stale results is to use a cut-off point and tell the server to use that as a base:

    RavenQueryStatistics stats;
    var results = session.Query<Product>()
    	.Statistics(out stats)
    	.Where(x => x.Price > 10)
    	.Customize(x => x.WaitForNonStaleResultsAsOf(new DateTime(2011, 5, 1, 10, 0, 0, 0)))
    	.ToArray();
    
    

This will make sure that you get the latest results up to that point in time. All pending tasks for changes occurred after this cut-off point will not be considered. And like before, a time-out can be set on that as well.

WaitForNonStaleResultsAsOfNow is also available, which is equivalent of calling WaitForNonStaleResultsAsOf(DateTime.Now).

Another option is to use WaitForNonStaleResultsAsOfLastWrite, which does exactly what it says it do. It tracks the last write by the application, and uses that as the cutoff point. This is usually recommended if you are working on machines where clock synchronization might be an issue, since WaitForNonStaleResultsAsOfLastWrite doesn't use the machine time, it uses the etag values for the writes.

You can also setup the document store to always wait for the last write, like so:

    documentStore.Conventions.DefaultQueryingConsistency = ConsistencyOptions.QueryYourWrites;
    
    

All queries in the store would behave as if WaitForNonStaleResultsAsOfLastWrite was applied to them.

Comments add new comment

The comments section is for user feedback or community content. If you seek assistance or have any questions, please post them at our support forums.

Mike Gleason
REPLY Posted by Mike Gleason on

Should this page discuss "QueryYourWrites"? And maybe the QueryYourWrites doesn't work with map/reduce.

Itamar Syn-Hershko
REPLY Posted by Itamar Syn-Hershko on

Yes to both questions... we will update the docs soon

Glenn Doten
REPLY Posted by Glenn Doten on

Does the ConsistencyOptions.MonotonicRead option set this behavior permanently for the database or is it just for the life of the IDocumentStore instance?

Itamar Syn-Hershko
REPLY Posted by Itamar Syn-Hershko on

Just for the IDocumentStore instance

Erik
REPLY Posted by Erik on

Uhhh... this page contradicts "https://groups.google.com/forum/?fromgroups#!topic/ravendb/zapvDGuO-AA".

If i wait to setup the store to always wait for nonstale results, is it monotonicRead or QueryYourWrites ?

QueryYourWrites sounds more like what i am looking for

Ayende Rahien
REPLY Posted by Ayende Rahien on

That would be QueryYourWrites, you are correct, we will fix the docs.

Federico Andres Lois
REPLY Posted by Federico Andres Lois on

From experience I wouldnt recommend to use in any case of:

documentStore.Conventions.DefaultQueryingConsistency = ConsistencyOptions.QueryYourWrites;

It is pretty easy to mistakenly believe that QueryYourWrites applies only to "you/session" instead of the datastore as a whole (because in a web environment probably your datastore means the entire database) can lead you to problems that are not easily seen in typical development environments.

Using QueryYourWrites is a recipe to shoot yourself in the foot when in production your traffic starts to rise.

Rajmund
REPLY Posted by Rajmund on

Hi. I use

documentStore.Conventions.DefaultQueryingConsistency = ConsistencyOptions.QueryYourWrites

statement in my code and it works fine. But how can I set global timeout for all queries for non stale indexes.

SUBMIT COMMENT