Session: Querying:
How to Filter by Non-Existing Field
-
There are situations where over time new fields are added to documents.
You may need to create a list of all of the documents that don't have these fields.- You can then write a patch to add the missing fields.
-
To find documents with a missing field you can either:
Query a Static Index
You can search for documents with missing fields by using a static index if it indexes the field which is suspected to be missing in some of the documents.
The index definition must also index a field that exists in every document (such as Id
) so that all documents will be indexed.
- For example, if you want to find documents that are missing the field
Freight
in theOrders
collection,
query an index that indexes the fieldsFreight
andId
. -
If your static index does not contain the desired field, either
- Modify your index definition to index the specific field. (This will trigger re-indexing.)
- Create an auto-index.
Example: Query a Static Index
In our example, we are looking for documents that are missing the field Freight
from the collection Orders
.
First we need an index that includes Freight
and a field that exists in every document
We index the missing field Freight
and the field Id
, which exists in every document.
This way, the index includes all of the documents in the collection,
including those that are missing the specified field.
// Create or modify a static index called Orders_ByFreight
public class Orders_ByFreight : AbstractIndexCreationTask<Order>
{
public Orders_ByFreight()
{
// Specify collection name
Map = orders => from doc in orders
select new
{
// Field that is missing in some documents
doc.Freight,
// Field that exists in all documents
doc.Id
};
}
}
Then we query the index to find documents where the field does not exist
SAMPLE QUERY:
Query the index Orders_ByFreight
and filter documents where freight
does not exist.
List<Order> results = session
.Advanced
// Query the static index
.DocumentQuery<Order, Orders_ByFreight>()
// Verify that the index is not stale (optional)
.WaitForNonStaleResults(TimeSpan.MaxValue)
// Negate the next method
.Not
// Specify the field that is suspected to be missing
.WhereExists(x => x.Freight)
.ToList();
// `results` will contain the list of incomplete documents.
from index "Orders/ByFreight"
where true and not exists("Freight")
// `not` cannot come immediately after `where`, thus we use `where true`.
LINQ SYNTAX:
List<T> results = session
.Advanced
.DocumentQuery<T, TIndexCreator>()
.Not
.WhereExists("missingFieldName")
.ToList();
Parameters | Type | Description |
---|---|---|
T | string | An object in a collection (singular of the collection name - e.g. Order from the collection Orders). |
TIndexCreator | string | The name of the index that you want to use. |
missingFieldName | string | The field that is missing in some of the documents. |
Query the Collection to Create an Auto-Index
Another option is to query the collection for the missing field.
This will either create a new auto-index or add the new field to an existing auto-index if it indexes the same collection.
See the example and query syntax descriptions below:
Example: A query that creates an Auto-Index
The following query will create an auto-index on the "Freight" field
that is missing in some documents in the Orders collection.
The query result will contain all documents that do not have this field.
List<Order> results = session
.Advanced
.DocumentQuery<Order>()
.Not
.WhereExists("Freight")
.ToList();
from "Orders"
where true and not exists("Freight")
// `not` cannot be used immediately after `where`, thus we use `where true`.
LINQ Query Syntax
List<T> results = session
.Advanced
.DocumentQuery<T>()
.Not
.WhereExists("missingFieldName")
.ToList();
Parameters | Type | Description |
---|---|---|
T | string | An object in a collection (singular of the collection name - e.g. Order from the collection Orders). |
missingFieldName | string | The field that is missing in some of the documents. |
Use Studio to filter by non-existing field
You can also use Studio to find missing fields with an RQL query such as:
from "Orders"
where exists("Company") and not exists("Freight")
In Studio we always use RQL syntax.
Like the LINQ syntax examples above,
we must first call a field that exists in every document in the collection
and then the field that does not exist in some of them.
List Documents Without a Specified Field
- Indexes
Click to see the Indexes menu items. - Query
Select to open the Query view. - Query editor
Write the query according to the RQL example described above. - Run Query
Click or press ctrl+enter to run the query. - Index used
This is the name of the auto-index created for this query.
You can click it to see the available Studio options for this index. - Results
This is the list of documents that do not have the specified field.
(The field "Freight" was explicitly removed from these Northwind documents for this example.)