How to Handle Document Relationships

The easiest solution is to denormalize the data within the containing entity, forcing it to contain the actual value of the referenced entity in addition to (or instead of) the foreign key.

Take this JSON document for example:

// Order document with ID: orders/1-A
{
    "customer": {
        "id": "customers/1-A",
        "name": "Itamar"
    },
    "items": [
        {
            "product": {
                "id": "products/1-A",
                "name": "Milk",
                "cost": 2.3
            },
            "quantity": 3
        }
    ]
}

As you can see, the Order document now contains denormalized data from both the Customer and the Product documents which are saved elsewhere in full. Note we won't have copied all the customer properties into the order; instead we just clone the ones that we care about when displaying or processing an order. This approach is called denormalized reference.

The denormalization approach avoids many cross document lookups and results in only the necessary data being transmitted over the network, but it makes other scenarios more difficult. For example, consider the following entity structure as our start point:

class Order {
    constructor(
        customerId = '',
        supplierIds = [],
        referral = null,
        lineItems = [],
        totalPrice = 0
    ) {
        Object.assign(this, {
            customerId,
            supplierIds,
            referral,
            lineItems,
            totalPrice
        });
    }
}

class Customer {
    constructor(
        id = '',
        name = ''
    ) {
        Object.assign(this, {
            id,
            name
        });
    }
}

If we know that whenever we load an Order from the database we will need to know the customer's name and address, we could decide to create a denormalized Order.customer field and store those details directly in the Order object. Obviously, the password and other irrelevant details will not be denormalized:

class DenormalizedCustomer {
    constructor(
        id = '',
        name = '',
        address = ''
    ) {
        Object.assign(this, {
            id,
            name,
            address
        });
    }
}

There wouldn't be a direct reference between the Order and the Customer. Instead, Order holds a DenormalizedCustomer, which contains the interesting bits from Customer that we need whenever we process Order objects.

But what happens when the user's address is changed? We will have to perform an aggregate operation to update all orders this customer has made. What if the customer has a lot of orders or changes their address frequently? Keeping these details in sync could become very demanding on the server. What if another process that works with orders needs a different set of customer properties? The DenormalizedCustomer will need to be expanded, possibly to the point that the majority of the customer record is cloned.

The Includes feature addresses the limitations of denormalization.
Instead of one object containing copies of the properties from another object, it is only necessary to hold a reference to the second object, which can be:

• a Document (as described below)
• a Document Revision
• a Counter
• a Time series
• a Compare exchange value

The server can then be instructed to pre-load the referenced object at the same time that the root object is retrieved, using:

const order = await session
     // Call 'include'
     // Pass the path of the document property that holds document to include
    .include("customerId")
    .load("orders/1-A");

const customer = await session
     // This call to 'load' will not require querying the server
     // No server request will be made
    .load(order.customerId);

Above we are asking RavenDB to retrieve the Order orders/1-A, and at the same time "include" the Customer referenced by the customerId property. The second call to load() is resolved completely client side (i.e. without a second request to the RavenDB server) because the relevant Customer object has already been retrieved (this is the full Customer object not a denormalized version).

There is also a possibility to load multiple documents:

const orders = await session
    .include("customerId")
    .load(["orders/1-A", "orders/2-A"]);

const orderEntities = Object.entries(orders);

for (let i = 0; i < orderEntities.length; i++) {
    // This will not require querying the server
    const customer = await session.load(orderEntities[i][1].customerId);
}

You can also use Includes with queries:

const orders = await session
    .query({ collection: "orders" })
    .whereGreaterThan("totalPrice", 100)
    .include("customerId")
    .all();

for (let i = 0; i < orders.length; i++) {
    // This will not require querying the server
    const customer = await session.load(orders[i].customerId);
}

const orders = await session
    .query({ collection: "orders" })
    .whereGreaterThan("totalPrice", 100)
    .include(i => i
        .includeDocuments("customerId")        // include document
        .includeCounter("OrderUpdateCounter")) // builder can include counters as well 
    .all();

for (let i = 0; i < orders.length; i++) {
    // This will not require querying the server
    const customer = await session.load(orders[i].customerId);
}

from "orders"
where totalPrice > 100
include customerId

from "orders" as o
where totalPrice > 100
include customerId, counters(o,'OrderUpdateCount')

This works because RavenDB has two channels through which it can return information in response to a load request. The first is the Results channel, through which the root object retrieved by the load() method call is returned. The second is the Includes channel, through which any included documents are sent back to the client. Client side, those included documents are not returned from the load() method call, but they are added to the session unit of work, and subsequent requests to load them are served directly from the session cache, without requiring any additional queries to the server.

One to many includes

Include can be used with a many to one relationship. In the above classes, an Order has a property supplierIds which contains an array of references to Supplier documents. The following code will cause the suppliers to be pre-loaded:

const order = await session
    .include("supplierIds")
    .load("orders/1-A");

for (let i = 0; i < order.supplierIds.length; i++) {
    // This will not require querying the server
    const supplier = await session.load(order.supplierIds[i]);
}

Alternatively, it is possible to use the fluent builder syntax.

const order = await session
    .load("orders/1-A", {
        includes: i => i.includeDocuments("supplierIds")
    });

for (let i = 0; i < order.supplierIds.length; i++) {
    // This will not require querying the server
    const supplier = await session.load(order.supplierIds[i]);
}

The calls to load() within the for loop will not require a call to the server as the Supplier objects will already be loaded into the session cache.

Multi-loads are also possible:

const orders = await session
    .include("supplierIds")
    .load(["orders/1-A", "orders/2-A"]);

const orderEntities = Object.entries(orders);

for (let i = 0; i < orderEntities.length; i++) {
    const suppliers = orderEntities[i][1].supplierIds;
    
    for (let j = 0; j < suppliers.length; j++) {
        // This will not require querying the server
        const supplier = await session.load(suppliers[j]);
    }
}

Secondary level includes

An Include does not need to work only on the value of a top level property within a document. It can be used to load a value from a secondary level. In the classes above, the Order contains a referral property which is of the type:

class Referral {
    constructor(
        customerId = '',
        commissionPercentage = 0
    ) {
        Object.assign(this, {
            customerId,
            commissionPercentage
        });
    }
}

This class contains an identifier for a Customer.
The following code will include the document referenced by that secondary level identifier:

const order = await session
    .include("referral.customerId")
    .load("orders/1-A");

// This will not require querying the server
const customer = await session.load(order.referral.customerId);

It is possible to execute the same code with the fluent builder syntax:

const order = await session
    .load("orders/1-A", {
        includes: i => i.includeDocuments("referral.customerId")
    });

// This will not require querying the server
const customer = await session.load(order.referral.customerId);

This secondary level include will also work with collections.
The lineItems property holds a collection of LineItem objects which each contain a reference to a Product:

class LineItem {
    constructor(
        productId = '',
        name = '',
        quantity = 0
    ) {
        Object.assign(this, {
            productId,
            name,
            quantity
        });
    }
}

The Product documents can be included using the following syntax:

const order = await session
    .include("lineItems[].productId")
    .load("orders/1-A");

for (let i = 0; i < order.lineItems.length; i++) {
    // This will not require querying the server
    const product = await session.load(order.lineItems[i].productId);
}

The fluent builder syntax works here too.

const order = await session
    .load("orders/1-A", {
        includes: i => i.includeDocuments("lineItems[].productId")
    });

for (let i = 0; i < order.lineItems.length; i++) {
    // This will not require querying the server
    const product = await session.load(order.lineItems[i].productId);
}

const order = await session
    .include("referral.customerId")
    .load("orders/1-A");

// This will not require querying the server
const customer = await session.load(order.referral.customerId);

curl -X GET "http://localhost:8080/databases/Northwind/docs?id=orders/1-A&include=Lines[].Product"

Dictionary includes

Dictionary keys and values can also be used when doing includes. Consider following scenario:

class Person {
    constructor(
        id = '',
        name = '',
        // attributes will be assigned a plain object containing key-value pairs
        attributes = {}
    ) {
        Object.assign(this, {
            id,
            name,
            attributes
        });
    }
}

const person1 = new Person();
person1.name = "John Doe";
person1.id = "people/1";
person1.attributes = {
    "mother": "people/2",
    "father": "people/3"
}

const person2 = new Person();
person2.name = "Helen Doe";
person2.id = "people/2";

const person3 = new Person();
person3.name = "George Doe";
person3.id = "people/3";

await session.store(person1);
await session.store(person2);
await session.store(person3);

await session.saveChanges();

Now we want to include all documents that are under dictionary values:

const person = await session
    .include("attributes.$Values")
    .load("people/1");

const mother = await session
    .load(person.attributes["mother"]);

const father = await session
    .load(person.attributes["father"]);

assert.equal(session.advanced.numberOfRequests, 1);

The code above can be also rewritten with fluent builder syntax:

const person = await session
    .load("people/1", {
        includes: i => i.includeDocuments("attributes.$Values")
    });

const mother = await session
    .load(person.attributes["mother"]);

const father = await session
    .load(person.attributes["father"]);

assert.equal(session.advanced.numberOfRequests, 1);

You can also include values from dictionary keys:

const person = await session
    .include("attributes.$Keys")
    .load("people/1");

Here, as well, this can be written with fluent builder syntax:

const person = await session
    .load("people/1", {
        includes: i => i.includeDocuments("attributes.$Keys")
    });

Dictionary includes: complex types

If values in dictionary are more complex, e.g.

class PersonWithAttribute {
    constructor(
        id = '',
        name = '',
        // attributes will be assigned a complex object
        attributes = {}
    ) {
        Object.assign(this, {
            id,
            name,
            attributes
        });
    }
}

class Attribute {
    constructor(
        ref = ''
    ) {
        Object.assign(this, {
            ref
        });
    }
}

const attr2 = new Attribute();
attr2.ref = "people/2";
const attr3 = new Attribute();
attr3.ref = "people/3";

const person1 = new PersonWithAttribute();
person1.name = "John Doe";
person1.id = "people/1";
person1.attributes = {
    "mother": attr2,
    "father": attr3
}

const person2 = new Person();
person2.name = "Helen Doe";
person2.id = "people/2";

const person3 = new Person();
person3.name = "George Doe";
person3.id = "people/3";

await session.store(person1);
await session.store(person2);
await session.store(person3);

await session.saveChanges();

We can also do includes on specific properties:

const person = await session
    .include("attributes.$Values[].ref")
    .load("people/1");

const mother = await session
    .load(person.attributes["mother"].ref);

const father = await session
    .load(person.attributes["father"].ref);

assert.equal(session.advanced.numberOfRequests, 1);

It is possible to combine the above techniques.
Using the DenormalizedCustomer from above and creating an order that uses it:

class Order2 {
    constructor(
        customer = {},
        supplierIds = '',
        referral = null,
        lineItems = [],
        totalPrice = 0
    ) {
        Object.assign(this, {
            customer,
            supplierIds,
            referral,
            lineItems,
            totalPrice
        });
    }
}

We have the advantages of a denormalization, a quick and simple load of an Order, and the fairly static Customer details that are required for most processing. But we also have the ability to easily and efficiently load the full Customer object when necessary using:

const order = await session
    .include("customer.id")
    .load("orders/1-A");

// This will not require querying the server
const customer = await session.load(order.customer.id);

This combining of denormalization and Includes could also be used with a list of denormalized objects.

It is possible to use Include on a query being a projection. Includes are evaluated after the projection has been evaluated. This opens up the possibility of implementing Tertiary Includes (i.e. retrieving documents that are referenced by documents that are referenced by the root document).

RavenDB can support Tertiary Includes, but before resorting to them you should re-evaluate your document model. Needing Tertiary Includes can be an indication that you are designing your documents along "Relational" lines.

There are no strict rules as to when to use which approach, but the general idea is to give it a lot of thought and consider the implications each approach has.

As an example, in an e-commerce application it might be better to denormalize product names and prices into an order line object since you want to make sure the customer sees the same price and product title in the order history. But the customer name and addresses should probably be references rather than denormalized into the order entity.

For most cases where denormalization is not an option, Includes are probably the answer.