see on GitHub

Indexes: Fanout Indexes

The fanout index is the index that outputs multiple index entries per each document. Here is an example of such one:

public static class Orders_ByProduct extends AbstractIndexCreationTask {
    public Orders_ByProduct() {
        map = "docs.Orders.SelectMany(order => order.Lines, (order, orderLine) => new { " +
            "    Product = orderLine.Product, " +
            "    ProductName = orderLine.ProductName " +
            "})";
    }
}
public static class Orders_ByProduct extends AbstractJavaScriptIndexCreationTask {
    public Orders_ByProduct() {
        setMaps(Sets.newHashSet("map('Orders', function (order){\n" +
            "    var res = [];\n" +
            "    order.Lines.forEach(l => {\n" +
            "        res.push({\n" +
            "            Product: l.Product,\n" +
            "            ProductName: l.ProductName\n" +
            "        })\n" +
            "    });\n" +
            "    return res;\n" +
            "})"));
    }
}

A large order, having a lot of line items, will create an index entry per each OrderLine item from the Lines collection. A single document can generate hundreds of index entries.

The fanout index concept is not specific for map-only indexes. It also applies to map-reduce indexes:

public static class Product_Sales extends AbstractIndexCreationTask {
    public static class Result {
        private String product;
        private int count;
        private double total;

        public String getProduct() {
            return product;
        }

        public void setProduct(String product) {
            this.product = product;
        }

        public int getCount() {
            return count;
        }

        public void setCount(int count) {
            this.count = count;
        }

        public double getTotal() {
            return total;
        }

        public void setTotal(double total) {
            this.total = total;
        }
    }

    public Product_Sales() {
        map = "docs.Orders.SelectMany(order => order.Lines, (order, line) => new { " +
            "    Product = line.Product, " +
            "    Count = 1, " +
            "    Total = (((decimal) line.Quantity) * line.PricePerUnit) * (1M - line.Discount) " +
            "})";

        reduce = "results.GroupBy(result => result.Product).Select(g => new {\n" +
            "    Product = g.Key,\n" +
            "    Count = Enumerable.Sum(g, x => ((int) x.Count)),\n" +
            "    Total = Enumerable.Sum(g, x0 => ((decimal) x0.Total))\n" +
            "})";
    }
}
public static class Product_Sales extends AbstractJavaScriptIndexCreationTask {
    public static class Result {
        private String product;
        private int count;
        private double total;

        public String getProduct() {
            return product;
        }

        public void setProduct(String product) {
            this.product = product;
        }

        public int getCount() {
            return count;
        }

        public void setCount(int count) {
            this.count = count;
        }

        public double getTotal() {
            return total;
        }

        public void setTotal(double total) {
            this.total = total;
        }
    }

    public Product_Sales() {
        setMaps(Sets.newHashSet("map('orders', function(order){\n" +
            "            var res = [];\n" +
            "            order.Lines.forEach(l => {\n" +
            "              res.push({\n" +
            "                Product: l.Product,\n" +
            "                Count: 1,\n" +
            "                Total:  (l.Quantity * l.PricePerUnit) * (1- l.Discount)\n" +
            "              })\n" +
            "            });\n" +
            "            return res;\n" +
            "        })"));

        setReduce("groupBy(x => x.Product)\n" +
            "    .aggregate(g => {\n" +
            "        return {\n" +
            "            Product : g.key,\n" +
            "            Count: g.values.reduce((sum, x) => x.Count + sum, 0),\n" +
            "            Total: g.values.reduce((sum, x) => x.Total + sum, 0)\n" +
            "        }\n" +
            "    })");
    }
}

The above index definitions are correct. In both cases this is actually what we want. However, you need to be aware that fanout indexes are typically more expensive than regular ones. RavenDB has to index many more entries than usual. What can result is higher utilization of CPU and memory, and overall declining performance of the index.

Note

Starting from version 4.0, the fanout indexes won't error when the number of index entries created from a single document exceeds the configured limit. The configuration options from 3.x:

  • Raven/MaxSimpleIndexOutputsPerDocument
  • Raven/MaxMapReduceIndexOutputsPerDocument

are no longer valid.

RavenDB will give you a performance hint regarding high fanout ratio using the Studio's notification center.

Performance Hints

Once RavenDB notices that the number of indexing outputs created from a document is high, the notification that will appear in the Studio:

Figure 1. High indexing fanout ratio notification

High indexing fanout ratio notification

The details will give you the following info:

Figure 2. Fanout index, performance hint details

Fanout index, performance hint details

You can control when a performance hint should be created using the PerformanceHints.Indexing.MaxIndexOutputsPerDocument setting (default: 1024).

Paging

Since the fanout index creates multiple entries for a single document and queries return documents by default (it can change if the query defines the projection) the paging of query results is a bit more complex. Please read the dedicated article about paging through tampered results.