Storage: Customizing RavenDB Data Files Locations

Storing RavenDB data files in different devices, by customizing their locations.

Overview

Motivation:

  • Avoiding traffic jams.

  • Better concurrency.

  • Directing each file or directory (e.g. Raven.voron file, Journals, etc.) to a data storage according to its speed, durability, etc.

Main components:

  • Journals - files for unbuffered, sequential writes, typically write only (except for recovery operations).
    Journals are also in the critical path for commit operations, since a commit is not confirmed until the write is done.

  • Raven.voron files - memory mapped, buffered files with random reads and writes.
    Write operations are async, but slow devices may cause slow downs (especially on reads).

Each database includes a Journals, a Temp and an Indexes folder, and a Raven.voron data file.
Each index folder has its own Journals and Temp folders, and a Raven.voron file.
All these components (Journals, Raven.voron, etc.) operate concurrently with each other and can be stored on different devices.
It allows you to organize the files based on their usage pattern and the performance of the different devices you own.

Practice

The structure of the RavenDB directories cannot be changed. An exception is locations of temporary files for documents and indexes, that can be changed by setting appropriate configuration options.

However, you can store any RavenDB data files or directories in whatever location you choose, by defining junction points (Windows) or mount points (Linux).

Example - Moving Journals

A common practice is to store the journals on a very fast drive to achieve better write performance. The following command will point the Journals directory of the Northwind database to a path on a different drive.

Windows

C:\RavenDB\Server\RavenData\Databases\Northwind>mklink /J Journals E:\Journals\Northwind

Linux

ln -s ~/RavenDB/Server/RavenData/Databases/Northwind/Journals /mnt/FastDrive/Databases/Northwind/Journals

Example - Moving Indexes

If you want to store the data of all indexes of the Northwind database in the custom location, you can use the following command:

Windows

C:\RavenDB\Server\RavenData\Databases\Northwind>mklink /J Indexes D:\Indexes\Northwind

Information

Creation of junction / mount points requires the database to be offline

Information

If data already exists in the directory, and you want to define the junction (Windows) or mount (Linux) point, you need to create a backup of the data first, and copy it back into the directory after executing the command.

Linux

ln -s ~/RavenDB/Server/RavenData/Databases/Northwind/Indexes /mnt/FastDrive/Databases/Northwind/Indexes

Information

Start RavenDB server after creating soft link to a faster drive mount point: