RavenDB version 2.5. Other versions:

Exporting and Importing data

In order to export or import data from a RavenDB server, you can use the Raven.Smuggler utility.

Raven.Smuggler is distributed in both the: - RavenDB distribution package. It is located under the /Smuggler folder. - RavenDB.Server nuget package. It is located under the /tools folder.

Using the Smuggler utility is necessary when trying to move a RavenDB Data folder around between servers. Simply copying it is not supported and can result in server errors.

Exporting

To Export data, use this command:

Raven.Smuggler out http://localhost:8080 dump.raven

This command will export all indexes, documents and attachments from the local RavenDB instance to a file named dump.raven.

The dump file will also include documents that were added during the export process, so you can make changes while the export is executing.

From RavenDB 2.5 the Smuggler is using document streaming to speed up the process. To maintain backward compatibility, the Smuggler will detect from what version it exports the documents and adjust behavior accordingly.

Importing

Raven.Smuggler in http://localhost:8080 dump.raven

This command will import all the indexes, documents and attachments from the file to the local instance.

This will overwrite any existing document on the local instance.

You can continue using that RavenDB instance while data is being imported to it.

To speed up the process, the Raven.Smuggler.exe is using bulk inserts and The Studio is using batching.

Incremental Export and Import

With the incremental export operation we can use in order to backup the database incrementally, on each export, we will only take the export the documents create or updated since the last export.

To export data with incremental we can use 2 options.
If it is the first run and the folder does not exist yet use (you can continue to use this command every time):

Raven.Smuggler out http://localhost:8080 folder_location --incremental

If you ran the command before or you created the folder you can use:

Raven.Smuggler out http://localhost:8080 folder_location

In order to import date that was exported with incremental operation, you can use either of the following:

1) Raven.Smuggler in http://localhost:8080 folder_location --incremental
2) Raven.Smuggler in http://localhost:8080 folder_location

Command line options

You can tweak the export/import process with the following parameters:

  • operate-on-types: Specify the types to export/import. Usage example: --operate-on-types=Indexes,Documents,Attachments,Transformers.
  • filter: Filter documents by a document property. Usage example: --filter=Property-Name=Value.
  • negative-filter: Filter documents NOT matching a document property. Usage example: --negative-filter=Property-Name=Value.
  • metadata-filter: Filter documents by a metadata property. Usage example: --metadata-filter=Raven-Entity-Name=Posts.
  • negative-metadata-filter: Filter documents NOT matching a metadata property. Usage example: --negative-metadata-filter=Raven-Entity-Name=Posts.
  • transform: Transform documents using a given script (import only).
  • transform-file: Transform documents using a given script file (import only).
  • timeout: The timeout (in milliseconds) to use for requests.
  • batch-size: The batch size for requests.
  • database: The database to operate on. If no specified, the operations will be on the default database.
  • username: The username to use when the database requires the client to authenticate.
  • password: The password to use when the database requires the client to authenticate.
  • domain: The domain to use when the database requires the client to authenticate.
  • api-key: The API-key to use, when using OAuth.
  • incremental: States usage of incremental operations.
  • wait-for-indexing: Wait until all indexing activity has been completed (import only).
  • excludeexpired: Excludes expired documents created by the expiration bundle.
  • help: You can use the help option in order to print the built-in options documentation.

Filtering

To filter out documents we introduced few filtering options that can be used during import or export process.

  1. filter is used to filter documents based on a property. E.g. if we want to export all documents with property Name and value John then we must apply command as follows: --filter=Name=John .
  2. negative-filter is an opposite to filter and will filter documents that does NOT match the given property.
  3. metadata-filter is similar to filter, but works on document metadata properties.
  4. negative-metadata-filter filters out documents that does NOT match given metadata property.

Transforms

Transforms can be used to modify or filter out documents, but only work during the import process. The scripts must use JavaScript syntax and be in following format:

    
        
    	function(doc) {
    		// custom code here
    	}
    
    

where doc will contain our document with it's metadata under @metadata property.

Change scripts:

E.g. To change document property Name value to the new one, the following script can be used:

    
        
    	function(doc) {
    		doc['Name'] = 'NewValue';
    		return doc;
    	}
    
    

Filter scripts:

If we return null then the document will be filtered out.

    
        
    	function(doc) {
    		var id = doc['@metadata']['@id']; 
    		if(id === 'orders/999')
    			return null;
    
    		return doc;
    	}
    
    

SmugglerApi

Alternatively, if you prefer to do export/import from code rather than from the console utility, you can use the SmugglerApi class. In order to use this class you need to reference the Raven.Smuggler.exe.

Usage example:

    var smugglerOptions = new SmugglerOptions { };
    
    var connectionStringOptions = new RavenConnectionStringOptions
    {
    	ApiKey = "ApiKey",
    	Credentials = new NetworkCredential("username", "password", "domain"),
    	DefaultDatabase = "database",
    	Url = "http://localhost:8080",
    };
    
    var smugglerApi = new SmugglerApi(smugglerOptions, connectionStringOptions);
    smugglerApi.ExportData(null, new SmugglerOptions { BackupPath = "dump.raven", OperateOnTypes = ItemType.Documents | ItemType.Indexes | ItemType.Attachments | ItemType.Transformers }, incremental: false);
    smugglerApi.ImportData(null, new SmugglerOptions { BackupPath = "dump.raven", OperateOnTypes = ItemType.Documents | ItemType.Indexes });
    
    

In the above code we exporting all of the data on the server, which is the documents, indexes, attachments and transformers, and than importing just the documents and the indexes. In this example the import would overwrite the existing documents, so if you want to import to another database you'll need to create another instance of the SmugllerApi with a different connection string options.

Comments add new comment

The comments section is for user feedback or community content. If you seek assistance or have any questions, please post them at our support forums.

Eugene  Safronov
REPLY Posted by Eugene Safronov on

Please update this page with description about import/export capabilities for Embedded database. I spent a lot of time before I found out that Smuggler API required http server up and running to get it to work: http://goo.gl/mHfcC. On the other hand there is a nice alternative with DataDumper class for Embedded ravenDb: http://goo.gl/TZ7i3

Dennis Kozora
REPLY Posted by Dennis Kozora on

When migrating from a newer Windows OS to an older Windows OS (ie Win8 to Win7), DO NOT use backup/restore . Use the import/export features. Also, for Embedded databases, use the Raven DataImporter class. The DataDumper does NOT require an embedded document store to initialize with UseEmbeddedHttpServer set to true (Smuggler API requires an http server).

class Program { static void Main(string[] args) { using (var documentStore = new EmbeddableDocumentStore { DataDirectory = @"C:\temp\ExistingDocumentStore", //DataDirectory = @"C:\temp\NewDocumentStore", }) {

            documentStore.Initialize();

            var options = new SmugglerOptions
                {
                    BackupPath = @"C:\temp\ravendb.backup"
                };

            var dumper = new DataDumper(documentStore.DocumentDatabase, options);

            //create the backup file; DataDirectory should point to the raven db to be backed up
            dumper.ExportData(options);

            //dumper.ImportData(options);
         }
    }
}

Raven Worshiper
REPLY Posted by Raven Worshiper on

Hey, I'm using raven.smuggler to Export from ravenDB version 1.0 and Import to ravenDB 2.0.2375. I'm getting file load exception due to lucene.net version differences (the analyzer which i'm using older than the current, the one in 2.0.2375 version). What should i do to migrate the data between two ravenDB versions correctly ?

Lorenzo Dieryckx
REPLY Posted by Lorenzo Dieryckx on

If you want to use the Smuggler API from code, you also need to reference Jint.Raven.

turkush
REPLY Posted by turkush on

SmugglerApi is not being referenced through raven.Abstraction.Smuggler. there is ISmugglerApi but where to mention connection if to use inteface export/import methods?

SUBMIT COMMENT