Sharding: Import and Export

When smuggler is called to export a sharded database:

• The client sends an export request to the orchestrator.
• The orchestrator forwards the request to all other shards.
• All shards stream their data to the client machine, where the data is gathered in a single .ravendbdump file.
  

  The amount of time it takes to complete the export process depends upon factors like shards' databases sizes, network performance, and the user-defined transform script.
  

See a code example here.

Smuggler can be used to import data into a database from either a .ravendbdump file or from backup files (full or incremental).

When data is imported into a sharded database one of the shard nodes is appointed orchestrator; the orchestrator retrieves items from the .ravendbdump or backup files, gathers the imported items into batches, and distributes them among the shards.

Importing data from a .ravendbdump file

There are no preliminary requirements regarding the structure or contents of the database the data is imported to.

• The data can be imported into a long-existing database, as well as into a database just created.
• The data can be imported into both sharded and non-sharded databases.
  In both cases, the data will be retrieved into the database from a local .ravendbdump file.
• If the database is sharded, the imported data will be distributed among the shards.
  In case the shard has several nodes, each shard database will be replicated to all the nodes of this shard.

See a code example here.

Importing data from backup files

• Backup files are given an extension that reflects the backup type.
  A full-backup file, for example, will be given a .ravendb-full-backup extension.
  Regardless of the extension, the internal structure of backup files is similar to that of .ravendbdump files.
  It is therefore possible to import backup files.
• Unlike the restore operation, which creates a new database from the backup files, importing a backup file adds the imported data to data already stored in the database.
• Import can be helpful in the following cases, among others:
  • a backup created for a non-sharded database can be imported into a sharded database.
    This allows the migration of data from a non-sharded database to a sharded one.
  • When a sharded database is backed up, each shard creates a backup of its own database. If you want to split the sharded database into multiple databases, you can simply import the backup of each shard into a newly created (sharded or not) database.