Bulk Insert: How to Work With Bulk Insert Operation

One of the features that is particularly useful when inserting large amount of data is bulk inserting. This is an optimized time-saving approach with few drawbacks that will be described later.

Syntax

BulkInsertOperation BulkInsert(string database = null, CancellationToken token = default);
Parameters
database string Name of database for which bulk operation should be performed. If null then the Database from DocumentStore will be used.
token CancellationToken Cancellation token used in order to halt the worker operation.
Return Value
BulkInsertOperation Instance of BulkInsertOperation used for interaction.

BulkInsertOperation

Methods

Signature Description
void Abort() Abort the operation
void Store(object entity, IMetadataDictionary metadata = null) store the entity, identifier will be generated automatically on client-side. Optional, metadata can be provided for the stored entity.
void Store(object entity, string id, IMetadataDictionary metadata = null) store the entity, with id parameter to explicitly declare the entity identifier. Optional, metadata can be provided for the stored entity.
void StoreAsync(object entity, IMetadataDictionary metadata = null) store the entity in an async manner, identifier will be generated automatically on client-side. Optional, metadata can be provided for the stored entity.
void StoreAsync(object entity, string id, IMetadataDictionary metadata = null) store the entity in an async manner, with id parameter to explicitly declare the entity identifier. Optional, metadata can be provided for the stored entity.
void Dispose() Dispose an object
void DisposeAsync() Dispose an object in an async manner

Limitations

There are a couple limitations to the API:

  • The bulk insert operation is broken into batches, each batch is treated in its own transaction so the whole operation isn't treated under a single transaction.
  • Bulk insert is not thread safe, a single bulk insert should not be accessed concurrently. The use of multiple bulk inserts, on the same client, concurrently is supported also the use in an async context is supported.

Example

Create bulk insert

Here we create a bulk insert operation and inserting a million documents of type Employee

using (BulkInsertOperation bulkInsert = store.BulkInsert())
{
    for (int i = 0; i < 1000 * 1000; i++)
    {
        bulkInsert.Store(new Employee
        {
            FirstName = "FirstName #" + i,
            LastName = "LastName #" + i
        });
    }
}
BulkInsertOperation bulkInsert = null;
try
{
    bulkInsert = store.BulkInsert();
    for (int i = 0; i < 1000 * 1000; i++)
    {
        await bulkInsert.StoreAsync(new Employee
        {
            FirstName = "FirstName #" + i,
            LastName = "LastName #" + i
        });
    }
}
finally
{
    if (bulkInsert != null)
    {
        await bulkInsert.DisposeAsync().ConfigureAwait(false);
    }
}