Bulk Insert: How to Work With Bulk Insert Operation
One of the features that is particularly useful when inserting large amount of data is bulk inserting
. This is an optimized time-saving approach with few drawbacks that will be described later.
Syntax
BulkInsertOperation BulkInsert(string database = null, CancellationToken token = default);
Parameters | ||
---|---|---|
database | string | Name of database for which bulk operation should be performed. If null then the Database from DocumentStore will be used. |
token | CancellationToken | Cancellation token used in order to halt the worker operation. |
Return Value | |
---|---|
BulkInsertOperation | Instance of BulkInsertOperation used for interaction. |
BulkInsertOperation
Methods
Signature | Description |
---|---|
void Abort() | Abort the operation |
void Store(object entity, IMetadataDictionary metadata = null) | store the entity, identifier will be generated automatically on client-side. Optional, metadata can be provided for the stored entity. |
void Store(object entity, string id, IMetadataDictionary metadata = null) | store the entity, with id parameter to explicitly declare the entity identifier. Optional, metadata can be provided for the stored entity. |
void StoreAsync(object entity, IMetadataDictionary metadata = null) | store the entity in an async manner, identifier will be generated automatically on client-side. Optional, metadata can be provided for the stored entity. |
void StoreAsync(object entity, string id, IMetadataDictionary metadata = null) | store the entity in an async manner, with id parameter to explicitly declare the entity identifier. Optional, metadata can be provided for the stored entity. |
void Dispose() | Dispose an object |
void DisposeAsync() | Dispose an object in an async manner |
Limitations
There are a couple limitations to the API:
- The bulk insert operation is broken into batches, each batch is treated in its own transaction so the whole operation isn't treated under a single transaction.
- Bulk insert is not thread safe, a single bulk insert should not be accessed concurrently. The use of multiple bulk inserts, on the same client, concurrently is supported also the use in an async context is supported.
Example
Create bulk insert
Here we create a bulk insert operation and inserting a million documents of type Employee
using (BulkInsertOperation bulkInsert = store.BulkInsert())
{
for (int i = 0; i < 1000 * 1000; i++)
{
bulkInsert.Store(new Employee
{
FirstName = "FirstName #" + i,
LastName = "LastName #" + i
});
}
}
BulkInsertOperation bulkInsert = null;
try
{
bulkInsert = store.BulkInsert();
for (int i = 0; i < 1000 * 1000; i++)
{
await bulkInsert.StoreAsync(new Employee
{
FirstName = "FirstName #" + i,
LastName = "LastName #" + i
});
}
}
finally
{
if (bulkInsert != null)
{
await bulkInsert.DisposeAsync().ConfigureAwait(false);
}
}