Generating Embeddings - Overview

Define an Embeddings Generation Task:
Specify a connection string that defines the AI provider and model for generating embeddings.
Define the source content - what parts of the documents will be used to create the embeddings.

Source content is processed:

1. The task extracts the specified content from the documents.
2. If a processing script is defined, it transforms the content before further processing.
3. The text is split according to the defined chunking method; a separate embedding will be created for each chunk.
4. Before contacting the provider, RavenDB checks the embeddings cache to determine whether an embedding already exists for the given content from that provider.
5. If a matching embedding is found, it is reused, avoiding unnecessary requests.
   If no cached embedding is found, the transformed and chunked content is sent to the configured AI provider.

Embeddings are generated by the AI provider:
The provider generates embeddings and sends them back to RavenDB.
If quantization was defined in the task, RavenDB applies it to the embeddings before storing them.

Embeddings are stored in your database:

• Each embedding is stored as an attachment in a dedicated collection.
• RavenDB maintains an embeddings cache, allowing reuse of embeddings for the same source content and reducing provider calls. Cached embeddings expire after a configurable duration.

Perform vector search:
Once the embeddings are stored, you can perform vector searches on your document content by:

• Running a dynamic query, which automatically creates an auto-index for the search.
• Defining a static index to store and query embeddings efficiently.

The query search term is split into chunks, and each chunk is looked up in the cache.
If not found, RavenDB requests an embedding from the provider and caches it.
The embedding (cached or newly created) is then used to compare against stored vectors.

Continuous processing:

• Embeddings generation tasks are Ongoing Tasks that process documents as they change.
  Before contacting the provider after a document change, the task first checks the cache to see if a matching embedding already exists, avoiding unnecessary requests.
• The requests to generate embeddings from the source text are sent to the provider in batches.
  The batch size is configurable, see the Ai.Embeddings.MaxBatchSize configuration key.
• A failed embeddings generation task will retry after the duration set in the
  Ai.Embeddings.MaxFallbackTimeInSec configuration key.

The following service providers are supported for auto-generating embeddings using tasks:

• OpenAI & OpenAI-compatible providers
• Azure Open AI
• Google AI
• Hugging Face
• Ollama
• Mistral AI
• bge-micro-v2 (a local embedded model within RavenDB)