One Database Behind the Agent: RavenDB as In-Cluster Working Memory

Highlights:

• Let the model fetch context through tool calls instead of pasting raw API responses into every prompt.
• Keep documents, full-text search, vector search, attachments, expiration, and Data Subscriptions in one RavenDB cluster.
• Declare the cluster on Kubernetes with the RavenDB Operator and run the whole sample with one command.

RavenDB is a multi-model database: documents, full-text search, vector search, attachments, expiration, and Data Subscriptions, all running inside one cluster, behind one connection, queried with one language. On its own, that's a convenient way to build most applications. Point an LLM agent at it and something sharper happens: the agent's entire working memory, the part of the stack usually assembled from four or five separate products, collapses into a database the agent already trusts.

We built a sample app that puts RavenDB to work as an AI agent database in a genuinely context-hungry use case: a conversational flight-search agent that spots routes where a connecting ticket beats a direct one. Many context data points like flight price history, route topology, airport geography, conversation state, and user preferences are all needed on nearly every turn.

The Bill Nobody Budgets For

Take an LLM agent to production, and the surprise is rarely the model. It's the two lines on the invoice that scale linearly with traffic: tokens and egress. 

A model is stateless, so every turn you hand it context: prices, history and user preferences. The easiest way to do that is to call an API, take the raw JSON, and paste it into the prompt. For a flight-price API, a single route search returns hundreds of records, which can amount to anywhere from 40,000 to 100,000 tokens. Every one of them is billed on the way in, and every megabyte of context assembled outside the cluster is billed again on the way out.

Our sample app runs against RavenDB in the same Kubernetes cluster and lets the model ask for exactly the context it needs. Compared to having everything pre-injected, each turn stays within a budget of around 2,500 tokens, with zero egress on retrieval. Same model, same provider, same answers.

The Shape of the Fix: RavenDB as a Tool, Not a Sidecar

The architectural decision that does all the work: the database is a tool the model calls, not a payload you inject.

The user sends a query to the Agent. The Agent passes only the prompt to the OpenAI API. Instead of receiving a large amount of pre-injected context, the model asks for a tool call, and the Agent runs it against a RavenDB instance in the same cluster. RavenDB responds locally, and only that small, structured result is sent on to the OpenAI API.

The agent exposes four tools: search_routes, get_live_prices, get_user_profile, update_constraints. The model decides which to call and with what arguments. RavenDB answers locally, in-cluster, and returns a pre-structured document: origin, destination, hubs, price range, hidden-city score. Roughly 400 tokens. That tool result rides back into the same API call.

Nothing is pre-injected. No sidecar quietly stuffing "relevant context" into every system prompt. The raw upstream response never reaches the model at all: a CronJob filters it on the way into the database every six hours. Each price is written with a 20-minute expiration, and anything older than two hours counts as stale, so the agent fetches a live price and writes it back to RavenDB. 

The per-turn budget is approximately 2,500 tokens: the system prompt (approx. 200 tokens), the user message (approx. 100 tokens), tool results (up to 1,800 tokens), and the model's response (up to 400 tokens). 

What One AI Agent Database Actually Covers

Here is the part worth dwelling on, because it is the reason this sample is a RavenDB sample and not an architecture diagram. An agent's context layer needs a surprising number of different data capabilities, and the industry answer is to buy one product per capability. This is what the sample uses instead: all of it in one cluster, one connection, one query language, one backup strategy.

Documents, no migrations

Routes, sessions, airports and users are documents. When hidden-city scoring gained hidden_city_via, hidden_city_decoy and a risk array, that was a code change, not a schema migration. Conversation state lives in a sessions/... document, so active_constraints ("carry-on only, max one stop") is an indexed field the agent can read without parsing turn text, and the history survives a pod restart. Killing the agent pod mid-conversation and continuing where you left off is a demo beat, not a caveat.

Full-text search with no index to define

The agent has to turn "flights from Warsaw to London" into WAW and LHR. That is a full-text lookup over the Airports collection:

session.query_collection("Airports").search("city", phrase, operator=SearchOperator.AND).take(3)

The AND operator makes a multi-word city like "Mexico City" match every term, not just one of them. No static index was ever written for this. RavenDB created the auto-index on the first query and maintains it from there. No Elasticsearch cluster, no ingest pipeline, no second copy of the data to keep in sync.

Vector search on the same documents

When no direct route exists between two airports, the agent first looks for a connecting hub among the cached routes. If that finds nothing, it falls back to "what is near here?". Every airport document carries a location_vector - a 3D unit-sphere projection of its lat/lng. The cosine similarity of two such vectors equals the cosine of the angle between the airports, so ranking by similarity is the same as ranking by great-circle distance, with no antimeridian or pole discontinuity:

session.query_collection("Airports")

       .vector_search("location_vector", origin_vector, number_of_candidates=20)

       .where_not_equals("iata", iata)

That is an ANN vector search query over a field stored on the document it describes. The sample then re-sorts the candidates by real distance and drops anything farther than 1,500 km. No embedding store to sync, no pgvector instance, no risk of the vector and the document disagreeing about what they represent. The same session that ran the full-text query runs this one.

Expiration instead of a cleanup job

Prices go stale fast. Route documents are written with an @expires timestamp 20 minutes out, and once document expiration is enabled for the database, RavenDB deletes them itself. There is no TTL-sweeper CronJob in this repo, and no Redis whose eviction policy you have to reason about under memory pressure. One thing to keep in mind: deletion runs on a schedule (every 60 seconds by default; in this sample every 36 hours, because of license limits), and until an expired document is deleted, loads and queries can still return it.

Attachments for the things that must never touch a query

A passport scan, a bag photo, an exported preference PDF. They hang off the user document as attachments, retrieved whole by the profile screen, and structurally incapable of leaking into a query result and inflating the token budget. No S3 bucket, no signed-URL plumbing.

Data Subscriptions instead of polling

The price-drop watcher is a subscription, not a scheduler:

SubscriptionCreationOptions(

    query="from Routes where hidden_city_score > 0.5",

    name="hidden-city-price-drops",

)

RavenDB pushes matching documents to the worker pod as they change. No burning API quota every five minutes, no cluster with a $500/month floor to move one event stream from one producer.

One round trip per turn

At the start of a turn the agent needs the conversation session and the user profile - two documents in two different collections. That is one call:

loaded = session.load([session_doc_id, user_doc_id])

Small thing. It is also the difference between a chat turn that feels instant and one that does not, and you get it without writing a join.

mTLS by default

In Kubernetes mode, every Python process - agent, worker, scraper - authenticates to RavenDB with an X.509 client certificate. Inside the cluster, not just at the edge. The start script generates the whole CA/server/client chain and applies it as Kubernetes secrets automatically, so in Kubernetes mode nothing talks to the database in plaintext. (The local docker-compose mode is a plain-HTTP development setup.)

The Kubernetes Half: One CRD

The cluster itself is declared, not assembled. The RavenDB Kubernetes Operator takes a RavenDBCluster resource and reconciles it:

apiVersion: ravendb.ravendb.io/v1

kind: RavenDBCluster

metadata:

  name: ravendb-cluster

spec:

  nodes:

    - tag: a

      publicServerUrl: https://a.hiddencity.local:443

      publicServerUrlTcp: tcp://a-tcp.hiddencity.local:443

    - tag: b

      publicServerUrl: https://b.hiddencity.local:443

      publicServerUrlTcp: tcp://b-tcp.hiddencity.local:443

  mode: None

  storage:

    data:

      size: 50Gi

The start script installs the operator and the certificates, then one helm upgrade --install  of the cluster chart and the operator does the rest: a one-shot bootstrapper Job forms the Raft cluster, certificates get wired in, rolling upgrades go node-by-node behind availability safety gates: halting on a failed gate, resuming automatically once it clears, and refusing downgrades outright. Admission webhooks reject an invalid topology before it can do damage.

Two honest notes. Node topology is fixed at bootstrap: adding or removing nodes later is a deliberate manual operation, not silent reconciliation. And the sample ships scaled to a single node because it runs without a multi-node license. The free Developer and Community licenses both allow three nodes, so with either one, restoring nodes b and c is two blocks of YAML in k8s/ravendb/values.yaml.

What It Costs, Stage by Stage

The sample's architecture doc walks the same use case through four infrastructure generations and prices each one at 1,000 requests/day with gpt-4o-mini.

Two things about stage 3 are worth saying out loud. It costs the most to operate and it still sends 20,000 tokens per turn, because caching a fat payload faster does not make it thin. And every one of those retrievals crosses a service boundary on the way to the prompt.

Stage 4's saving is not "we found a cheaper model." It is $5.65/day at 1,000 requests, $564/day at 100,000. From sending less, and from five components collapsing into one.

Run it yourself: python -m scripts.measure_tokens prints tokens per request against the naive baseline, and python -m scripts.measure_egress prints the line the whole design exists to produce "Retrieval egress: 0 KB".

One Deliberate Exception

Honesty about the pattern: there is exactly one place where the agent pre-fetches instead of waiting to be asked. If a message is unambiguously "from X to Y" and both cities resolve to exactly one airport each, the agent resolves them and calls search_routes itself before the first model call, turning a two-call turn into one, measured at around 2,100 tokens down to 750.

The discipline that keeps this from becoming the sidecar we rejected is strict: exact match on both airports, or nothing. Anything ambiguous falls back to the model calling the tool itself, unchanged. Worth copying. And worth copying the guardrail with it.

Takeaways

An agent's context bill is an architecture decision, not a model decision. Where the data lives and how it reaches the prompt decide most of the cost.

• Let the model ask. Context delivered as a tool result is context you can budget, filter, and measure. Context injected into every prompt is context you pay for whether it was needed or not.
• Filter on the way in, once. The 40,000-token upstream response belongs in your database, shaped, not in your prompt, raw.
• Co-location is an architecture choice, not an optimization. Retrieval that never leaves the cluster has no egress line and no round-trip tail latency to hide.
• Count your products, not just your bills. Documents, full-text, vectors, attachments, expiration, and change subscriptions in one AI agent database means one thing to secure, monitor, back up, and upgrade.

Ready to Give Your Agent One Database?

The sample is on GitHub and comes up with one command: a kind cluster, cert-manager, ingress-nginx, the operator, RavenDB, and the app, all versions pinned. All you need is Docker and an OpenAI API key.

• Sample: ravendb/samples-hidden-city
• Operator: ravendb/ravendb-operator
• Reference: RavenDB Vector Search Documentation

Come tell us what your agent's context layer looks like, and share your bill-shock stories with us and the community on our Discord server.

We are looking forward to your feedback!

Frequently Asked Questions

How does RavenDB reduce token costs for LLM agents?

The agent calls RavenDB as a tool instead of pasting raw API responses into the prompt. RavenDB returns a pre-structured document of roughly 400 tokens, and the per-turn budget is about 2,500 tokens. A raw flight-price response can take 40,000 tokens or more.

What does zero retrieval egress mean?

RavenDB runs in the same Kubernetes cluster as the agent, so looking up context never crosses the cluster boundary. What goes out to the LLM provider is the prompt plus a small, filtered tool result, never the raw upstream data.

What do I need to run the sample?

Docker and an OpenAI API key. A Travelpayouts token is optional; without it, the sample uses bundled fixture data. To run the 3-node cluster instead of one node, register a free Developer or Community license.