Complex distributed transactions with RavenDB

An interesting question was raised in the mailing list, how do we handle non trivial transactions within a RavenDB cluster? The scenario in the question is money transfer between accounts, which is almost the default example for transactions.

The catch here is that now we have a set of business rules to apply.

• Neither account may be blocked
• There should be sufficient funds in the source account
• The funds involved are not tainted

Let’s see how this can look like in code, shall we?

I’m assuming am model similar to bitcoin, where you can break apart coins, but not merge them, by the way. What we can see is that we have non trivial logic going on here, especially as we are hiding all the actual work involved here behind the scenes. The code, as written, will work great, as long as we have a single thread of execution. We can change it easily to support concurrent work using:

With this call: session.Advanced.UseOptimisticConcurrency = true; we tell RavenDB that it should do an optimistic concurrency check on the documents when we save them. If any of the documents has changed while the code is running a concurrency exception will be raised and the whole transaction will be aborted.

This works, until you are running on a distributed cluster and have to support transactions running on different nodes. Luckily, RavenDB has an answer for that as well, with cluster wide transactions.

It is important to understand that with cluster wide transactions, we have two separate layers here. The documents are part of the overall transaction, but do not impact it. We need to use compare exchange values to allow optimistic concurrency on the whole operation. Here is the code,  it is a bit dense, but the key here is that we added a compare exchange value that mirrors every account. Whenever we modify the account, we also modify the compare exchange value. Then we can use a cluster wide transaction like so:

With this in place, RavenDB will ensure that this transaction will only be accepted if both compare exchange values are the same as they were when we evaluated them. Because any change touch both the document and the compare exchange values, and because they are modified in a cluster wide transaction, we are safe even in a clustered environment. If there is a change to the data, we’ll get a concurrency exception and have to re-compute.

Important: You’ll note that we are comparing the documents and the compare exchange values to make sure that they have the same funds. Why do we need to do that? Compare exchange and documents live on separate stores and are replicated across the cluster using different mechanisms. This is because compare exchange values are using consensus and documents are using gossip for replication. It is possible that they won’t be in sync when we read them, which is why we do the extra check to ensure that we start from a level playing field.

Note that I’m using API 5.0 here, with 4.2, you need to explicitly call UpdateCompareExchangeValue, but in 5.0, we have fixed things so we do change tracking on the values. That means that it is important that the values would actually change, of course.

And this is how you write a distributed transaction with RavenDB to transfer funds in a clustered environment without needing to really worry about any of the details. Handle & retry a concurrency exception and you are pretty much done.