What’s New RavenDB 5.3

Introduced in RavenDB 5.0, Distributed Time Series is a powerful tool that allows you to collect numerical values from sources like monitoring devices and stock trading apps, and order them by time in a single series — one entry of up to 32 values per time frame.

In RavenDB 5.3, we’ve introduced enhancements that enable users to alter stored time series values, allowing entries to be used as counters that can be incremented and decremented. This provides a handy way to accept and update values in real-time and on an ongoing basis, without the worry of conflicts and performance drops due to expensive synchronization.

As an example, think of an ebook store where you need to collect and update statistics in real-time. For every new hour, you’d create an initial tally of zero sold books, then update this value in real-time with each new sale. Not only that, you can now go back in time and make corrections for any cancellations. All of these operations are guaranteed to be executed concurrently on multiple cluster nodes, without any danger of data loss.

Sometimes, rather than simply executing a query and getting back results, you want to define the query and let RavenDB deliver the results on an ongoing basis. This is done using Data Subscriptions — ongoing cluster-managed operations in which user-defined “Subscription Workers” consume (receive and process) batches of data from the server. RavenDB guarantees the processing of these batches; if the subscription worker reports a processing error, the batch will be re-delivered until processing is successful.

RavenDB 5.3 enhances this feature with Concurrent Data Subscriptions. Now you can have more than one Subscription Worker consuming documents from the same subscription. RavenDB Servers will provide multiple Workers with batches of documents to process, taking care to distribute different batches to different Workers, and ensuring all of them are under a similar load. Servers will always ensure that none of the data is left unprocessed.

Depending on your needs, Workers can reside on the same machine or run on separate ones. As a result, you can now scale out your processing infrastructure and tremendously increase your capacity for processing documents arriving from subscriptions.

Before adopting a new piece of technology you need to know that it will be a team player in your existing ecosystem. If you’re using Elasticsearch as a solution for full-text search, you can now add RavenDB 5.3 to the list of applications populating your organization’s Elasticsearch database. Through the powerful ETL (Extract, Transform, Load) mechanism, you can reshape RavenDB data before pushing it to Elasticsearch and avoid building time-expensive custom solutions.

External contribution provided by Djordje Djukic

From version 5.3, RavenDB is supporting integration with Power BI: a powerful business intelligence service by Microsoft. This enables you to push your data to Power BI, where you can analyze, process, transform and remodel it. Using this functionality you can create intelligent reports and gain new insights into your data and business. Additionally, you can now run queries from your BI client to fetch any data from RavenDB that you want to process, analyze and report on.

Starting with version 5.3, writing RQL in the RavenDB Studio will be a much more intuitive experience. As you type, the studio will actively help you with suggestions, offering you a list of possible keywords dynamically filtered based on the current context. RQL queries and patch scripts will be validated before you run them, helping you master RQL and improve your coding skills over time.

Features like autocomplete, code hints, context-aware suggestions, and lists of data element names will smooth the learning curve and allow you to focus on your objectives without getting caught up in the syntax of your queries and patch scripts.

In addition to existing compression methods for data stored on disk, version 5.3 introduces TCP traffic compression for data in-transport.

TCP is a backbone protocol for all data exchanges between RavenDB servers and the underlying protocols used by Data Subscriptions. It’s used extensively whenever your documents change: pushing the data to client applications with associated Subscriptions and synchronizing data between RavenDB servers whether they’re in the same cluster or linked through one of our External Replication features.

We’ve made our TCP compression process extremely fast and efficient, in turn making our TCP data exchanges faster and more efficient. The process is applied to documents transferred both internally and externally, significantly reducing the amount of data transported — in some cases by a factor of 10. This also reduces the amount of data you transfer across the wire, so your operational costs will be lower too.

Simply put, leveraging TCP compression will reduce your bandwidth usage, decrease the amount of data in transit, lower network transfer costs and latency in general, and provide you with a more responsive system. Best of all, no configuration is needed and this compression is completely transparent to the end-user.