Building a Log Aggregation on Top of RavenDB
In most software systems today, there is simply too much going on to inspect the behavior of the system manually. You end up throwing the data somewhere and if there is an error or something that you need to do, you have to run queries.
You need to aggregate the logs of your system operations to get a clear picture of what's happening.
RavenDB enables you to put out fires quickly by just turning on the fire hose.
Even if you are sorting out information you aren't familiar with that is coming from different locations with different structures, you can manage it quickly and produce relevant metrics with ease.
One of the most important features RavenDB puts in your hands is a full text search where you can apply that to the logs.
Another is Time series.
Answering questions like "Show me all of the past 12 hours of query latencies on all my system that deal with external integration", becomes easy.
You can track patterns, changes, and outliers in CPU utilization and memory usage over time to optimize everything. Combine this with our native mapReduce for real time aggregations and you can produce metrics that monitor the overall health of your systems quickly, allowing you to act on red flags before they become red alerts.
Effective log aggregation in your database requires the combination of a lot of features you would normally have to add-on, integrate with, or develop yourself. However, with RavenDB it's all inside your database.
In this webinar, Microsoft MVP and RavenDB CEO Oren Eini will walk you through all the considerations you need to create a log aggregation system.
You will see:
- Optimal patterns for log aggregation
- Utilizing full text search, mapReduce, time series data modeling and documents compression
- Executing live ingestion so every time a system generates a log message, it goes where you want it to
- Solutions for effective monitoring of your logs to produce the metrics you need.