Time Series Overview
A huge number of systems, including an expanding variety of IoT devices, produce continuous streams of values that can be collected and used for various needs. Time series are vectors of data points that are designated to collect values over time, store them consecutively, and manage them with high efficiency and performance.
Time series data is compressed to lower storage usage and transaction time.
In this page:
Time series can be aggregated and queried to illustrate process behavior, predict future developments, track noticeable value changes, and create other helpful statistics.
Here are a few examples for value streams that can be easily and effectively handled by time series.
- A sequence of heart rate values can be collected from a smart wrist-watch, and be used to build a person's training program.
- Weather-stations' measurements collected over a chosen time period can be compared to equivalent past periods to predict the weather.
- Bandwidth usage reports of a home cable modem monitor can be used to build a better charging plan.
- Coordinates sent by delivery trucks' GPS trackers can be collected and analyzed to secure the vehicles and improve the service.
- Daily changes in stock prices can be used to build investment plans.
RavenDB's Time Series Implementation
Time series functionality is fully integrated into RavenDB's distributed environment and document model.
Distributed Time Series
Distributed clients and nodes can modify time series concurrently; the modifications are merged by the cluster without conflict.
Time Series as Document Extensions
A time series always extends a single specific document.
The context and source of the time series can be kept clear this way, and time series management can use the comfort and strength of the document interface.
A barometer's specifications document, for example, can be the parent document for a time series that is populated with measurements taken by a barometer of this specification.
Like the other document extensions, time series can take part in fully transactional operations.
Time Series Features
Notable time series features include -
- Highly-Efficient Storage Management
Time series data is compressed and segmented to minimize storage usage and transmission time.
- A Thorough Set of API Methods
The time series API**
includes a variety of
- Full GUI Support
Time series can be viewed and managed using the Studio.
Time Series Querying and Aggregation
- High-performance common queries
The results of a set of common queries are prepared in advance in time series segments' headers, so the response to querying for a series minimum value, for example, is returned nearly instantly.
- LINQ and raw RQL queries
Flexible queries and aggregations can be executed using LINQ expressions and raw RQL over time series timestamps, tags and values.
- High-performance common queries
- Time Series Indexing
Time series can be indexed.
Rollup and Retention Policies
- Rollup Policies
You can set time series rollup policies to aggregate large series into smaller sets by your definitions.
- Retention Policies
You can set time series retention policies to automatically remove time series entries that have reached their expiration date/time.
- Rollup Policies
- Including Time Series
You can include (pre-fetch) time series data while loading documents.
Included data is held by the client's session, and is delivered to the user with no additional server calls.
You can patch time series data to your documents.
(visit the API documentation to learn more).
Time Series Data
Separate Name and Data Storage
The separation of names and data prevents time series value updates from invoking document-change events, keeping documents' availability and performance whatever size their time series grow to be and however frequent their value-updates are.
Time Series Segments
Time series are composed of consecutive segments.
When a time series is created, its values are held in a single segment.
As the number of values grows (or when a certain amount of time has passed since the last entry appendage), segments are added to the series.
Segments are managed automatically by RavenDB, clients do not need to do
anything in this regard.
Time series segmentation heightens performance and minimizes transaction and query time, since only the relevant segments of even a very long series would be retrieved and queried, and only relevant segments would be updated.
Common Queries Performance
Segmentation also helps provide results for common queries extremely
fast, since results for such queries as
Max and others are
automatically stored and updated in segment headers, and are always
available for instant retrieval.
Time Series Entries
Each time series segment is composed of consecutive time series entries.
Each entry is composed of a timestamp, 1 to 32 values, and an optional tag.
DateTime timestamp marks each entry in millisecond precision.
Timestamps are always indicated using UTC.
Timestamps, and therefore time series entries, are always ordered by time,
from the oldest timestamp to the newest.
E.g. in a heart rate time series, timestamps would indicate the time in which each heart rate measurement has been taken.
Up to 32
double values can be appended per-entry.
We allow storing as many as 32 values per entry, since appending multiple values may be a requirement for some time series. Here are a few examples.
A heart-rate time series
An entry with a single value (the heart-rate measurement taken by a smart wrist-watch) is added to the time series every minute.
A truck-route time series
An entry with 2 values (the latitude and longitude reported by a GPS device) is added to the time series every minute.
A stock-price time series
An entry with 5 values (stock price when the trade starts and ends, its highest and lowest prices during the day, and its daily trade volume) is added to the time series every day.
A single optional
string tag can be added per entry.
Tags are designated to provide information regarding their entries.
A tag can be a short descriptive text.
A tag can also contain a document's ID, and function as a reference to this document.
A reference-tag is preferable when we want the tag to be very short and yet refer us to an unlimited source of information.
Reference-tags can be used to filter time series data during a query.
E.g., the query can -
1. load time series entries whose tags refer to device-specification documents.
2. retrieve and examine the specification document referred to by each entry.
3. project to the client only values measured by Japanese devices.
Prefer re-using a few tags many times over using many unique tags,
to minimize memory and storage usage and optimize time series performance.