Aggregate metrics
Chronosphere Observability Platform ingests metrics from many sources. These sources can supply multiple metric types, and are generally ingested as raw data.
Although raw data might provide detailed information about a specific point in time, that data grows rapidly and is expensive to store. Control costs by aggregating and downsampling your data.
Aggregation rules
Aggregation rules combine downsampling (lower sampling rate of metrics) and aggregation (focusing on the metrics important to a use case) to reduce the quantity of stored metrics data, and improve the performance of queries.
Observability Platform offers these types of aggregation rules:
- Rollup rules aggregate metric data by grouping based on a label or downsample metric data in memory on the streaming ingest path before storing the results in the database. Rollup rules perform better, are more scalable, and require fewer steps to create aggregations.
- Recording rules ingest the raw metric data first before reading it and then storing the aggregated and downsampled metric data back into the database. Recording rules are backward-compatible with Prometheus setups and support all PromQL functions.
- Mapping rules downsample in-memory metric data on the streaming ingest path before storing any results in the database. The interval is the time between aggregated data points.
Review your metrics to determine if they're useful because they have references, or because some of their labels may be useful (dimensions).
To step through this process, see Reducing cardinality.
Compare data reduction methods
The following chart compares different methods of reducing incoming or persisted data. Your system needs might dictate a particular use, or indicate changing from one method to another.
Derived metrics and drop rules aren't aggregation rules, but their purpose is to reduce incoming data or cardinality.
| Rule Type | Purpose | Benefits |
|---|---|---|
| Drop rules | Drop unused metrics. | Fast, persisted data reduction with quick implementation. |
| Rollup rules | Reduce cardinality, downsample data, and perform simple aggregations. | Scalable, can drop raw data, and handle late arriving data. |
| Mapping rules | Downsample or drop metrics. | Scalable downsampling of unaggregated metrics. |
| Recording rules | Improve query performance by pre-executing queries. | Execute any PromQL query and add labels. |
| Derived metrics | Execute complex queries in real-time. | Replaces most recording rule use cases and doesn't add new time series. |
Drop rules apply during collection, rollup and mapping rules apply during aggregation, and derived metrics and recording rules apply at query time.
View your aggregation rules
View, sort, and filter existing rollup and mapping rules from the Aggregation Rules page in Observability Platform.
Boilerplate rules
Chronoctl provides boilerplate code for you to base aggregation rules upon. Use the following command to generate the boilerplate:
chronoctl mapping-rules scaffold
chronoctl recording-rules scaffold