Reduce stored metric data
Chronosphere Observability Platform ingests metrics from many sources. These sources can supply multiple metric types, and are generally ingested as raw data.
Although raw data might provide detailed information about a specific point in time, that data grows rapidly and is expensive to store. Control costs by aggregating, downsampling, or dropping unneeded data.
There are several ways to reduce metrics persisted by Observability Platform, depending on your use case:
- Send no metrics: To not send any metrics to Observability Platform, adjust what you send from the client-side by using mechanisms like relabel rules in the Chronosphere Collector.
- Send metrics without persisting: To send metrics to Observability Platform so you can experiment with them in tools such as the Live Telemetry Analyzer, but not persist them, use drop rules.
- Send metrics and persist: To send metrics to Observability Platform and persist them, but optimize their storage with aggregation, use mapping rules.
Compare data reduction methods
The following chart compares different methods of reducing incoming or persisted data. Your system needs might dictate a particular use, or indicate changing from one method to another.
Rollup, mapping, and recording rules aggregate data. Derived metrics and drop rules don't aggregate data. Their purpose is to reduce incoming data or cardinality prior to aggregation.
| Rule Type | Purpose | Benefits |
|---|---|---|
| Drop rules | Drop unused metrics. | Fast, persisted data reduction with quick implementation. |
| Rollup rules | Reduce cardinality, downsample data, and perform simple aggregations. | Scalable, can drop raw data, and handle late arriving data. |
| Mapping rules | Downsample or drop metrics. | Scalable downsampling of unaggregated metrics. |
| Recording rules | Improve query performance by pre-executing queries. | Execute any PromQL query and add labels. |
| Derived metrics | Execute complex queries in real-time. | Replaces most recording rule use cases and doesn't add new time series. |
Order of rule application
Drop rules are applied at ingestion time. After metrics are sent but before they're stored, rollup rules and mapping rules are applied. Derived metrics and recording rules are then applied at query time.
Aggregation rule types
Aggregation rules combine downsampling (lower sampling rate of metrics) and aggregation (focusing on the metrics important to a use case) to reduce the quantity of stored metrics data, and improve the performance of queries.
Observability Platform offers these types of aggregation rules:
- Rollup rules aggregate metric data by grouping based on a label or downsample metric data in memory on the streaming ingest path before storing the results in the database. Rollup rules perform better, are more scalable, and require fewer steps to create aggregations.
- Recording rules ingest the raw metric data first before reading it and then storing the aggregated and downsampled metric data back into the database. Recording rules are backward-compatible with Prometheus setups and support all PromQL functions.
- Mapping rules downsample in-memory metric data on the streaming ingest path before storing any results in the database. The interval is the time between aggregated data points.
Review your metrics to determine if they're useful because they have references, or because some of their labels can be useful (dimensions).
To step through this process, see Reducing cardinality.
View your aggregation rules
View, sort, and filter existing rollup and mapping rules from the Aggregation Rules page in Observability Platform.
Boilerplate rules
Chronoctl provides boilerplate code for you to base aggregation rules upon. Use the following command to generate the boilerplate:
chronoctl mapping-rules scaffold
chronoctl recording-rules scaffold