Managing the cardinality of persisted data can be challenging when determining which
metrics to send to Chronosphere Observability Platform. Metric cardinality is the
number of unique time series produced by a combination of metric names and their
associated labels. Cardinality is the total number of these combinations. The more
combinations there are, the greater a metric’s cardinality is, and the more data
Observability Platform persists. This can affect
your license.
After sending data to Observability Platform, use the
Live Telemetry Analyzer to identify
opportunities to reduce the overall volume of metrics. If you’re sending Prometheus
metrics to Observability Platform, you can also use the
Telemetry Usage Analyzer to identify obsolete or
unnecessary data, and understand the impact of a proposed shaping rule to users of
that data.Use the Chronosphere Control Plane to reduce the amount of data retained in your
system over time. Create shaping rules to drop data before
it reaches Observability Platform, aggregate and rewrite data into more manageable
and usable statistics, and alias expressions to improve data queries and references.After creating shaping rules, use the
Aggregation Rules UI
to understand your existing shaping rules and how they affect your environment. Use the
shaping impact preview to
preview the impact of a rule on your overall system, which helps prevent breaking
changes and ensure the rules you create operate as expected.After configuring shaping rules, define metrics
quotas and pools to assign specific
percentages of your total persisted writes limit to pools of metrics.If you identify data you absolutely don’t need and want to reduce the data you send
to Observability Platform, adjust what metric data you send from the client-side by
using mechanisms like relabel rules in the
Chronosphere Collector.
Collectors ingest metrics at specific intervals, based on system
configuration. This granularity of metric data can be helpful in diagnostic efforts,
but certain issues might not be served by such granularity. If you can diagnose
production issues by using a coarser granularity of metric data, downsampling the
data reduces the amount of data persisted to the Observability Platform database.Downsample incoming data in Observability Platform using these methods:
Change the Chronosphere Collector configuration by
changing the rate
at which the Collector publishes metrics to the server.
Use mapping rules to downsample metrics
that aren’t aggregated.
Use rollup rules to downsample aggregated
metrics.
Over time, persisted data uses significant storage capacity. Observability Platform
performs long-term downsampling to control data
storage costs while retaining important statistics.Be aware that late-arriving data
can affect downsampling.
