Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.chronosphere.io/llms.txt

Use this file to discover all available pages before exploring further.

Prometheus makes experimental functions and operators available behind a feature flag. The Prometheus project might change the name, syntax, or semantics of experimental functions or remove them entirely. Chronosphere Observability Platform accepts a subset of PromQL experimental functions determined to be relatively stable, and which provide significant value while also being at a low risk of being removed by the Prometheus project. When the upstream Prometheus project makes breaking changes, Observability Platform will either preserve backward compatibility if possible or notify users and provide a deprecation timeline. The following experimental functions are available in Observability Platform:

double_exponential_smoothing()

Use the double_exponential_smoothing() experimental function to smooth a time series based on the importance you assign to older data and possible trends. See the PromQL documentation.

fill() | fill_left() | fill_right()

The fill() modifiers control how binary operations (arithmetic and comparison operators) handle series that don’t have matching label sets. By default, PromQL only produces results for label combinations that exist on both sides of an operation. Fill modifiers let you specify default values for missing series, allowing operations to produce results even when label sets don’t perfectly overlap. Observability Platform supports the following fill() modifier variants:
  • fill(value): Fills both sides with the specified value when a matching series is missing on either side.
  • fill_left(value): Only fills the left side with the value when left series are missing, but right series exist.
  • fill_right(value): Only fills the right side with the value when right series are missing, but left series exist.
The value can be any scalar expression, including special values like NaN, Inf, or -Inf. These modifiers can help to compute arithmetic with delta metrics, compare metrics from different sources without complete label overlap, and let missing series behave like there’s a specific default value (like zero) rather than being absent. For example, to add two metrics where you want to treat missing series as zero:
requests_by_endpoint + fill(0) errors_by_endpoint
If requests_by_endpoint has series for endpoints A, B, and C, but errors_by_endpoint has series for only A and B, this query produces results for all three endpoints, treating the missing error count for endpoint C as zero. To fill only one side, use the directional variants, which helps when you want asymmetric behavior:
active_connections + fill_right(0) connection_limit
This query produces results for all active connections (treating missing limits as zero), but won’t create results for limits that have no corresponding active connections.

Combine fill variants

You can specify different fill values for each side:
metric_a + fill_left(5) fill_right(10) metric_b
This query fills missing left-side series with a value of 5, and missing right-side series with a value of 10.

Create comparisons

Fill modifiers work with comparison operators (==, !=, >, <, >=, <=), including the bool modifier:
actual_usage > fill(0.8) usage_threshold
This query compares usage against thresholds, treating missing thresholds as 0.8. When used with bool, the result is 1 (true) or 0 (false):
actual_usage > bool fill(0.8) usage_threshold

Match labels

Fill modifiers can include on() and ignoring() for label matching:
requests + on(status) fill(0) request_limits
This query matches on only the status label, filling missing series with 0.

Match one to many

Fill modifiers work with group_left and group_right for many-to-one and one-to-many joins:
requests / on(status) group_left fill_right(1) limits
When using group_left, the fill_right modifier fills the side designated by 1 (right side), and fill_left fills the “many” side (left side). This creates results for all request series, using 1 as the limit when no matching limit exists.

Edge cases

When using the fill() modifier, consider the following edge cases:
  • When one vector is completely empty, the fill() modifier and the directional variants that would fill the empty side will produce results using only the non-empty vector’s series.
  • When both sides have the same label set, the fill() modifier has no effect (all series match).
  • When label sets have no overlap at all, the fill() modifier creates results for all series from both sides.

histogram_quantiles()

The histogram_quantiles(v instant-vector, label_name string, quantile1 scalar,[quantile2 scalar, ...]) function computes multiple quantiles (up to ten) from both native and classic histogram metrics in a single operation. Results are labeled with the specified label name. This function is more efficient than repeatedly calling histogram_quantile(). Use the following query to compute the fiftieth, ninetieth, and ninety-ninth percentiles of request duration with results labeled by percentile:
histogram_quantiles(http_request_duration_seconds, "percentile", 0.5, 0.9, 0.99)
Each quantile value is returned as a separate series with the specified label. From the example, the percentiles are 0.5, 0.9, and 0.99. Use the label_name parameter to choose the label that distinguishes the different quantile series.

ts_of_*(range-vector)

Use the ts_of_... (timestamp of) experimental functions to aggregate timestamps of samples within a specified interval. For details, see <aggregation>_over_time() in the Prometheus documentation. These experimental functions were introduced by specific Prometheus versions:
Function nameDescriptionVersion added
ts_of_first_over_timeReturns the timestamp of the earliest sample in the range vector for each series.3.7.0
ts_of_last_over_timeReturns the timestamp of the last sample in the specified interval, per series.3.5.0
ts_of_min_over_timeReturns the timestamp of the last float sample that has the minimum value of all float samples in the specified interval, per series.3.5.0
ts_of_max_over_timeReturns the timestamp of the last float sample that has the maximum value of all float samples in the specified interval, per series.3.5.0

holt_winters() (deprecated)

As of Prometheus 3.0, this function is named double_exponential_smoothing(). Although Observability Platform aliases holt_winters() to provide backward compatibility, you should replace it with the new name to avoid issues when Observability Platform removes the holt_winters() alias.