Derived metrics

OBSERVABILITY PLATFORM

Derived metrics

Derived metrics let you create user-friendly names for queries. Use derived metrics to implement query aliasing, which reduces the need to write complex queries.

View your derived metrics

View derived metrics with the Chronoctl command chronoctl derived-metrics list, filtered by their slugs with the --slugs flag.

For example, to list all derived metrics:

chronoctl derived-metrics list

To list derived metrics with slugs slug_name_1 and slug_name_2:

chronoctl derived-metrics list --slugs slug_name_1,slug_name_2

Create a derived metric

Here’s a Chronoctl example of a derived metric with two underlying expressions:

api_version: v1/config
kind: DerivedMetric
spec:
  name: my derived metric
  slug: my-derived-metric
  metric_name: test_metric
  description: This is a test derived metric
  queries:
  - query:
      prometheus_expr: scrape_duration_seconds{$job}
      variables:
        - name: job
          default_prometheus_selector: job=~".*"
    selector:
      labels:
      - name: id
        type: EXACT
        value: abc
  - query:
      prometheus_expr: scrape_series_added{$job}
      variables:
        - name: job
          default_prometheus_selector: job=~".*"

Use a selector in your derived metric definition when you want a metric name that’s used by different queries based on the selector, or when you want the same derived metric to map to different underlying metrics. This can cause performance issues if you have a large number of id items in use. Chronosphere doesn’t support sums across id items. If you have many selectors, a recording rule is often a better option.

If you want to map only a derived metric name to a query, you don’t need a selector.

Delete a derived metric

Users can modify Terraform-managed resources only by using Terraform. Learn more.

To delete a derived metric with Chronoctl, use the chronoctl derived-metrics delete command with the slug of the derived metric you want to delete:

chronoctl derived-metrics delete SLUG_NAME

You can delete more than one metric at a time by providing a comma-separated list of slugs. For example:

chronoctl derived-metrics delete SLUG_NAME_1,SLUG_NAME_2

Replace a recording rule

When you have a complex or slow recording rule, in some cases you can replace the rule with a derived metric.

For example, this recording rule queries for a number of HTTP status codes:

 - record: slo:sli_error:ratio_rate5m
    expr: |  (sum(rate(flask_http_request_duration_seconds_count{job="default/productservice-servicemonitor/0", status=~"(5..|4..)"}[5m])))
      /    (sum(rate(flask_http_request_duration_seconds_count{job="default/productservice-servicemonitor/0"}[5m])))
    labels:
      owner: customersuccess
      repo: chronosphereio/productservice
      sloth_id: productservice-requests-availability
      sloth_service: productservice
      sloth_slo: requests-availability
      sloth_window: 5m
      tier: "2"

Replacing the recording rule with this derived metric can reduce query load:

resource "chronosphere_derived_metric" "slo-error-rate-5m" {
  name     = "slo-error-rate-5m"
  slug     = "slo-error-rate-5m"
  description = "Service Error Rate - 5m"
  metric_name = "slo:sli_error:ratio_rate5m"
 
  queries {
     query {
        expr = """
         sum(rate(flask_http_request_duration_seconds_count{
             $job,
             status=~"(5..|4..)"
         }[1m]))
         /
         sum(rate(flask_http_request_duration_seconds_count{
             $job
         }[1m]))
       """
   variables {
        name = "job"
        default_selector = "job=~\".*\""
   }
    }
  }
}

Derived telemetry Derived labels