Derived metrics

Derived metrics let you create user-friendly names for queries. Use derived metrics to implement query aliasing, which reduces the need to write complex queries.

View your derived metrics

Chronoctl
API

View derived metrics with the Chronoctl command chronoctl derived-metrics list, filtered by their slugs with the --slugs flag.For example, to list all derived metrics:

chronoctl derived-metrics list

To list derived metrics with slugs slug_name_1 and slug_name_2:

chronoctl derived-metrics list --slugs slug_name_1,slug_name_2

To complete this action with the Chronosphere API, use the ListDerivedMetric endpoint.Because the Chronosphere API requires authentication, include an API token with your curl request, as shown in the following example. For more details, see Create an API token.

export CHRONOSPHERE_API_TOKEN="TOKEN"
export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"

curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
     -X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"

Replace the following:

TOKEN: Your API token.
INSTANCE: The subdomain name for your organization’s Observability Platform instance.
METHOD: The HTTP method to use with the request, such as GET or POST.
ENDPOINT_PATH: The specific endpoint you want to access.

Create a derived metric

Chronoctl
Terraform
API

Here’s a Chronoctl example of a derived metric with two underlying expressions:

api_version: v1/config
kind: DerivedMetric
spec:
  name: my derived metric
  slug: my-derived-metric
  metric_name: test_metric
  description: This is a test derived metric
  queries:
  - query:
      prometheus_expr: scrape_duration_seconds{$job}
      variables:
        - name: job
          default_prometheus_selector: job=~".*"
    selector:
      labels:
      - name: id
        type: EXACT
        value: abc
  - query:
      prometheus_expr: scrape_series_added{$job}
      variables:
        - name: job
          default_prometheus_selector: job=~".*"

Create a derived metric with Terraform by using the chronosphere_derived_metric type followed by a name in the resource declaration:

resource "chronosphere_derived_metric" "my-derived-metric" {
    # The name of the derived metric.
    name        = "my derived metric"
    # Slug is the unique identifier of the derived metric.
    slug        = "my-derived-metric"
    # Description about the derived metric.
    description = "This is a test derived metric"
    # Metric name to be used to query for the derived metric.
    metric_name = "test_metric"

    # List of queries corresponding to the derived metric. If more than one query is specified,
    # all but one need to have selectors defined.
    queries {
        # Optional selector indicates which query to select for a derived metric based on the selector.
        # For example, the selector will match a query like test_metric{id="abc"}.
        selector {
            # The set of labels that need to match for the selector to be considered a match.
            # The value here can be a regex value.
            labels = {
                id = "abc"
            }
        }
        # The query corresponding to the previously described selector.
        query {
            # expr has the expression meant to be run for the derived metric, and must contain the
            # list of selector variables that will be applied / propagated from the derived metrics query.
            # Selectors not specified in the query apply after the query has been processed in a post-processing step.
            expr = "scrape_duration_seconds{$job}"

            # variables contains information about selectors used in the expression. Each selector specified in the
            # expression must have a variables block.
            variables {
                # The name of the variable.
                name             = "job"
                # The default selector expression to fill in the query if selector isn't provided
                # on the derived metrics query.
                default_selector = "job=~\".*\""
            }
        }
    }


    # An example of another query without any selectors. Note how it points to a different expression.
    queries {
        query {
            expr = "scrape_series_added{$job}"
            variables {
                name             = "job"
                default_selector = "job=~\".*\""
            }
        }
    }
}

To complete this action with the Chronosphere API, use the CreateDerivedMetric endpoint.Because the Chronosphere API requires authentication, include an API token with your curl request, as shown in the following example. For more details, see Create an API token.

export CHRONOSPHERE_API_TOKEN="TOKEN"
export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"

curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
     -X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"

Replace the following:

TOKEN: Your API token.
INSTANCE: The subdomain name for your organization’s Observability Platform instance.
METHOD: The HTTP method to use with the request, such as GET or POST.
ENDPOINT_PATH: The specific endpoint you want to access.

Use a selector in your derived metric definition when you want a metric name that’s used by different queries based on the selector, or when you want the same derived metric to map to different underlying metrics. This can cause performance issues if you have a large number of id items in use. Chronosphere doesn’t support sums across id items. If you have many selectors, a recording rule is often a better option. If you want to map only a derived metric name to a query, you don’t need a selector.

Delete a derived metric

Users can modify Terraform-managed resources only by using Terraform. Learn more.

Chronoctl
Terraform
API

To delete a derived metric with Chronoctl, use the chronoctl derived-metrics delete command with the slug of the derived metric you want to delete:

chronoctl derived-metrics delete SLUG_NAME

You can delete more than one metric at a time by providing a comma-separated list of slugs. For example:

chronoctl derived-metrics delete SLUG_NAME_1,SLUG_NAME_2

To delete a derived metric with Terraform:

Remove the derived metric’s definition from the Terraform file.
Run terraform apply.

To complete this action with the Chronosphere API, use the DeleteDerivedMetric endpoint.Because the Chronosphere API requires authentication, include an API token with your curl request, as shown in the following example. For more details, see Create an API token.

export CHRONOSPHERE_API_TOKEN="TOKEN"
export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"

curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
     -X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"

Replace the following:

TOKEN: Your API token.
INSTANCE: The subdomain name for your organization’s Observability Platform instance.
METHOD: The HTTP method to use with the request, such as GET or POST.
ENDPOINT_PATH: The specific endpoint you want to access.

Replace a recording rule

When you have a complex or slow recording rule, in some cases you can replace the rule with a derived metric. For example, this recording rule queries for a number of HTTP status codes:

 - record: slo:sli_error:ratio_rate5m
    expr: |  (sum(rate(flask_http_request_duration_seconds_count{job="default/productservice-servicemonitor/0", status=~"(5..|4..)"}[5m])))
      /    (sum(rate(flask_http_request_duration_seconds_count{job="default/productservice-servicemonitor/0"}[5m])))
    labels:
      owner: customersuccess
      repo: chronosphereio/productservice
      sloth_id: productservice-requests-availability
      sloth_service: productservice
      sloth_slo: requests-availability
      sloth_window: 5m
      tier: "2"

Replacing the recording rule with this derived metric can reduce query load:

resource "chronosphere_derived_metric" "slo-error-rate-5m" {
  name     = "slo-error-rate-5m"
  slug     = "slo-error-rate-5m"
  description = "Service Error Rate - 5m"
  metric_name = "slo:sli_error:ratio_rate5m"

  queries {
     query {
        expr = """
         sum(rate(flask_http_request_duration_seconds_count{
             $job,
             status=~"(5..|4..)"
         }[1m]))
         /
         sum(rate(flask_http_request_duration_seconds_count{
             $job
         }[1m]))
       """
   variables {
        name = "job"
        default_selector = "job=~\".*\""
   }
    }
  }
}

Observability Platform

Telemetry Pipeline

Tools

More information

View your derived metrics

Create a derived metric

Delete a derived metric

Replace a recording rule

Observability Platform

Telemetry Pipeline

Tools

More information

​View your derived metrics

​Create a derived metric

​Delete a derived metric

​Replace a recording rule

View your derived metrics

Create a derived metric

Delete a derived metric

Replace a recording rule