> ## Documentation Index > Fetch the complete documentation index at: https://docs.chronosphere.io/llms.txt > Use this file to discover all available pages before exploring further. # Recording rules Recording rules are a type of [aggregation rule](/control/shaping/shape-metrics/rules) used primarily to improve the performance of frequent queries. Recording rules let you compute frequently used or expensive queries ahead of time, and save the results as a new set of time series. Instead of running the expensive or complex query, Chronosphere Observability Platform queries for the time series generated by the recording rule. This concept is similar to using a precomputed lookup table to avoid complex calculations and find data faster. Recording rules let you alias repeated expressions used in dashboards, and also alias other recording rules. For example, a recording rule might compute the total usage percentage for a resource, which can be referenced in other recording rules and dashboards. Recording rules run based on a fixed interval. They ingest raw metric data into the database before reading it, save the results in a new time series, generate the aggregated and downsampled metric data, and store the data in the database. ## Limitations Although recording rules are powerful, they have the following limitations: * Recording rules might be delayed because they run in a batch format. * Recording rules apply only to individual metrics, not broad aggregations. * Recording rules don't have the ability to discard raw data after aggregation. * Recording rules don't capture [late-arriving data](/administer/limits-licensing/limits/metric-limits#late-arriving-metrics). * If a recording rule fails to run, there's no way to backfill the data. To avoid these limitations, use [rollup rules](/control/shaping/shape-metrics/rules/rollup) or [derived telemetry](/investigate/querying/metrics/derived-telemetry) instead. For example, if you have issues with late-arriving data, consider using rollup rules instead of recording rules. ## Attributes Recording rules support only Prometheus metrics. Configure a recording rule with a PromQL statement executed against the metrics data with the result stored in a new time series with a unique metric name. PromQL statements in recording rules can include any PromQL function. See the [Recording rule API](/tooling/api-info/definition/operations/CreateRecordingRule) for a full definition and list of attributes. ## Best practices Recording rules support adding labels to the resultant aggregated metrics, which rollup rules don't support. Rollup rules also require using either a Prometheus relabel rule, or a derived metric with a `label_replace` function in conjunction with the rollup rule, to accomplish the same goal. Due to architectural differences between Observability Platform and Prometheus, defining recording rules is sometimes different, especially for expensive recording rules that span many metrics. For example, recording rules in Observability Platform are part of a rule group, whereas recording rules in Prometheus aren't guaranteed to be run sequentially. Observability Platform uses a single data store. To enhance performance, use the following recommendations: * Break up the recording rules to scope to different clusters, or another label that scopes your metrics. * Use the `metric_name` field so they all get written back into the same name. With a Prometheus or Thanos setup, Chronosphere recommends [scoping the rules to the local Prometheus server](https://thanos.io/tip/components/rule.md/#rule-aka-ruler) to avoid cross-Prometheus queries. ## View recording rules Select from the following methods to view your recording rules. In the navigation menu, click ** Go to Admin** and then select ** Control > Recording Rules**. The recording rules page is searchable by rule **Name** or **Execution group**. The following fields display: * **Name:** The rule name. * **Execution Group:** The execution group this rule is assigned to. Rules in the same execution group run at intervals. The entire group must complete an execution before the rules in that group will run again. * **Metric Name:** The time series to output to. * **Interval:** How often the rule evaluates. * **Labels:** Label names added to the output metric. * **Query:** Click the `<>` to display the query used for this rule. To list your existing recording rules using [Chronoctl](/tooling/chronoctl), use the `chronoctl recording-rules list` command: ```shell theme={null} chronoctl recording-rules list ``` To complete this action with the Chronosphere API, use the [`ListRecordingRules`](/tooling/api-info/definition/operations/ListRecordingRules) endpoint. Because the Chronosphere API requires authentication, include an API token with your `curl` request, as shown in the following example. For more details, see [Create an API token](/tooling/api-info#create-an-api-token). ```shell /"TOKEN"/ /INSTANCE/ /METHOD/ /ENDPOINT_PATH/ theme={null} export CHRONOSPHERE_API_TOKEN="TOKEN" export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io" curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \ -X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH" ``` Replace the following: * *`TOKEN`*: Your API token. * *`INSTANCE`*: The subdomain name for your organization's Observability Platform instance. * *`METHOD`*: The HTTP method to use with the request, such as `GET` or `POST`. * *`ENDPOINT_PATH`*: The specific endpoint you want to access. ## Create or update a recording rule Select from the following methods to create or update recording rules. Users can modify Terraform-managed resources only by using Terraform. [Learn more](/tooling/infrastructure/terraform#prevent-changes-to-managed-resources). To create a recording rule with [Chronoctl](/tooling/chronoctl), define the rule in a YAML file and apply it. If you don't already have a YAML configuration file, use the `scaffold` Chronoctl parameter to generate a template for a specific resource type: ```shell theme={null} chronoctl recording-rules scaffold ``` You can redirect the results (using the redirection operator `>`) to a file for editing. 1. Create or edit a YAML configuration file to configure the recording rule. See the [Chronoctl example](#chronoctl-example) for a complete configuration file. 2. Apply the recording rule: ```shell theme={null} chronoctl apply -f FILE_NAME.yaml ``` Replace *`FILE_NAME`* with the name of the YAML configuration file. When you run `terraform plan` to generate an execution plan, Chronosphere automatically tests configurations that include notification policies by submitting them as dry runs. For details, see the [Terraform provider](/tooling/infrastructure/terraform#validate-plans-with-dry-runs) documentation. Create a recording rule with [Terraform](/tooling/infrastructure/terraform) by using the `chronosphere_recording_rule` type followed by a name in a resource declaration. For more information, see the [Terraform recording rule example](#terraform-example). 1. Add the definition to a Terraform file. 2. Run this command to create the resource: ```shell theme={null} terraform apply ``` To complete this action with the Chronosphere API, use the [`CreateRecordingRule`](/tooling/api-info/definition/operations/CreateRecordingRule) endpoint. Because the Chronosphere API requires authentication, include an API token with your `curl` request, as shown in the following example. For more details, see [Create an API token](/tooling/api-info#create-an-api-token). ```shell /"TOKEN"/ /INSTANCE/ /METHOD/ /ENDPOINT_PATH/ theme={null} export CHRONOSPHERE_API_TOKEN="TOKEN" export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io" curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \ -X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH" ``` Replace the following: * *`TOKEN`*: Your API token. * *`INSTANCE`*: The subdomain name for your organization's Observability Platform instance. * *`METHOD`*: The HTTP method to use with the request, such as `GET` or `POST`. * *`ENDPOINT_PATH`*: The specific endpoint you want to access. ### Chronoctl example The following YAML example includes three recording rules that calculate the average rate of increase per second for jobs that contain a value for `node`. The results display for `instance` and `container` as measured over one minute. This example uses the `metric_name` field to specify the output name of the time series, and the `name` field to display the human readable name. For backwards compatibility, the example uses the `name` field for the time series if `metric_name` isn't specified, like in the third rule. ```yaml theme={null} api_version: v1/config kind: RecordingRule spec: name: cpu-usage-seconds-sum-rate-1m slug: instance-container-cpu-usage-seconds-sum-rate1m prometheus_expr: sum(rate(container_cpu_usage_seconds_total{node=""}[1m])) by (instance, container) metric_name: instance_container:cpu_usage_seconds:sum_rate1m interval_secs: 60 label_policy: add: resource: cpu --- api_version: v1/config kind: RecordingRule spec: name: network-receive-bytes-sum-rate-1m slug: instance-container-network-receive-bytes-sum-rate1m metric_name: instance_container:network_receive_bytes:sum_rate1m prometheus_expr: sum(rate(container_network_receive_bytes_total{node=""}[1m])) by (instance, container) interval_secs: 60 label_policy: add: resource: network-receive --- api_version: v1/config kind: RecordingRule spec: name: instance_container:network_transmit_bytes:sum_rate1m slug: instance-container-network-transmit-bytes-sum-rate1m prometheus_expr: sum(rate(container_network_transmit_bytes_total{node=""}[1m])) by (instance, container) interval_secs: 60 label_policy: add: resource: network-transmit ``` ### Terraform example The following code creates a recording rule that Terraform refers to as `scrape_duration_recording_rule` with the name `60s rule`, and defines the other data needed to create a recording rule. ```terraform theme={null} resource "chronosphere_recording_rule" "scrape_duration_recording_rule" { # Name for the rule # Value forms the metric name output if metric_name is not specified. name = "60s rule" metric_name = "scrape_duration_seconds:max_60s" # Arbitrary labels to attach to the rule # These labels end up being part of the output metric, # and take precedence over any labels that the expression would have created. # For example, if the expression had metrics with "foo=bar", but the value was # "foo=test", the final metric generated would have "foo=test". labels = { "owner" = "infra" } # Interval at which to evaluate the rule interval = "60s" # The PromQL expression to evaluate expr = "max(scrape_duration_seconds)" } ``` ## Delete a recording rule Select from the following methods to delete a recording rule. To delete a recording rule using [Chronoctl](/tooling/chronoctl), use the `chronoctl recording-rules delete` command: ```shell theme={null} chronoctl recording-rules delete SLUG ``` Replace *`SLUG`* with the slug of the rule you want to delete. To delete a resource that's managed by [Terraform](/tooling/infrastructure/terraform): 1. Edit your Terraform configuration file to remove the pre-existing resource definition. 2. Run this command to remove the resource from Observability Platform: ```shell theme={null} terraform apply ``` To complete this action with the Chronosphere API, use the [`DeleteRecordingRule`](/tooling/api-info/definition/operations/DeleteRecordingRule) endpoint. Because the Chronosphere API requires authentication, include an API token with your `curl` request, as shown in the following example. For more details, see [Create an API token](/tooling/api-info#create-an-api-token). ```shell /"TOKEN"/ /INSTANCE/ /METHOD/ /ENDPOINT_PATH/ theme={null} export CHRONOSPHERE_API_TOKEN="TOKEN" export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io" curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \ -X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH" ``` Replace the following: * *`TOKEN`*: Your API token. * *`INSTANCE`*: The subdomain name for your organization's Observability Platform instance. * *`METHOD`*: The HTTP method to use with the request, such as `GET` or `POST`. * *`ENDPOINT_PATH`*: The specific endpoint you want to access.