Control your log data

This feature isn’t available to all Chronosphere Observability Platform users and might not be visible in your app. For information about enabling this feature in your environment, contact Chronosphere Support.

Chronosphere Observability Platform provides control rules that let you manage log data by transforming, reshaping, retaining, or excluding data before it’s stored. These rules let you control the volume of persisted log data and improve data quality. By controlling which data you keep or drop, you can reduce noise, better control costs, and focus on data that matters most to your business. Observability Platform offers the following control rules:

Drop logs that match a specific filter to remove unnecessary or low-value data.
Drop fields from logs that match specified conditions, which helps reduce the size of individual logs. If a log contains no data after fields are dropped, you can choose to drop the entire log.
Sample logs to retain a certain percentage of matching logs and dropping the remainder, which can significantly reduce volume and maintain visibility.

View control rules

Select from one of the following methods to view control rules.

To view control rules:

In the navigation menu, click Go to Admin and then select Control > Logs Control Rules.

All defined control rules display.
To view the code definition for all configured control rules, click the Code config tab.

Create control rules

Use control rules to drop certain logs at ingestion, or fields that you don’t want to persist. For example, dropping log data from a specific cluster, test data in a development environment, or unnecessary individual fields across your logs.

Use one of the following methods to create control rules for log data:

Define control rules in Observability Platform, and then use the Code Config tool to apply the rule definition.

In the navigation menu, click Go to Admin and then select Control > Logs Control Rules.
Click Add control rule to create a control rule.
In the Create control rule page, enter a name for the control rule.
From the Action menu, select the type of rule to create:
- Drop logs: Completely drop logs that match the specified filter.
- Drop fields: Remove specific fields from logs that match the specified filter.
- Sample logs: Retain a certain percentage of matching logs and drop the remainder.
Enter a filter to return log data for the drop rule, then press Windows+Return (Command+Return on macOS) to submit the filter. This rule applies only to logs that match this filter at the time the log data was ingested.
Review the returned data from preview filter preview and make changes as necessary.
Enter additional information for the specified action:
- Drop fields: In the Parent path field, enter the level of depth in the log to apply the regular expression to. For example, if you wanted to apply the regular expression to fields under httpRequest, such as httpRequest.status, enter httpRequest in the Parent path field.
  
  In the Field regex field, enter the regular expression that determines which fields to match on.
- Sample logs: In the Logs to keep field, enter a percentage of logs you want to keep. For example, 25 for 25%.
Click Save to save the control rule definition.
In the Code Config tab, use the Code Config tool to apply the definition.

Chronoctl example

The following example defines a control rules file for Chronoctl, and contains the following rules:

The first rule samples 10% of logs that match the defined filter.
The second rule drops all logs for the nginx service of severity INFO where the http.Request.status equals 200.
The third rule drops all kubernetes fields that match the defined filter. The regular expression indicates that any fields matching kubernetes.[FIELD] are dropped from all logs.

api_version: v1/config
kind: LogControlConfig
spec:
  # The ordered list of control rules.
  rules:
    - name: "Sample billing service with missing user ID"
      mode: ENABLED
      # Log filter syntax to select logs. Only matching logs will have control action applied.
      filter: "cluster_name:'production'\n
              AND service='billing-service'\n
              AND message = 'missing user ID'"
      type: SAMPLE
      sample:
        # Percentage of matching logs to keep. Must be in the range [0, 1].
        rate: 0.10
    - name: "Drop NGINX information logs"
      mode: ENABLED
      filter: "service = 'nginx' and severity = 'INFO' and httpRequest.status = '200'"
      type: DROP
    - name: 'Drop Kubernetes fields from development environments'
      mode: ENABLED
      filter: "namespace = 'development' and kubernetes.cluster_name = 'test'"
      # The configuration for a drop field action.
      type: DROP_FIELD
      drop_field:
        # Regular expression to match the field names to drop.
        field_regex: .*
        # Selector to indicate field path. Use 'parent[child]' syntax to
        # indicate nesting.
        parent_path:
          selector: kubernetes

Terraform example

The following example defines a control rules resource for Terraform, and contains the following rules:

The first rule samples 10% of logs that match the defined filter.
The second rule drops all logs for the nginx service of severity INFO where the http.Request.status equals 200.
The third rule drops all kubernetes fields that match the defined filter. The regular expression indicates that any fields matching kubernetes.[FIELD] are dropped from all logs.

resource "chronosphere_log_control_config" "log_control_config" {
  # The ordered list of control rules.
  rules {
    name   = "Sample billing service with missing user ID"
    # Log filter syntax to select logs. Only matching logs will have control action applied.
    filter = chomp(<<-EOF
    cluster_name:"rc"
    AND service="billing-service"
    AND message = "missing user ID"
    EOF
    )
    mode = "ENABLED"
 
    sample {
      # Percentage of matching logs to keep. Must be in the range [0, 1].
      rate = 0.01
    }
    # The configuration for a drop field action.
    type = "SAMPLE"
  }
 
  rules {
    name   = "Drop NGINX information logs"
    filter = "service = \"nginx\" and severity = \"INFO\" and httpRequest.status = \"200\""
    mode   = "ENABLED"
    type   = "DROP"
  }
 
  rules {
    name = "Drop Kubernetes fields from development environments"
 
    drop_field {
      # Regular expression to match the field names to drop.
      field_regex = ".*"
      # Selector to indicate field path. Use 'parent[child]' syntax to
      # indicate nesting.
      parent_path {
        selector = "kubernetes"
      }
    }
 
    filter = "chronosphere_namespace = \"rc-logging\" and kubernetes.cluster_name = \"rc-b\""
    mode   = "ENABLED"
    type   = "DROP_FIELD"
  }
}

Edit control rules

When creating or editing control rules, you can use the Code Config tool to view code representations of the rules. The displayed code also responds to changes you make in the Visual editor tab.

Select from the following methods to edit control rules.

In the navigation menu, click Go to Admin and then select Control > Logs Control Rules.
In the row of the control rule you want to edit, click the three vertical dots icon and then click Edit control rule.
In the Edit control rule drawer, make changes to your control rule, and then click Save.

Observability Platform saves changes to your control rule.

Delete control rules

Select from the following methods to delete control rules.

In the navigation menu, click Go to Admin and then select Control > Logs Control Rules.
In the row of the control rule you want to delete, click the three vertical dots icon and then click Delete control rule.

Observability Platform deletes your control rule.

Parse logs Administer