Control your log data

Chronosphere Observability Platform provides control rules that let you manage log data by transforming, reshaping, retaining, or excluding data before it’s stored. These rules let you control the volume of persisted log data and improve data quality. By controlling which data you keep or drop, you can reduce noise, better control costs, and focus on data that matters most to your business. Observability Platform offers the following control rules:

• Drop logs that match a specific filter to remove unnecessary or low-value data.
• Drop fields from logs that match specified conditions, which helps reduce the size of individual logs. If a log contains no data after fields are dropped, you can choose to drop the entire log.
• Sample logs to retain a certain percentage of matching logs and dropping the remainder, which can significantly reduce volume and maintain visibility.

View control rules

Select from one of the following methods to view control rules.

Create control rules

Use control rules to drop certain logs at ingestion, or fields that you don’t want to persist. For example, dropping log data from a specific cluster, test data in a development environment, or unnecessary individual fields across your logs.

Use one of the following methods to create control rules for log data:

Chronoctl example

The following example defines a control rules file for Chronoctl, and contains the following rules:

• The first rule samples 10% of logs that match the defined filter.
• The second rule drops all logs for the nginx service of severity INFO where the http.Request.status equals 200.
• The third rule drops all kubernetes fields that match the defined filter. The regular expression indicates that any fields matching kubernetes.[FIELD] are dropped from all logs.

api_version: v1/config
kind: LogControlConfig
spec:
  # The ordered list of control rules.
  rules:
    - name: "Sample billing service with missing user ID"
      mode: ENABLED
      # Log filter syntax to select logs. Only matching logs will have control action applied.
      filter: "cluster_name:'production'\n
              AND service='billing-service'\n
              AND message = 'missing user ID'"
      type: SAMPLE
      sample:
        # Percentage of matching logs to keep. Must be in the range [0, 1].
        rate: 0.10
    - name: "Drop NGINX information logs"
      mode: ENABLED
      filter: "service = 'nginx' and severity = 'INFO' and httpRequest.status = '200'"
      type: DROP
    - name: 'Drop Kubernetes fields from development environments'
      mode: ENABLED
      filter: "namespace = 'development' and kubernetes.cluster_name = 'test'"
      # The configuration for a drop field action.
      type: DROP_FIELD
      drop_field:
        # Regular expression to match the field names to drop.
        field_regex: .*
        # Selector to indicate field path. Use 'parent[child]' syntax to
        # indicate nesting.
        parent_path:
          selector: kubernetes

Terraform example

The following example defines a control rules resource for Terraform, and contains the following rules:

• The first rule samples 10% of logs that match the defined filter.
• The second rule drops all logs for the nginx service of severity INFO where the http.Request.status equals 200.
• The third rule drops all kubernetes fields that match the defined filter. The regular expression indicates that any fields matching kubernetes.[FIELD] are dropped from all logs.

resource "chronosphere_log_control_config" "log_control_config" {
  # The ordered list of control rules.
  rules {
    name   = "Sample billing service with missing user ID"
    # Log filter syntax to select logs. Only matching logs will have control action applied.
    filter = chomp(<<-EOF
    cluster_name:"rc"
    AND service="billing-service"
    AND message = "missing user ID"
    EOF
    )
    mode = "ENABLED"
 
    sample {
      # Percentage of matching logs to keep. Must be in the range [0, 1].
      rate = 0.01
    }
    # The configuration for a drop field action.
    type = "SAMPLE"
  }
 
  rules {
    name   = "Drop NGINX information logs"
    filter = "service = \"nginx\" and severity = \"INFO\" and httpRequest.status = \"200\""
    mode   = "ENABLED"
    type   = "DROP"
  }
 
  rules {
    name = "Drop Kubernetes fields from development environments"
 
    drop_field {
      # Regular expression to match the field names to drop.
      field_regex = ".*"
      # Selector to indicate field path. Use 'parent[child]' syntax to
      # indicate nesting.
      parent_path {
        selector = "kubernetes"
      }
    }
 
    filter = "chronosphere_namespace = \"rc-logging\" and kubernetes.cluster_name = \"rc-b\""
    mode   = "ENABLED"
    type   = "DROP_FIELD"
  }
}

Edit control rules

When creating or editing control rules, you can use the Code Config tool to view code representations of the rules. The displayed code also responds to changes you make in the Visual editor tab.

Select from the following methods to edit control rules.

Delete control rules

Select from the following methods to delete control rules.