Log dataset budgets

OBSERVABILITY PLATFORM

Log dataset budgets

This feature is available only to specific Chronosphere Observability Platform users, and has not been announced or officially released. Do not share or discuss this feature, or information about it, with anyone outside of your organization.

After defining log datasets, you can create budgets to allocate a percentage of your log license limit to each dataset. This allocation is similar to pool allocations for metrics. Budgets ensure you don't accidentally exceed your log license limit if a dataset starts generating data excessively and unexpectedly.

If a log or single service, pod, or environment begins to emit a high volume of data and pushes the system over its persisted ingest limit, Chronosphere Observability Platform applies a penalty to the dataset containing the emitted log. Datasets that otherwise haven't exceeded their quota aren't penalized.

You can use the Logging License Consumption dashboard to monitor your persisted log data consumption against your license capacity. This dashboard monitors your persisted writes, persisted volume, and ingest limit consumption against your logging license quotas over a given time span, including for each of your datasets. A panel in the Per Dataset panel group displays received, persisted, and dropped data in bytes against your license limit for each dataset.

View budgets

Select from one of the following methods to view budgets for log datasets.

To complete this action with the Chronosphere API, use the ReadLogAllocationConfig endpoint.

Because the Chronosphere API requires authentication, include an API token with your curl request, as shown in the following example. For more details, see Create an API token.

export CHRONOSPHERE_API_TOKEN="TOKEN"
export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"
 
curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
     -X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"

Replace the following:

TOKEN: Your API token.
INSTANCE: The subdomain name for your organization's Observability Platform instance.
METHOD: The HTTP method to use with the request, such as GET or POST.
ENDPOINT_PATH: The specific endpoint you want to access.

Create budgets

You can create budgets for datasets using the following methods. When creating budgets, first define a default dataset and assign an allocation percentage, which must be a value between zero and 100. Then, define individual datasets that map to your organization.

If your organization exceeds the defined limit for persisted writes of log data, Observability Platform applies a penalty to the dataset containing the emitted log. If the log isn't assigned to a defined dataset, then the budget for the default dataset applies.

The percentage defined in the default dataset (and any additional datasets) must both add up to and not exceed 100.

Prerequisites

Before creating budgets, define log datasets. Use the slug or ID of your dataset to assign budgets.

When you run terraform plan to generate an execution plan, Chronosphere automatically tests configurations that include notification policies by submitting them as dry runs. For details, see the Terraform provider documentation.

To create a budget with Terraform:

Create or edit a Terraform file and add the definition by using the chronosphere_log_allocation_config type.
Define the default_dataset, which applies to logs that are not in any other dataset. Set a license percentage, and optionally define priorities for individual queries.
Define dataset_allocations to allocate budgets to individual datasets. Datasets are evaluated in order. Set a license percentage, and optionally define priorities for individual queries.
Run this command to apply the changes:
```
terraform apply
```

See the Terraform budget example for a completed budget resource.

Terraform budget example

The following Terraform resource creates a budget that Terraform refers to by my_log_allocation_config. The budget defines the allocation percentage and priority for the default_dataset, and for an individual dataset named shopper_production_us.

resource "chronosphere_log_allocation_config" "my_log_allocation_config" {
  # Defines the default dataset, which applies if logs don't match any defined datasets.
  default_dataset {
    # Sets the percentage of the license limit to allocate to the default dataset.
    allocation {
      percent_of_license = 90.1
    }
    # Defines explicit high and low priority match criteria to define which logs to
    # drop first (low) and last (high).
    priorities {
      # A list of search filters defining which logs are considered high priority in
      # this dataset. Filters are combined as OR statements so only one filter needs to
      # match.
      high_priority_filter {
        query = "severity='ERROR'"
      }
      # A list of search filters defining which logs are considered low priority in
      # this dataset. Filters are combined as OR statements so only one filter needs to
      # match.
      low_priority_filter {
        query = "severity='DEBUG'"
      }
    }
  }
 
  dataset_allocation {
  # Unique ID of the dataset to allocate a percentage of the license limit to.
    dataset_id = shopper_production_us
    # Sets the percentage of the license limit to allocate to this dataset.
    allocation {
      percent_of_license = 9.9
    }
    priorities {
      high_priority_filter {
        query = "severity='ERROR'"
      }
      low_priority_filter {
        query = "severity='DEBUG'"
      }
    }
  }
}

Update budgets

Select from one of the following methods to update budgets for available log datasets.

To edit a budget using Terraform:

Create or edit a Terraform file that updates the resource's existing properties.
Run this command to apply the changes:
```
terraform apply
```

Delete budgets

Select from one of the following methods to delete budgets for log datasets.

Users cannot modify Terraform-managed resources in the user interface, with Chronoctl, or by using the API. Learn more.

To delete a resource with Terraform:

Remove its definition from the Terraform file.
Run this command to apply the changes:
```
terraform apply
```

External tasks Live Telemetry Analyzer