Configure ingestion and processing of OpenTelemetry Metrics

For prerequisites to configuring the OpenTelemetry Collector for metrics data ingestion, see Configure your OpenTelemetry Collector.

OpenTelemetry ingestion configuration options

The OpenTelemetry Protocol (OTLP) metric data model (opens in a new tab) defines separate Resource and Data Point attribute contexts.

• The Resource context (opens in a new tab) contains attributes, such as host.name or k8s.cluster.name, that are common to all time series produced by the metric writer.
• The Data Point context contains attributes unique to the time series, such as http.route.

Chronosphere Observability Platform provides a configuration interface to control OpenTelemetry Metric Resource attribute processing. The default OpenTelemetry Metrics processing configuration in Observability Platform merges all Resource and Data Point attributes to the time series as labels. If there are conflicting attribute keys, Observability Platform uses the Data Point value.

You can configure how Observability Platform processes OpenTelemetry Metrics to:

• Prevent Observability Platform from merging resource attributes.
• Enable the creation and writing of a target_info metric using all resource attributes. The target_info metric is functionally equivalent to the up metric, the presence of which indicates that a resource is available.
• Change how Observability Platform handles conflicts when merging Resource and Data Point attributes.
• Define an exclusion list of resource attribute keys to exclude from merging.

Resource attribute configuration options

The OpenTelemetry Metrics Ingestion configuration has a top-level resource_attributes field with additional sub-fields that define how Observability Platform processes resource attributes.

flatten_mode

The flatten_mode field determines how Observability Platform merges resource attributes into the time series. Its valid values are:

• MERGE (default): Observability Platform merges Resource attributes into every data point associated with that resource, so that both sets end up as Prometheus labels on each individual metric time series. If attribute keys conflict, Observability Platform uses the value from the data point.
• IGNORE: Observability Platform doesn't copy resource attributes to the resulting time series.
• OVERWRITE: Observability Platform copies resource attributes onto every data point. If attribute keys conflict, Observability Platform uses the value from the resource.

filter_mode

The filter_mode field determines which resource attributes Observability Platform processes when generating target_info metrics and merging resource attributes into time series as labels. Its valid values are:

• APPEND_DEFAULT_EXCLUDE_KEYS (default): Observability Platform includes all attributes unless they're defined in your exclude_keys list or the default Observability Platform-defined resource attribute exclusions list.

  With this setting, Observability Platform appends any exclude_keys you define to the default Observability Platform-defined list.

• CUSTOM_EXCLUDE_KEYS: Observability Platform includes all attributes unless they're specifically defined in your exclude_keys list.

  With this setting, Observability Platform applies only your defined exclude_keys list.

exclude_keys

Use the exclude_keys field to list key names that you want Observability Platform to exclude from both target_info generation and resource attribute merging.

The field's format is an array of strings, such as:

[
  "example.exclude",
  "another.example.exclude"
]

The resource attribute exclusions that Observability Platform applies by default aren't returned in this response. See Observability Platform-defined resource attribute exclusions for the latest list of default key exclusions.

Observability Platform-defined resource attribute exclusions

Some OpenTelemetry Resource attributes are likely to exceed Observability Platform attribute value size limits, resulting in rejected time series. Observability Platform excludes the following Resource attributes by default:

In cases where the OpenTelemetry Collector truncates these attributes to conform with Observability Platform ingestion requirements, you can change the merge behavior to include all resource attributes by setting filter_mode = CUSTOM_EXCLUDE_KEYS.

Manage OpenTelemetry metrics ingestion configurations

The Chronosphere API Config Endpoint provides methods to create, read, update, and delete the OpenTelemetry Metrics Configuration.

View the configuration

You can view OTLP ingestion configurations using one of the following methods.

chronoctl otel-metrics-ingestion read

Create or update the configuration

You can change the default OpenTelemetry Metrics ingestion configuration using either Chronoctl or the Terraform provider.

chronoctl otel-metrics-ingestion scaffold

api_version: v1/config
kind: OtelMetricsIngestion
spec:
  resource_attributes:
    generate_target_info: true
    flatten_mode: MERGE
    filter_mode: APPEND_DEFAULT_EXCLUDE_KEYS
    exclude_keys:
      - example.exclude
      - another.example.exclude

chronoctl apply -f <FILE_NAME.YML>

Delete the configuration

Use one of the following methods to delete an OpenTelemetry configuration.

chronoctl otel-metrics-ingestion delete

chronoctl otel-metrics-ingestion read