Managing service discovery

Chronosphere Observability Platform provides a view into your applications based on services (opens in a new tab). If a service doesn't match what you expect, or there are panel groups missing from a service, you can edit the service configuration in Observability Platform to include any missing items.

Understand service configuration

The Service configuration page displays information about which services have data, and how Observability Platform maps the data together.

Edit a service configuration

If you have administrative privileges, click Go to Admin and then select Platform > Service Configuration.

Service discovery preview

The Service discovery preview reflects information selected, updated, or removed in the Service discovery templates and Service rules.

The preview updates as you make changes, and displays the current state of the configuration of discovered services.

This includes:

• Status: The status field indicates a change from the state of the service when discovered. Toggle Show diff only to filter the list for services with a status change
  • Inactivated indicates a previously discovered service which no longer has data, usually because the data is gone, or the service configuration changed.
  • Added indicates a newly discovered service.
  • Modified indicates a service changed by a user.
• Service slug: The slug name for the service.
• Service name: The service name.
• Templates: The Service discovery template used to map this service.

Click Apply changes at any time to save changes made to service configurations. If you have unsaved changes, a warning message displays:

Changes have not yet been applied.

Click Reset changes to discard any unsaved changes.

Service discovery templates

Service discovery templates are templates that manually match metrics to a discovered service.

• Status: Toggle the switch to enable or disable the template. Click Enable all to enable all templates. If all templates are enabled, this toggle changes to Disable all.
• Slug: The template's slug.
• Template name: The user-readable template name.
• Service label mapping: Labels identifying the service name. Provided by service label or instance label + identifying label from the Edit template form.
• Discovered services: The number of discovered services.

Edit or delete a discovery template

You can edit existing templates to ensure your configurations match your services when those services change.

1. Click the vertical dots and then select Edit config to change or Delete to remove the template.

2. Available edits to template details include:

   • Toggle Enable for service discovery to disable the template.

   • Template name: A name describing what the template found.

   • Template query: The query used to create the template.

   • Target name: The metric label containing the service name.

   • Target regex: The regular expression used to search for a label value.

   Focus your template using one of the following groups:

   • Service mapping: Use a Service label filter the template to only that service label.

   • Instance mapping:

     • Instance label: This label usually contains information about the instance or pod. Label values are used as keys to map to the service name.
     • Transform regex: This regular expression allows modification of the value in the instance label to match a value in the service instance mappings.
     • Transform replacement: This regular expression is replacing the original label, allowing modification of the value in the instance label so that it matches a value in the service instance mappings.
     • Identifying label: This label contains information that uniquely identifies which service the time series belongs to.

3. Click Stage changes to preview your updated configuration.

Service rules

When a service doesn't match expectations, create service rules to correct them. Service rules define how to group or match service data when service names or labels might not be the same across multiple instances.

Use the Search rules box to find a specific rule.

Create a service rule

To create a service rule:

1. Click + Create.
2. Add the following fields:
   • Rule Description: Describe what this service rule does. Displays on configuration page as Description.
   • Merge action: Displays on configuration page as Type. Select a rule type:
     • Merge rules group services together.
     • Ignore rules remove them from the interface display, and stop collecting data.
     • Keep only rules keep only the selected data, and discard all other data.
       ⚠️

       Data discarded by a keep rule isn't recoverable.

   • Source labels: Specify which label's values the rules apply to.
   • Regex: Add a new regular expression to replace the original. On the service configuration page, the original Regex displays, along with the Replacement, which is the value added here.
3. Click Save. The Service configuration page displays, with your updates displayed.
4. Click Preview and apply.
5. The Service changes preview screen displays. Preview your changes using the table. The Service and the Metric components included in that service show what metrics are merged into a given service name. Services and metrics being removed from display have a [-] before them.
6. Click Run service discovery when finished.

Edit or delete a service rule

To edit or delete a service rule:

1. Click the 3 vertical dots.
2. Click Edit rule to change the rule, or Delete to remove the rule.
3. Edit the fields, per their descriptions in the create rules section.
4. Click Stage changes. The Service discovery preview displays your updates.
5. Click Apply changes to save or Reset changes to cancel.

Rule order

Service rules apply from the top to the bottom of the list. A service matches the first rule applying to it, and won't match additional rules. Use the drag icon to move the rule up or down in the list to change the order of rule application.

Service instance discovery

The initial Chronosphere service discovery maps services to pod instances (opens in a new tab). Pod instances usually have labels explaining what they do, and what service they belong to. Chronosphere uses those labels to match the physical pods to a discovered service. Configurations in this section aren't editable.

• Slug: The slug name of the discovery rule defining the service pod mapping.
• Name: The rule name.
• Service labels: One or more labels containing information about the service name.
• Pod label: This label usually contains information about the pod or instance. The pod label is used as a key to map to the service name.

Click the vertical dots icon and then Open in Metrics Explorer to open the query for that service instance in Metrics Explorer.