OBSERVABILITY PLATFORM
Control
Log controls
Shape logs

Shape log data

This feature isn’t available to all Chronosphere Observability Platform users and might not be visible in your app. For information about enabling this feature in your environment, contact Chronosphere Support.

Your log data can be formatted differently depending on which service generates that data. Some data is structured, such as valid JSON. Other data might be somewhat structured, or not have any structure at all. In these instanced, you can create parsers to create structured logs from plaintext logs, and parse individual fields to extract meaningful data without changing your instrumentation.

For example, if your instrumentation emits unstructured plaintext logs, create a plaintext parser to provide structure by separating the individual fields.

If your logs contain JSON or key/value pairs buried within a single string field, create a field parser to extract the values. The extracted fields can then be queried in Logs Explorer and used in aggregations and transformations.

View parsers

Select from the following methods to view and order available parsers. Parsers display in match order.

To view parsers:

  1. In the navigation menu, click Go to Admin and then select Control > Logs Configuration.

    All parsers display, including plaintext and field parsers.

  2. To view the code definition for all configured parsers, click the Code Config tab.

Create parsers

Parsers operate on both plaintext logs and individual fields. All configured parsers are included in a single definition, and evaluated in match order. The first parser in the definition that matches a log is applied, and any subsequent parsers are ignored.

Although you can define parsers in the Observability Platform web interface, you must use the Code Config tool to apply the definition.

Create plaintext parsers

Plaintext parsers run against all ingested logs with the plaintext mimetype, and apply parsing logic to modify the original record. These parsers affect the entire log, whereas field parsers operate on individual fields within a log.

Select either a regular expression or a key/value matching pattern to parse logs. When using a key/value pattern, you can extract any number of key value pairs from a plaintext log. Plaintext parsers also have an option to keep the original log after applying parsing logic.

Plaintext parsers apply only when the media type is set to plaintext for logs. Regular expressions for the matching type use RE2 syntax (opens in a new tab) and named capture groups. Each named capture group becomes the key name in the structured log. Observability Platform requires at least one named capture group in the regular expression.

Use one of the following methods to create plaintext parsers.

Define the plaintext parser definition in Observability Platform, and then use the Code Config tool to apply the definition.

  1. In the navigation menu, click Go to Admin and then select Control > Logs Configuration.
  2. In the Plaintext Parsers section, click Create plaintext parser.
  3. Enter a name for the parser.
  4. Select the matching type for the parser.
    • REGEX uses a regular expression to match logs. In the Template field, select the regular expression template to apply to your logs. Use a predefined template such as Nginx, or specify a Custom template.
    • Key/value uses a specified pattern to separate key/value pairs in log data. Enter the following values:
      • Pair separator is the character used to split matching pairs into its key and value. For example, a colon (:) if your data is in the format key: value.
      • Delimiter is the character used to split the input log data into key/value pairs. For example, a comma (,) if your data is in the format key: value, key: value.
      • Trim set identifies the leading and trailing characters to trim from the key/value pairs. For example, a space (" ") between key/value pairs if your data is in the format key: value, key: value.
  5. If you want to keep the original log after parsing, select the corresponding checkbox.
  6. Click Save to save the parser definition.
  7. On the Logs Configuration page, click View code config.
  8. In the Code Config tab, use the Code Config tool to apply the definition.

Create field parsers

Field parsers operate on an individual field within a log, and run on all ingested logs, including JSON and plaintext logs. Field parsers run after plaintext parsers. When defining a field parser, choose which matching pattern to apply and specify the destination of the parsed field.

Define the field parser definition in Observability Platform, and then use the Code Config tool to apply the definition.

  1. In the navigation menu, click Go to Admin and then select Control > Logs Configuration.

  2. In the Field Parsers section, click Create field parser.

  3. Enter a source field for the parser to operate on, such as message.

  4. Enter a destination for the parsed data, which can be one of these locations:

    • Use the same field name, which overwrites the existing field data with the parsed data.
    • Specify a new field name to contain the parsed data.
    • Leave the destination field empty, which populates the parsed data to the top level of your logs.

  5. Select the matching type for the parser.

    • JSON matches on valid JSON logs.
    • REGEX uses a regular expression to match logs. In the Template field, select the regular expression template to apply to your logs. Use a predefined template such as Nginx, or specify a Custom template.
    • Key/value uses a specified pattern to separate key/value pairs in log data. Enter the following values:
      • Pair separator is the character used to split matching pairs into its key and value. For example, a colon (:) if your data is in the format key:value.
      • Delimiter is the character used to split the input log data into key/value pairs. For example, a comma (,) if your data is in the format key: value, key: value.
      • Trim set identifies the leading and trailing characters to trim from the key/value pairs. For example, a space (" ") between key/value pairs if your data is in the format key: value, key: value.

  6. Click Save to save the parser definition.

  7. On the Logs Configuration page, click View code config.

  8. In the Code Config tab, use the Code Config tool to apply the definition.

Edit parsers

When creating or editing a parser, you can use the Code Config tool to view code representations of a parser. The displayed code also responds to changes you make in the Visual Editor tab.

Select from the following methods to edit parsers.

  1. In the navigation menu, click Go to Admin and then select Control > Logs Configuration.
  2. In the row of the parser you want to edit, click the three vertical dots icon and then click Edit parser.
  3. In the Edit parser drawer, make changes to your parser, and then click Save.

Observability Platform saves changes to your parser.

Delete parsers

Select from the following methods to delete parsers.

  1. In the navigation menu, click Go to Admin and then select Control > Logs Configuration.
  2. In the row of the parser you want to edit, click the three vertical dots icon and then click Delete parser.

Observability Platform deletes your parser.

Examples

The following example shows how to apply both plaintext and field parsers to create structured output from an unstructured log record.

Consider this input log data. The record is a single string that contains multiple key/value pairs in a single message field. In this state, querying the data is difficult, and you can’t complete aggregations or transformations.

2025-09-22 09:15:45.101 2025/09/22 13:15:45 [info] 34567#34567: *9876543210 a new
user session was started, client: 203.0.113.55, service: app.example.net, activity:
"GET /data/dashboard/overview?appVersion=1.2.3 HTTP/1.1", host: "app.example.net",
source: "https://app.example.net/home"

By applying a plaintext parser and a field parser, you can separate the fixed fields and extract the key/value pairs into a new field that you can operate on in Logs Explorer.

First, create a plaintext parser that uses a regular expression to extract the key/value pairs. Next, add a field parser to parse out the values. The following Terraform definition includes both parsers and their definitions:

resource "chronosphere_log_ingest_config" "log_ingest_config" {
 
  plaintext_parser {
    name = "metadata"
    mode = "ENABLED"
 
    parser {
      parser_type = "REGEX"
 
      regex_parser {
        regex = "^(?P<timestamp>\\S+ \\S+)\\s+(?P<log_time>\\S+ \\S+) \\[(?P<log_level>\\w+)\\] (?P<pid>\\d+)#(?P<tid>\\d+): \\*(?P<connection_id>\\d+) (?P<message>.*?), (?P<metadata>.+)$"
      }
    }
  }
 
  field_parser {
    destination {
      selector = "metadata"
    }
 
    mode = "ENABLED"
 
    parser {
      parser_type = "KEY_VALUE"
 
      key_value_parser {
        delimiter      = ","
        pair_separator = ":"
        trim_set       = " "
      }
    }
 
    source {
      selector = "metadata"
    }
  }
}

After applying both parsers, the output data is output to a new metadata field, and is structured as shown in the following example. The new field includes key/value pairs that can be queried in Logs Explorer, and used in aggregations and transformations.

{
      "timestamp": "2025-08-22 13:36:18.648",
      "log_time": "2025/08/22 17:36:18",
      "log_level": "info",
      "pid": "34567",
      "tid": "34567",
      "connection_id": "9876543210",
      "message": "a new user session was started",
      "metadata": {
        "client": " 203.0.113.55",
        "service": " app.example.net",
        "activity": " \"GET /data/dashboard/overview?appVersion=1.2.3\"",
        "host": " \"app.example.net\"",
        "source": " \"https://app.example.net/home\""
      }
    }
BudgetsAdminister