> ## Documentation Index
> Fetch the complete documentation index at: https://docs.chronosphere.io/llms.txt
> Use this file to discover all available pages before exploring further.
# Parse your log data
Your log data can be formatted differently depending on which service generates that
data. Some data is structured, such as valid JSON. Other data might be somewhat
structured, or not have any structure at all. In these instances, you can create
parsers to create structured logs from plain text logs, and parse individual fields to
extract meaningful data without changing your instrumentation.
For example, if your instrumentation emits unstructured plain-text logs, create a
[plain text parser](#create-plain-text-parsers) to provide structure by separating the
individual fields.
If your logs contain JSON or key/value pairs buried contained in a single string field,
create a [field parser](#create-field-parsers) to extract the values. The extracted
fields can then be queried in [Logs Explorer](/investigate/querying/query-logs)
and used in aggregations and transformations.
Parsers run after ingestion, but before applying normalization rules and control
rules. If you want to additionally parse log data after initial parsing
and normalization complete,
[create a control rule](/control/shaping/shape-logs/control-logs#create-control-rules)
to parse fields.
```mermaid actions={false} theme={null}
flowchart LR
Send["Send data"]
subgraph OP["Observability Platform"]
Parse[Parse data]
Normalize[Normalize data]
Control[Control data]
Format[Format data]
Store[Store data]
end
Send --> Parse --> Normalize --> Control --> Format --> Store
click Normalize "/control/shaping/shape-logs/normalize-logs"
click Control "/control/shaping/shape-logs/control-logs"
click Format "/control/shaping/shape-logs/lookup-logs/"
classDef stages fill:#2fbf71,stroke-width:2px,stroke:#2fbf71,color:#FFFFFF;
class Parse stages
```
## View parsers
Select from the following methods to view and order available parsers. Parsers
display in match order.
To view parsers:
1. In the navigation menu, click ** Go to Admin**
and then select
** Control > Logs Ingest**.
All parsers display, including plain-text and field parsers.
2. To view only plain-text parsers or field parsers, from the **View** dropdown,
select **Plaintext parsers** or **Field parsers**.
3. To view the code definition for all configured parsers, click the **Code Config**
tab.
The **Logs parsing and mapping** section includes all defined parsers.
To use [Chronoctl](/tooling/chronoctl) to return all parsers, use the
`chronoctl log-ingest-config read` command:
```shell theme={null}
chronoctl log-ingest-config read
```
[Use the Code Config tool](/tooling/gitops#use-the-code-config-tool) in Observability
Platform to view a parser's Terraform representation.
To complete this action with the Chronosphere API, use the
[`ReadLogIngestConfig`](/tooling/api-info/definition/operations/ReadLogIngestConfig)
endpoint.
Because the Chronosphere API requires authentication, include an API token with your
`curl` request, as shown in the following example. For more details, see
[Create an API token](/tooling/api-info#create-an-api-token).
```shell /"TOKEN"/ /INSTANCE/ /METHOD/ /ENDPOINT_PATH/ theme={null}
export CHRONOSPHERE_API_TOKEN="TOKEN"
export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"
curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
-X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"
```
Replace the following:
* *`TOKEN`*: Your API token.
* *`INSTANCE`*: The subdomain name for your organization's Observability Platform instance.
* *`METHOD`*: The HTTP method to use with the request, such as `GET` or `POST`.
* *`ENDPOINT_PATH`*: The specific endpoint you want to access.
## Create parsers
Parsers operate on both [plain-text logs](#create-plain-text-parsers) and
[individual fields](#create-field-parsers). All configured parsers are included in a
single definition, and evaluated in match order. Observability Platform applies the
first parser in the definition that matches a log, and ignores any subsequent
parsers.
Although you can define parsers in the Observability Platform web interface, you must
[use the Code Config tool](/tooling/gitops#use-the-code-config-tool) to apply the definition.
### Create plain-text parsers
Plain text parsers run against all ingested logs with the `plaintext` mimetype, and
apply parsing logic to modify the original record. These parsers affect the entire
log. Use field parsers to operate on individual fields within a log.
Select either a regular expression or a key/value matching pattern to parse logs.
When using a key/value pattern, you can extract any number of key value pairs from a
plain text log. Plain text parsers also have an option to keep the original log after
applying parsing logic.
Plain text parsers apply only when the media type is set to `plaintext` for logs.
Regular expressions for the matching type use
[RE2 syntax](https://github.com/google/re2/wiki/Syntax) and named capture
groups. Each named capture group becomes the key name in the structured log.
Observability Platform requires at least one named capture group in the regular
expression.
Use one of the following methods to create plain text parsers.
Define the plain text parser definition in Observability Platform, and then
[use the Code Config tool](/tooling/gitops#use-the-code-config-tool) to apply the definition.
1. In the navigation menu, click ** Go to Admin**
and then select
** Control > Logs Ingest**.
2. In the **Plaintext parsers** section, click **Create**.
3. Enter a name for the parser.
4. Select the matching type for the parser:
* **Regex**: Use a regular expression to match logs. In the **Template** field,
select the regular expression template to apply to your logs. Use a predefined
template such as **Nginx**, or specify a **Custom** template. For example, this
regular expression is used in subsequent steps for previewing changes:
```text wrap theme={null}
^(?P[^ ]*) [^ ]* (?P[^ ]*) \[(?P
To use [Chronoctl](/tooling/chronoctl) to create plain text parsers, use the
`chronoctl log-ingest-config create` command:
```shell theme={null}
chronoctl log-ingest-config create
```
1. Run the following command to generate a sample parser configuration you can use as
a template:
```shell theme={null}
chronoctl log-ingest-config scaffold
```
In the template, `kind: LogIngestConfig` defines an individual parser configuration.
2. Use the `plaintext_parsers` object to define the plain text parser to run on your
log data.
3. With a completed definition, submit it with:
```shell /FILE_NAME/ theme={null}
chronoctl log-ingest-config create -f FILE_NAME
```
Replace *`FILE_NAME`* with the name of the YAML definition file you want to use.
When you run `terraform plan` to generate an execution plan, Chronosphere automatically
tests configurations that include notification policies by submitting them as dry runs.
For details, see the
[Terraform provider](/tooling/infrastructure/terraform#validate-plans-with-dry-runs)
documentation.
To create a plain text parser with [Terraform](/tooling/infrastructure/terraform):
1. Create a Terraform resource with the `chronosphere_log_ingest_config` resource
type, followed by a resource name.
2. Use the `plaintext_parser` object to define the plain text parser to run on your
log data.
3. Add the definition to a Terraform file.
4. Run this command to create the resource:
```shell theme={null}
terraform apply
```
To complete this action with the Chronosphere API, use the
[`CreateLogIngestConfig`](/tooling/api-info/definition/operations/CreateLogIngestConfig)
endpoint.
Use the `plaintext_parsers` object to define the plain text parser to run against
your log data.
Because the Chronosphere API requires authentication, include an API token with your
`curl` request, as shown in the following example. For more details, see
[Create an API token](/tooling/api-info#create-an-api-token).
```shell /"TOKEN"/ /INSTANCE/ /METHOD/ /ENDPOINT_PATH/ theme={null}
export CHRONOSPHERE_API_TOKEN="TOKEN"
export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"
curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
-X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"
```
Replace the following:
* *`TOKEN`*: Your API token.
* *`INSTANCE`*: The subdomain name for your organization's Observability Platform instance.
* *`METHOD`*: The HTTP method to use with the request, such as `GET` or `POST`.
* *`ENDPOINT_PATH`*: The specific endpoint you want to access.
### Create field parsers
Field parsers operate on an individual field within a log, and run on all ingested
logs, including JSON and plain text logs. Field parsers run after plain text parsers.
When defining a field parser, choose which matching pattern to apply and specify the
destination of the parsed field.
Define the field parser definition in Observability Platform, and then
[use the Code Config tool](/tooling/gitops#use-the-code-config-tool) to apply the
definition.
1. In the navigation menu, click ** Go to Admin**
and then select
** Control > Logs Ingest**.
2. In the **Field Parsers** section, click **Create**.
3. Enter a source field for the parser to operate on, such as `message`.
4. Enter a destination for the parsed data, which can be one of these locations:
* Use the same field name, which overwrites the existing field data with the
parsed data.
* Specify a new field name to contain the parsed data.
* Leave the destination field empty, which populates the parsed data to the top
level of your logs.
5. Select the matching type for the parser.
* **JSON** matches on valid JSON logs.
* **REGEX** uses a regular expression to match logs. In the **Template** field,
select the regular expression template to apply to your logs. Use a predefined
template such as **Nginx**, or specify a **Custom** template. For example, this
regular expression is used in subsequent steps for previewing changes:
```text theme={null}
user=(?P\w+) action=(?P\w+)
```
* **Key/value** uses a specified pattern to separate key/value pairs in log data.
Enter the following values:
* **Pair separator** is the character used to split matching pairs into its key
and value. For example, a colon (`:`) if your data is in the format `key:value`.
* **Delimiter** is the character used to split the input log data into key/value
pairs. For example, a comma (`,`) if your data is in the format
`key: value, key: value`.
* **Trim set** identifies the leading and trailing characters to trim from the
key/value pairs. For example, a space (`" "`) between key/value pairs if your
data is in the format `key: value, key: value`.
6. To test the parser, in the **Preview log parser** section, enter a portion of your
log payload. For example:
```json theme={null}
{"message": "user=john action=login timestamp=12345"}
```
7. Click **Test parser** to view a preview of the generated output after the parser
runs. For example, the following output is generated by the input from the
previous step:
```json theme={null}
{
"message": "user=john action=login timestamp=12345",
"parsed_message": {
"action": "login",
"user": "john"
}
}
```
8. Click **Save** to save the parser definition.
9. On the **Logs Ingest** page, click **View code config**.
10. Click the **Code config** tab and [use the Code Config tool](/tooling/gitops#use-the-code-config-tool)
to apply the definition.
To use [Chronoctl](/tooling/chronoctl) to create field parsers, use the
`chronoctl log-ingest-config create` command:
```shell theme={null}
chronoctl log-ingest-config create
```
1. Run the following command to generate a sample parser configuration you can use as
a template:
```shell theme={null}
chronoctl log-ingest-config scaffold
```
In the template, `kind: LogIngestConfig` defines an individual parser configuration.
2. Use the `field_parsers` object to define the plain text parser to run on your
log data.
3. With a completed definition, submit it with:
```shell /FILE_NAME/ theme={null}
chronoctl log-ingest-config create -f FILE_NAME
```
Replace *`FILE_NAME`* with the name of the YAML definition file you want to use.
When you run `terraform plan` to generate an execution plan, Chronosphere automatically
tests configurations that include notification policies by submitting them as dry runs.
For details, see the
[Terraform provider](/tooling/infrastructure/terraform#validate-plans-with-dry-runs)
documentation.
To create a plain text parser with [Terraform](/tooling/infrastructure/terraform):
1. Create a Terraform resource with the `chronosphere_log_ingest_config` resource
type, followed by a resource name.
2. Use the `field_parser` object to define the field parser to run on your log data.
3. Add the definition to a Terraform file.
4. Run this command to create the resource:
```shell theme={null}
terraform apply
```
To complete this action with the Chronosphere API, use the
[`CreateLogIngestConfig`](/tooling/api-info/definition/operations/CreateLogIngestConfig)
endpoint.
Use the `field_parsers` object to define the field parser to run against your log data.
Because the Chronosphere API requires authentication, include an API token with your
`curl` request, as shown in the following example. For more details, see
[Create an API token](/tooling/api-info#create-an-api-token).
```shell /"TOKEN"/ /INSTANCE/ /METHOD/ /ENDPOINT_PATH/ theme={null}
export CHRONOSPHERE_API_TOKEN="TOKEN"
export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"
curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
-X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"
```
Replace the following:
* *`TOKEN`*: Your API token.
* *`INSTANCE`*: The subdomain name for your organization's Observability Platform instance.
* *`METHOD`*: The HTTP method to use with the request, such as `GET` or `POST`.
* *`ENDPOINT_PATH`*: The specific endpoint you want to access.
## Edit parsers
When creating or editing a parser, you can
[use the Code Config tool](/tooling/gitops#use-the-code-config-tool)
to view code representations of a parser. The displayed code also responds to changes
you make in the **Visual editor** tab.
Select from the following methods to edit parsers.
1. In the navigation menu, click ** Go to Admin**
and then select
** Control > Logs Ingest**.
2. In the row of the parser you want to edit, click the
three vertical dots icon and then click **Edit parser**.
3. In the **Edit parser** drawer, make changes to your parser, and then click
**Save**.
Observability Platform saves changes to your parser.
4. Click the **Code config** tab and
[use the Code Config tool](/tooling/gitops#use-the-code-config-tool) to apply your
changes.
To update parsers with [Chronoctl](/tooling/chronoctl), use the
`log-ingest-config update` command:
```shell theme={null}
chronoctl log-ingest-config update
```
1. Update the parser definition file.
2. Run the following command to submit the changes:
```shell /FILE_NAME/ theme={null}
chronoctl log-ingest-config update -f FILE_NAME.yaml
```
Replace *`FILE_NAME`* with the name of the YAML definition file you want to use.
Users can modify Terraform-managed resources only by using Terraform.
[Learn more](/tooling/infrastructure/terraform#prevent-changes-to-managed-resources).
To edit parsers using [Terraform](/tooling/infrastructure/terraform):
1. Create or edit a Terraform file that updates the resource's existing properties.
2. Run this command to apply the changes:
```shell theme={null}
terraform apply
```
You can also [use the Code Config tool](/tooling/gitops#use-the-code-config-tool) to
view the parser's Terraform representation.
To complete this action with the Chronosphere API, use the
[`UpdateLogIngestConfig`](/tooling/api-info/definition/operations/UpdateLogIngestConfig)
endpoint.
Because the Chronosphere API requires authentication, include an API token with your
`curl` request, as shown in the following example. For more details, see
[Create an API token](/tooling/api-info#create-an-api-token).
```shell /"TOKEN"/ /INSTANCE/ /METHOD/ /ENDPOINT_PATH/ theme={null}
export CHRONOSPHERE_API_TOKEN="TOKEN"
export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"
curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
-X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"
```
Replace the following:
* *`TOKEN`*: Your API token.
* *`INSTANCE`*: The subdomain name for your organization's Observability Platform instance.
* *`METHOD`*: The HTTP method to use with the request, such as `GET` or `POST`.
* *`ENDPOINT_PATH`*: The specific endpoint you want to access.
## Delete parsers
Select from the following methods to delete parsers.
1. In the navigation menu, click ** Go to Admin**
and then select
** Control > Logs Ingest**.
2. In the row of the parser you want to edit, click the
three vertical dots icon and then click **Delete parser**.
3. Click the **Code config** tab and
[use the Code Config tool](/tooling/gitops#use-the-code-config-tool) to apply your
changes.
Observability Platform deletes your parser.
To delete a parser with [Chronoctl](/tooling/chronoctl), use the
`chronoctl log-ingest-config delete` command:
```shell theme={null}
chronoctl log-ingest-config delete
```
Users can modify Terraform-managed resources only by using Terraform.
[Learn more](/tooling/infrastructure/terraform#prevent-changes-to-managed-resources).
To delete a resource that's managed by [Terraform](/tooling/infrastructure/terraform):
1. Edit your Terraform configuration file to remove the pre-existing resource
definition.
2. Run this command to remove the resource from Observability Platform:
```shell theme={null}
terraform apply
```
To complete this action with the Chronosphere API, use the
[`DeleteLogIngestConfig`](/tooling/api-info/definition/operations/DeleteLogIngestConfig)
endpoint.
Because the Chronosphere API requires authentication, include an API token with your
`curl` request, as shown in the following example. For more details, see
[Create an API token](/tooling/api-info#create-an-api-token).
```shell /"TOKEN"/ /INSTANCE/ /METHOD/ /ENDPOINT_PATH/ theme={null}
export CHRONOSPHERE_API_TOKEN="TOKEN"
export CHRONOSPHERE_DOMAIN="INSTANCE.chronosphere.io"
curl -H "API-Token: ${CHRONOSPHERE_API_TOKEN}" \
-X METHOD "https://${CHRONOSPHERE_DOMAIN}/ENDPOINT_PATH"
```
Replace the following:
* *`TOKEN`*: Your API token.
* *`INSTANCE`*: The subdomain name for your organization's Observability Platform instance.
* *`METHOD`*: The HTTP method to use with the request, such as `GET` or `POST`.
* *`ENDPOINT_PATH`*: The specific endpoint you want to access.
## Examples
The following example shows how to apply both plain text and field parsers to create
structured output from an unstructured log record.
Consider the following input log data. The record is a single string that contains multiple
key/value pairs in a single `message` field. In this state, querying the data is
difficult, and you can't complete aggregations or transformations.
```text theme={null}
2025-09-22 09:15:45.101 2025/09/22 13:15:45 [info] 34567#34567: *9876543210 a new
user session was started, client: 203.0.113.55, service: app.example.net, activity:
"GET /data/dashboard/overview?appVersion=1.2.3 HTTP/1.1", host: "app.example.net",
source: "https://app.example.net/home"
```
By applying a plain text parser and a field parser, you can separate the fixed fields
and extract the key/value pairs into a new field that you can operate on in Logs
Explorer.
First, create a [plain text parser](#create-plain-text-parsers) that uses a regular
expression to extract the key/value pairs. Next, add a [field parser](#create-field-parsers)
to parse out the values. The following Terraform definition includes both parsers and
their definitions:
```terraform wrap theme={null}
resource "chronosphere_log_ingest_config" "log_ingest_config" {
plain text_parser {
name = "metadata"
mode = "ENABLED"
parser {
parser_type = "REGEX"
regex_parser {
regex = "^(?P\\S+ \\S+)\\s+(?P\\S+ \\S+) \\[(?P\\w+)\\] (?P\\d+)#(?P\\d+): \\*(?P\\d+) (?P.*?), (?P.+)$"
}
}
}
field_parser {
destination {
selector = "metadata"
}
mode = "ENABLED"
parser {
parser_type = "KEY_VALUE"
key_value_parser {
delimiter = ","
pair_separator = ":"
trim_set = " "
}
}
source {
selector = "metadata"
}
}
}
```
After applying both parsers, the generated data is output to a new `metadata` field, and
is structured as shown in the following example. The new field includes key/value
pairs that can be queried in [Logs Explorer](/investigate/querying/query-logs) and
used in aggregations and transformations.
```json theme={null}
{
"timestamp": "2025-08-22 13:36:18.648",
"log_time": "2025/08/22 17:36:18",
"log_level": "info",
"pid": "34567",
"tid": "34567",
"connection_id": "9876543210",
"message": "a new user session was started",
"metadata": {
"client": " 203.0.113.55",
"service": " app.example.net",
"activity": " \"GET /data/dashboard/overview?appVersion=1.2.3\"",
"host": " \"app.example.net\"",
"source": " \"https://app.example.net/home\""
}
}
```