Ingest log data

Before you can explore and query your log data, you need to parse and ingest that data correctly before it enters Chronosphere Observability Platform.

Parsing log data provides structure to your data and removes information you don't need. For example, parsing can:

• Add a schema or format to your logs.
• Remove sensitive information, such as credit card numbers.
• Exclude noisy logs or fields that aren't useful.
• Add context, such as hostname, GeoIP address, or Kubernetes metadata.

Chronosphere recommends using Core Agent or a combination of Core Agent plus Telemetry Pipeline, but also supports using your existing ingest pipeline, such as Fluent Bit, LogStash, or the OpenTelemetry Collector. You have the flexibility of sending log data from your existing ingest pipeline directly to Observability Platform, or sending that data to Chronosphere Telemetry Pipeline for parsing and then to Observability Platform.

Whichever ingestion method you choose, Chronosphere recommends putting your primary parsing logic on the client side. This configuration means you can use your existing parsing logic or parse data in Telemetry Pipeline without needing to also manage parsing logic in LogScale. LogScale still requires you to have a parser to use Logs, but you can use one of the built-in parsers or make minor modifications.

Choose one of the following options to ingest your log data: