OBSERVABILITY PLATFORM
Investigate
Logs
Query logs

Query log data

After you ingest and parse your log data, you can make queries against the fields available in logging events. Use the LogScale Query Language (opens in a new tab) within Logs to define search queries for retrieving, processing, and analyzing data.

LogScale stores data as events, which are discrete pieces of data with an associated timestamp. Each search query consists of a chain of data processing commands, as shown in the following diagram. Each expression passes its result to the next expression in the sequence, which narrows the events that the query returns. You can create complex queries by combining expressions.

LogScale queries are similar to SQL queries. You build a search to include or exclude values from a data source. LogScale queries have the added ability to perform calculations and transform data as part of the query. The two most common queries in LogScale are transformation and aggregation queries. See Transformation and aggregation queries (opens in a new tab) in the LogScale documentation for more information.

When writing a query, you can search either the raw event data or the fields extracted from parsing. Events can have these different fields types:

  • Tag fields have a hashtag (#) prefix and define how to physically store and index events. For example, #repo and #type are common tag fields.
  • Metadata fields have an at (@) prefix to indicate event metadata extracted from ingestion. All events have default fields, such as @id or @timestamp.
  • User fields are any fields that aren't tag fields or metadata fields.

After signing in to Chronosphere Observability Platform, in the navigation menu, select Explorers > Logs Explorer to start querying log data.

Query best practices

The more specific your query is, the fewer results you need to search. Writing efficient queries reduces CPU cost and memory usage, and returns results faster. When writing LogScale queries, follow these best practices:

  • Use tag filters to reduce the dataset and limit results to a smaller information pool. Tags tell LogScale where to search for data, and using more specific tags can create more performant queries.
  • Add transformation expressions after aggregation expressions whenever possible.
  • Always aggregate data.

When applied, these best practices map to a query workflow resembling the following steps:

  1. Configure one or more tag filters to narrow search results to only certain events. For example, #repo or #type might be tags you can use to narrow your search to specific events you're interested in.
  2. Use filters such as an asterisk (*), double quotes (""), and the does not equal operator (!=) to narrow your search to a specific data type.
  3. Add more filters to include or exclude specific fields. You can use transformation functions, such as format(), eval(), or any function with an as parameter.
  4. Aggregate the filtered results using the aggregate functions, such as sum(), top(), or groupBy().
  5. Post process the aggregate result, using functions such as table() or format(), among others.

Case sensitivity

Fields and values in your query are case sensitive. For example, if you want to match on STATUS_CODE_UNSET, the following query doesn't return any results:

scopeSpan.span.status.code = STATUS_CODE_unset

To write a query that isn't case sensitive, enclose your query term with forward slashes, followed by the letter i. The following query returns results for both STATUS_CODE_unset and STATUS_CODE_UNSET:

scopeSpan.span.status.code = /STATUS_CODE_unset/i

Combine filters and expressions

You use the and and or logical operators to combine filters and add or exclude conditions in your search. If you don't specify an operator, the and operator is implied. The following query returns results with GET or POST in the method field:

method="GET" or
method="POST"

You can also combine query expressions by using the pipe operator (|), which takes the expression output from the left side and uses it as input on the right side of the expression. For example, the following query combines several expressions:

status >=400
| method != PATCH
| top(method, limit=3)

Looking at each expression, this combined query:

  1. Searches for status codes greater than or equal to 400.
  2. Selects all events with an HTTP method, such as GET or POST, but doesn't include PATCH.
  3. Selects the top three HTTP methods with the highest number of events matching the previous conditions.
  4. Returns the total count for each method.

Write a query

To write a query in Logs:

  1. In the Data Source dropdown, select the data source you want to search.
  2. Enter one or more search terms in the Search box and click Run.
  3. If needed, adjust the size of the box by dragging manually or clicking the arrows on the right to make it fit the query.

See the query examples section for common query examples and links to the LogScale documentation for additional examples.

After writing a query, you can create a link from trace span details to your log data. These links can use templated variables that make the links context-sensitive to the selected span details.

Add a query to a dashboard

After writing a query, you can add the query to a panel in an existing Observability Platform standard dashboard. You can also create a new dashboard and add the query to a panel in that dashboard.

To add a query to a dashboard:

  1. In the navigation menu, select Explorers > Logs Explorer.

  2. Write a query that returns the log data you want to view.

  3. Click Add to dashboard.

  4. In the Add to dashboard dialog, enter a name for your query. The query you defined is input in the Name field by default.

  5. Choose one of the following options to add the panel to a dashboard:

    • From the dropdown, select a dashboard where you want to add the panel.
    • Click create a new dashboard to create a new dashboard containing a panel with the query you defined.

  6. Click Add to dashboard to add the panel to your dashboard.

Observability Platform adds the panel to the selected dashboard. You can change the panel's type if you want the query to display as a different visualization.

Share a URL to a query

When investigating issues, you might want to share a defined query with other users, or include a URL to a defined query in monitor annotations, runbooks, or other on-call tools. Logs includes the ability to copy a short URL to a defined query using either a relative or absolute time range.

Relative time is useful for understanding the results of a query in a past period of time relative to the current time, such as in the past 30 minutes. Absolute time is better suited for comparing results across a fixed point in time, such as the results of a query from last Monday at 8:00 AM versus that same query run today.

To copy a URL to a defined query:

  1. In the navigation menu select Explorers > Logs Explorer.

  2. Write a query that returns the log data you want to view.

  3. Click Copy URL and choose type of time range to use for the URL:

    • Copy with absolute time range: Create a link that runs a query against the date and time interval from when you copied the link. For example, if your query uses 1h as the time interval, the time range is exactly one hour ago, based on the date and time you ran the query.

    • Copy with relative time range: Create a link that runs a query against the current time. For example, if your query uses 1h as the time interval, the time range is exactly one hour ago from the current time.

The URL is copied to your clipboard based on your selection.

Access recent and saved queries

When investigating issues, you might use the same query frequently. Rather than redefining your query, use recent and saved queries to access previously defined queries.

To apply a fully defined query from a previous time period:

  1. In the navigation menu select Explorers > Logs Explorer.
  2. Click the Queries dropdown to display all available queries.
  3. Click the Recent tab or the Saved tab to display the queries you want to view.
  4. Locate the query you want to apply and click it.

The parameters in the query override any parameters in the Search box.

Save a query

To save a query in Logs:

  1. In the Data Source dropdown, select the data source you want to search.
  2. Enter one or more search terms in the Search box and click Run.
  3. In the Results panel, click Save > Saved search.
  4. In the Save query window, enter a name for your query and click Save.

Your query displays in the Saved tab of the Queries dropdown.

Query examples

In Logs, you can search for strings, such as environment=production, and filter based on values of specific fields, such as method="GET". Beyond these basic queries, you can combine query expressions and add functions to modify data or create new fields.

The most common query types in LogScale are filter queries, transformation queries, and aggregation queries. You must create a filter query to create a transformation or aggregation query, but you can create transformation and aggregation queries without a filter query.

LogScale provides a guide that includes common query functions (opens in a new tab) with associated use cases that cover both transformation and aggregation queries.

For examples of common queries, see frequently used query operations (opens in a new tab) in the LogScale documentation.

Filter queries

Filtering scopes your query to events with specific fields or values without modifying the data, such as returning all events where method="GET". Filter queries can use free text, field matches, and regular expressions.

The simplest filter query is searching for free text, such as error. You can add field filters (opens in a new tab) to a free text query to match on specific event fields, such as error!=200. To match fields using regular expressions, enclose the expression in two forward slashes. For example, url=/login/ matches any fields where the url field contains login.

Transformation queries

Transformation queries include functions that modify input records by adding, removing, or updating fields in the results set. Unlike aggregation queries, transformation queries don't produce additional events as output.

For example, the following query uses the time:dayOfWeekName() function to extract the day of the week from the timestamp of each event, format that value to the name of the day, and enter the value in a field named the_day:

time:dayOfWeekName(field=@timestamp, as=the_day)

Similarly, you can use the eval() function to modify fields or dynamically create new fields. For example, the following query returns the responsesize field in kilobytes instead of bytes:

eval(sizeInKb=responsesize / 1000)

See transforming or modifying data (opens in a new tab) in the LogScale documentation for a list of common transformation functions and examples.

The CrowdStrike® Falcon LogScale™ GitHub repository (opens in a new tab) contains a list of predefined transformation queries you can copy for converting data in various use cases.

Aggregation queries

Aggregation queries include function calls that can combine their input into a new structure, or emit new events into the output stream.

An aggregation query uses at least one aggregate function, such as sum(), groupBy(), count(), or avg().

You might want calculate the total revenue of all products sold on your website. The following query includes some aggregation expressions you can use to create new output data:

url=/^\/checkout_complete\/(?<product_id>\d+)/
| match(file="products.csv", column=product_id, field=product_id)
| sum(product_price, as="Total revenue")

Looking at each expression, this combined query:

  1. Finds anything after /checkout_complete in the URL and transform the data into a product_id that consists of one or more numbers.
  2. Uses the product_id to find the product and associated product_name and product_price in the products.csv file.
  3. Sums all product_price values and return the result in a field named Total revenue.

In a different example, customers are experiencing slow checkout times, and some can't submit their order. As part of your investigation, you can build a query to inspect HTTP status codes by region and timezone:

resource.attributes.hostname=~/^mysql-order-uswest1-*/
| statuscode >= 400
| groupBy([resource.attributes.hostname])
| sort(@timezone)

Looking at each expression, this combined query:

  1. Matches any log events with mysql-order-uswest1 in the resource.attributes.hostname field.
  2. Filters status codes to only error codes greater than or equal to 400.
  3. Groups the results by the resource.attributes.hostname field.
  4. Sorts the results by the @timezone field.

See aggregating data (opens in a new tab) in the LogScale documentation for a list of common aggregation functions and examples.

Configure LogScaleConfigure alerting