Create links to related information

When viewing span details in Trace Explorer for a selected trace, you can add links to related information, such as dashboards within Chronosphere Observability Platform, related logs in Logs Explorer, external services such as related tracing logs stored in your cloud provider, or links to other observability tools.

You can dynamically generate links to external services using templated variables, such as {{ trace_id }}, {{ service }}, and {{ operation }}. When you click one of these links that contains a variable, Observability Platform interpolates the variables with information from the selected span. For example, Observability Platform replaces the {{ service }} variable with the name of the service from the selected span.

These links persist across all traces in Observability Platform.

  1. In Trace Explorer, define the criteria for your search.

  2. To view a specific trace related to your search, click the Traces tab and then select an individual trace to open the trace details page.

  3. On the trace details page, in the Links section, click + Add link.

  4. Enter a display name for the link, and then define a URL for your link.

    For example, the following link opens the Services Overview dashboard scoped to the service in the selected span. When you click the link, Observability Platform replaces the {{ service }} variable with the name of the service from the selected span.

    https://example.chronosphere.io/dashboards/services-overview/services-overview?orgId=1&var-svc={{ service }}&var-root_svc={{ service }}
  5. Click Save to save your link.

Link to data in Logs Explorer

If you want to link from trace span details to related logs in Logs Explorer, copy the query, decode it, and replace values with variables in the encoded URL. The following URL is an example of the decoded base URL:

https://ADDRESS.chronosphere.io/logs-explorer?repo=REPOSITORY&query={"queryString":"QUERY","live":false,"parameterArgs":{},"start":"TIME"}

Replace the following:

  • ADDRESS: your company name prefixed to your Observability Platform instance. For example, MY_COMPANY.chronosphere.io.
  • REPOSITORY: the name of the repository you want to query, such as default.
  • QUERY: the Logs Explorer query you want to run when someone clicks the link. See logging query examples for examples.
  • TIME: the time interval you want the query to use, such as 1h for one hour or 1d for one day.

Queries in Logs Explorer separate query parameters with a pipe (|), as shown in the following example:

service = "payment_gateway_service"
| operation = "/auth.Auth/AuthCheck"
| severity = ERROR
| traceID = 9CBAF7F3B77ED87A8F0D2A477F54C1C6

The URL in Logs Explorer uses URL-encoded characters that can be difficult to parse visually. You can use the base URL to create your query, or define your query in Logs Explorer and then decode the URL with an online tool such as URL Decoder (opens in a new tab). The following steps follow the URL Decoder-based approach.

To create the URL for your query:

  1. In the navigation menu, select Explorers > Logs Explorer.

  2. In Logs Explorer, write a query to define the data you're searching for.

  3. In your browser address bar, copy the resulting URL from the Logs Explorer page.

  4. In a new tab, navigate to URL Decoder (opens in a new tab).

  5. Paste the URL you copied and click Decode.

    Review the decoded URL and identify the values you want to replace with variables. For example, the decoded URL might contain the name of a service like payment_gateway_service:

    service+=+\"payment_gateway_service\"
  6. In the encoded URL you pasted into URL Decoder, replace any values where you want to use variables. For example, the encoded URL might contain the service parameter with a value of payment_gateway_service:

    service+%3D+%5C%22payment_gateway_service%5C%22

    Replace payment_gateway_service with the {{ service }} variable:

    service+%3D+%5C%22{{ service }}%5C%22

    See logging query examples for examples.

  7. After replacing values with variables, copy the encoded URL, complete with all URL-encoded characters.

  8. Create a link in the span details.

Logging query examples

The following examples show both the encoded and decoded URLs for a query in Logs Explorer to help explain the syntax and how to replace parameter values with variables.

Each of the following examples uses carriage returns to improve readability on the page. If you copy any examples, remove these characters before including them in a span link so the URL is valid.

The following URL defines a query in the default repository. This query includes the following parameters:

  • A service named payment_gateway_service.
  • An operation named /auth.Auth/AuthCheck.
  • An environment named production.
  • A severity equal to ERROR.
https://ADDRESS.chronosphere.io/logs-explorer?start=1h&repo=defaul&query=%7B%22queryString%22
%3A%22service+%3D+%5C%22payment_gateway_service%5C%22%5Cn%7Coperation+%3D+%5C%22%2F
auth.Auth%2FAuthCheck%5C%22%5Cn%7Cenvironment+%3D+%5C%22production%5C%22%5Cn%7C+severity+%3D+%5C
%22ERROR%5C%22%2C%22live%22%3Afalse%2C%22parameterArgs%22%3A%7B%7D%2C%22languageVersion%22
%3A%22xdr1%22%2C%22start%22%3A%221d%22%7D

Without URL encoding, the URL looks like this:

https://ADDRESS.chronosphere.io/logs-explorer?repo=default&query={"queryString":"service+=
+\"payment_gateway_service\"\n|+operation+=+"/auth.Auth/AuthCheck"\n|+environment+=+production\n
|+severity+=+ERROR","live":false,"parameterArgs":{},"start":"1d"}

Viewing this version of the URL helps to determine where the values are you want to replace, such as payment_gateway_service or /auth.Auth/AuthCheck.

You can then replace these values for service and operation with the {{ service }} and {{ operation }} templated variables. The resulting URL with variables should be similar to the following:

https://ADDRESS.chronosphere.io/logs-explorer?start=1h&repo=default&query=%7B%22queryString%22
%3A%22service+%3D+%5C%22{{ service }} %5C%22%5Cn%7Coperation+%3D+%5C%22%2F{{ operation }}%5C%22
%5Cn%7Cenvironment+%3D+%5C%22production%5C%22%22%2C%22live%22%3Afalse%2C%22parameterArgs%22%3A
%7B%7D%2C%22languageVersion%22%3A%22xdr1%22%2C%22start%22%3A%221d%22%7D

Copy the resulting encoded URL and then create a link in the span details. When users click the link, Observability Platform replaces the templated {{ service }} and {{ operation }} variables with the service and operation of the selected span.

Similarly, you can use the {{ span_id }} and {{ trace_id }} variables in your templated link.

https://ADDRESS.chronosphere.io/logs-explorer?repo=default&query={"queryString":"service+=+
\"payment_gateway_service\"\n|+traceID+=+9CBAF7F3B77ED87A8F0D2A477F54C1C6\n+|spanID+=+05BDC8EFAB57B1EA
\n|+environment+=+production\n|+severity+=+ERROR","live":false,"parameterArgs":{},"start":"1d"}

Replace the values for traceID and spanID and with the templated variables. The resulting URL with variables looks like this:

https://ADDRESS.io/logs-explorer?start=1h&repo=default&query=%7B%22queryString%22%3A%22
service+%3D+%5C%22{{ service }}%5C%22%5Cn%7C+traceID+%3D+{{ trace_id }}%5Cn%7C+spanID+%3D
+{{ span_id }}%22%2C%22live%22%3Afalse%2C%22parameterArgs%22%3A%7B%7D%2C%22languageVersion
%22%3A%22xdr1%22%2C%22start%22%3A%221d%22%7D

Link to tracing data from a dashboard

You can create data links from a Grafana dashboard to relevant traces. These data links use Grafana variables (opens in a new tab) and Observability Platform parameters to refer to series fields, labels, and values. You define a URL in a data link to express a search in Trace Explorer. Observability Platform takes the contextual information in dashboard metrics and uses it to build links to search for traces. Clicking the link navigates you to Trace Explorer and replaces the variables with matching criteria defined in the search.

Data links operate on both native metrics and metrics derived from traces within Observability Platform.

The Trace Metrics dashboard Observability Platform provides by default contains a data link for each of the included panels. Use this URL as the basis for creating a data link:

${chrono_domain}/traces/?d_closeto=${__value.time}&d_metricname=${metric}

To create a data link:

  1. In any dashboard panel, hold the pointer over the panel, click the dropdown arrow, and click Edit.

  2. In the Edit Panel screen, click the Field tab.

  3. Under Data links, click Add link.

  4. In the Edit link window, define the Title and URL for your data link, which can link to traces and other external data sources that use the context in the graph as inputs.

    For example, the following URL uses the value of the to_svc and to_op labels as inputs, based on the specific series in the chart (if the chart shows multiple series). This example also includes the value-specific Grafana variables value.time, which is the point in time on the graph, and value.numeric, which is the value of the metric at that point.

    ${chrono_domain}/traces/explorer?d_closeto=${__value.time}&d_minduration=${__value.numeric}&d_service=${__field.labels.to_svc}&d_operation=${__field.labels.to_op}
  5. Click Save to save your changes.

  6. On the main dashboard screen, click Save to apply and save your changes.

  7. In the Save dashboard window, add a note about your changes and then click Save.

After saving your data link, you can click the time series in any chart, and then click Query Traces to view the contextual link to traces.

Data link parameters

Use any of the following parameters when creating a data link from a dashboard to tracing data. You can specify parameters in conjunction with Grafana data link variables (opens in a new tab) to refer to series fields, labels, and values.

Any parameters in the Trace Explorer search that you define when creating the trace metric override parameters you specify in the data link URL.

  • d_closeto: Identifies a value ten minutes before and after the specified point you select in the graph. For example, d_closeto=${__value.time} identifies a time window that includes ten minutes before and after the time you select in the graph.

  • d_error: Specifies whether to include the error count in the data link. For example, d_error=true.

  • d_metricname: The trace metric name to pull from the dashboard panel. For example, d_metricname=${metric}.

  • d_minduration: The minimum duration for the selected metric. Use this parameter for charts that measure duration or latency, such as p99 or p50 duration. For example, &d_minduration=${__value.numeric}.

  • d_operation: The operation to include in the data link. For example, &d_operation=${__field.labels.to_op} identifies the value of the to_op label for the selected time series on the dashboard panel.

  • d_service: The service to include in the data link. For example, d_service=${__field.labels.to_svc}.

  • d_tagname: The name of a tag. For example, d_tagname=environment.

  • d_tagvalue: The value of a tag. For example, d_tagvalue=production.