This feature is in Early Access (EA), and might not be visible in your app. To learn
more about this program and the features it contains, see the
Early access page.
Chronosphere Observability Platform generates detailed visualizations of each triggered
alert, and provides tools to help you analyze when, why, and how the alert was triggered.You can access alert detail pages wherever a triggered alert is referenced, including:
From the list of alerts, which you can access
by selecting Alerting > Alerts
in the navigation menu.
Each alert presents its alert details in a section that includes the alert’s
status, a visualization of the queries that triggered the alert and any change events,
and a table of the series and conditions that triggered the alert and serve as a
legend for the chart.The page designates the alert’s status with an icon, text, and color:
Icon
Text
Description
Critical
Actively triggered alert that exceeds the defined critical conditions.
Warning
Actively triggered alert that exceeds the defined warning conditions.
Muted
Alert that’s muted by an active muting rule.
Passing
An alert that’s no longer triggered.
The page also lists any triggering Signal as chips
that identify its keys and their values.The query results that triggered the alert are visualized as a
time series chart. You can optionally:
Show thresholds, which draws dotted horizontal lines on the time series chart
depicting the alert’s triggering thresholds
Show query, which displays the query that triggered the alert and populates
the time series chart
Open in explorer, which opens Metrics Explorer
and populates it with the triggering query
In the time series table that follows, you can select from visualized series to
highlight them in the visualization, and you can search series to filter the
list. You can also toggle between the default table view,
which lists the time series’ status as an icon and its labels and their values,
or a list view that lists only each series’
status and the query that produces it.You can optionally reveal the Conditions that triggered the alert. Observability Platform
lists these conditions as a table of each condition’s status, operator, sustained duration,
time to resolution, and the signals to which the alert’s conditions apply.
If the alert’s source is an SLO, the alert details page also reproduces the SLO’s
SLI breakdown and Burn/Error rates sections for reference. For more information,
see Service level objectives.
When an alert is resolved, document its resolution in the alert’s Resolution notes
section. These notes can provide context to identify recurring issues and capture
actions for incident reviews.
The alert details page’s Resolution notes section lists any resolution notes
associated with an alert.In addition to the note’s contents, each resolution note includes:
The user ID of the note’s author.
The date and time that the note was added.
An (edited) indicator, if the note was edited.
The note’s associated signals.
The date and time of the alert’s most recent resolution at the time the note was
added.
You can add multiple resolution notes to an alert. To add a resolution note from
an alert details page:
Click + Add to open the Create resolution note panel.
Enter the note in the Resolution note field. The field accepts Markdown formatting,
with shortcut buttons for Bold, Italics, code, and links.To preview formatting, enable the Show preview toggle.
Click Create to create the note, or Cancel to return to the alert details
page.
By default, the alert details page filters resolution notes by the actively firing
signal.To display notes for all related alerts, turn off the Show current signal only
toggle. This might result in duplicated notes if the same note was added to multiple
related alerts.
To mute an alert, create a muting rule. You
can do this directly from an alert details page by clicking the Mute alert button.
This opens a panel to create a muting rule
that’s populated with the alert’s relevant source and name, and without requiring
you to leave the alert details page.If a muting rule is already muting a triggered alert, clicking this button instead
opens a panel to edit the associated muting rule.Alternatively, you can select
Alerting > Muting Rules
in the navigation menu to create new muting rules or edit existing muting rules.
However, doing so doesn’t populate the muting rule with details from a specific
alert’s source.
Each alert detail page includes a link to return to the source of its trigger.
For instance, an alert triggered by a monitor has a Return to source monitor link,
and an alert triggered by an SLO has a Return to source SLO link.If you navigate to an alert’s source, you can return to the alert either by clicking
the triggered alert in the source entity. You can also navigate to the
list of alerts and filter it by the alert’s title.
You can customize the chronological scope of an alert’s details by selecting a
time range, which defaults to the Last 1 hour. The
chart and time series table update to depict time series data only within the selected
time range, but the alert’s status always displays its current state.
Selecting a time span that lacks data related to the alert can cause the alert’s
tables or visualizations to report No data. Confirm that the time range selector
is set to a range relevant to the alerting event.
You can also use differential diagnosis (DDx)
from an alert’s detail page. This helps you identify correlations within specific
alerts while investigating the root causes of their triggers.
If you’ve enabled change events, the alert details page
displays associated events in the time series chart and lists them in the Change events
section.To toggle or filter change events in the chart, click Events
to open the Display events panel. Toggle their Status, Events list,
and Categories to your preferences, and then click Save to confirm your
changes.The change events list includes each Alert change event
related to the displayed alert. The section’s table lists events that occurred during
the selected time span in chronological order, starting with the most recent event.To view a change event’s details, click the event’s row in the list to open its
Change event panel. This displays the event’s title, category, source, type,
time of occurrence, and label names and values. It also links to the alert’s source
and provides tabs to Comments and the change event entity’s JSON depiction.To further explore all of the listed change events, click Open explorer in the
alert details page’s Change events section to open
Changes Explorer and populate it with the same
time range and query used to populate the alert details page’s list of change events.
The Alert information sidebar identifies the collection
identified as the alert’s Owner and its associated Team.
If the alert’s source is an SLO, this section also lists the SLO’s Runbook link
if one is defined.When designing collections and teams, add contextual links, details, and default
notification policies to ensure that responders can quickly identify and notify
responsible colleagues or follow established policies and processes when an alert
is triggered.The Additional information section of the sidebar lists any Annotations
defined on the alert’s triggering source. For example, if you defined
annotations on a monitor, the alert’s
additional information lists each of the monitor’s annotations and populates any
variables defined in them.When designing something that can trigger an alert, such as a monitor or SLO, use
annotations to conditionally contextualize these alert views with information pertinent
to responders.
An additional sidebar section identifies any other alerting signals triggered by
the same source. For example, if the source is a monitor, the section is titled
Monitor information.