Skip to main content
When a monitor generates an alert in Chronosphere Observability Platform, it triggers notifiers based on the assigned notification policy. Notifiers are the endpoints receiving notifications, including where to deliver alerts and who to notify. You can configure notifiers to send details of the alert to a channel such as email, PagerDuty, or Slack. Observability Platform also supports unfurling of monitor graphs in Slack. This integration enables Slack users to view or share monitors in Slack channels aimed at mitigating alerts. Notification policies are rules that determine how to route notifications to notifiers based on signals that trigger from monitors. Teams own notification policies, and assign them either to a specific monitor or set as a default policy for all monitors in a specified collection. For programmatic configuration, the Chronosphere API can route alerts with inline destinations that reference notifier connections, the default approach for new credentials and routes. The same API still accepts notifier_slugs that reference separate notifiers until you finish moving those routes. You can’t set both on the same notifier list. While you migrate, follow the notifier resource migration process. Optionally, you can use signals to create groups of notifications. Signals determine how to group alerts, which affects how many notifications Observability Platform sends. To customize the title and description of notifications generated by a monitor, see Notification templates.

​
Identify failed notification delivery

Observability Platform reports repeated delivery failures so that you can identify alerts that aren’t reaching their destinations. A delivery failure can appear in the following locations:
  • A Delivery failing badge marks an affected entry in the Notifiers list or the Notifier connections tab. Hold the pointer over the badge to view the cause.
  • The details view for an affected notifier or notifier connection displays one or more banners with the cause, the number of affected monitors, and how long the failures have occurred.
  • An affected monitor’s detail page displays banners for failing notifiers or notifier connections on routes that match the monitor.
  • An SLO detail page displays banners for failing notification targets when burn rate alerting is enabled. Because Observability Platform evaluates the notification policy at the policy level for an SLO, these banners can include targets from overrides that don’t match the SLO.
Each banner describes the failure and how Observability Platform handles it:
  • For a configuration problem, review the diagnostic message, then correct invalid credentials or settings. If the banner identifies a notifier that sends through a notifier connection, inspect both configurations.
  • If the destination rejects notifications, Observability Platform retries delivery until the destination recovers.
  • If the destination rate limits notifications, route fewer notifications through the affected notifier or connection, or request a higher rate limit from the receiving service.
  • For an internal delivery problem, Observability Platform retries delivery. Contact Chronosphere Support if the failure continues.
After correcting a configuration problem, send a test notification through the same route. Delivery failure indicators reflect recent failures and can remain visible while Observability Platform refreshes their status. To view a history of alerts for a monitor:
  1. In the navigation menu, select Alerting > Monitors.
  2. Select a monitor.
  3. Click Alert history in the actions menu.