Customize monitor notification messages

This feature isn’t available to all Chronosphere Observability Platform users and might not be visible in your app. For information about enabling this feature in your environment, contact Chronosphere Support.

Use notification templates to control the title and description of notifications generated by a monitor. Templates are Go template strings expanded at notification time with alert context, so the notification can include signal data such as severity, labels, and threshold values. A notification template has two optional fields:

Title: The subject line or heading of the notification.
Description: The body text of the notification.

To add a notification template to a monitor, see Create a monitor or Edit a monitor. For the notification_template field definition, see the monitor data model.

Set a notification template

Add a notification_template block when creating or updating a monitor.

Chronoctl
Terraform
API

Include a notification_template block in your monitor YAML when running chronoctl monitors create or chronoctl monitors update. For more information, see Chronoctl.

notification_template:
  title: "[{{.Severity}}] {{.Labels.service}} — error rate alert"
  description: >-
    Service {{.Labels.service}} in {{.Labels.env}} fired
    {{.ThresholdOp}} {{.Threshold}}. Status: {{.Status}}.

Add a notification_template block to your chronosphere_monitor resource. For more information, see the Terraform provider documentation.

Terraform example

resource "chronosphere_monitor" "example" {
  name = "High error rate"

  notification_template {
    title       = "[{{.Severity}}] {{.Labels.service}} — error rate alert"
    description = "Service {{.Labels.service}} in {{.Labels.env}} fired {{.ThresholdOp}} {{.Threshold}}. Status: {{.Status}}."
  }
}

Set the notification_template field on the monitor body when calling CreateMonitor or the equivalent update endpoint.

{
  "monitor": {
    "name": "High error rate",
    "notification_template": {
      "title": "[{{.Severity}}] {{.Labels.service}} — error rate alert",
      "description": "Service {{.Labels.service}} in {{.Labels.env}} fired {{.ThresholdOp}} {{.Threshold}}. Status: {{.Status}}."
    }
  }
}

Error rate alert example

The following template uses severity and label variables to produce a notification title and description.

{
  "title": "[{{.Severity}}] {{.Labels.service}} threshold exceeded",
  "description": "{{.Labels.env}} crossed {{.ThresholdOp}} {{.Threshold}}"
}

For a critical alert on the payments service in production, Chronosphere Observability Platform renders the notification as:

[critical] payments threshold exceeded
production crossed >= 0.95

Template variables

Reference variables with dot notation, for example, {{.Severity}}.

Variable	Type	Description
`.Severity`	string	Alert severity: `"critical"` or `"warn"`.
`.Status`	string	Alert state: `"firing"` or `"resolved"`.
`.Labels.<name>`	string	Signal labels attached to the alert. For example, `.Labels.env` or `.Labels.service`.
`.Threshold`	number	Threshold value for the active severity condition.
`.ThresholdOp`	string	Comparison operator for the active condition. For example, `">="` or `"<"`.
`.ResolveThreshold`	number	Effective resolve threshold. Defaults to `.Threshold` when no separate resolve threshold is configured.
`.StartsAtEpoch`	integer	Start time of the earliest alert in `.Alerts`, in epoch milliseconds.
`.Alerts`	list	All alerts in the notification group. See Alert object fields.
`.Alerts.Firing`	list	Subset of `.Alerts` that are currently firing.
`.Alerts.Resolved`	list	Subset of `.Alerts` that have resolved.

Alert object fields

Each item in .Alerts, .Alerts.Firing, and .Alerts.Resolved exposes the following fields:

Field	Type	Description
`.Status`	string	Alert state: `"firing"` or `"resolved"`.
`.Labels`	map	Per-alert label key-value pairs.
`.Annotations`	map	Per-alert annotation key-value pairs.
`.Threshold`	number	Per-alert threshold value.
`.ThresholdOp`	string	Per-alert comparison operator.
`.ResolveThreshold`	number	Per-alert effective resolve threshold.
`.StartsAt`	timestamp	Time the alert started firing.
`.EndsAt`	timestamp	Time the alert resolved, if applicable.

Template functions

In addition to the standard Go template built-ins, the following functions are available.

Arithmetic

Function	Description	Example
`add`	Adds two integers.	`{{add 1 2}}` returns `3`
`sub`	Subtracts two integers.	`{{sub 5 3}}` returns `2`
`mul`	Multiplies two integers.	`{{mul 3 4}}` returns `12`

String manipulation

Function	Description	Example
`toUpper`	Converts a string to uppercase.	`{{toUpper .Labels.env}}` returns `PRODUCTION`
`toLower`	Converts a string to lowercase.	`{{toLower .Labels.env}}` returns `production`
`title`	Title-cases a string.	`{{title .Labels.env}}` returns `Production`
`join`	Joins a string slice with a separator.	`{{join .Labels.regions ", "}}`
`match`	Returns `true` if a string matches a regular expression.	`{{match "^prod" .Labels.env}}`
`reReplaceAll`	Replaces all regexp matches in a string.	`{{reReplaceAll "-" "_" .Labels.service}}`

Template examples

Count firing and resolved alerts

{{len .Alerts.Firing}} firing, {{len .Alerts.Resolved}} resolved

List all firing alerts

{{range .Alerts.Firing -}}
- {{.Labels.service}} ({{.Labels.env}})
{{end}}

Show the resolve threshold conditionally

Threshold: {{.ThresholdOp}} {{.Threshold}}
{{- if ne .ResolveThreshold .Threshold}} (resolves at {{.ResolveThreshold}}){{end}}

Reference the alert start time

Alert started at epoch ms: {{.StartsAtEpoch}}

Observability Platform

Telemetry Pipeline

More information

Set a notification template

Error rate alert example

Template variables

Alert object fields

Template functions

Arithmetic

String manipulation

Template examples

Count firing and resolved alerts

List all firing alerts

Show the resolve threshold conditionally

Reference the alert start time

​Set a notification template

​Error rate alert example

​Template variables

​Alert object fields

​Template functions

​Arithmetic

​String manipulation

​Template examples

​Count firing and resolved alerts

​List all firing alerts

​Show the resolve threshold conditionally

​Reference the alert start time

Set a notification template

Error rate alert example

Template variables

Alert object fields

Template functions

Arithmetic

String manipulation

Template examples

Count firing and resolved alerts

List all firing alerts

Show the resolve threshold conditionally

Reference the alert start time