# Customize notifications **** This documentation topic is designed for Grafana workspaces that support **Grafana version 12.x**. For Grafana workspaces that support Grafana version 10.x, see [Working in Grafana version 10](using-grafana-v10.md). For Grafana workspaces that support Grafana version 9.x, see [Working in Grafana version 9](using-grafana-v9.md). For Grafana workspaces that support Grafana version 8.x, see [Working in Grafana version 8](using-grafana-v8.md). Customize your notifications with notifications templates. You can use notification templates to change the title, message, and format of the message in your notifications. Notification templates are not tied to specific contact point integrations, such as Amazon SNS or Slack. However, you can choose to create separate notification templates for different contact point integrations. You can use notification templates to: + Add, remove, or re-order information in the notification including the summary, description, labels and annotations, values, and links + Format text in bold and italic, and add or remove line breaks You cannot use notification templates to: + Change the design of notifications in instant messaging services such as Slack and Microsoft Teams **Topics** + [Using Go’s templating language](v12-alerting-notifications-go-templating.md) + [Create notification templates](v12-alerting-create-templates.md) + [Using notification templates](#v12-alerting-use-notification-templates) + [Template reference](v12-alerting-template-reference.md) # Using Go’s templating language **** This documentation topic is designed for Grafana workspaces that support **Grafana version 12.x**. For Grafana workspaces that support Grafana version 10.x, see [Working in Grafana version 10](using-grafana-v10.md). For Grafana workspaces that support Grafana version 9.x, see [Working in Grafana version 9](using-grafana-v9.md). For Grafana workspaces that support Grafana version 8.x, see [Working in Grafana version 8](using-grafana-v8.md). You write notification templates in Go’s templating language, [text/template](https://pkg.go.dev/text/template). This section provides an overview of Go’s templating language and writing templates in text/template. ## Dot In text/template there is a special cursor called dot, and is written as `.`. You can think of this cursor as a variable whose value changes depending where in the template it is used. For example, at the start of a notification template `.` refers to the `ExtendedData` object, which contains a number of fields including `Alerts`, `Status`, `GroupLabels`, `CommonLabels`, `CommonAnnotations` and `ExternalURL`. However, dot might refer to something else when used in a `range` over a list, when used inside a `with`, or when writing feature templates to be used in other templates. You can see examples of this in [Create notification templates](v12-alerting-create-templates.md), and all data and functions in the [Template reference](v12-alerting-template-reference.md). ## Opening and closing tags In text/template, templates start with `{{` and end with `}}` irrespective of whether the template prints a variable or runs control structures such as if statements. This is different from other templating languages such as Jinja where printing a variable uses `{{` and `}}` and control structures use `{%` and `%}`. ## Print To print the value of something use `{{` and `}}`. You can print the value of dot, a field of dot, the result of a function, and the value of a [variable](#v12-go-variables). For example, to print the `Alerts` field where dot refers to `ExtendedData` you would write the following: ``` {{ .Alerts }} ``` ## Iterate over alerts To print just the labels of each alert, rather than all information about the alert, you can use a `range` to iterate the alerts in `ExtendedData`: ``` {{ range .Alerts }} {{ .Labels }} {{ end }} ``` Inside the range dot no longer refers to `ExtendedData`, but to an `Alert`. You can use `{{ .Labels }}` to print the labels of each alert. This works because `{{ range .Alerts }}` changes dot to refer to the current alert in the list of alerts. When the range is finished dot is reset to the value it had before the start of the range, which in this example is `ExtendedData`: ``` {{ range .Alerts }} {{ .Labels }} {{ end }} {{/* does not work, .Labels does not exist here */}} {{ .Labels }} {{/* works, cursor was reset */}} {{ .Status }} ``` ## Iterate over annotations and labels Let’s write a template to print the labels of each alert in the format `The name of the label is $name, and the value is $value`, where `$name` and `$value` contain the name and value of each label. Like in the previous example, use a range to iterate over the alerts in `.Alerts` such that dot refers to the current alert in the list of alerts, and then use a second range on the sorted labels so dot is updated a second time to refer to the current label. Inside the second range use `.Name` and `.Value` to print the name and value of each label: ``` {{ range .Alerts }} {{ range .Labels.SortedPairs }} The name of the label is {{ .Name }}, and the value is {{ .Value }} {{ end }} {{ range .Annotations.SortedPairs }} The name of the annotation is {{ .Name }}, and the value is {{ .Value }} {{ end }} {{ end }} ``` ## The index functions To print a specific annotation or label use the `index` function. ``` {{ range .Alerts }} The name of the alert is {{ index .Labels "alertname" }} {{ end }} ``` ## If statements You can use if statements in templates. For example, to print `There are no alerts` if there are no alerts in `.Alerts` you would write the following: ``` {{ if .Alerts }} There are alerts {{ else }} There are no alerts {{ end }} ``` ## With With is similar to if statements, however unlike if statements, `with` updates dot to refer to the value of the with: ``` {{ with .Alerts }} There are {{ len . }} alert(s) {{ else }} There are no alerts {{ end }} ``` ## Variables Variables in text/template must be created within the template. For example, to create a variable called `$variable` with the current value of dot you would write the following: ``` {{ $variable := . }} ``` You can use `$variable` inside a range or `with` and it will refer to the value of dot at the time the variable was defined, not the current value of dot. For example, you cannot write a template that use `{{ .Labels }}` in the second range because here dot refers to the current label, not the current alert: ``` {{ range .Alerts }} {{ range .Labels.SortedPairs }} {{ .Name }} = {{ .Value }} {{/* does not work because in the second range . is a label not an alert */}} There are {{ len .Labels }} {{ end }} {{ end }} ``` You can fix this by defining a variable called `$alert` in the first range and before the second range: ``` {{ range .Alerts }} {{ $alert := . }} {{ range .Labels.SortedPairs }} {{ .Name }} = {{ .Value }} {{/* works because $alert refers to the value of dot inside the first range */}} There are {{ len $alert.Labels }} {{ end }} {{ end }} ``` ## Range with index You can get the index of each alert within a range by defining index and value variables at the start of the range: ``` {{ $num_alerts := len .Alerts }} {{ range $index, $alert := .Alerts }} This is alert {{ $index }} out of {{ $num_alerts }} {{ end }} ``` ## Define templates You can define templates that can be used within other templates, using `define` and the name of the template in double quotes. You should not define templates with the same name as other templates, including default templates such as `__subject`, `__text_values_list`, `__text_alert_list`, `default.title` and `default.message`. Where a template has been created with the same name as a default template, or a template in another notification template, Grafana might use either template. Grafana does not prevent, or show an error message, when there are two or more templates with the same name. ``` {{ define "print_labels" }} {{ end }} ``` ## Execute templates You can execute defined template within your template using `template`, the name of the template in double quotes, and the cursor that should be passed to the template: ``` {{ template "print_labels" . }} ``` ## Pass data to templates Within a template dot refers to the value that is passed to the template. For example, if a template is passed a list of firing alerts then dot refers to that list of firing alerts: ``` {{ template "print_alerts" .Alerts }} ``` If the template is passed the sorted labels for an alert then dot refers to the list of sorted labels: ``` {{ template "print_labels" .SortedLabels }} ``` This is useful when writing reusable templates. For example, to print all alerts you might write the following: ``` {{ template "print_alerts" .Alerts }} ``` Then to print just the firing alerts you could write this: ``` {{ template "print_alerts" .Alerts.Firing }} ``` This works because both `.Alerts` and `.Alerts.Firing` are lists of alerts. ``` {{ define "print_alerts" }} {{ range . }} {{ template "print_labels" .SortedLabels }} {{ end }} {{ end }} ``` ## Comments You can add comments with `{{/*` and `*/}}`: ``` {{/* This is a comment */}} ``` To prevent comments from adding line breaks use: ``` {{- /* This is a comment with no leading or trailing line breaks */ -}} ``` ## Indentation You can use indentation, both tabs and spaces, and line breaks, to make templates more readable: ``` {{ range .Alerts }} {{ range .Labels.SortedPairs }} {{ .Name }} = {{ .Value }} {{ end }} {{ end }} ``` However, indentation in the template will also be present in the text. Next we will see how to remove it. ## Remove spaces and line breaks In text/template use `{{-` and `-}}` to remove leading and trailing spaces and line breaks. For example, when using indentation and line breaks to make a template more readable: ``` {{ range .Alerts }} {{ range .Labels.SortedPairs }} {{ .Name }} = {{ .Value }} {{ end }} {{ end }} ``` The indentation and line breaks will also be present in the text: ``` alertname = "Test" grafana_folder = "Test alerts" ``` You can remove the indentation and line breaks from the text changing `}}` to `-}}` at the start of each range: ``` {{ range .Alerts -}} {{ range .Labels.SortedPairs -}} {{ .Name }} = {{ .Value }} {{ end }} {{ end }} ``` The indentation and line breaks in the template are now absent from the text: ``` alertname = "Test" grafana_folder = "Test alerts" ``` # Create notification templates **** This documentation topic is designed for Grafana workspaces that support **Grafana version 12.x**. For Grafana workspaces that support Grafana version 10.x, see [Working in Grafana version 10](using-grafana-v10.md). For Grafana workspaces that support Grafana version 9.x, see [Working in Grafana version 9](using-grafana-v9.md). For Grafana workspaces that support Grafana version 8.x, see [Working in Grafana version 8](using-grafana-v8.md). Create reusable notification templates to send to your contact points. You can add one or more templates to your notification template. Your notification template name must be unique. You cannot have two templates with the same name in the same notification template or in different notification templates. Avoid defining templates with the same name as default templates, such as: `__subject`, `__text_values_list`, `__text_alert_list`, `default.title` and `default.message`. In the Contact points tab, you can see a list of your notification templates. ## Creating notification templates **To create a notification template** 1. Choose **Alerting**, **Contact points**. 1. Choose the **Notification Templates** tab, and then **\$1 Add notification template**. 1. Choose a name for the notification template, such as `email.subject`. 1. Write the content of the template in the content field. For example: ``` {{ if .Alerts.Firing -}} {{ len .Alerts.Firing }} firing alerts {{ end }} {{ if .Alerts.Resolved -}} {{ len .Alerts.Resolved }} resolved alerts {{ end }} ``` 1. Save your changes. `{{ define "email.subject" }}` (where `email.subject` is the name of your template) and `{{ end }}` is automatically added to the start and end of the content. **To create a notification template that contains more than one template** 1. Choose **Alerting**, **Contact points**. 1. Choose the **Notification Templates** tab, and then **\$1 Add notification template**. 1. Enter a name for the overall notification template. For example, `email`. 1. Write each template in the Content field, including `{{ define "name-of-template" }}` and `{{ end }}` at the start and end of each template. You can use descriptive names for each of the templates in the notification template, for example, `email.subject` or `email.message`. In this case, do not reuse the name of the notification template you entered above. Later sections show detailed examples for templates you might create. 1. Click Save. ## Preview notification templates Preview how your notification templates will look before using them in your contact points, helping you understand the result of the template you are creating as well as giving you a chance to fix any errors before saving the template. **Note** Notification previews are only available for Grafana Alertmanager. **To preview your notification templates** 1. Choose **Alerting**, **Contact points**. 1. Choose the **Notification Templates** tab, and then **\$1 Add notification template**, or edit an existing template. 1. Add or update your template content. Default data is provided and you can add or edit alert data to it as well as alert instances. You can add alert data directly in the Payload data window itself, or click **Select alert instances** or **Add custom alerts**. 1. [Optional] To add alert data from existing alert instances: 1. Choose **Select alert instances**. 1. Hover over the alert instances to view more information about each alert instance/ 1. Choose **Confirm** to add the alert instance to the payload. 1. [Optional] To add alert data using the Alert data editor, choose **Add custom data**: 1. Add annotations, custom labels, or set a dashboard or panel. 1. Toggle Firing or resolved, depending on whether you want to add firing or resolved alerts to your notification. 1. Choose **Add alert data**. 1. Choose **Refresh preview** to see what your template content will look like and the corresponding payload data. If there are any errors in your template, they are displayed in the Preview and you can correct them before saving. 1. Save your changes. ## Creating a template for the subject of message Create a template for the subject of an email that contains the number of firing and resolved alerts, as in this example: ``` 1 firing alerts, 0 resolved alerts ``` **To create a template for the subject of an email** 1. Create a template called `email.subject` with the following content: ``` {{ define "email.subject" }} {{ len .Alerts.Firing }} firing alerts, {{ len .Alerts.Resolved }} resolved alerts {{ end }} ``` 1. Use the template when creating your contact point integration by putting it into the **Subject** field with the `template` keyword. ``` {{ template "email.subject" . }} ``` ## Creating a template for the message of an email Create a template for the message of an email that contains a summary of all firing and resolved alerts, as in this example: ``` There are 2 firing alerts, and 1 resolved alerts Firing alerts: - alertname=Test 1 grafana_folder=GrafanaCloud has value(s) B=1 - alertname=Test 2 grafana_folder=GrafanaCloud has value(s) B=2 Resolved alerts: - alertname=Test 3 grafana_folder=GrafanaCloud has value(s) B=0 ``` **To create a template for the message of an email** 1. Create a notification template called `email` with two templates in the content: `email.message_alert` and `email.message`. The `email.message_alert` template is used to print the labels and values for each firing and resolved alert while the `email.message` template contains the structure of the email. ``` {{- define "email.message_alert" -}} {{- range .Labels.SortedPairs }}{{ .Name }}={{ .Value }} {{ end }} has value(s) {{- range $k, $v := .Values }} {{ $k }}={{ $v }}{{ end }} {{- end -}} {{ define "email.message" }} There are {{ len .Alerts.Firing }} firing alerts, and {{ len .Alerts.Resolved }} resolved alerts {{ if .Alerts.Firing -}} Firing alerts: {{- range .Alerts.Firing }} - {{ template "email.message_alert" . }} {{- end }} {{- end }} {{ if .Alerts.Resolved -}} Resolved alerts: {{- range .Alerts.Resolved }} - {{ template "email.message_alert" . }} {{- end }} {{- end }} {{ end }} ``` 1. Use the template when creating your contact point integration by putting it into the **Text Body** field with the `template` keyword. ``` {{ template "email.message" . }} ``` ## Creating a template for the title of a Slack message Create a template for the title of a Slack message that contains the number of firing and resolved alerts, as in the following example: ``` 1 firing alerts, 0 resolved alerts ``` **To create a template for the title of a Slack message** 1. Create a template called `slack.title` with the following content: ``` {{ define "slack.title" }} {{ len .Alerts.Firing }} firing alerts, {{ len .Alerts.Resolved }} resolved alerts {{ end }} ``` 1. Execute the template from the title field in your contact point integration. ``` {{ template "slack.title" . }} ``` ## Creating a template for the content of a Slack message Create a template for the content of a Slack message that contains a description of all firing and resolved alerts, including their labels, annotations, and Dashboard URL. **Note** This template is for Grafana managed alerts only. To use the template for data source managed alerts, delete the references to DashboardURL and SilenceURL. For more information about configuring Prometheus notifications, see the [Prometheus documentation on notifications](https://prometheus.io/docs/alerting/latest/notifications/). ``` 1 firing alerts: [firing] Test1 Labels: - alertname: Test1 - grafana_folder: GrafanaCloud Annotations: - description: This is a test alert Go to dashboard: https://example.com/d/dlhdLqF4z?orgId=1 1 resolved alerts: [firing] Test2 Labels: - alertname: Test2 - grafana_folder: GrafanaCloud Annotations: - description: This is another test alert Go to dashboard: https://example.com/d/dlhdLqF4z?orgId=1 ``` **To create a template for the content of a Slack message** 1. Create a template called `slack` with two templates in the content: `slack.print_alert` and `slack.message`. The `slack.print_alert` template is used to print the labels, annotations, and DashboardURL while the `slack.message` template contains the structure of the notification. ``` {{ define "slack.print_alert" -}} [{{.Status}}] {{ .Labels.alertname }} Labels: {{ range .Labels.SortedPairs -}} - {{ .Name }}: {{ .Value }} {{ end -}} {{ if .Annotations -}} Annotations: {{ range .Annotations.SortedPairs -}} - {{ .Name }}: {{ .Value }} {{ end -}} {{ end -}} {{ if .DashboardURL -}} Go to dashboard: {{ .DashboardURL }} {{- end }} {{- end }} {{ define "slack.message" -}} {{ if .Alerts.Firing -}} {{ len .Alerts.Firing }} firing alerts: {{ range .Alerts.Firing }} {{ template "slack.print_alert" . }} {{ end -}} {{ end }} {{ if .Alerts.Resolved -}} {{ len .Alerts.Resolved }} resolved alerts: {{ range .Alerts.Resolved }} {{ template "slack.print_alert" .}} {{ end -}} {{ end }} {{- end }} ``` 1. Execute the template from the text body field in your contact point integration: ``` {{ template "slack.message" . }} ``` ## Template both email and Slack with shared templates Instead of creating separate notification templates for each contact point, such as email and Slack, you can share the same template. For example, if you want to send an email with this subject and Slack message with this title `1 firing alerts, 0 resolved alerts`, you can create a shared template. **To create a shared template** 1. Create a template called `common.subject_title` with the following content: ``` {{ define "common.subject_title" }} {{ len .Alerts.Firing }} firing alerts, {{ len .Alerts.Resolved }} resolved alerts {{ end }} ``` 1. For email, run the template from the subject field in your email contact point integration: ``` {{ template "common.subject_title" . }} ``` 1. For Slack, run the template from the title field in your Slack contact point integration: ``` {{ template "common.subject_title" . }} ``` ## Using notification templates Use templates in contact points to customize your notifications. **To use a template when creating a contact point** 1. From the **Alerting** menu, choose the **Contact points** tab to see a list of existing contact points. 1. Choose **New**. Alternately, you can edit an existing contact point by choosing the **Edit** icon. 1. Enter the templates you wish to use in a field, such as **Message** or **Subject**. To enter a template, use the form `{{ template "template_name" . }}`, replacing *template\$1name* with the name of the template you want to use. 1. Choose **Save contact point**. # Template reference **** This documentation topic is designed for Grafana workspaces that support **Grafana version 12.x**. For Grafana workspaces that support Grafana version 10.x, see [Working in Grafana version 10](using-grafana-v10.md). For Grafana workspaces that support Grafana version 9.x, see [Working in Grafana version 9](using-grafana-v9.md). For Grafana workspaces that support Grafana version 8.x, see [Working in Grafana version 8](using-grafana-v8.md). This section provides reference information for creating your templates. **Alert (type)** The alert type contains the following data. | Name | Kind | Description | Example | | --- | --- | --- | --- | | Status | string | `firing` or `resolved`. | \$1\$1 .Status \$1\$1 | | Labels | KeyValue | A set of labels attached to the alert. | \$1\$1 .Labels \$1\$1 | | Annotations | KeyValue | A set of annotations attached to the alert. | \$1\$1 .Annotations \$1\$1 | | Values | KeyValue | The values of all expressions, including Classic Conditions | \$1\$1 .Values \$1\$1 | | StartsAt | time.Time | Time the alert started firing. | \$1\$1 .StartsAt \$1\$1 | | EndsAt | time.Time | Only set if the end time of an alert is known. Otherwise set to a configurable timeout period from the time since the last alert was received. | \$1\$1 .EndsAt \$1\$1 | | GeneratorURL | string | A back link to Grafana or external Alertmanager. | \$1\$1 .GeneratorURL \$1\$1 | | SilenceURL | string | A link to silence the alert (with labels for this alert pre-filled). Only for Grafana managed alerts. | \$1\$1 .SilenceURL\$1\$1 | | DashboardURL | string | Link to grafana dashboard, if alert rule belongs to one. Only for Grafana managed alerts. | \$1\$1 .DashboardURL \$1\$1 | | PanelURL | string | Link to grafana dashboard panel, if alert rule belongs to one. Only for Grafana managed alerts. | \$1\$1 .PanelURL \$1\$1 | | Fingerprint | string | Fingerprint that can be used to identify the alert. | \$1\$1 .Fingerprint \$1\$1 | | ValueString | string | A string that contains the labels and value of each reduced expression in the alert. | \$1\$1 .ValueString \$1\$1 | **ExtendedData** The ExtendedData object contains the following properties. | Name | Kind | Description | Example | | --- | --- | --- | --- | | Receiver | `string` | The name of the contact point sending the notification. | `{{ .Receiver }}` | | Status | `string` | The status is `firing` if at least one alert is firing, otherwise `resolved`. | `{{ .Status }}` | | Alerts | `[]Alert` | List of all firing and resolved alerts in this notification. | `There are {{ len .Alerts }} alerts` | | Firing alerts | `[]Alert` | List of all firing alerts in this notification. | `There are {{ len .Alerts.Firing }} firing alerts` | | Resolved alerts | `[]Alert` | List of all resolved alerts in this notification. | `There are {{ len .Alerts.Resolved }} resolved alerts` | | GroupLabels | `KeyValue` | The labels that group these alerts int his notification. | `{{ .GroupLabels }}` | | CommonLabels | `KeyValue` | The labels common to all alerts in this notification. | `{{ .CommonLabels }}` | | CommonAnnotations | `KeyValue` | The annotations common to all alerts in this notification. | `{{ .CommonAnnotations }}` | | ExternalURL | `string` | A link to the Grafana workspace or Alertmanager that sent this notification. | `{{ .ExternalURL }}` | **KeyValue type** The `KeyValue` type is a set of key/value string pairs that represent labels and annotations. In addition to direct access of the data stored as a `KeyValue`, there are also methods for sorting, removing, and transforming the data. | Name | Arguments | Returns | Notes | Example | | --- | --- | --- | --- | --- | | SortedPairs | | Sorted list of key and value string pairs | | `{{ .Annotations.SortedPairs }}` | | Remove | []string | KeyValue | Returns a copy of the Key/Value map without the given keys. | `{{ .Annotations.Remove "summary" }}` | | Names | | []string | List of names | `{{ .Names }}` | | Values | | []string | List of values | `{{ .Values }}` | **Time** Time is from the Go [https://pkg.go.dev/time#Time](https://pkg.go.dev/time#Time) package. You can print a time in a number of different formats. For example, to print the time that an alert fired in the format `Monday, 1st January 2022 at 10:00AM`, you write the following template: ``` {{ .StartsAt.Format "Monday, 2 January 2006 at 3:04PM" }} ``` You can find a reference for Go’s time format [here](https://pkg.go.dev/time#pkg-constants).