Notifications Engine is here!

The Argoproj community has been working on notifications functionality for a while now. We’ve tried several different approaches and learned a lot from our early users. Based on our learnings, we’ve come up with the idea of a notifications engine that solves various notifications-related use cases for all Argo projects and even beyond.

The first notifications engine consumer was the argocd-notifications project that provides notifications for the Argo CD Application CRD and soon for Argo Rollouts as well. Today, we are happy to announce the first version of Notifications Engine 🎉 !

What is Notifications Engine exactly?

The Notifications Engine is a Golang based library that implements notifications functionality for Kubernetes controllers. First-class notifications support is often a second thought in Kubernetes controllers. This is challenging to solve generically because notifications are very opinionated by nature. It is hard to predict what kind of events end-users want to be notified about and especially what the notification should look like. Additionally, there are lots of notifications services so it is hard to decide which one to support first. The Notifications Engine is trying to tackle these challenges:

• Provide a flexible configuration-driven mechanism of triggers and templates that allows CRD controller administrators to accommodate end-user requirements without making any code changes;
• Provide out-of-the-box integration with dozens of notifications services (Slack, SMTP, Telegram, etc.) with many more integrations yet to come;

The features provided by the notifications engine enable the following use cases:

• Notify users about important events that happened in the Kubernetes cluster. The events examples are degraded deployment; invalid certificate configuration; successful completion of a job etc.
• Built custom integration between resources in your cluster and external company tools. For example, you might trigger a CI pipeline after Argo CD Application successfully deployed or open the Jira ticket when external secret controller failed to retrieve the secret from AWS Secret Manager.
• Set up an audit system that catches and pushes information about important events into persistent storage.

Features

Once the notifications engine is integrated into your project, the project end-users get a powerful configuration-driven mechanism to send notifications to dozens of notifications services.

Notifications Triggers and Templates

The engine introduces notifications triggers and templates that allow catching important custom resource events and sending fully customizable notifications. The trigger is a named condition that monitors your Kubernetes resource and decides if it is time to send the notification, and the notification template is a stateless function that generates the notification content.

Both triggers and templates are typically configured once by the controller maintainers and customized by the administrators who run the controller. The example below demonstrates a trigger that notifies users about a problem with Argo CD Application:

Notifications Services

In addition to the triggers and templates, the administrator needs to configure integration with supported notification services. The notifications engine out of the box supports Slack, MS Teams, Email, Mattermost, Telegram, Rocket Chat, OpsGenie. The list of supported integration does not end with only text-based notifications. The users can leverage the notifications engine to update commits status in Github, create Grafana annotations or configure totally custom integration using generic Webhook-based service. The example below demonstrates integration with Slack:

User-Friendly Subscriptions

Once the triggers, templates, and services are configured by the administrator the end-users just need to “subscribe” to the triggers they are interested in. In order to subscribe the user needs to add the notifications.argoproj.io/subscribe.<trigger>.<service> annotation and specifies the notification recipient in the annotation value. The following example creates on-sync-succeeded trigger subscription that sends notifications to two Slack channels:

How to Integrate Notifications Engine?

Are you excited about the notifications engine and want to integrate it into your project? You have to write some Golang code but the good news is that you won’t need to write much. Typically, you would have to create one main.go file that contains boilerplate code to talk to Kubernetes and few lines of code that bootstraps the notifications controller.

Demo!

A demo is worth a thousand words. In the case of a library, the demo is a tutorial that demonstrates how to use the library. The following paragraphs explain how you can build notifications for the Cert-Manager Certificate CRD. The full example is available in examples/certmanager directory in the notifications engine repository.

Controller

The work required to monitor custom resources and deliver notifications is performed by the notifications controller. The machinery required to build the controller is provided by pkg/controller and pkg/api packages. Before jumping into implementing the controller we need to write boilerplate code that is required to talk to the Kubernetes cluster and provide access to the Kubernetes ConfigMap and Secret with the triggers and templates:

The next step is to write code that instantiates the controller itself:

The last step is to start the controller:

We are pretty much done! The controller is ready. To start using it we need to configure triggers, templates and set up the integration with some notifications service. The following YAML integrates our notifications controller with Slack and messages us when any certificate successfully provisioned by the Cert Manager:

To see the controller in action we still need to set up the demo environment. Unfortunately, it won’t fit into a blog but here are some links:

• Get yourself Minikube cluster: https://minikube.sigs.k8s.io/docs/start/
• Install Cert Manager: https://cert-manager.io/docs/installation/
• Set up Slack integration: https://github.com/argoproj/notifications-engine/blob/master/docs/services/slack.md

Finally, run the controller and enjoy the Slack notifications!

What Is Next?

In addition to the controller toolkit, the notifications engine includes troubleshooting tools. The tools include Prometheus metrics and CLI that help to create and validate triggers and templates. Learn more about these features in the notifications engine documentation. Don’t hesitate to share your feedback in CNCF Slack channel or create Github issue to request more integrations or report a bug!