BookmarkSubscribeRSS Feed

Sending SAS Viya alert notifications to MS Teams

Started ‎03-16-2023 by
Modified ‎03-16-2023 by
Views 570

Effective alerting requires that the right people are notified of issues at the right time and via the most appropriate channels. SAS Viya Administrators and DevOps engineers using Alertmanager to manage alerts in the SAS Viya platform can integrate Alertmanager with several third party party applications to distribute alert notifications. While native support is built-in for some applications, many others are capable of receiving information via webhooks (e.g. alert notifications sent by Alertmanager). In this post, we'll look at how Alertmanager can integrate with Microsoft Teams via webhook, allowing alert notifications to be automatically posted to a Microsoft Teams channel when alerts are triggered.


There are several ways to set up the integration. In any case, as there is no native support for Teams integration in Alertmanager, and because Teams requires messages to be in a certain format (one that Alertmanager doesn't produce its alerts in), a third-party tool must be deployed to act as an intermediary. There are many to choose from, but in this case I used an open-source utility called prometheus-msteams. It is essentially a lightweight web server which receives alert messages from Alertmanager and sends them to a Teams channel in the required format.


But first, the Teams channel must be configured. Generate the unique incoming webhook URL (which is where prometheus-msteams will send messages) in the desired Teams channel by right-clicking on the channel name and clicking Connectors.  



Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.


If it's not already added, click Add on the Incoming Webhook Connector. When it's added, click Configure.  




Provide a name and then click create to generate the unique URL. Copy/take note of this, as it is required later.  




Next, set up prometheus-msteams. There are several ways to deploy it, including as a binary or as a docker container. In my lab, I deployed it to my Kubernetes cluster (in the monitoring namespace deployed by SAS Viya Monitoring for Kubernetes) using the Helm chart. First, I added the chart repo as per the instructions with:


helm repo add prometheus-msteams


Then prepare the configuration file before deployment. This is where webhook URL from Teams must be specified.


tee ~/config.yaml > /dev/null << EOF
replicaCount: 1
  tag: v1.5.1

# in alertmanager, this will be used as http://prometheus-msteams:2000/alertmanager
– alertmanager:
# # Enable metrics for prometheus operator

    enabled: true
      release: prometheus # change this accordingly
    scrapeInterval: 30s


To deploy, run:


helm upgrade --install prometheus-msteams --namespace v4mmon -f config.yaml  prometheus-msteams/prometheus-msteams


That results in: 


Release "prometheus-msteams" does not exist. Installing it now. 
NAME: prometheus-msteams
LAST DEPLOYED: Wed Mar 15 06:52:49 2023
NAMESPACE: v4mmon STATUS: deployed
NOTES: ** Please be patient while the chart is being deployed **

To monitor the deployment, execute the following command:

kubectl get pods -l app=prometheus-msteams --namespace v4mmon -w


Check that it has been deployed and is running:


kubectl get po -n v4mmon|grep msteams


The result:


prometheus-msteams-588777df8f-btmq2 1/1 Running 0 92s


The next step is to define a new receiver in the Alertmanager configuration for prometheus-msteams. In my lab, I made it the default receiver for all routes, so all alert notifications will get sent to Teams.


tee ~/alertmanager.yaml > /dev/null << EOF
  smtp_smarthost: $(hostname -f):1025
  smtp_from: ''
  smtp_require_tls: false
  resolve_timeout: 5m
  receiver: prometheus-msteams
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 12h
- name: prometheus-msteams
  - url: ""
    send_resolved: true


Note that in this example, the URL is referring to the IP & port of the node where the prometheus-msteams pod is running. In my lab, I set up the prometheus-msteams service with a NodePort so that I could simulate test alerts using cURL (from outside the cluster).


Apply the updated configuration:


cat <<EOF | kubectl apply -f -
apiVersion: v1
  alertmanager.yaml: $(cat ~/alertmanager.yaml | base64 -w0)
kind: Secret
  name: alertmanager-v4m-alertmanager
  namespace: v4mmon
type: Opaque


Once the Alertmanager config is applied, it may take a minute to take effect (hint: keep an eye on the Status page in the Alertmanager UI). When it does, notifications will start appearing as posts in the Teams channel when the alerts begin firing.  




If you want to customize the way messages are posted, you can do that too by following the documented instructions and examples.


For more information refer to the GitHub page. Alternative approaches are also widely documented online.


If you'd rather use Slack than Teams, check out this previous post. Thanks for reading.


Find more articles from SAS Global Enablement and Learning here.

Version history
Last update:
‎03-16-2023 05:22 PM
Updated by:



Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.

If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website. 

Register now!

Free course: Data Literacy Essentials

Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning  and boost your career prospects.

Get Started

Article Tags