In a default deployment of SAS Viya Monitoring for Kubernetes, Prometheus and Alertmanager are deployed without any authentication mechanism. That is, anyone with the URL can access the Prometheus Expression Browser and Alertmanager UIs. But there is a way to make things a little more secure. In this article, we'll look at how basic authentication can be configured for the monitoring applications.
Applications are not certainly considered 'secure' simply with the addition of basic authentication, but doing so does add a layer of simple security and access control. There are some gaps and caveats, including but not limited to:
Even so, basic authentication is probably better than no authentication, at least in some cases. Note though, that in many cases, there may not be a need to expose Prometheus and Alertmanager at all. They can function just fine in-cluster without ingress (though NodePorts should be disabled as a security measure, which will be soon be the default setting for SAS Viya Monitoring for Kubernetes) and in cases where external access is required for short periods, port-forwarding can be a good option.
This page of the GitHub repository provides more information on deploying logging and monitoring components with ingress. Samples are also provided for both host and path ingresses.
We will need to make some adjustments to the ingress configuration to implement basic-auth. But first, we need to create/define credentials in a password file. If installed, we can use htpasswd:
# below command will prompt for a password htpasswd ./auth marcelo
... or we can run something like the following for the same result:
printf "marcelo:$(openssl passwd -crypt Biel$a)\n" >> auth
To add more user credentials, so that multiple users can access the applications with different credentials, simply re-run the command with the additional usernames and passwords. The resulting 'auth' file, which contains a list of users and encoded passwords (remember to limit access to it!), will be used to create a new Kubernetes secret. Run the following to create the secret in the monitoring namespace ("ops4viyamon" in this example):
kubectl create secret generic basic-auth --from-file auth -n ops4viyamon
Next, the ingress needs to be modified to use the credentials in the secret to implement basic authentication. Edit the ingresses (in the monitoring namespace) for Prometheus and Alertmanager by adding the following keys to the
nginx.ingress.kubernetes.io/auth-realm: Authentication Required nginx.ingress.kubernetes.io/auth-secret: ops4viyamon/basic-auth nginx.ingress.kubernetes.io/auth-type: basic
The quick/lazy way to test this is to use Lens or
kubectl edit ing, but the changes will not persist if you redeploy/upgrade. The better way would be to edit the ingress configuration in
user-values-prom-operator.yaml. In any case, the result should end up looking something like:
Select any image to see a larger version.
Mobile users: To view the images, select the "Full" version at the bottom of the page.
Once that's done for both the Prometheus and Alertmanager ingresses, we're ready test with a cURL command or by hitting the URL in a browser. This time, we should get a prompt for credentials:
Now, users will only be able to access the interfaces if they authenticate using the credentials specified.
Grafana has its own authentication layer, but there is something else to consider for Grafana. While Grafana connects to Prometheus (the datasource for the dashboards), the dashboards still work without additional configuration after basic authentication has been implemented, because the connection to Prometheus uses the internal ClusterIP service address to connect directly, and not via the ingress controller. Remember that we've configured basic-auth for the NginX controller rather than for Prometheus. The latter is also possible though, as Prometheus has built-in support for basic authentication, which doesn't require ingress at all (although there seems to be an issue getting it working for Alertmanager). Refer to this guide and this example for more information. Note that using another method may require changes to the connection information in the Grafana datasource config (to supply basic auth credentials).
The NginX documentation also contains more information, but of course it's also possible to use another ingress controller (e.g. Apache), or to configure the ingress controller to start in a container in the Prometheus pod in a Sidecar proxy pattern. Possibly a topic for a future article.
Find more articles from SAS Global Enablement and Learning here.
Registration is open! SAS is returning to Vegas for an AI and analytics experience like no other! Whether you're an executive, manager, end user or SAS partner, SAS Innovate is designed for everyone on your team. Register for just $495 by 12/31/2023.
If you are interested in speaking, there is still time to submit a session idea. More details are posted on the website.
Data Literacy is for all, even absolute beginners. Jump on board with this free e-learning and boost your career prospects.