The sudo requirement explained to your customer (Viya setup)
With the release of Viya there is a new approach for the user account required for the installation and configuration (deployment) process. This in part reflect's the use of Ansible to deploy Viya across multiple machines for an effective and consistent deployment experience.
As stated here in the official deployment guide, the requirement for this installation account is : “Super user (sudo) or root access.” (For those familiar with 9.4 deployments, the use of root was required for parts of the deployment process. A familiar example comes from setuid root requirements on “caslaunch”, “elssrv”, “sasauth” ,”sasperm” files on Linux/Unix platforms.)
Moreover, for multi-machines deployment we need this account to be able to connect with SSH from the Ansible controller host to all the other machines of the deployment.
SAS recognizes that customers will want to ring fence the use of the root account for security reasons. Therefore when customers wish to have their deployment teams install Viya or third party technical experts deploy Viya, they will most like perform the installation with a “sudo” account.
What is "sudo"?
“sudo” (aka “--- Super user do ---””) allows a permitted user to execute a command as the super user (root, the administrator account in Linux/Unix) or another user, as specified by the security policy. The default security policy is “sudoers”, which is configured via the file /etc/sudoers, or via LDAP.
Basically “sudo” enables you to define commands that a specific user is allowed to run as root, or as another user.
When you run the sudo command, an OS level check is done against the policy to ensure that your account is defined as a “sudoer”. If not you might see a quite worrying message that “this incident will be reported”. If yes then either your command is executed or you are prompted to confirm your identify with your own password and then the command is executed.
The “sudo” privileges can be very limited (e.g.: just allow one specific command as a specific user) or, on the contrary much extended (e.g.: allow ALL commands as root).
It’s all down to the sudoers policy.
You can implement “Black list” or “White list” approaches.
- With the “Black list” approach you typically give access to any commands except a few ones that seems risky to you (such as shell, su, or infrastructure related commands).
- Whereas with the “White list” approach you list all the authorized commands (such as: “useradd”, “mkdir”, “chmod”, “chown”, “yum install”, “cp”, etc…), so the “sudo” account can only run the commands in this list.
In the policy, you can also restrict the list of accounts that can be impersonated by the sudoers account and require the “sudoer” to provide his password before allowing the command to be executed.
As the Viya installation playbook requires the installation account to execute specific commands that are not allowed for a standard user, we leverage “sudo” to perform the Viya deployment.
If you want to know more on how to configure the commands, allowed users, allowed “run as” users, etc… this page provides some useful examples.
Why do we (SAS) need it?
A core feature of the Viya deployment is the use of RPM (RedHat Package Manager).
But an RPMs installation requires the root account.
In addition, the Viya installation process also needs to create users, set ownerships on files and folders, register services, run commands as “sas” or as “cas”, etc…
And finally we have our long time requirement with the setuid root requirements on “caslaunch”, “elssrv”, “sasauth” ,”sasperm” files. This one still here with Viya. It has been around for long time, which reminds us that the root account is also required in SAS 9 and earlier versions.
During the Viya deployment, Ansible drives the download and installation of the RPM packages via YUM and also performs all the privileged and impersonated commands through its “become” option.
The Ansible documentation explains in the “Become (Privilege Escalation)” section that privilege escalation has to be general. Trying to allow only specific commands (such as “/sbin/service” or “/bin/chmod”) will fail because of the way Ansible dynamically builds and executes the tasks. We will see later what it means in detail.
The ability to run commands as the privileged user can be seen as the price to pay for the increased automation of our software that Ansible and Yum bring.
How?
Let's focus a little bit on the sudoers policy as it will likely be the most frequently used option.
The most permissive access for an installation account are given with a line as below:
The "viyadep" user can run any command without being prompted. It is the simplest case, however the sudoers configuration file supports a lot of subtleties to have a better control on who can run what.
For example, if the customer IT is more concerned with the security, the security administrator might want to implement something more like the below:
Here, we allow the "viyadep" account to run any executable located in "/usr/bin" folder as "sas", "cas" or "root", but the access to "shell" and "su" executables is restricted.
It is a classic sudoers restriction because access to "shell" or "su" commands would then allow a global unrestricted access on the machine. In addition, as we are not using "NOPASSWD", it means that each time, the "viyadep" user issues a sudo command, he has to confirm his identity with his own password.
Let's see what happens with the sudoers configuration above, when we want to use Ansible to run privileged commands.
Note that we need to use the "ask-become-pass" parameter and provide a password because in the "sudoers" configuration our "viyadep" account needs to provide a password to confirm his identity.
Unfortunately, as you can see, Ansible generates an executable from the Ansible task and tries to run it, as the "become" user with "/bin/sh", which is not allowed, so the task fails. (Which is consistent with the Ansible documentation on the “become” mode and its requirements – explained above) It is the reason why we cannot give to our customers a list of privileged commands that are run during the Viya deployment.
This simple test shows that the execution of our Viya installation Ansible playbook only works if we allow the execution of "/usr/bin/sh" (which is equivalent to authorize ALL commands).
So most of the time, what we want to ask to our customer, to be on the safe side, is something like:
With that, we allow the "viyadep" account (as long as he can provide his password to confirm his identity) to run any commands as "root", "sas" or "cas".
Mitigations measures, best practices and other options
So, unfortunately as we’ve just seen, we have no other choice than to open quite widely the range of commands that our Viya installer account can run as the super-user.
Here are a few things you could do to ease the acceptance of the sudoers requirement for the Viya installation:
- Disable or restrict the sudoers permissions right after the installation
- During the installation the use of viyadep could be considered a security risk, but we can limit the time this is a risk: Once the installation is completed, we don’t need this sudo privilege anymore. A best practice is to comment the viyadep authorization line (in case we need it later for any clean-up/reinstallation/upgrade) and to create a new line allowing only the services management, for example :
- Encourage the customer to monitor sudo actions during the deployment, implement “auditd” and keep track of the changes.
- RedHat and CentOS based systems keeps track of all the commands ran with sudo in “/var/log/secure”.
- The sudo actions are calling transient dynamic scripts created during the playbook execution – then the scripts are deleted. Ansible can be configured to keep all the dynamically generated scripts that are generated and executed (ANSIBLE_KEEP_REMOTE_FILES=1) – So the IT team can always go in “/tmp/.ansible/tmp/” and look at what has been run in the system during the Viya installation.
- Discuss, explain and propose the “bin-echo” approach (see below)
If our sudoers requirement is really a NO GO for your IT team, there are a couple of other options…
- Negotiate the root access
Sometimes, it might be easier to involve a system administrator in the deployment process and have them seated next to you and run the Viya deployment with their own root access. Just prepare all the commands that need to be run as root and let them type the commands!
- Use YUM only deployment?
Instead of using Ansible which cannot limit the escalation to certain commands, you might have the idea to follow the “Use YUM only” installation path. So the IT team would have a better view of what is done and how.
However this method has many drawbacks and is, moreover extremely limited: it does not allow the configuration of MPP CAS, and it is also restricted to “Programming” environment only – no installation and configuration of Visual Interfaces and Microservices.
It is not an exhaustive list, I’m sure there are other things that I am missing, please feel free to contribute here if you have other ideas!
Bin-echo
If after all the explanations and the mitigation measures you customer still unhappy with our sudoers requirement, there is maybe one more card that you can play.
If we look carefully at the sudo commands executed by Ansible during a Viya deployment playbook execution, for example in the "/var/log/secure" file we notice that Ansible is always working the same way to run tasks.
Here's an excerpt :
What do you notice?
Yes, it is always the same pattern:
Or
As Ansible is always using the same pattern to run its tasks, we could actually leverage this to have something a little bit more restrictive. For example, if we add this line in the sudoers:
In this case, the "viyadep" user is NOT allowed to run any privileged commands as "root", "sas" or "cas" UNLESS the command starts with : "/bin/sh -c echo"
For example:
BUT in the same time, the "viyadep" user will be able to run Ansible to perform the whole SAS Viya deployment!
This looks very promising, however … it is not the “magic silver bullet”. So before you try this please consider the following:
- If the sudoers account is aware of such configuration (which is not easy as the sudoers is well protected), then he can find a way to exploit this command to run any commands (he just need to make them start with “/bin/sh -c echo”.
For example:
So as per the previous recommendation , once the deployment of Viya has been completed, the root user should update the sudoers file to tighten the security of the viyadep user to the required level.
- This approach relies on the way Ansible is executing the tasks at the current version. For this blog, we used Ansible 2.4 as the version for the research. It was tested with success with system assessment and Viya deployment playbooks in a multi-machine configuration. Our research was not exhaustive so it may not work in all use cases and/or with the future releases of Ansible.
However if it is working in your situation, it might be a good "trade off" that you can reach with the customer security responsible.
Conclusion
Because of the way Ansible is working, we don’t have a complete workaround to reduce the exposure of the "sudo" configuration (like a list of commands performed during the deployment), but with the appropriate explanations, mitigations and best practices, showing that we are in control of the implications, the customer will, hopefully, temporary accept this risk to enable the deployment of Viya.
I’d like to thank @ErwanGranger for his contribution to this blog post and also our RnD colleagues for their expertise, review and comments. Hopefully, this blog post gave you enough details and information on the root/sudo requirement to improve the way you can handle it in a real world deployment.
Don't miss out on SAS Innovate - Register now for the FREE Livestream!
Can't make it to Vegas? No problem! Watch our general sessions LIVE or on-demand starting April 17th. Hear from SAS execs, best-selling author Adam Grant, Hot Ones host Sean Evans, top tech journalist Kara Swisher, AI expert Cassie Kozyrkov, and the mind-blowing dance crew iLuminate! Plus, get access to over 20 breakout sessions.
