In this post, we'll see how to change the SAS WORK path for the SAS Compute Server globally and individually for each compute context, and also how to set SAS WORK for older-style SAS Workspace servers.
No serious SAS Viya deployment should be left with the SAS WORK directories pointing to the default location.
The default path was chosen by the developers because we can be confident that it exists. It eliminates a reason your deployment might have problems when first deployed, but the default SAS WORK path is definitely not the ideal SAS WORK path. I'll explain why too.
[My thanks to Jessica Franklin, Olof Smith, Mark Miskiman and Alex Yang for their feedback on an earlier SAS-internal version of this post.]
I suspect many readers will know why you should change the SAS WORK path from the default, so we will get straight to how you do it. If you're not sure why you should do it, or if you are interested in learning more about optimising SAS WORK, read on past the how part and we will cover some other context and considerations towards the end of this post.
How to change the SAS WORK path
The following examples suppose you have created a better file system directory for temporary SAS WORK files called /saswork. Substitute the directory that you actually created wherever you see /saswork in the examples below.
Tip: You can of course use another file system path, as best suits your needs. Other examples I have seen include /sastemp/saswork, /sas/tmp and a separate, dedicated volume mounted at /opt/sas/viya/temp. However, if you do choose to use a path under /opt/sas/viya for SAS WORK, it is very important not have the path physically stored on same disk volume that is used for other content in /opt/sas/viya!
SAS Compute Server
For SAS Compute Server sessions launched by SAS Studio 5 or SAS Model Studio for example, here are two options, one global and one specific to a single Compute Context.
Globally, before deployment, in SAS Viya 3.5 on Linux: As described in SAS Usage Note 63087: Change the WORK location for the compute server in SAS Viya, to change the SASWORK and other temporary directory path globally for all sessions run by the Compute Server before deployment of SAS Viya 3.5 on Linux, add or modify a line like the line in red below, in the file sas_viya_playbook/vars.yml in your deployment playbook directory:
# Creates a workspaceserver sasenv_deployment file
FOUNDATION_CONFIGURATION:
X: COMPUTESERVER_TMP_PATH=/saswork
Note: The X above stands for the next consecutive number under FOUNDATION_CONFIGURATION.
If you make this change post-deployment, you need to re-run the playbook to incorporate this change. Changing the WORK location in this way prevents you from having to make the change again after an upgrade is performed.
Globally, after deployment: To change the SASWORK and other temporary directory path globally for all sessions run by the Compute Server after deployment, you can either follow the method described immediately above and re-run the deployment playbook. Or, you can add or modify a line in the file /opt/sas/viya/config/etc/sysconfig/compsrv/default/sas-compsrv on each ComputeServices host machine, like this:
export COMPUTESERVER_TMP_PATH=/saswork
There is no need to restart the SAS Launcher Service (or any other service) on hosts where you edited sas-compsrv. It is re-read each time a compute server session is launched. [Updated 21 Nov 2019: thanks to Jessica Franklin for correcting me on this point, and for writing Usage Note 63087: Change the WORK location for the compute server in SAS Viya, which I have partially copied here.]
This changes the SASWORK temporary directory for all Compute Contexts, unless an individual Compute Context has a SAS option overriding it, as described in the step below.
For a single Compute Context, after deployment: Open SAS Environment Manager and open the Contexts page. Change to the Compute Contexts view, and then edit one, or all of the existing contexts. Each time, in the Edit Compute Context dialog box, on the Advanced tab, enter the following in the SAS options text box:
-WORK /saswork
Then click Save.
Tip: If nothing happens when you hit the Save button, see Problem Note 63759: There is no response when you click "Save" after you edit default compute contex..., which references a hot fix that resolves this.
SAS Workspace Server, Batch Server, Compute Server and Connect Server
For the SAS Workspace Server launched from SAS Studio 4 for example, and also for the other SAS 9 style SAS servers (Batch Server, Compute Server and Connect Server), you can set the SAS Work path before deployment, or change it after deployment.
Before deployment, edit vars.yml, and change the /tmp to /saswork (or the equivalent directory that you created) in the lines that look something like the following:
SASV9_CONFIGURATION:
1: '/* this will send the SASWORK to the /sastmp location */'
2: 'WORK /tmp'
This change will affect all of the SAS 9 style servers: SAS Workspace Server, Batch Server, Compute Server and Connect Server.
After deployment, edit the sasv9_usermods.cfg file for the SAS Workspace Server, SAS Batch Server, SAS Compute Server and SAS Connect Server, all under /opt/sas/viya/config/etc/ [Updated 21Nov2019: with thanks to Mark Miskiman for his comment]:
workspaceserver/default/sasv9_deployment.cfg
batchserver/default/sasv9_deployment.cfg
compsrv/default/sasv9_deployment.cfg
connectserver/default/sasv9_deployment.cfg
For example, for the SAS Workspace Server, look in /opt/sas/viya/config/etc/workspaceserver/default/ on the hosts in the ComputeServices host group where it runs. In the sasv9_usermods.cfg file, add (your equivalent of):
-WORK /saswork
After making a change to a sasv9_usermods.cfg file, re-start the SAS Object Spawner so that it takes effect. Use systemd’s systemctl, or SysV’s service command as appropriate for your platform:
For a version of UNIX or Linux using systemd:
systemctl restart sas-viya-spawner-default
For a version of UNIX or Linux using SysV:
service sas-viya-spawner-default restart
Remember that all users of the SAS Compute Server or older-style SAS Workspace, Batch, Connect and Compute Servers require Write access to the new SAS WORK directory. Use chown and chgroup as root (or sudo chown and sudo chgroup as another user who is in sudoers) to set the ownership of the directory, and chmod (or sudo chmod) to set the directory access mode to grant those users access as follows:
chmod 777 /saswork (all users get full permissions)
chmod +t /saswork (sticky bit; ownership cannot be changed)
Confirmation
Confirm that work files are being written to the new temporary SASWORK directory (for example, /saswork) by navigating to it and inspecting its contents when SAS sessions are running in SAS Studio 4, SAS Studio V, SAS Model Studio, and so on. Confirm that all users can correctly run SAS sessions and SAS jobs, to validate the permissions on the new directory are correct.
From a SAS session running in SAS Studio 4 or SAS Studio V, signed in as a user who can run SAS sessions (for example, their account is able to log on to the host OS on the relevant host machines in the ComputeServices host group, and they have a home directory), run the following code to show the value of the WORK SAS option:
proc options option=WORK;
run;
In the log output, you should see the current WORK library path, something like:
WORK=/saswork/SAS_workuniqueidA_hostname/SAS_workuniqueidB_hostname
Why you should change the SAS WORK path
For SAS Viya deployments on Linux, the default SAS WORK path is /tmp, which is often on the boot volume along with the OS binaries and maybe the SAS binaries. If the volume containing /tmp becomes full, the OS is going to have all sorts of problems. The OS likely has a scheduled job that will clear down old files in /tmp after a couple of days or so, which is not helpful for very long-running SAS jobs that use SAS WORK. In practice this seems to be less of an issue that one might imagine - comparatively few SAS jobs run for so long that their work files get deleted, but still, it's far from ideal that they could be deleted. Also, for most serious SAS Viya deployments, the SAS WORK path needs very high I/O throughput for SAS to perform SPRE tasks efficiently, and the boot volume is unlikely to be configured for very high I/O throughput.
I have not discussed Windows so far in this post. The methods for changing the SAS WORK path on Windows are pretty much the same as on Linux, but there are a few differences. For SAS Viya deployments on Windows, the default SAS WORK path is the user's own home directory. Again, it is probably mounted either on the boot partition or on some network-accessible shared storage such as a SAN or NAS drive. SAN storage can be high-performance, but high-performance SAN storage is very expensive, and we see fewer architectures with multi-million dollar SANs than we used to five years ago, especially with the increasing shift to cloud hosting and commodity or cloud storage. The user's home directory may be on relatively low-performance disk. The same issues arise if it is the same disk as the Windows OS: filling it will cause a whole host of problems.
The two main reasons to use a separate, high-I/O throughput volume for SAS WORK are therefore:
- To improve performance of SAS compute (or SPRE) processing, which is every bit as important as CAS processing and much faster than CAS for some tasks.
- To protect other volumes in your deployment from being filled up if users fill the disk used by SAS WORK.
My GEL colleague Raphaël Poumarede recently wrote about Viya Architecture Design in GCP: the basics. In his excellent article, he recommends (specifically for GCP)... SAS WORK and SASUTIL use a local SSD / Zonal SSD persistent disk. For local SSD, stripe at least 4 preferably 8 together to get the IO throughput for each instance*. (*) Use RAID0 as using parity or mirroring RAID levels across local SSD devices does not provide any actual reliability or redundancy benefit. See this page for guidance. When placed on striped SSDs, SAS WORK and SAS UTILLOC should share a single file system.
Also linked early in Raphaël's post, see Margaret Crevar's SAS Paper 1866-2018 Important Performance Considerations When Moving SAS to a Public Cloud (for both SAS 9 and SAS Viya). Many of the considerations and recommendations are valid for on-premise deployments, or could be translated into a similar on-premise concept quite easily.
If you are interested in learning more about other useful SAS Administration tasks, have a look at my Checklist of SAS Viya Administration Tasks (task number 34).
See you next time!
