08-25-2016 08:57 AM
We are facing an issue in which the SAS WORK directory is getting fulled or near to full quite often. In order to track that which SAS program or SAS user is causing the issue, we have configured SAS Environment Manager. But SAS environment manager gives us the trend of usage in last few day but we are not able to find who(SAS program and SAS User) is using SAS WORK directory more. Please let us know if someone has done similar kind of tracking before..Thanks!!
08-25-2016 09:10 AM
I tackled the problem from a different angle.
For the filesystem that contains the WORK location (and only the WORK location!), I have specified a reasonable user quota class as default that allows each user to go about their work. When a user exceeds his/her quota, the step fails with a "file is full and maybe damaged" note.
Usually they then get in contact with me (I'm also the data warehouse helpdesk), giving me the opportunity to fix their mistake.
(Most of the time, such events are caused by improper SQL joins or erroneously included columns like those resulting from a faulty Excel import)
In my experience, all shared disk resources must be protected with quotas; some people simply eat disk space just because it is there.
08-25-2016 09:21 AM
With regards to automatically find offending jobs/sessions, it is quite hard to infer the physical dataset size from what is written to the SAS log.
As Environment Manager can "only" trigger an alert if SASWORK is exceeding a certain treshold value, it is still up to you to determine which SAS session was/is the culprit.
This usually means using the operating system's tools, like
# cd /data/saswork # du -sk *|sort -nr|head
This gives you the "biggest" WORKs; you can then find the process responsible, as the process number is coded in hex into the directory name.
# ls -l SAS_workXXXXXXXXXXX
will also reveal the owner of the offending directory
08-25-2016 09:33 AM
This depends mainly on the operating system of your SAS server.
I could only give you hints for AIX, as we use that here.
You will need Adminstrator/superuser privileges to do it, anyway.
So you may be served best by taking this to your IT people.
08-25-2016 09:40 AM
might be good starting points.
08-25-2016 03:40 PM
Are you running the SAS Cleanwork utility on your SAS Server? It is best practice to run this daily to delete WORK folders left by SAS sessions that did not close cleanly. If you do not run this, orphan WORK folders can accumulate over time and fill up your WORK space.
08-25-2016 01:17 PM
not sure this helps .. I found a paper: http://support.sas.com/resources/papers/proceedings12/099-2012.pdf, which might be helpful.
It is from 2012 - so pretty old - but the code suggested works, I ran it.
Maybe this helps to get closer to what you are looking for?
Another paper - old too: http://support.sas.com/resources/papers/proceedings15/3196-2015.pdf
08-25-2016 04:06 PM
08-25-2016 10:59 PM
OK, good to hear. Also in our Windows SAS environment we have SAS dataset compression turned on by default so that includes WORK libraries. In many datasets we get over 70% compression due to having long character columns. The CPU overhead for us is worthwhile due to the IO and space saving . I suggest you check this out too.
08-29-2016 08:59 AM
If you're interested in a third party commercial solution, our Enterprise Session Monitor for SAS addresses the exact challenge you're having (easily finding the user or program that is currently filling your SASWORK/UTIL disk). It measures the SASWORK size for each session in real time and at any time gives you a breakdown of exactly how that space is being used with one click. It can also tell you how that space was being used at any arbitrary point in time over the last 4 weeks, as well as doing a bunch of other stuff like automatically associating workspace sessions to the metadata users that own them, letting users keep an eye on what resources the sessions they own are currently using on the server, allowing superusers to clean up unwanted sessions very easily, etc. etc.
If you were on a UNIX-based OS, as a workaround I would suggest a couple of script-based solutions similar to what Kurt has suggested. Turning on SASWORK compression is a good idea but I'd definitely want to profile your disk throughput performance and look at CPU headroom before flicking the switch in production.
Hope that helps.