Architecting, installing and maintaining your SAS environment

SAS Work directory usage

Reply
Contributor
Posts: 22

SAS Work directory usage

Hi,

 

We are facing an issue in which the SAS WORK directory is getting fulled or near to full quite often. In order to track that which SAS program or SAS user is causing the issue, we have configured SAS Environment Manager. But SAS environment manager gives us the trend of usage in last few day but  we are not able to find who(SAS program and SAS User) is using SAS WORK directory more. Please let us know if someone has done similar kind of tracking before..Thanks!!

Super User
Posts: 6,928

Re: SAS Work directory usage

I tackled the problem from a different angle.

For the filesystem that contains the WORK location (and only the WORK location!), I have specified a reasonable user quota class as default that allows each user to go about their work. When a user exceeds his/her quota, the step fails with a "file is full and maybe damaged" note.

Usually they then get in contact with me (I'm also the data warehouse helpdesk), giving me the opportunity to fix their mistake.

(Most of the time, such events are caused by improper SQL joins or erroneously included columns like those resulting from a faulty Excel import)

 

In my experience, all shared disk resources must be protected with quotas; some people simply eat disk space just because it is there.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Super User
Posts: 6,928

Re: SAS Work directory usage

With regards to automatically find offending jobs/sessions, it is quite hard to infer the physical dataset size from what is written to the SAS log.

As Environment Manager can "only" trigger an alert if SASWORK is exceeding a certain treshold value, it is still up to you to determine which SAS session was/is the culprit.

This usually means using the operating system's tools, like

# cd /data/saswork
# du -sk *|sort -nr|head

(UNIX)

This gives you the "biggest" WORKs; you can then find the process responsible, as the process number is coded in hex into the directory name.

# ls -l SAS_workXXXXXXXXXXX

will also reveal the owner of the offending directory

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Contributor
Posts: 22

Re: SAS Work directory usage

Thanks Kurt!!

Could you provide the steps you did for defining user quota class then, we would try that out. It seems to be a feasible solution through.
Super User
Posts: 6,928

Re: SAS Work directory usage

This depends mainly on the operating system of your SAS server.

I could only give you hints for AIX, as we use that here.

You will need Adminstrator/superuser privileges to do it, anyway.

So you may be served best by taking this to your IT people.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Contributor
Posts: 22

Re: SAS Work directory usage

Thanks we are using windows 10 and I have administrator rights on the server.
Super User
Posts: 6,928

Re: SAS Work directory usage

Then this

https://msdn.microsoft.com/en-us/library/windows/desktop/aa365228%28v=vs.85%29.aspx

and this

http://www.isunshare.com/windows-10/enable-quota-management-for-hard-disk-in-windows-10.html

might be good starting points.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Contributor
Posts: 22

Re: SAS Work directory usage

Thanks I will check these and come back in case of any issue or question!!
Super User
Posts: 3,101

Re: SAS Work directory usage

Are you running the SAS Cleanwork utility on your SAS Server? It is best practice to run this daily to delete WORK folders left by SAS sessions that did not close cleanly. If you do not run this, orphan WORK folders can accumulate over time and fill up your WORK space.

 

http://support.sas.com/documentation/cdl/en/hostwin/67962/HTML/default/viewer.htm#n0j43jculk2kasn1qh...

 

 

 

 

SAS Super FREQ
Posts: 291

Re: SAS Work directory usage

Hi,

 

not sure this helps .. I found a paper: http://support.sas.com/resources/papers/proceedings12/099-2012.pdf, which might be helpful.

It is from 2012 - so pretty old - but the code suggested works, I ran it.

Maybe this helps to get closer to what you are looking for?

 

Another paper - old too:  http://support.sas.com/resources/papers/proceedings15/3196-2015.pdf

(old) http://www.okstate.edu/sas/v8/sashtml/cms/zsf-work.htm

 

Thanks

Anja

 

 

Contributor
Posts: 22

Re: SAS Work directory usage

Thanks for the link..I will check these..

And yes we are running SAS cleanwork utility every night. But the issue comes now and then in day time and the challenge is to find the user or program which makes WORK directory out of space. And just for information, we have around 644 GB space available in SAS WORK directory.
Super User
Posts: 3,101

Re: SAS Work directory usage

OK, good to hear. Also in our Windows SAS environment we have SAS dataset compression turned on by default so that includes WORK libraries. In many datasets we get over 70% compression due to having long character columns. The CPU overhead for us is worthwhile due to the IO and space saving . I suggest you check this out too. 

Frequent Contributor
Posts: 91

Re: SAS Work directory usage

Amit,

 

If you're interested in a third party commercial solution, our Enterprise Session Monitor for SAS addresses the exact challenge you're having (easily finding the user or program that is currently filling your SASWORK/UTIL disk). It measures the SASWORK size for each session in real time and at any time gives you a breakdown of exactly how that space is being used with one click. It can also tell you how that space was being used at any arbitrary point in time over the last 4 weeks, as well as doing a bunch of other stuff like automatically associating workspace sessions to the metadata users that own them, letting users keep an eye on what resources the sessions they own are currently using on the server, allowing superusers to clean up unwanted sessions very easily, etc. etc.

 

Check out this screencast on Vimeo - relevant bits are around 50 seconds in, and then again around 3 minutes in. Product page can be found here.

 

 

If you were on a UNIX-based OS, as a workaround I would suggest a couple of script-based solutions similar to what Kurt has suggested. Turning on SASWORK compression is a good idea but I'd definitely want to profile your disk throughput performance and look at CPU headroom before flicking the switch in production. 

 

Hope that helps.

 

Nik

Introduction to Enterprise Session Monitor for SAS, demonstrating basic monitoring and session management capabilities. For more information visit boemskats.com/esm
Ask a Question
Discussion stats
  • 12 replies
  • 710 views
  • 2 likes
  • 5 in conversation