Desktop productivity for business analysts and programmers

File system /sas ,/sas/saswork /sas/sasutil reached 100%

Reply
Frequent Contributor
Posts: 148

File system /sas ,/sas/saswork /sas/sasutil reached 100%

Hi Friends.

We are using Solaris machine for our SAS.

and I found suddenly File system /sas, /sas/saswork, /sas/sasutil reached 100%.not sure how suddenly this happened.

anyone have any idea how can i check which process or job or which user has occupied more space and that made the file system 100%.

which command I need to use,I have used several commands like du-a,du -s,du -hd,du -k, du -a /sas | sort -n -r | head -n 10...etc

but these did not help me.

I want to find which job or which user occupied more space because of which suddenly the space reached 100%.

I would really appreciate if any one can provide your valuable suggestion which commands i should use and where i need to check

Thanks in advance

Grand Advisor
Posts: 17,311

Re: File system /sas ,/sas/saswork /sas/sasutil reached 100%

Run a file listing on sas/work and check the owners of the largest files or who owns the largest portion. It may not be one user, it could be having 3 users running large jobs at the same time. Or the space has not been cleaned up in a while and there lots of stuff lying around.

Respected Advisor
Posts: 3,059

Re: File system /sas ,/sas/saswork /sas/sasutil reached 100%

I wouldn't necessarily assume that it is one user / job that has filled up SASWORK. Look at the number of of _TDnnnn folders in SASWORKIf there are a lot and the dates they are created are spread over time it could be that these are just SAS jobs/sessions that did not properly clear their WORK space.

Many SAS sites run the SAS CLEANUP process daily to deal with this very issue.

Frequent Contributor
Posts: 148

Re: File system /sas ,/sas/saswork /sas/sasutil reached 100%

Thanks for your prompt replies

Till evening the space was at 30 % and suddenly it reached above 90% and slowly within couple of hours it reached 100 %.

and suddenly after some time all the space came to normal 30 %.

/sas is the main mounting point on which all other mounting points falls as below

/sas/guide

/sas/saswork

/sas/util

when I watched  all these locations were at 90 % and slowly they reached 100%.

so In this kind of situation what is the immediate step i need to take and how can i find which files or users occupying more space and how can i resolve this issue immediately.

and what are the file systems i need to check

/saswork location i did'nt find any files with the name _TDnnnn folders and i did'nt find any files with bigger than in mega bytes

I want to find exactly how can i find which file system and job or user occupied more space which has increased  space  suddenly to maximum

Respected Advisor
Posts: 3,059

Re: File system /sas ,/sas/saswork /sas/sasutil reached 100%

It is not clear from your response whether it is /saswork that is the problem or not. Please clarify.

Every SAS job will create a directory under SASWORK like /saswork/_TDnnnn_servername_    If there aren't any directories like this then where is SAS writing its WORK files?

In a SAS session check the properties of the WORK libname to confirm your WORK location.

Frequent Contributor
Posts: 148

Re: File system /sas ,/sas/saswork /sas/sasutil reached 100%

Yes the problem is with all below mounting points which reached 100 %

/sas

/sas/saswork

/sas/sasutil

Esteemed Advisor
Posts: 5,194

Re: File system /sas ,/sas/saswork /sas/sasutil reached 100%

Talk to your UNIX admin for a way to monitor disk usage per user.

Data never sleeps
Esteemed Advisor
Posts: 6,636

Re: File system /sas ,/sas/saswork /sas/sasutil reached 100%

For shared directories (one of them being the SAS work location, the other /home, where the sasuser libs are usually located), quota management for the users is a must. And I mean MUST.

Quota management will mean that a user with an ill-formed request will experience an individual loss of service, without denying the service to all others.

Once you have a near-to-full or full condition, I would run something like

find /sas -type f -ls

in a filename pipe and read the output into a SAS dataset. From that, you can use your SAS skills to find the offending file(s).

Be aware that files without a directory entry (but that still use disk space) may exist in UNIX, they can be found by executing

fuser -dcV path

where path must be in the filesystem (not just path) in question. Gives you a list of processes that hold a handle on a completely unlinked file, if such exist.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Respected Advisor
Posts: 3,823

Re: File system /sas ,/sas/saswork /sas/sasutil reached 100%

It could be that a user run some "insane" job, let's say something like a cartesian product between two very big tables. That could fill-up saswork & sasutil quite a bit. And once the session finishes all the space would get freed-up almost immediately.

What "disturbs" me a bit: You say that space usage "jumped" from 30% to 90% in almost no time. Assuming your disk space is considerable writing to disk filling up 60% of space should take some time. So I'm asking myself if there had been some infrastructure work been going on in the background and you simply didn't always have the same amount of disk space available. I would have a chat with your UNIX system admin to find out what happened.

Frequent Contributor
Posts: 148

Re: File system /sas ,/sas/saswork /sas/sasutil reached 100%

The space has not jumped immediately from 30% to 100%,it slowly increasing % by %  within 1/2 hrs it went to 100%.

Esteemed Advisor
Posts: 6,636

Re: File system /sas ,/sas/saswork /sas/sasutil reached 100%

In this kind of situation, there should be at least one user complaining about failing jobs/steps. How many users are working on your installation? Maybe just asking all users if they have problems could reveal the source of the disk full condition.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Frequent Contributor
Posts: 148

Re: File system /sas ,/sas/saswork /sas/sasutil reached 100%

Its on Development environment and we have very few users working may be 4 or 5 users on this and no one has complained about failing jobs the space is going back to normal immediately in few mins after it touches 100%.

Esteemed Advisor
Posts: 6,636

Re: File system /sas ,/sas/saswork /sas/sasutil reached 100%

Then I'd write a shell script that checks for a certain usage level, and if a treshold is reached, executes a du -sk in the filesystem, appends the result to a file, and notifies you. Run this script every minute using cron, and, once you have identified where the large files are placed, modify the script to dig deeper. Sooner or later you will find the file(s), and the user responsible.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Valued Guide
Posts: 3,206

Re: File system /sas ,/sas/saswork /sas/sasutil reached 100%

When your users are of the type analytics miners, then do not use any quotas you can isolate them on a dedicated saswork filesystem.
I am against quotas because you are going into a fight against users needing to do their work. When hurring them they will look for other ways to achieve their goals.

One side of the company is keeping the other side busy without having any business efficiency (job creation). 


When you have predictable workload that is have verified and tested sas programs running. Their behavior is mostly planning. Only not noticed program bugs will cause errors. The user having problems will complain about those.

---->-- ja karman --<-----
Esteemed Advisor
Posts: 6,636

Re: File system /sas ,/sas/saswork /sas/sasutil reached 100%

Well, it's my job to identify the needs of users and set their quotas accordingly. Powerusers are assigned to a different class and get additional workspaces, but I still prevent them from clogging up resources that they share with other users.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Ask a Question
Discussion stats
  • 42 replies
  • 1235 views
  • 2 likes
  • 8 in conversation