BookmarkSubscribeRSS Feed
odmhx
Calcite | Level 5
I have a long and big production job run on the weekends. The space used at the peak (sort and merge) varies from 200 gb to 240 gb. However when it's close to the year end, as the input files become bigger and bigger, causing the peak use of space exceeding the 250 gb limit thus abend.

Can anyone give me some suggestion/coding to monitor the peak space usage of the program, i.e, write it to the log so I can pro-actively monitor the space issue and request more space before it reaches the limit?

I am running this job in the background on an AIX server.

Thank you in advance
4 REPLIES 4
DanielSantos
Barite | Level 11
Hi.

There is more than a way of doing this, but I think, any of them will rely on reporting of your OS.

The following AIX command will report total, available and used storage under a specified /dir

[pre]df -g /dir[/pre]
the -g modifier is for GB unit

I guess you could pipe the result into SAS and output it to the log, something very simple like this:

[pre]%macro df(DIR);
data _null_;
infile 'df -g data' pipe;
input;
put _infile_;
run;
%mend;[/pre]
Then you just have to call the macro, for example, before/after every datasetp/proc submited in your program, like this:

[pre]proc sort data=...
...
run;

%df(/home/c085442/data);

data ...
...
run;

%df(/home/c085442/data);[/pre]
Another way would be to write a shell script that using a specified time interval would issue the command and then write the result (timestamped) into a file.
From there you could issue some kind of warning or even send a mail to a list of specified recipients.

Hope it helps.

Cheers from Portugal

Daniel Santos @ www.cgd.pt
odmhx
Calcite | Level 5
Thank you very much for your reply. I also thought of your first solution, but didn't use it because it doesn't run in the middle of the data step or proc sort where the abend happens. When the input file size increases, it seems that the working storage expands exponentially in the merge and sort steps, that makes gestimate of the space usage very hard.

Your second suggestion is very helpful. I'll try that.


> Hi.
>
> There is more than a way of doing this, but I think,
> any of them will rely on reporting of your OS.
>
> The following AIX command will report total,
> available and used storage under a specified /dir
>
> [pre]df -g /dir[/pre]
> the -g modifier is for GB unit
>
> I guess you could pipe the result into SAS and output
> it to the log, something very simple like this:
>
> [pre]%macro df(DIR);
> data _null_;
> infile 'df -g data' pipe;
> input;
> put _infile_;
> run;
> %mend;[/pre]
> Then you just have to call the macro, for example,
> before/after every datasetp/proc submited in your
> program, like this:
>
> [pre]proc sort data=...
> ...
> run;
>
> %df(/home/c085442/data);
>
> data ...
> ...
> run;
>
> %df(/home/c085442/data);[/pre]
> Another way would be to write a shell script that
> using a specified time interval would issue the
> command and then write the result (timestamped) into
> a file.
> From there you could issue some kind of warning or
> even send a mail to a list of specified recipients.
>
> Hope it helps.
>
> Cheers from Portugal
>
> Daniel Santos @ www.cgd.pt
sbb
Lapis Lazuli | Level 10 sbb
Lapis Lazuli | Level 10
Investigate using the SAS-maintained view identified as SASHELP.VTABLE, where you will find useful statistics on this type of information. Search the SAS support website and do a PROC CONTENTS on the view for details.

Scott Barry
SBBWorks, Inc.
odmhx
Calcite | Level 5
Thank you. I will.

> Investigate using the SAS-maintained view identified
> as SASHELP.VTABLE, where you will find useful
> statistics on this type of information. Search the
> SAS support website and do a PROC CONTENTS on the
> view for details.
>
> Scott Barry
> SBBWorks, Inc.

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 4 replies
  • 881 views
  • 0 likes
  • 3 in conversation