Issues with SAS Code created for Server space

[swati428]

Hi All,

I am having  requirement with the space management,for that i got below code.which has developed in such a way that

it triggers with sas program when threshold reaches max point and from there it alerts user who is occupying max space.

but i have trouble in generating the output which am expecting.

when am executing the first code its not giving the output of all files in /sas directory,its giving only files which are with sas user id

and there are so many datasets generetaed by other users which are very large datasets like 1GB-60 GB datasets are not showing.

I mean no datasets it is showing in the output created by users.

please help me and let me know whether the code am executing i did something wrong

For the first line of below code  I have  replaced it with " df -h /sas |tail -1 " ( /sas is the file system on our server)

and last line of the codee x '/XXXXXXX/XXXXXX/every4hour/unix_user_report.sas &'; I replaced with

/sas/eg/coding/unix_user_report.sas &"; ( here I saved  the second program unix_user_report.sas program on sas server directory /sas/eg/coding)

/*First step: Find current available space on a File System, if it breaches to threshold point (considered here is
85%) it triggers sas program to capture all users file content*/

%macro filesyssize;
filename pro11 pipe " df -h /<filesystem> |tail -1 "; /******** captures available space percentage on a file
system*******/;
*** parse information from unix ***;
data usr;
length filesys $20 gb_blocks 8 free_space 8 percen_used $8 iused 8 percen_iused $8 mount $20;
***** read standard output from unix command ********;
infile pro11 lrecl=162 missover pad;
input filesys gb_blocks free_space percen_used iused percen_iused mount;
run;
data usr1;
set usr;
date_time= put(datetime(),datetime18.);
percen_used1 = input(substr(percen_used,1,length(percen_used)-1),8.);
run;
proc sql noprint;
select percen_used1 into :percen from usr1;
quit;
%if &percen > 85 %then
%do;
x '/XXXXXXX/XXXXXX/every4hour/unix_user_report.sas &';
%end;
%mend filesyssize;
%filesyssize;

here in the 2nd program below the 1st line of the code x "find /<filesystem> -xdev -size +1024 -exec ls -l {} ';' > /tmp/test.txt"; i replaced with x "find /sas -xdev -size +1024 -exec ls -l {} ';' > /tmp/test.txt";

and 4th line of the code infile "/tmp/XXXXXXXX " lrecl=2000 truncover firstobs=1; I replaced it with 

infile "/tmp/test.txt " lrecl=2000 truncover firstobs=1;

/*Second Step: create data step and generate the final report.
/**** unix_user_report.sas start from here *******/
/*** Get sudo access on find commnad or run with root to avoid permission issues on user folders and here
considered to get all files size more than 1 GB *****/

x "find /<filesystem> -xdev -size +1024 -exec ls -l {} ';' > /tmp/test.txt";
data processes (drop=__: fper t filesize t);
format UID $20. Filesize Location $500. Gb;
infile "/tmp/XXXXXXXX " lrecl=2000 truncover firstobs=1;
input;
__line=_infile_;
Fper=compress(scan(__line,1,' '),' ');
t=compress(scan(__line,2,' '),' ');
UID=compress(scan(__line,3,' '),' ');
Group=compress(scan(__line,4,' '),' ');
Filesize=compress(scan(__line,5,' '),' ');
Month=compress(scan(__line,6,' '),' ');
Date = compress(scan(__line,7,' '),' ');
Time=compress(scan(__line,8,' '),' ');
Location=compress(scan(__line,9,' '),' ');
GB=(filesize)/(1024*1024*1024);
File_Name=scan(location,-1,'/');
run;

and here for the below macro code in the ods html body where i should create this "/aalsasmon_apm/Reports/&UID..html"

do i need to create this directory on sas server or EG and after i create do i need to create file as UID.html in this directory 

aalsasmon_apm/Reports

%macro create(UID);
ods html body="/aalsasmon_apm/Reports/&UID..html" style=sasweb;
proc print data=processes (keep=UID GB Location File_Name Month Date Time) noobs;
where UID="&UID";
title "Report for a &UID on /aalpprg filesystem ";
run;
ods html close;
%mend;

proc sort data=processes out=newdsn nodupkey;
by UID;
data _null_ ;
set newdsn;
by UID;
call execute('%create('||UID||')');
run;
data one;
set newdsn;
retain fmtname '$test';
rename UID=start ;
label='<A '||compress('HREF="'||UID||'.html')
||'">'||trim(UID)||'</A>';
run;

proc format cntlin=one;
run;

proc sql;
create table users as select distinct(UID) from processes;
select count(*) as obscnt into :cnt from users;
select UID into: uid1 separated by '","' from users;
quit

Thank you

[Tom]

I think that you might have a question in there, but it is hard to see. Are you having trouble reading the output of your FIND command into a SAS dataset? Are you having trouble making a report from the dataset? Or if not that then what exactly is the question?

[swati428]

Tom,


Thanks for your prompt response.

I apologize for not clear with the Question.


basically am looking for Managing space for our server,I found below artcile.


The developer has developed the code for Managing space on the AIX Server.

This code actually finds current available space on a File System, if it reaches to threshold point (considered here is 85%) it triggers sas program to capture all users file content and it will alert the user via email who is consuming more space.

actually am trying to make it work for our file system on the Solaris server like for /sas mounting point but its not giving the output as expected.

I would really appreciate if you can assist me with this.

here is the below article.



http://support.sas.com/resources/papers/proceedings15/3100-2015.pdf


Thank you
Swathi

[Kurt_Bremser]

First of all, the best method (IMO) for space management in a shared resource is the introduction of individual quotas. Be proactive instead of reactive.

Second, to find the files of all users in a given filesystem, you need to have the permissions to read their directories, so you need to take a look at the umask setting(s) of the users, and which are their primary groups.

Now, if you allow users access to other users workspaces, you degrade the overall security of the system. My preferred solution is to have a script running periodically from the superuser's crontab (that eleminates the permission problem) and place the result in a dedicated location where only the (system or data warehouse) administrators have access to (that reduces the security problem). From there, you can process the data further. Or you could do the treshold detection in the script itself (keep in mind that AIX is, like all UNIXes, extremely powerful out of the box and provides multiple scripting options like perl or python in addition to just bash) and also send the mails from there.