topic Re: SAS on AWS shows insufficient space in Administration and Deployment

SAS on AWS shows insufficient space

BalassaSamuelso — Mon, 15 Feb 2016 22:40:42 GMT

Dear all,

I uploaded a few files to SAS University edition on AWS and tried a few calculations. The last proc freq shows me

 ERROR: Insufficient space in file WORK.'SASTMP-000000072'n.UTILITY.
 ERROR: File WORK.'SASTMP-000000072'n.UTILITY is damaged. I/O processing did not complete.

I even deleted some of my files and resized the EBS to 80 GB. I don't know how you can check the free space on that EBS in SAS, but there should still be plenty left!

Re: SAS on AWS shows insufficient space

Kurt_Bremser — Tue, 16 Feb 2016 11:12:11 GMT

Are you sure that a PROC FREQ causes this? Could you please post more of your log?

In my experience, PROC FREQ does not create a utility file, but does it's job in memory.

Re: SAS on AWS shows insufficient space

BalassaSamuelso — Tue, 16 Feb 2016 12:21:07 GMT

I executed

proc freq data=abc.cjreimport NLEVELS;
	tables OrderID UserID;
run;

and got this in the log

 1          OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 55         
 56         proc freq data=abc.cjreimport NLEVELS;
 57         tables OrderID UserID;
 58         run;
 
 ERROR: Insufficient space in file WORK.'SASTMP-000000025'n.UTILITY.
 ERROR: File WORK.'SASTMP-000000025'n.UTILITY is damaged. I/O processing did not complete.
 NOTE: The SAS System stopped processing this step because of errors.
 NOTE: There were 7045546 observations read from the data set ABC.CJREIMPORT.
 NOTE: PROCEDURE FREQ used (Total process time):
       real time           1:23.24
       cpu time            13.63 seconds
       
 59         
 60         OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 72

Re: SAS on AWS shows insufficient space

Kurt_Bremser — Tue, 16 Feb 2016 12:57:54 GMT

Try

PROC SQL;
select count(distinct(OrderID) from abc.cjreimport;
quit;

To get the cardinality of OrderID. I guess you try to count something that isn't suited for counting with FREQ, as almost every observation belongs to a different OrderID, so FREQ needs much more memory than usual.

How many observations does abc.cjreimport have?

Re: SAS on AWS shows insufficient space

BalassaSamuelso — Tue, 16 Feb 2016 13:07:41 GMT

Perfect, thank you so much! That was the solution.

The dataset has around 7 million observations. I even tried updating my instance to 4GB RAM, same error there.