SAS on AWS shows insufficient space

[BalassaSamuelso]

Dear all, 

I uploaded a few files to SAS University edition on AWS and tried a few calculations. The last proc freq shows me

 ERROR: Insufficient space in file WORK.'SASTMP-000000072'n.UTILITY.
 ERROR: File WORK.'SASTMP-000000072'n.UTILITY is damaged. I/O processing did not complete.

I even deleted some of my files and resized the EBS to 80 GB. I don't know how you can check the free space on that EBS in SAS, but there should still be plenty left!

[Kurt_Bremser]

Are you sure that a PROC FREQ causes this? Could you please post more of your log?

In my experience, PROC FREQ does not create a utility file, but does it's job in memory.

[BalassaSamuelso]

I executed 

proc freq data=abc.cjreimport NLEVELS;
	tables OrderID UserID;
run;

and got this in the log

 1          OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 55         
 56         proc freq data=abc.cjreimport NLEVELS;
 57         tables OrderID UserID;
 58         run;
 
 ERROR: Insufficient space in file WORK.'SASTMP-000000025'n.UTILITY.
 ERROR: File WORK.'SASTMP-000000025'n.UTILITY is damaged. I/O processing did not complete.
 NOTE: The SAS System stopped processing this step because of errors.
 NOTE: There were 7045546 observations read from the data set ABC.CJREIMPORT.
 NOTE: PROCEDURE FREQ used (Total process time):
       real time           1:23.24
       cpu time            13.63 seconds
       
 59         
 60         OPTIONS NONOTES NOSTIMER NOSOURCE NOSYNTAXCHECK;
 72         

[Kurt_Bremser · Accepted solution]

Try

PROC SQL;
select count(distinct(OrderID) from abc.cjreimport;
quit;

To get the cardinality of OrderID. I guess you try to count something that isn't suited for counting with FREQ, as almost every observation belongs to a different OrderID, so FREQ needs much more memory than usual.

How many observations does abc.cjreimport have?

[BalassaSamuelso]

Perfect, thank you so much! That was the solution.

The dataset has around 7 million observations. I even tried updating my instance to 4GB RAM, same error there.