DATA Step, Macro, Functions and more

Insufficient Memory Issue?

Reply
Super Contributor
Posts: 436

Insufficient Memory Issue?

When I try to sum the data I get an error (below).  I’ve tried closing the EGuide session, tried logging off and on, and just now tried a Select Distinct of only the 3 columns that are just text.

 

It is a big, but not crazy huge table 14MM rows, only 9 columns.

 

53         SELECT PARENT_NM, COUNT(DISTINCT PID_KEY) FORMAT=COMMA11. AS IDS, SUM(DOLLARS) FORMAT=DOLLAR17. AS DOLLARS
54         FROM DAS.FINAL
55         GROUP BY PARENT_NM
56         ;
ERROR: File WORK.'SASTMP-000000136'n.INDEX is damaged. I/O processing did not complete.
ERROR: :Unable to insert into temporary index while processing summary functions.

It’s occurring with any of the datasets that stem from the transaction table I’m bringing back.  I did run this within EG. 

 

The original log file I did not save, other than copying the error above.  It takes hours for the server to run its course and return the error. Wondering if there is corrupt data somewhere in that field.'

 

DEFINITELY IT’S THE GROUP BY PARENT_NM that is causing issue. Because when I remove 'GROUP BY PARENT_NM' , code is running fine.

 

I actually don’t have access to run in UNIX, so I can't test it there. Appreciate if someone guide me to overcome this issue.

 

 

Super User
Posts: 19,878

Re: Insufficient Memory Issue?

Try using a data step or Proc to do your calculation instead? 

 

I'd also check with your IT group, the issue may not be your computer. I'm assuming that EG is actually passing the code to a server and not running locally m

Respected Advisor
Posts: 4,173

Re: Insufficient Memory Issue?

[ Edited ]

SAS EG is the client. Your code executes on the Server side (=Unix for you).

 

I strongly assume that the error you encounter is caused by insufficient disc space (not memory) for WORK or UTILLOC.

Contact the SAS Admin at your site so these disk areas get either cleaned up or get additional space allocated.

 

Super User
Posts: 7,868

Re: Insufficient Memory Issue?

Run (or have run) the cleanwork utility on the work location.

If that doesn't fix it, optimize the code by using a proc sort and a data step (SQL creates an overly large utility file).

If that doesn't work, you need more disk space in work.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Ask a Question
Discussion stats
  • 3 replies
  • 404 views
  • 0 likes
  • 4 in conversation