BookmarkSubscribeRSS Feed
Gregor
Obsidian | Level 7

I have inherited a complex macro structure which nested loops used to work fine with 70 iterations of the outermost loop.

Now I have 86 iterations and sas colapses (I/O -erros) after 20 iterations.

 

I solve this in only executing 15 iterations at a time:

 


%loopy_a(beginloop=1, endloop= 5,libi  = VALI_OOS);
%put xxxxx '6 - > 10';
%loopy_a(beginloop=6, endloop= 10,libi = VALI_OOS);
%put xxxxx '11 - > 15';
%loopy_a(beginloop=11, endloop= 15,libi = VALI_OOS);
%put xxxxx '16 - > 20';
%loopy_a(beginloop=16, endloop= 20,libi = VALI_OOS);
%put xxxxx '21 - > 25';
%loopy_a(beginloop=21, endloop= 25,libi = VALI_OOS);
%put xxxxx '26 - > 30';
%loopy_a(beginloop=26, endloop= 30,libi = VALI_OOS);
%put xxxxx '31 - > 35';
%loopy_a(beginloop=31, endloop= 35,libi = VALI_OOS);
%put xxxxx '36 - > 40';
%loopy_a(beginloop=36, endloop= 40,libi = VALI_OOS);

This solution is all but elegant. I would very much prefer to execute loopy_a from begin_loop =1 to end_loop = 86 at a time.

I suspect the amount of data handled is not the problem, but the looped expansion of the macro structure.

Is there a way to get around this? Can I ask sas somehow not to expand the macro structure? Or am I having another problem?

 

 

7 REPLIES 7
Tom
Super User Tom
Super User

There is not a general answer to this question. It depends on what the macro is actually doing.

It might be simple to recode the macro to work in a way that does not cause I/O errors, but we can't even make many suggestions without knowing what the macro is doing.

Gregor
Obsidian | Level 7

You aked for it :-).

 

The macro runs a loop of sub macros that also run loops that create about 300 small temporary files in the work directory:

 


%macro loopy_a(beginloop=, endloop= ,libi=);

%do aaaa = &beginloop %to &endloop;
%let bbbb = %eval(&aaaa+13);

%Let FirstMonth = &aaaa;
%Let LastMonth = &bbbb;


%put loop &aaaa. 1;

%Select_and_Sort(libi=&libi.,acht10='no',FILTERT=(TYP ne ''),TYP=G);
%put xxxxx loop &aaaa mit beginloop = &beginloop  endloop = &endloop 
%Inner_Loop1(libi=&libi.,acht10='no',FILTERT=(TYP ne ''),TYP=G);
[inner_loop 2-18]
%Inner_Loop19(libi=&libi.,acht10='no',FILTERT=(TYP ne ''),TYP=G);
%Inner_Loop_Final(libi=&libi.,acht10='no',FILTERT=(TYP ne ''),TYP=G);

%end;


%mend;

Do you need more details on the inner loops?

 

PaigeMiller
Diamond | Level 26

Also, within the macro you can delete some temporary datasets when they are no longer needed.

 

Are you sure it is I/O that causes SAS to crash? How do you know?

--
Paige Miller
Gregor
Obsidian | Level 7

Could it be helpful, to save the temporary files somewhere else, maybe even on a different server?

 

I am not sure about the REAL cause of the errors.

The I/O errors are jsut the first that appear.

 

jakarman
Barite | Level 11

The macro as generated text code is not the issue it is  compilded than fiexed in size.
The macro is generating soms SAS code and the problem is somwhere there. 

 

It seems the clean-up on every turn is not done inside the looping of the loop there  but after that.

Fix that ....

For fixing the macro you will need to go to the owner of the macro. That is your local company. Maybe your are the owner yourself now.
Get to the macro code and analyze that one instead of just using/calling it.   

---->-- ja karman --<-----
Gregor
Obsidian | Level 7

So can I stop sas somehow from compiling the big snowball?

Can I somehow implement a clean up within the loop?

Would this clean up be about the compiled code?

Can the data in work space be the problem? - Are there to many files in the work space?

Would it help to store the temporary files in another libary on the server, not locally?

Thanks.

 

 

 

 

 

Patrick
Opal | Level 21

Without seing your code and log it's not possible to give you real advice. If you can provide both to us (as attachments).

 

SAS WORK tables are physically written to a file location. They get automatically deleted when the SAS session closes. It's also possible to delete some or all tables in WORK via code.

 

I've seen in rare cases I/O error occur when there was no more space on the disk. Given that you get this error based on the number of "loops" it could be that every single loop creates huge tables in WORK and that you simply run out of disk space.

 

One thing you could try: On top of your code before you call the macros execute "options compress=yes;"

 

That's not the real solution but may-be it could get you over this hurdle for the moment.

 

If it's a space problem with WORK then the "real solution" would be:

-  get more disk space for WORK

-  or even better: amend your code so that it doesn't use that much space

 

 

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

SAS Enterprise Guide vs. SAS Studio

What’s the difference between SAS Enterprise Guide and SAS Studio? How are they similar? Just ask SAS’ Danny Modlin.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 7 replies
  • 1433 views
  • 3 likes
  • 5 in conversation