topic Re: Clear log every N interation in SAS Programming

Clear log every N interation

hhchenfx — Mon, 28 Feb 2022 00:46:15 GMT

Hi,

I have a macro and I run this marcro for every row in VAR_LIST dataset.

The thing is, there are thousands of row in Var_list and the log will get full quickly during the EXECUTE step in my code.

Is there any way we can put a counting variables and request SAS to clear log every, say 1000 execution.

I don't want to save log in a separated txt file.

Can you please help?

Thanks,

HHC

*GET VAR_LIST;
ods select none;
proc contents data=have out=var_list (keep = name);
run;
ods select all;

*EXECUTE------------------------- Can we clear log every 1000 execution?;
data execute_all;
set var_list;
where name ne 'n';

str = catt('%lastdate(column=', name, ');');
call execute(str);
run;

Re: Clear log every N interation

whymath — Mon, 28 Feb 2022 01:11:17 GMT

How about using "options nonotes" to prevent to print notes?

Re: Clear log every N interation

hhchenfx — Mon, 28 Feb 2022 01:15:44 GMT

Oh, I still want to see the log once in a while.

Re: Clear log every N interation

SASKiwi — Mon, 28 Feb 2022 01:27:12 GMT

If you run your program in batch mode your SAS log never becomes "full" unless you run out of disk space.

Re: Clear log every N interation

whymath — Mon, 28 Feb 2022 02:15:46 GMT

You may want to know the "dmslogsize=" system option(link). You can set it to 999999 and this is the maximum value.

Re: Clear log every N interation

Tom — Mon, 28 Feb 2022 02:32:04 GMT

Why don't you want the log in a separate file?

Have you done normal steps to prevent the log from being filled with unneeded lines?
Turn off FULLSTIMER option.

Turn off MLOGIC and SYMBOLGEN option.

If your macro is stable then turn off MPRINT option also.

Prevent CALL EXECUTE() from pushing the code the macro generates into the stack to run by using %NRSTR() in your string.

Also don't append that extra ';' after the macro call. You don't need it since obviously the macro is generating multiple statements already or you wouldn't be worried about the size of the log.

call execute(cats('%nrstr(%lastdate)(column=', name, ')'));

And of course the most obvious thing, change your process to do the whole dataset at once instead of variable by variable.

What does %LASTDATE() do?

Re: Clear log every N interation

Quentin — Mon, 28 Feb 2022 14:56:18 GMT

I'm generally opposed to the idea of turning off writing to the log or automatically clearing a log.

People who do that often think "I know this code works now, so I don't need the log." I think that is a dangerous thought. Much safer to write the log to a file.

That said, can you tell us which IDE you are using? If you are using Display Manager (PC SAS) I believe you can clear the log by submitting

dm 'log;clear;';

And since you are using CALL EXECUTE to generate your macro calls, you could also use CALL EXECUTE to execute the DM command after N loops of the data step, e.g. :

data _null_ ;
  do i=1 to 10 ;
    put i= ;

    call execute('data foo ; x=' || i ||' ;run ;') ;

    if mod(i,4)=0 then do ;
      call execute("dm 'log ;clear ;';") ;
    end ;
  end ;
run ;

I doubt there is a similar way to dynamically clear the log while a DATA step is executing in Studio or EG, because they are essentially batch submissions rather than interactive (as I understand it).

And generally, I would think clearing the log in the middle of a DATA step is a #VeryBadIdea. You never know what can go wrong (in a program you wrote, the data being read, or the computer it's running on.

Re: Clear log every N interation

PaigeMiller — Mon, 28 Feb 2022 15:52:56 GMT

@Quentin wrote:

I'm generally opposed to the idea of turning off writing to the log or automatically clearing a log.

Yes, I agree completely, Quentin.

And generally, I would think clearing the log in the middle of a DATA step is a #VeryBadIdea. You never know what can go wrong (in a program you wrote, the data being read, or the computer it's running on.

Yes, I agree completely, Quentin.

So if your program is likely to create a very very very very big log, write the log to a file instead of to the LOG window.

I have set up, in Base SAS Display Manager a keystroke combination that clears the log whenever I want (but it doesn't work while the program is running). So its a manual thing. I also choose the submit command as part of this keystroke combination, so it clears the previous log and a new log is created. Of course I only do this after checking the log from my previous run, and have fixed (I hope) the errors, and am ready to run it again and I don't need the old log.

If you type Keys into the command bar at the top left, a window appears that lets you assign specific commands to specific keystrokes. Next the SHF F1 (Shift-F1), I have entered

cle log; submit

and so now when I press Shift-F1, my previous log is cleared and the code in the Enhanced editor window (or the selected code in the Enhanced Editor window) starts running. Slick trick.

Re: Clear log every N interation

hhchenfx — Mon, 28 Feb 2022 16:39:22 GMT

I am really appreciate your help and insight.

They are very helpful!

HHC