SAS_Clean Log

[Khaladdin]

Hi all,

I have a large dataset with a comprehensive cleaning code. The code works well and pretty fast. The only issue is it creates a huge amount of log file (more than 500 GB). As a first step, I decided to use  proc printto and save log file in my hard disk. However, as I said, even log file became more than 500 GB. Thus, I need something which prevents SAS to create any information in the log. 

Much appreciated. 

[PeterClemmensen]

See if this gives you some info you can use

Suppressing Information in the SAS Log

[FreelanceReinh · Accepted solution]

Hi @Khaladdin,

You can also use PROC PRINTTO to discard the log.

Example:

filename nirvana dummy;
proc printto log=nirvana;
run;

[Khaladdin]

I am currently trying this. Will let you know once I have the result.

[Khaladdin]

It works. As suggested by other contributors, it is a bit dangerous to send all log to nirvana. However, I have checked my cleaned data and everything is perfect.

[noling]

Just out of curiosity, what are you printing to the log? Is it only the default notes from the procedures or have you specified additional information to print?

[Khaladdin]

Actually, it is just the default notes from the procedures. I have run the same code previously. It has never printed anything like that. I am using the same type of data, however, with large size. It now starts to print a large log. I do not know the reason actually.

[noling]

Interesting. I'd think you'd have to be running 1000's of procedures to generate a 500GB log. I'd examine the log to see if there is any unexpected output. 500GB is alarming.

[Quentin]

Agree, I'd look at the log and see what's going on.

I've never in my life had enough confidence in a SAS program to feel good about the idea of sending a log to nirvana.  

Even if you have the perfect program, there's too much opportunity to something to go wrong with your data, or network, or .....

A 500GB log sounds like something may have gone horribly wrong.

[ErikLund_Jensen]

Hi @Khaladdin 

You don't say anything about your setup, and if you are running your job locally I have no good ideas. But in a client/server somebody could have made changes that to the SAS installation, that can make the log explode, if you are running many steps, e.g. with a macro loop.

Have a look at the log options and see if mprint, mlogic, symbolgen or source2 are enabled. And check ARM logging. After an upgrade we had ARM logging and mprint enabled, and out logs exploded with a factor 10 or more.

[Khaladdin]

Hi all,

Many thanks for your comments.

I have looked at the log. The log file does not show any error. However, it automatically copies all my data into the log file as a text. 

[Tom]

---

@Khaladdin wrote:

Hi all,

Many thanks for your comments.

I have looked at the log. The log file does not show any error. However, it automatically copies all my data into the log file as a text. 

---

SAS does NOT automatically copy your data to the log.  Even if you have in-line data SAS does not echo that data to the log.

646  data test;
647    input x;
648  cards;

NOTE: The data set WORK.TEST has 3 observations and 1 variables.
NOTE: DATA statement used (Total process time):
      real time           0.09 seconds
      cpu time            0.02 seconds

652  ;

[Ksharp]

Try this option . EDITED 

options notes=no source=no source2=no;

data have;
set sashelp.class;
run;