topic Re: SAS_Clean Log in SAS Programming

SAS_Clean Log

Khaladdin — Thu, 24 Oct 2019 14:59:14 GMT

Hi all,

I have a large dataset with a comprehensive cleaning code. The code works well and pretty fast. The only issue is it creates a huge amount of log file (more than 500 GB). As a first step, I decided to use proc printto and save log file in my hard disk. However, as I said, even log file became more than 500 GB. Thus, I need something which prevents SAS to create any information in the log.

Much appreciated.

Re: SAS_Clean Log

PeterClemmensen — Thu, 24 Oct 2019 15:12:01 GMT

See if this gives you some info you can use

Suppressing Information in the SAS Log

Re: SAS_Clean Log

FreelanceReinh — Thu, 24 Oct 2019 15:15:00 GMT

Hi @Khaladdin,

You can also use PROC PRINTTO to discard the log.

Example:

filename nirvana dummy;
proc printto log=nirvana;
run;

Re: SAS_Clean Log

noling — Thu, 24 Oct 2019 15:42:08 GMT

Just out of curiosity, what are you printing to the log? Is it only the default notes from the procedures or have you specified additional information to print?

Re: SAS_Clean Log

Khaladdin — Thu, 24 Oct 2019 15:45:30 GMT

Actually, it is just the default notes from the procedures. I have run the same code previously. It has never printed anything like that. I am using the same type of data, however, with large size. It now starts to print a large log. I do not know the reason actually.

Re: SAS_Clean Log

Khaladdin — Thu, 24 Oct 2019 15:45:50 GMT

I am currently trying this. Will let you know once I have the result.

Re: SAS_Clean Log

noling — Thu, 24 Oct 2019 16:51:27 GMT

Interesting. I'd think you'd have to be running 1000's of procedures to generate a 500GB log. I'd examine the log to see if there is any unexpected output. 500GB is alarming.

Re: SAS_Clean Log

Quentin — Thu, 24 Oct 2019 18:16:58 GMT

Agree, I'd look at the log and see what's going on.

I've never in my life had enough confidence in a SAS program to feel good about the idea of sending a log to nirvana.

Even if you have the perfect program, there's too much opportunity to something to go wrong with your data, or network, or .....

A 500GB log sounds like something may have gone horribly wrong.

Re: SAS_Clean Log

ErikLund_Jensen — Thu, 24 Oct 2019 19:11:57 GMT

Hi @Khaladdin

You don't say anything about your setup, and if you are running your job locally I have no good ideas. But in a client/server somebody could have made changes that to the SAS installation, that can make the log explode, if you are running many steps, e.g. with a macro loop.

Have a look at the log options and see if mprint, mlogic, symbolgen or source2 are enabled. And check ARM logging. After an upgrade we had ARM logging and mprint enabled, and out logs exploded with a factor 10 or more.

Re: SAS_Clean Log

Khaladdin — Fri, 25 Oct 2019 07:16:18 GMT

Hi all,

Many thanks for your comments.

I have looked at the log. The log file does not show any error. However, it automatically copies all my data into the log file as a text.

Re: SAS_Clean Log

Khaladdin — Fri, 25 Oct 2019 07:27:15 GMT

It works. As suggested by other contributors, it is a bit dangerous to send all log to nirvana. However, I have checked my cleaned data and everything is perfect.

Re: SAS_Clean Log

Ksharp — Fri, 25 Oct 2019 12:29:02 GMT

Try this option . EDITED

options notes=no source=no source2=no;

data have;
set sashelp.class;
run;

Re: SAS_Clean Log

Tom — Fri, 25 Oct 2019 13:04:01 GMT

@Khaladdin wrote:

Hi all,

Many thanks for your comments.

I have looked at the log. The log file does not show any error. However, it automatically copies all my data into the log file as a text.

SAS does NOT automatically copy your data to the log. Even if you have in-line data SAS does not echo that data to the log.

646  data test;
647    input x;
648  cards;

NOTE: The data set WORK.TEST has 3 observations and 1 variables.
NOTE: DATA statement used (Total process time):
      real time           0.09 seconds
      cpu time            0.02 seconds

652  ;