topic Re: Turning Log Data into Dataset in SAS Data Management

Turning Log Data into Dataset

jei — Fri, 04 Aug 2017 01:47:17 GMT

Hi,

I have logs from machines which is in a txt file format and I would like to convert it into SAS dataset or in a tabular form. I have SAS Data Integration and SAS Enterprise Guide in our environment. May I know how it will be possible?

Thank you!

Re: Turning Log Data into Dataset

TomKari — Fri, 04 Aug 2017 02:59:01 GMT

It depends on how sophisticated you want to be. At the most basic, this will work (untested, you'll need to check to see what the longest log records are, I'm assuming 256):

data LogData;

length LogRecord $256;

infile "/dir1/dir2/SASlog.log" lrecl=32767;

input;

LogRecord = _infile_;

run;

You can then get into all sorts of sophisticated fiddles. Try googling "SAS log scraper", and that should give you a ton of ideas.

Tom

Re: Turning Log Data into Dataset

Kurt_Bremser — Fri, 04 Aug 2017 06:48:03 GMT

I guess you would have a hard time creating the import step with just a point-and-click interface. You most probably won't get around writing the code yourself, in a code node either in DI Studio or EG.

I have done such things, reading performance reports from mainframe into SAS, or from the log of the SAS/SHARE server or the webserver. To give you hints or code snippets, we would need an example of the log.

Re: Turning Log Data into Dataset

TomKari — Fri, 04 Aug 2017 16:59:57 GMT

Thanks, @Kurt_Bremser.

I hadn't thought of trying it using the point and click options. I just gave it a quick test, and assuming all you want is one variable with the entire contents of the log line, the point and click import task works beautifully. I just defined the data as tab-delimited, and I don't believe there are ever any tabs in SAS logs.

Tom

Re: Turning Log Data into Dataset

LinusH — Sun, 06 Aug 2017 13:01:40 GMT

Anything is possible, especially when you have SAS.
The more that you know about the log files the better. Perhaps there's a file specification?
If it's a common machine source chances are that someone have experience with this task.
If you have DI Studio that's the preferred tool if this will be a continuous setup.

Re: Turning Log Data into Dataset

jei — Mon, 07 Aug 2017 02:01:03 GMT

Hi @Kurt_Bremser,

I masked the log details of the attached samle file but still the format is the same. Hope you can give me a hint to transform it into SAS dataset.

Thank you!

Re: Turning Log Data into Dataset

Kurt_Bremser — Mon, 07 Aug 2017 06:16:49 GMT

Try the following code. I added comments to describe what it does. Note that my detection of lines broken by a CRLF in the text might need modification.

data want;
/* The infile comes from DOS and uses CRLF and the pipe character as delimiter */
infile '$HOME/sascommunity/sample_log.txt' lrecl=500 termstr=CRLF dlm='|' truncover;
input f1 :$1. dt_str :$27. textstr :$200.;
if f1 ne ' '; /* filters emtpy lines */
dt = input(substr(dt_str,1,23),e8601dt23.3);
tz = substr(dt_str,25,3);
/* you might want to correct the timestamp according to the timezone, with intnx() */
format dt e8601dt23.3; /* ISO-standard display */
/* now let's test if we have a continuation line caused by a spurious CRLF in the text */
input teststr $200. @1 @@; /* hold the line pointer and reset the column to 1 */
if substr(teststr,1,1) = '08'x
then do;
  textstr = trim(textstr) !! teststr;
  input; /* finish reading the continuation line */
end;
drop dt_str teststr;
run;

PS I edited line 11 of your file before importing as it contains a typo in the year (22017).