SAS code to read multiple .log files to produce a dataset

[Babloo]

Can someone explain why dataset with created with 0 observations? See the log below.

3          data log_analysis;

4          length fname filename $200;

5          infile '/usr/sas/sas_config/Lev1/SASApp/StoredProcessServer/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_*.log'

5        ! truncover filename=fname;

6          input var : $ 3000.;

7          filename=fname;

8          var1 = _infile_;

9          if var1 = :'2015';

10         Date_With_TimeStamp = scan(var1,1," ");

11         Status = scan(var1,2," ");

12         Processid = scan(var1,3," ");

13         userid = scan(var1,4," ");

14         Details = scan(var1,-1,'-');

15         drop var var1;

16         run;

2                                                          The SAS System                              03:07 Monday, August 24, 2015

NOTE: The infile '/usr/sas/sas_config/Lev1/SASApp/StoredProcessServer/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_*.log' is:

      Filename=/usr/sas/sas_config/Lev1/SASApp/StoredProcessServer/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_10891.log,

      File List=/usr/sas/sas_config/Lev1/SASApp/StoredProcessServer/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_*.log,

      Owner Name=sassrv,Group Name=sas,

      Access Permission=rwxrwxr-x,

      Last Modified=Tue Nov  4 17:05:00 2014,

      File Size (bytes)=2148

NOTE: The infile '/usr/sas/sas_config/Lev1/SASApp/StoredProcessServer/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_*.log' is:

      Filename=/usr/sas/sas_config/Lev1/SASApp/StoredProcessServer/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_11086.log,

      File List=/usr/sas/sas_config/Lev1/SASApp/StoredProcessServer/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_*.log,

      Owner Name=sassrv,Group Name=sas,

      Access Permission=rwxrwxr-x,

      Last Modified=Tue Nov  4 17:05:00 2014,

      File Size (bytes)=2089

NOTE: The infile '/usr/sas/sas_config/Lev1/SASApp/StoredProcessServer/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_*.log' is:

      Filename=/usr/sas/sas_config/Lev1/SASApp/StoredProcessServer/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_11215.log,

      File List=/usr/sas/sas_config/Lev1/SASApp/StoredProcessServer/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_*.log,

      Owner Name=sassrv,Group Name=sas,

      Access Permission=rwxrwxr-x,

      Last Modified=Tue Nov  4 17:05:00 2014,

      File Size (bytes)=2089

NOTE: 15 records were read from the infile

      '/usr/sas/sas_config/Lev1/SASApp/StoredProcessServer/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_*.log'.

      The minimum record length was 83.

      The maximum record length was 256.

NOTE: 15 records were read from the infile

      '/usr/sas/sas_config/Lev1/SASApp/StoredProcessServer/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_*.log'.

      The minimum record length was 83.

      The maximum record length was 256.

NOTE: 15 records were read from the infile

      '/usr/sas/sas_config/Lev1/SASApp/StoredProcessServer/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_*.log'.

      The minimum record length was 83.

      The maximum record length was 256.

      One or more lines were truncated.

NOTE: The data set WORK.LOG_ANALYSIS has 0 observations and 6 variables.

NOTE: DATA statement used (Total process time):

      real time           0.00 seconds

      cpu time            0.00 seconds

[Tom]

I would assume because of line 9. None of the 45 records you read from the 3 files started with the four characters '2015'.

Perhaps there are leading spaces on the lines?  Perhaps the lines are longer and are wrapping when you look at the files in your terminal or editor window and the '2015' is in the middle of the line?

[Babloo]

I slightly tweaked my code to search for '201' and it worked. However, if there are any missing values in any of the variables (status,processid,userid), I don't know to handle.

Input File:

2014-11-04T17:04:59,710 INFO  [00000009] :sassrv - Client connection 2 for user sassrv closed.

2014-11-04T17:04:59,905 DEBUG [00001133] 3:sassrv - START Perf.ARM.IOM.StoredProcessServer.ServerAdministration.DeferredStopServer 7fd96afe7290 0 0

2014-11-04T17:04:59,925       [00001133] 3:sassrv - STP: Stored Process Server Shutting Down.

My code:

data log_analysis;

length fname filename $200;

infile '/usr/sas/sas_config/Lev1/SASApp/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_*.log' dsd truncover filename=fname;

input var : $ 3000.;

filename=fname;

var1 = _infile_;

if var1 = :'201';

Date_With_TimeStamp = scan(var1,1," ");

Status = scan(var1,2," ");

Processid = scan(var1,3," ");

userid = scan(var1,4," ");

Details = scan(var1,-1,'-');

drop var var1;

run;

Output:

filename	Date_With_TimeStamp	Status	Processid	userid	Details
/usr/sas/sas_config/Lev1/SASApp/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_10891.log	2014-11-04T17:04:59,710	INFO	[00000009]	:sassrv	Client connection 2 for user sassrv closed.
/usr/sas/sas_config/Lev1/SASApp/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_10891.log	2014-11-04T17:04:59,905	DEBUG	[00001133]	3:sassrv	START Perf.ARM.IOM.StoredProcessServer.ServerAdministration.DeferredStopServer 7fd96afe7290 0 0
/usr/sas/sas_config/Lev1/SASApp/Logs/SASApp_STPServer_2014-11-04_tmptcmsaslva2_10891.log	2014-11-04T17:04:59,925	[00001133]	3:sassrv	-	STP: Stored Process Server Shutting Down.

Here for the third record, variable 'status' should be missing, but I got 'processid' value in the 'status' variable. Also I need only the file name in the variable 'filename' instead of full directory.

Desired Output:

filename	Date_With_TimeStamp	Status	Processid	userid	Details
SASApp_STPServer_2014-11-04_tmptcmsaslva2_10891.log	2014-11-04T17:04:59,710	INFO	[00000009]	:sassrv	Client connection 2 for user sassrv closed.
SASApp_STPServer_2014-11-04_tmptcmsaslva2_10891.log	2014-11-04T17:04:59,905	DEBUG	[00001133]	3:sassrv	START Perf.ARM.IOM.StoredProcessServer.ServerAdministration.DeferredStopServer 7fd96afe7290 0 0
SASApp_STPServer_2014-11-04_tmptcmsaslva2_10891.log	2014-11-04T17:04:59,925		[00001133]	3:sassrv	STP: Stored Process Server Shutting Down.

Thanks again for any help I receive.

[Ksharp]

Just found your file is fixed column length file which make it easy to import by the column input method.

data have;
infile '/folders/myfolders/x.txt' lrecl=32767 truncover;
input Date_With_TimeStamp $ 1-24 Status $ 25-30 Processid $ 31-41 userid $ 42-50 Details $ 51-300 ;
run;

Xia Keshan

[jakarman]

The header is indicating the ARM logging is active. That is where those INFO messages have their origin.
See SAS(R) Environment Manager 2.4: User's Guide and SAS(R) 9.4 Logging: Configuration and Programming Reference, Second Edition

Your question  is about something doing yourself that is delivered as a service.

[Ron_MacroMaven]

This tool is for Windows logs.

http://www.sascommunity.org/wiki/Reading_job_logs

[jakarman]

There are many tools to process log4j log4net logs. The top one in this market is: Splunk makes machine data accessible, usable and valuable to everyone.

The reason is it is accepted by CSO (Centrsl Security oiificers) departments. SAS logging the arm messages are log4j messages......