BookmarkSubscribeRSS Feed
🔒 This topic is solved and locked. Need further help from the community? Please sign in and ask a new question.
Saurabh13
Fluorite | Level 6

I have a sas log . For example :

NOTE: The data set WORK.A has 1 observations and 1 variables.
NOTE: DATA statement used (Total process time):
real time 0.01 seconds
user cpu time 0.01 seconds
system cpu time 0.01 seconds
memory 764.06k
OS Memory 18596.00k
Timestamp 05/08/2017 11:26:34 PM
Step Count 6 Switch Count 30
Page Faults 0
Page Reclaims 351
Page Swaps 0
Voluntary Context Switches 73
Involuntary Context Switches 1
Block Input Operations 0
Block Output Operations 312

28
29 data B;
30 set A;
31 x=3;
32 run;

NOTE: There were 1 observations read from the data set WORK.A.
NOTE: The data set WORK.B has 1 observations and 2 variables.
NOTE: DATA statement used (Total process time):
real time 0.00 seconds
user cpu time 0.01 seconds
2 The SAS System 23:11 Monday, May 8, 2017

system cpu time 0.00 seconds
memory 1032.62k
OS Memory 18856.00k
Timestamp 05/08/2017 11:26:34 PM
Step Count 7 Switch Count 40
Page Faults 0
Page Reclaims 297
Page Swaps 0
Voluntary Context Switches 111
Involuntary Context Switches 0
Block Input Operations 0
Block Output Operations 328

33
34 proc sql;
35 create table XYZ as select * from B;
NOTE: Table WORK.XYZ created, with 1 rows and 2 columns.

36 quit;
NOTE: PROCEDURE SQL used (Total process time):
real time 0.00 seconds
user cpu time 0.00 seconds
system cpu time 0.00 seconds

 

After Each data step or Proc step there is statistics of real time and CPU time. I want to parse and create dataset which will have dataset name and real time . Like

 

Table            Time     records 

Work.A         0.01      1

Work.B         0.00      1

Work.XYZ    0.00      1

 

I am trying like this but not able to capture Table /dataset name .

 

filename in "log_path";

 

data x;
infile inn missover truncover;
input text $1000. ;
b=strip(text);
x=compress(text);
c=substr(x,1,8);
zz=compress(x,'realtime');
if text eq '' then delete;
if c='realtime';
run;

 

could someone please help ?

 

Thanks,

1 ACCEPTED SOLUTION

Accepted Solutions
Saurabh13
Fluorite | Level 6

Tried but in 2 steps.

data x_wo_step(keep= text ds_name ds_count real_time ) w_step(keep=ds_name step_used);
infile inn truncover end=eof;
input text $1000. ;
retain ds_name ds_count ;
if indexw(text,"NOTE: The data set") eq 1 then do;
ds_name = scan(text,5," ");
ds_count = scan(text,7," ");
output x_wo_step;
end;
else if indexw(text,"NOTE: Table") eq 1 then do;
ds_name = scan(text,3," ");
ds_count = scan(text,6," ");
output x_wo_step;
end;
else if indexw(text,'real') eq 1 then do; 
real_time = scan(text,3," "); 
output x_wo_step;
end;
else if indexw(text,'used') gt 1 then do;
step_used = scan(text,2," ")||""|| scan(text,3," "); /*--It's data/proc step --*/
output w_step;
end;
else do;
delete;
end;
run;
data x_wo_step_1;
set x_wo_step;
if real_time eq '' then delete;
run;
data Final_stats(drop=text);
merge x_wo_step_1(in=a) w_step(in=b);
by ds_name;
run;

View solution in original post

6 REPLIES 6
Saurabh13
Fluorite | Level 6
25         data A;
26          s=2;
27         run;

NOTE: The data set WORK.A has 1 observations and 1 variables.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      user cpu time       0.01 seconds
      system cpu time     0.00 seconds
      memory              765.00k
      OS Memory           18340.00k
      Timestamp           05/09/2017 12:41:45 AM
      Step Count                        2  Switch Count  29
      Page Faults                       0
      Page Reclaims                     425
      Page Swaps                        0
      Voluntary Context Switches        67
      Involuntary Context Switches      0
      Block Input Operations            0
      Block Output Operations           304
      

28         
29         data B;
30          set A;
31          x=3;
32         run;

NOTE: There were 1 observations read from the data set WORK.A.
NOTE: The data set WORK.B has 1 observations and 2 variables.
NOTE: DATA statement used (Total process time):
      real time           0.00 seconds
      user cpu time       0.00 seconds
2                                                          The SAS System                                 00:41 Tuesday, May 9, 2017

      system cpu time     0.00 seconds
      memory              1030.53k
      OS Memory           18600.00k
      Timestamp           05/09/2017 12:41:45 AM
      Step Count                        3  Switch Count  39
      Page Faults                       0
      Page Reclaims                     314
      Page Swaps                        0
      Voluntary Context Switches        92
      Involuntary Context Switches      0
      Block Input Operations            0
      Block Output Operations           320
      

33         
34         proc sql;
35          create table XYZ as select * from B;
NOTE: Table WORK.XYZ created, with 1 rows and 2 columns.

36         quit;
NOTE: PROCEDURE SQL used (Total process time):
      real time           0.00 seconds
      user cpu time       0.01 seconds
      system cpu time     0.00 seconds
      memory              5419.12k
      OS Memory           23464.00k
      Timestamp           05/09/2017 12:41:45 AM
      Step Count                        4  Switch Count  29
      Page Faults                       0
      Page Reclaims                     149
      Page Swaps                        0
      Voluntary Context Switches        67
      Involuntary Context Switches      0
      Block Input Operations            0
      Block Output Operations           304
Kurt_Bremser
Super User

Concur with @Shmuel. use the index() and scan() functions; index() to detect the presence of a certain string, and scan() to extract a certain "word" from the string.

The data step would look like that:

- declare the infile with truncover and a end=done option

- retain variables for datasetname, number of records and real time

- set length for datasetname to $32

- when you encounter a index(text,"NOTE: The data set") = 1, do:

  - if datasetname is already set (indicating that you've been collecting data already), output

  - set datasetname with scan(text,5," ") and extract nrecs with nrecs = input(scan(text,7," ",best.)

  - set realtime to missing

when you encounter "real time" in text, extract realtime like nrecs (experiment until you get the correct "word" number)

when if done is true and datasetname > " ", output (to write the last occurence)

 

The check for the NOTEs from SQL look similar to that for a data step, just the words and positions are different

 

Saurabh13
Fluorite | Level 6

Tried but in 2 steps.

data x_wo_step(keep= text ds_name ds_count real_time ) w_step(keep=ds_name step_used);
infile inn truncover end=eof;
input text $1000. ;
retain ds_name ds_count ;
if indexw(text,"NOTE: The data set") eq 1 then do;
ds_name = scan(text,5," ");
ds_count = scan(text,7," ");
output x_wo_step;
end;
else if indexw(text,"NOTE: Table") eq 1 then do;
ds_name = scan(text,3," ");
ds_count = scan(text,6," ");
output x_wo_step;
end;
else if indexw(text,'real') eq 1 then do; 
real_time = scan(text,3," "); 
output x_wo_step;
end;
else if indexw(text,'used') gt 1 then do;
step_used = scan(text,2," ")||""|| scan(text,3," "); /*--It's data/proc step --*/
output w_step;
end;
else do;
delete;
end;
run;
data x_wo_step_1;
set x_wo_step;
if real_time eq '' then delete;
run;
data Final_stats(drop=text);
merge x_wo_step_1(in=a) w_step(in=b);
by ds_name;
run;
Kurt_Bremser
Super User

My take would be this:

data log (keep=datasetname nrecs realtime);
infile in truncover end=eof;
retain datasetname nrecs realtime;
format
  datasetname $32.
  nrecs comma12.
  realtime time11.2
;
input text $100.;
if index(text,"NOTE: The data set") = 1
then do;
  if datasetname ne " " then output;
  datasetname = scan(text,5," ");
  nrecs = input(scan(text,7," "),best.);
  realtime = .;
end;
else if index(text,"NOTE: Table") = 1
then do;
  if datasetname ne " " then output;
  datasetname = scan(text,3," ");
  nrecs = input(scan(text,6," "),best.);
  realtime = .;
end;
if index(text,"real time") > 0 then realtime = input(scan(text,3," "),best.);
if eof and datasetname ne " " then output;
run;

Since PROC SQL issues a "Note: Table" for every dataset created, but only writes the statistics when it quits, one must not rely on "real time" for any output.

Also note that your approach would create wrong data when the same dataset is created in several steps within one log file.

Shmuel
Garnet | Level 18

Each log line has its own format.

Use functions index or find to check for wanted keywords like DATA, OUT=, observations, real time, etc.

When found use function scan  to get the desired information (dataset name or number of observations or measured time).

 

In your code, you used function compress. Change it to function compbl or even skip it.

Use function scan  instead function substr.

 

Try it and come with your new code and results.

 

sas-innovate-2024.png

Join us for SAS Innovate April 16-19 at the Aria in Las Vegas. Bring the team and save big with our group pricing for a limited time only.

Pre-conference courses and tutorials are filling up fast and are always a sellout. Register today to reserve your seat.

 

Register now!

How to Concatenate Values

Learn how use the CAT functions in SAS to join values from multiple variables into a single value.

Find more tutorials on the SAS Users YouTube channel.

Click image to register for webinarClick image to register for webinar

Classroom Training Available!

Select SAS Training centers are offering in-person courses. View upcoming courses for:

View all other training opportunities.

Discussion stats
  • 6 replies
  • 5782 views
  • 0 likes
  • 3 in conversation