topic Re: info. from log in SAS Enterprise Guide

info. from log

jimksas — Wed, 12 Feb 2014 23:39:48 GMT

Hi friends - i am totally new @ SAS...

I am taking training from one local institute online and one question raise from one student that,

if we have very big SAS log file and there are around 10 SAS library have been assigned and if we wants to figure out how many data sets has been created from code in log, what is the best and quick way to do it?

Please give me some time and direction...

Thanks a lot!!!

Re: info. from log

LinusH — Thu, 13 Feb 2014 07:32:04 GMT

Interesting question. But what is the issue/requirement, what's the purpose?

Do you want to do this programmatically?

I think a better idea is to go to the source code, and understand the logic there. There are a tool called SCAPROC, I don't know if that can give you what you are looking for...?

Re: info. from log

jimksas — Thu, 13 Feb 2014 17:24:45 GMT

Yes Linus - i want to see if we can do this programmatically...

I am at very beginning stage of SAS but I have an imagination and i am thinking if we can pull out every string from log and create one variable where we find below NOTE, which would related to a very specific line.

/*so may be this variable would have observations like this*/

Datasets

NOTE: The data set work.test has 50 observations and 10 variables.

NOTE: The data set jim.batchout has 100000 observation and 50 variables.

NOTE: The data set work.test5 has 6000 observations and 5 variables.

Thanks a lot for your interest...!!!

Re: info. from log

Amir — Thu, 13 Feb 2014 17:58:09 GMT

Hi,

How about trying something like the following before you finish your SAS session.

It should create a SAS data set (new_ds) showing the SAS data sets created in the current session on the day you run it.

proc sql noprint;

create table new_ds as

select *

from dictionary.tables

where datepart(crdate) eq today()

;

quit;

Regards,

Amir.

Re: info. from log

jimksas — Thu, 13 Feb 2014 18:50:57 GMT

Hi Amir - i cannot change existing coding in our prod envi. - i don't have permission to do that as well as they will not allow me to do that .

I must write code separately/ unscratched...

Thanks!

Re: info. from log

Amir — Thu, 13 Feb 2014 18:56:42 GMT

Hi,

In that case I think it would also help if you explain how you want your final output, e.g. flat file of data, and supply an example of what it should look like.

Regards,

Amir.

Re: info. from log

jimksas — Thu, 13 Feb 2014 19:15:09 GMT

i have explain it in my previous post - see above///

Basically i am expecting one variable and all string from log showing below. But again this is my thinking but you guys have more than one way to do it. I am just trying to figure easiest way to do it...

Thanks!!!

/*here data set would look like, one variable and no. of observation with all string starting from "NOTE: the data set .....has....obs and ....vars." */

Datasets

NOTE: The data set work.test has 50 observations and 10 variables.

NOTE: The data set jim.batchout has 100000 observation and 50 variables.

NOTE: The data set work.test5 has 6000 observations and 5 variables.

Re: info. from log

Cynthia_sas — Fri, 14 Feb 2014 13:47:38 GMT

Hi:

You can always use PROC PRINTTO to write the LOG to a .TXT file and then write a DATA step program to read the .TXT file and extract only the strings pertinent to dataset creation. There have been quite a few usergroup papers about that approach. Generally, though, you'd have to run your "new" program after every step that created datasets -- so that is saving a lot of LOG files and running your program to parse the LOG files. It is do-able. But, personally, I think the SQL example is more elegant and less work. You don't have to do a "Create table" in the SQL, you could just do a report by starting with the SELECT instead of the CREATE.

cynthia

Re: info. from log

Amir — Fri, 14 Feb 2014 18:49:57 GMT

Hi,

If you plan on reading the log then there might be more than one way the log informs you of when a new data set is created, e.g. if a SAS data set was created via proc sql then you might see a slightly different note in the log, e.g.:

NOTE: Table WORK.TEST created, with 10 rows and 3 columns.

Further, you'll need to consider things such as proc append which will also put out a note like you've shown but would you want to consider it as a new data set to add to your count. The same question applies if a data set is created and overwrites an existing same-named data set.

And so on.

Regards,

Amir.

Re: info. from log

jimksas — Wed, 19 Feb 2014 00:54:29 GMT

So friends - I am thinking to pull out two different kind of string from log.

/*This is my imagination - please forgive me if i am wrong*/

%if string strating from ="NOTE: The data set"

%if string starting from="NOTE: Table"

/*then keep all these string into new data set "a" under variables "dataset"*/

dataset (variable)

NOTE: The data set work.test1 has 5 observations and 10 variables

NOTE: The data set market.test2 has 100 observation and 10 variables

NOTE: The data set work. test5 has 1000000 observation and 500 variables

NOTE: Table WORK.TEST created, with 10 rows and 3 columns

- Can someone please write code? Really appreciate it...

Thanks.

Re: info. from log

Reeza — Wed, 19 Feb 2014 01:30:37 GMT

Google: SAS Log Analyzer Site: Lexjansen.com

For several examples of programs that do this.

Also, worth looking into proc scaproc with the attr option on the record statement, though I don't think it gives you what you want.

Re: info. from log

jimksas — Wed, 19 Feb 2014 17:24:44 GMT

Thanks Reeza - I have found below parent child relationship code but don't know how to interpret with my scenario...

/*if we have below scenario*/

/*there are 2 basic statements which contain the word observation: the "FROM" statement and "HAS" statement*/

NOTE: There were 3 observations read from the data set WORK.RANDDATE.

NOTE: There were 16 observations read from the data set WORK.CONMED1.

NOTE: The data set WORK.CONMED2 has 16 observations and 22 variables.

/*then we can apply this code*/

proc sql;

create table parent7 as select parent6.child, parent6.parent1, parent6.parent2, parent6.parent3, parent6.parent4, parent6.parent5, dsets.parent as parent6

from dsets right join parent6 on dsets.child=parent.parent5;

quit;

where;

- collect information about parent dataset which we store in variable "parent"

- keep track of the obs count from the parent data sets - "parcount"

- from has statement, collect name of the child dataset - "child"

- final obs count - "chdcount"

- line at which this occurred in sas log - "lineid"

- this info. contain in data set "dset"

/*Output would be like this*/

REPORT1

CHILD parent1 parent2 parent3

dataset obs dataset obs source obs source obs

conmed2 16 conmed1 16 conmed 16 raw.med 16

randdate 3

Re: info. from log

jimksas — Wed, 19 Feb 2014 18:04:12 GMT

I would say, if SAS itself implement output which include below information from new upcoming version - then it would be awesome feature!!!

- How many data sets and tables has been created at end of the each log file

- data set observation and variables /*we have this info. available but along with new feature */

I will keep searching on this...until i found something...!!!

Re: info. from log

Astounding — Wed, 19 Feb 2014 18:18:59 GMT

Understanding the question is actually more complex than it sounds. Here are some considerations ...

If a data set gets created and later deleted by the same program, should it be listed?

If a permanent data set gets deleted and later created by the same program, should it be listed?

If a permanent data set gets replaced by the program, should it be listed?

If a permanent data set gets modified by the program, should it be listed? Several variations exist on what "modified" means.

If a view gets created instead of a data set, should that be listed? Would the answer change if the view actually gets used to feed data to a procedure?

If a data set gets created and replaced 5 times during the course of the program, do you want a count of how many times it has been "created"?

I'm not suggesting answers here ... just wanted to point out that the question becomes complex when you dig into it.

Re: info. from log

jimksas — Wed, 19 Feb 2014 18:39:50 GMT

Main goal is : list out how many data sets stored in permanent libraries at end of the log file.

After careful reviewing your questions - my answers would be BUT still i would stay with my main goal -

If a data set gets created and later deleted by the same program, should it be listed? NO -

If a permanent data set gets deleted and later created by the same program, should it be listed? - YES

If a permanent data set gets replaced by the program, should it be listed? YES

If a permanent data set gets modified by the program, should it be listed? - YES. Several variations exist on what "modified" means.

If a view gets created instead of a data set, should that be listed? YES. Would the answer change if the view actually gets used to feed data to a procedure? NO

If a data set gets created and replaced 5 times during the course of the program, do you want a count of how many times it has been "created"? NO

Re: info. from log

jimksas — Thu, 20 Feb 2014 19:25:36 GMT

/*If i have to pull out strings like below from a complete log file - do we have any method to do it?

NOTE: The data set work.test has 100 observations and 10 variables.

NOTE: The data set market.test2 has 500 observation and 50 variables.

/*just a thought*/

filename logfile 'c:\test.log';

%let a=work | market;

%let b=any character;

%let c=any number;

%macro check;

if string in : ("NOTE: The data set &a.&b &c &b &c &b") then

do;

data test;

length string $100;

infile logfile;

string="NOTE: The data set &a.&b &c &b &c &b";

run;

proc sql;

select loginfo into :string separated by ' ' from test;

quit;

end;

%mend;

%check;

/*thanks*/

Re: info. from log

Cynthia_sas — Thu, 20 Feb 2014 19:55:40 GMT

Hi:

I'm not sure you have the macro code correct. Your macro variables are not doing what you think they are going to do. Try this test:

%let a=work | market;

%let b=any character;

%let c=any number;

%macro check;

%put NOTE: The data set &a.&b &c &b &c &b;

%mend;

%check;

and then look in the SAS log (see screen shot). Have you tried to actually RUN your code???? Your IF statement is incorrect.

cynthia

Re: info. from log

jimksas — Thu, 20 Feb 2014 20:12:50 GMT

Cynthia - i mentioned in my comment that this is just thought - I am not expert at all on macro, i am at very beginning stage in macro as well as SAS. I just wanted to know if somehow we can pull those particular strings

(

NOTE: The data set work.test has 100 observations and 10 variables.

NOTE: The data set market.test2 has 500 observation and 50 variables.

)

from log.

- Thanks for your time!!!

Re: info. from log

ChrisHemedinger — Thu, 20 Feb 2014 20:13:56 GMT

If the goal is to create a summary of "net data sets created/modified" by the current program, then perhaps you could instrument your program for this. Here's an approach that uses the data table datetime stamp to detect which tables were modified after a given point in time (captured at the program start):

libname perm "c:\temp"; 

/* start the timer ... NOW! */
%let cutoff = %sysfunc(datetime());

/* create some tables in various libraries */
data perm.a perm.b perm.c work.ignore;
  set sashelp.class;
run; 

/* create a table with entries for */
/* each table created since the program started */
proc sql; 
create table net_tables as
select * from sashelp.vtable t1
where t1.modate > &cutoff
 and libname not in ('WORK' 'SASHELP' 'MAPS' /* additional libraries to "ignore" */);
quit;

Make sense?

Chris

Re: info. from log

jimksas — Thu, 20 Feb 2014 21:09:34 GMT

WOW Chris ... ... this is awesome...This will definitely help. I didn't think someone would come up with just 2 steps to fulfill this requirement.

Thanks a lot...!!! i really appreciate your time...!!!

I will drop unwanted variables from "net_tables"...

- Thanks Again Chris.

- Thanks ALL for your time and comments.