info. from log

[jimksas]

Hi friends - i am totally new @ SAS...

I am taking training from one local institute online and one question raise from one student that,

if we have very big SAS log file and there are around 10 SAS library have been assigned and if we wants to figure out how many data sets has been created from code in log, what is the best and quick way to do it?

Please give me some time and direction...

Thanks a lot!!!

[LinusH]

Interesting question. But what is the issue/requirement, what's the purpose?

Do you want to do this programmatically?

I think a better idea is to go to the source code, and understand the logic there. There are a tool called SCAPROC, I don't know if that can give you what you are looking for...?

[jimksas]

Yes Linus - i want to see if we can do this programmatically...

I am at very beginning stage of SAS but I have an imagination and i am thinking if we can pull out every string from log and create one variable where we find below NOTE, which would related to a very specific line.

/*so may be this variable would have observations like this*/

Datasets

NOTE: The data set work.test has 50 observations and 10 variables.

NOTE: The data set jim.batchout has 100000 observation and 50 variables.

NOTE: The data set work.test5 has 6000 observations and 5 variables.

Thanks a lot for your interest...!!!

[Amir]

Hi,

How about trying something like the following before you finish your SAS session.

It should create a SAS data set (new_ds) showing the SAS data sets created in the current session on the day you run it.

proc sql noprint;

  create table new_ds as

  select *

  from dictionary.tables

  where datepart(crdate) eq today()

  ;

quit;

Regards,

Amir.

[jimksas]

Hi Amir - i cannot change existing coding in our prod envi. - i don't have permission to do that as well as they will not allow me to do that .

I must write code separately/ unscratched...

Thanks!

[Amir]

Hi,

In that case I think it would also help if you explain how you want your final output, e.g. flat file of data, and supply an example of what it should look like.

Regards,

Amir.

[jimksas]

i have explain it in my previous post - see above///

Basically i am expecting one variable and all string from log showing below. But again this is my thinking but you guys have more than one way to do it. I am just trying to figure easiest way to do it...

Thanks!!!

/*here data set would look like, one variable and no. of observation with all string starting from "NOTE: the data set .....has....obs and ....vars." */

Datasets

NOTE: The data set work.test has 50 observations and 10 variables.

NOTE: The data set jim.batchout has 100000 observation and 50 variables.

NOTE: The data set work.test5 has 6000 observations and 5 variables.

[Cynthia_sas]

Hi:

  You can always use PROC PRINTTO to write the LOG to a .TXT file and then write a DATA step program to read the .TXT file and extract only the strings pertinent to dataset creation. There have been quite a few usergroup papers about that approach. Generally, though, you'd have to run your "new" program after every step that created datasets -- so that is saving a lot of LOG files and running your program to parse the LOG files. It is do-able. But, personally, I think the SQL example is more elegant and less work. You don't have to do a "Create table" in the SQL, you could just do a report by starting with the SELECT instead of the CREATE.

cynthia

[Amir]

Hi,

If you plan on reading the log then there might be more than one way the log informs you of when a new data set is created, e.g. if a SAS data set was created via proc sql then you might see a slightly different note in the log, e.g.:

NOTE: Table WORK.TEST created, with 10 rows and 3 columns.

Further, you'll need to consider things such as proc append which will also put out a note like you've shown but would you want to consider it as a new data set to add to your count. The same question applies if a data set is created and overwrites an existing same-named data set.

And so on.

Regards,

Amir.

[jimksas]

So friends - I am thinking to pull out two different kind of string from log.

/*This is my imagination - please forgive me if i am wrong*/

%if string strating from ="NOTE: The data set"

OR

%if string starting from="NOTE: Table"

/*then keep all these string into new data set "a" under variables "dataset"*/

dataset (variable)

NOTE: The data set work.test1 has 5 observations and 10 variables

NOTE: The data set market.test2 has 100 observation and 10 variables

NOTE: The data set work. test5 has 1000000 observation and 500 variables

NOTE: Table WORK.TEST created, with 10 rows and 3 columns

- Can someone please write code? Really appreciate it...

Thanks.

[Reeza]

Google: SAS Log Analyzer Site: Lexjansen.com

For several examples of programs that do this.

Also, worth looking into proc scaproc with the attr option on the record statement, though I don't think it gives you what you want.

[jimksas]

Thanks Reeza - I have found below parent child relationship code but don't know how to interpret with my scenario...

/*if we have below scenario*/

/*there are 2 basic statements which contain the word observation: the "FROM" statement and "HAS" statement*/

NOTE: There were 3 observations read from the data set WORK.RANDDATE.

NOTE: There were 16 observations read from the data set WORK.CONMED1.

NOTE: The data set WORK.CONMED2 has 16 observations and 22 variables.

/*then we can apply this code*/

proc sql;

  create table parent7 as select parent6.child, parent6.parent1, parent6.parent2, parent6.parent3, parent6.parent4, parent6.parent5, dsets.parent as parent6

  from dsets right join parent6 on dsets.child=parent.parent5;

quit;

/*

where;

- collect information about parent dataset which we store in variable "parent"

- keep track of the obs count from the parent data sets - "parcount"

- from has statement, collect name of the child dataset - "child"

- final obs count - "chdcount"

- line at which this occurred in sas log - "lineid"

- this info. contain in data set "dset"

*/

/*Output would be like this*/

REPORT1

CHILD                       parent1                     parent2                     parent3      

dataset          obs     dataset        obs         source      obs           source        obs

conmed2        16       conmed1      16            conmed    16              raw.med     16

                                randdate       3

[jimksas]

I would say, if SAS itself implement output which include below information from new upcoming version - then it would be awesome feature!!!

                 - How many data sets and tables has been created at end of the each log file

                 - data set observation and variables /*we have this info. available but along with new feature */

I will keep searching on this...until i found something...!!!

[Astounding]

Understanding the question is actually more complex than it sounds.  Here are some considerations ...

If a data set gets created and later deleted by the same program, should it be listed?

If a permanent data set gets deleted and later created by the same program, should it be listed?

If a permanent data set gets replaced by the program, should it be listed?

If a permanent data set gets modified by the program, should it be listed?  Several variations exist on what "modified" means.

If a view gets created instead of a data set, should that be listed?  Would the answer change if the view actually gets used to feed data to a procedure?

If a data set gets created and replaced 5 times during the course of the program, do you want a count of how many times it has been "created"?

I'm not suggesting answers here ... just wanted to point out that the question becomes complex when you dig into it.

[jimksas]

Main goal is : list out how many data sets stored in permanent libraries at end of the log file.

After careful reviewing your questions - my answers would be BUT still i would stay with my main goal -

If a data set gets created and later deleted by the same program, should it be listed? NO -

If a permanent data set gets deleted and later created by the same program, should it be listed? - YES

If a permanent data set gets replaced by the program, should it be listed? YES

If a permanent data set gets modified by the program, should it be listed? - YES.  Several variations exist on what "modified" means.

If a view gets created instead of a data set, should that be listed? YES. Would the answer change if the view actually gets used to feed data to a procedure? NO

If a data set gets created and replaced 5 times during the course of the program, do you want a count of how many times it has been "created"? NO