Read sas program to create metadata

[mnjtrana]

Hi All,

@Reeza @KurtBremser @ChrisHemedinger @Patrick @LinusH @Tom 

I am trying to read few sas programs and create a dataset from it and extract some metadata from it. 

So basically i am trying to read the whole program step by step and add some flags like below.

We are trying to flag the step_no: which is the count for the complete step, whether its a proc or a proc sql or datastep.

data_Step_no- for each value present for the step_no, assign incremental count till the datastep end.

proc_Step_no- for each value present for the step_no, assign incremental count till the proc end(or basically a run;).

We are trying to do this using @@ and mutiple find statements, however couldn't make it working for the data_step_no and proc_step_no. Do you think there is any such way to accomplish this task. We are trying to automate our migration. we are using sas EG7.1 on unix and sas 9.2

code	rec_no	step_no	data_step_no	proc_step_no
data want;	1	1	1	.
infile tmp truncover end=done;	2	1	2	.
input @1 father $9. mother $10. address $40.;	3	1	3	.
input child $9. age ?? 3.0 @@;	4	1	4	.
run;	5	1	5	.
DATA COMMAS;	6	2	1	.
INFILE DATALINES DLM=',';	7	2	2	.
INPUT ID HEIGHT WEIGHT GENDER $ AGE;	8	2	3	.
DATALINES;	9	2	4	.
1,68,144,M,23	10	2	5	.
2,78,202,M,34	11	2	6	.
3,62,99,F,37	12	2	7	.
4,61,101,F,45	13	2	8	.
;	14	2	9	.
run;	15	2	10	.
PROC PRINT;	16	3	.	1
TITLE 'Example 2.1';	17	3	.	1
RUN;	18	3	.	1
DATA AMPERS;	18	4	1	.
INPUT NAME & $25. AGE GENDER : $1.;	18	4	2	.
DATALINES;	18	4	3	.
RASPUTIN    45 M	18	4	4	.
BETSY ROSS  62 F	18	4	5	.
ROBERT LOUIS STEVENSON  75 M	18	4	6	.
;	18	4	7	.
run;	18	4	1	.
PROC PRINT;	18	5	.	2
TITLE 'Example 4';	18	5	.	2
RUN;	18	5	.	2
/* this is comment*/	19	.	.	.

[SASKiwi]

How does analysing the structure of your SAS programs programmatically help to automate migration? I'm assuming you are migrating from SAS 9.2 to 9.4.

In my experience, the best way to migrate is to set up your target SAS servers as similar as possible to your source SAS servers then test all of your key programs in the new environment one by one manually. Running your jobs in batch mode is the most productive way. The way you are trying to approach it simply won't work because it is near impossible to analyse SAS programs with high accuracy.

[mnjtrana]

We are actually migrating to sas 9.4 along with new sources of input. So we need to change few of the dataset names through lookup and few of the variable names being used in the assignment statement inside the datastep. so we need to know where the dataset starts, where it ends, what is the input datasdet name and what are the assignment or other statements inside dataset.

I know we wont be able to automate it 100% , but the program list is huge, so any manual effort reduced will help a lot.

[Reeza]

Try PROC SCAPROC for the higher level information automated.
Search on lexjansen.com for code and log analyzers. I'm more in the camp that you cannot automate your way out of bad documentation - especially if you have any macros or shortcut variable or data set lists.
Ensure you account for things like the following:

data want;
set have;
array _demo(5) demo1-demo5;
array _demo2(4) _temporary_ new_demo4-new_demo7;

rename demo1-demo5 = random1-random5;
run;

[SASKiwi]

@mnjtrana - Editors like Notepad++ can be very helpful here. They can search all SAS programs in a particular folder for example and list all occurrences of a search string like a dataset name that needs to change. 

[Patrick]

I second what @SASKiwi writes. May be tell us what you're trying to get out of such analysis. There might be other ways to get there.

It wouldn't be hard to write some code for the simple sample program you've posted but what when it comes to dynamic and data driven code (macros, call execute() etc.).

[Kurt_Bremser]

When I want to know in which program a specific dataset or infile is used, I find all the logs from the previous month (because that covers ALL program runs) and grep for the dataset/filename:

find /sas/log -type f -name \*.log.202003\* -exec grep "MYLIB.MYDATA" {} \; -print

That's much easier than doing it with SAS.