topic Re: Read sas program to create metadata in SAS Programming

Read sas program to create metadata

mnjtrana — Sun, 05 Apr 2020 17:37:21 GMT

Hi All,

@Reeza @KurtBremser @ChrisHemedinger @Patrick @LinusH @Tom

I am trying to read few sas programs and create a dataset from it and extract some metadata from it.

So basically i am trying to read the whole program step by step and add some flags like below.

We are trying to flag the step_no: which is the count for the complete step, whether its a proc or a proc sql or datastep.

data_Step_no- for each value present for the step_no, assign incremental count till the datastep end.

proc_Step_no- for each value present for the step_no, assign incremental count till the proc end(or basically a run;).

We are trying to do this using @@ and mutiple find statements, however couldn't make it working for the data_step_no and proc_step_no. Do you think there is any such way to accomplish this task. We are trying to automate our migration. we are using sas EG7.1 on unix and sas 9.2

code	rec_no	step_no	data_step_no	proc_step_no
data want;	1	1	1	.
infile tmp truncover end=done;	2	1	2	.
input @1 father $9. mother $10. address $40.;	3	1	3	.
input child $9. age ?? 3.0 @@;	4	1	4	.
run;	5	1	5	.
DATA COMMAS;	6	2	1	.
INFILE DATALINES DLM=',';	7	2	2	.
INPUT ID HEIGHT WEIGHT GENDER $ AGE;	8	2	3	.
DATALINES;	9	2	4	.
1,68,144,M,23	10	2	5	.
2,78,202,M,34	11	2	6	.
3,62,99,F,37	12	2	7	.
4,61,101,F,45	13	2	8	.
;	14	2	9	.
run;	15	2	10	.
PROC PRINT;	16	3	.	1
TITLE 'Example 2.1';	17	3	.	1
RUN;	18	3	.	1
DATA AMPERS;	18	4	1	.
INPUT NAME & $25. AGE GENDER : $1.;	18	4	2	.
DATALINES;	18	4	3	.
RASPUTIN 45 M	18	4	4	.
BETSY ROSS 62 F	18	4	5	.
ROBERT LOUIS STEVENSON 75 M	18	4	6	.
;	18	4	7	.
run;	18	4	1	.
PROC PRINT;	18	5	.	2
TITLE 'Example 4';	18	5	.	2
RUN;	18	5	.	2
/* this is comment*/	19	.	.	.

Re: Read sas program to create metadata

SASKiwi — Sun, 05 Apr 2020 20:31:09 GMT

How does analysing the structure of your SAS programs programmatically help to automate migration? I'm assuming you are migrating from SAS 9.2 to 9.4.

In my experience, the best way to migrate is to set up your target SAS servers as similar as possible to your source SAS servers then test all of your key programs in the new environment one by one manually. Running your jobs in batch mode is the most productive way. The way you are trying to approach it simply won't work because it is near impossible to analyse SAS programs with high accuracy.

Re: Read sas program to create metadata

Patrick — Sun, 05 Apr 2020 21:43:14 GMT

I second what @SASKiwi writes. May be tell us what you're trying to get out of such analysis. There might be other ways to get there.

It wouldn't be hard to write some code for the simple sample program you've posted but what when it comes to dynamic and data driven code (macros, call execute() etc.).

Re: Read sas program to create metadata

mnjtrana — Mon, 06 Apr 2020 07:00:50 GMT

We are actually migrating to sas 9.4 along with new sources of input. So we need to change few of the dataset names through lookup and few of the variable names being used in the assignment statement inside the datastep. so we need to know where the dataset starts, where it ends, what is the input datasdet name and what are the assignment or other statements inside dataset.

I know we wont be able to automate it 100% , but the program list is huge, so any manual effort reduced will help a lot.

Re: Read sas program to create metadata

Kurt_Bremser — Mon, 06 Apr 2020 13:06:27 GMT

When I want to know in which program a specific dataset or infile is used, I find all the logs from the previous month (because that covers ALL program runs) and grep for the dataset/filename:

find /sas/log -type f -name \*.log.202003\* -exec grep "MYLIB.MYDATA" {} \; -print

That's much easier than doing it with SAS.

Re: Read sas program to create metadata

Reeza — Mon, 06 Apr 2020 15:22:46 GMT

Try PROC SCAPROC for the higher level information automated.
Search on lexjansen.com for code and log analyzers. I'm more in the camp that you cannot automate your way out of bad documentation - especially if you have any macros or shortcut variable or data set lists.
Ensure you account for things like the following:

data want;
set have;
array _demo(5) demo1-demo5;
array _demo2(4) _temporary_ new_demo4-new_demo7;

rename demo1-demo5 = random1-random5;
run;

Re: Read sas program to create metadata

SASKiwi — Mon, 06 Apr 2020 20:51:37 GMT

@mnjtrana - Editors like Notepad++ can be very helpful here. They can search all SAS programs in a particular folder for example and list all occurrences of a search string like a dataset name that needs to change.