How to retain summary information across runs

[noling]

Here's a method I use to retain summary information across many SAS runs. Summary information could include information like number of records, run time, or file sizes. In this example I want to track how many records are in 3 tables, each time the SAS job executes.

General approach:

1. Make an empty table with the desired structure to hold desired information.
   1. Store on disk, not in saswork, since we want to read/write to this table often across batch jobs and SAS sessions
2. Loop though tables/fields/whatever you want to process and grab the desired information
3. Place summary information into temporary table, then add temporary table to permanent table
   1. Include timestamp of rundate
4. Monitor/reference permanent table as desired!

libname local 'xxxxx';

%macro log_summary_information(tables_to_process);
	
	*1. make empty summary table with correct metadata if it does not already exist;
	%if not %sysfunc(exist(local.stacked_counts)) %then %do;
		data local.stacked_counts;
			length table_name $50 record_count run_time 8.;
			format run_time datetime.;
			stop;
		run;
	%end;

	%let to_loop_count = %eval(%sysfunc(countc(&tables_to_process," "))+1);
	%put will process &to_loop_count tables &tables_to_process;

	*2. loop through tables and get desired information;
	%do i = 1 %to &to_loop_count;
		%let table = %scan(&tables_to_process,&i," ");
		%put Processing &table;

		*2. grab record counts/desired information;
		%let dsid = %sysfunc(open(&table));
		%let count =%sysfunc(attrn(&dsid,nobs));
		%let rc = %sysfunc(close(&dsid));
		
		*3. create summary record and append to growing summary table;
		data temp_record;
			table_name ="&table";
			record_count=&count;
			run_time=datetime();
		run;
		proc append base=local.stacked_counts data=temp_record force;
		run;
	%end;
	
%mend;
%let tables_to_process=sashelp.cars sashelp.class sashelp.baseball;
%log_summary_information(&tables_to_process)

I ran this twice, and the record counts are the same (duh - referencing static SASHELP tables), but this could change depending on your context.

Thoughts?

[tomrvincent]

Cool...though I think I'd want to add a check of the date created/modified so I didn't add duplicate records (replacing 'run_time' with the max of created/modified).

dateCreate = finfo(_fid, "Create Time");
dateModify = finfo(_fid, "Last Modified");