topic SAS Connect - Monitoring Rsubmit program progress in SAS Programming

SAS Connect - Monitoring Rsubmit program progress

buechler66 — Fri, 13 May 2016 15:08:42 GMT

Hi. I'm wondering if you have suggestions for ways to monitor the progress of a Rsubmit SAS Connect Program. Once I submit my program all I see is:

NOTE: Remote submit to ISASPROD commencing.

A good deal of my program is a looping macro that creates a dataset named QueryData and appends it to FinalData. This looping can happen up to 100+ times. Depending on server resources my program can take anywhere from 20 minutes or 8+ hours. I'd like to be able to check in and see how far along the program is. Any suggestions?

Re: SAS Connect - Monitoring Rsubmit program progress

LinusH — Fri, 13 May 2016 15:28:57 GMT

One idea if you have necessary rights is to create a file with the current loop counter.

Also, with a job taking up to 8 hours, I would suspect that is a candidate for overnight scheduling.

And, I'm also suspect that you could optimize this job quite a lot. 100+ looping often can be rewritten in a more efficient way (generally speaking, not knowing about your specifics of course).

Re: SAS Connect - Monitoring Rsubmit program progress

buechler66 — Fri, 13 May 2016 15:37:50 GMT

Hmm..how can I write out a filename? Maybe I could write a filename and append on the &rule_order somehow. Like QueryData_&rule_order perhaps?

Yah, each iteration is a Rule loop. And each record can only be flagged with the first rule's criteria it meets. And the rules are run in order of precedence from 1 to 100+. So if a record qualifies for Rule 1 then it cannot also qualify for Rule 2 on.

* RULE QUERIES. Build and execute rule queries based on the rules defined in QueryRules (one at a time looping) ;
%macro BuildQueryData(analysis_desc= , rule= , rule_order= );

	proc sql;
	create table QueryData as 
	(       select DISTINCT %str(%')&analysis_desc.%str(%') as RULE_NM length = 58, 
			b.actual_dlvry_date as AD_DT, 
			b.imb_code length = 31, 
			&rule_order as RULE_ORDER,
			b.spm_calc_batch_date
            from iv_ora.bi_spm_piece_recon a,  bids_ora.bi_spm_piece_recon b                                                                                                                                                                                                                                                                                                                                                                                                                                                   
            where a.spm_calc_batch_date = b.spm_calc_batch_date
              and a.imb_code = b.imb_code
            and &rule
	); 
	quit;

	* Append datasets to final dataset ;
	proc append base=FinalData data=QueryData force;
	run;

	* Create unique index (if not exist), to meet the criteria that only one record (imb) per ;
	* rule is kept and counted in the data ;
	data _null_;
		set sashelp.vmember (where=(Libname='WORK' and memname='FINALDATA'));
		if upcase(index) = 'NO' then call execute('%CreateUniqueIndex');
	RUN;

%mend BuildQueryData;