topic Re: autorun scheduled background job in parallel process in Developers

autorun scheduled background job in parallel process

rsb7678 — Thu, 21 May 2009 18:37:12 GMT

Hi

Have little knowledge on Submitting background jobs in Unix.
We have a script that run automatically as soon as the data gets updated.
I parallel process a job into three parallel process, see below. Each is independent of the other, uses different input files but follows the same logic in generating the output datasets.

script_file:

nohup sas9 /pgms_rx/process1_v2.sas &
nohup sas9 /pgms_rx/process2_v2.sas &
nohup sas9 /pgms_rx/process3_v2.sas &

I would like to run another sas program that convers the output of all these three processes into a single pipe delimited zipped file. I did wrtie a program to do this .
but how can i automate it, i.e., include it in the script file. Once all the above three process generate output datasets successfully run the pipe_delimited.sas program that generated the desired output.

Let me know if you have further questions.
Thank you.
RB

Re: autorun scheduled background job in parallel process

sbb — Sat, 30 May 2009 13:13:35 GMT

A few techniques come to mind, however I found a technical conference paper at the SAS support http://support.sas.com/ website which discusses parallel processing. And for your successor SAS program, you can use a technique that detects the presence (or lack) of "gate" file(s), such that when all are present you proceed with your follow-on SAS program execution.

Or, maybe your enterprise already has a job scheduler specifically for this purpose - maybe ask around.

Regardless, I'm sure you can find a suitable tool/technique with SAS - take advantage of the support website and its resources.

Scott Barry
SBBWorks, Inc.

Parallel Processing on the Cheap: Using Unix Pipes to Run SAS® Programs in Parallel
Ted Conway, Ted Conway Consulting, Inc., Chicago, IL
http://www2.sas.com/proceedings/sugi28/114-28.pdf