batch jobs simultaniously

[rsb7678]

Hi

Have little knowledge on Submitting background jobs in Unix.
We have a script that run automatically as soon as the data gets updated.
I parallel process a job into three parallel process, see below. Each is independent of the other, uses different input files but follows the same logic in generating the output datasets.

script_file:

nohup sas9 /pgms_rx/process1_v2.sas &
nohup sas9 /pgms_rx/process2_v2.sas &
nohup sas9 /pgms_rx/process3_v2.sas &

I would like to run another sas program that convers the output of all these three processes into a single pipe delimited zipped file. I did wrtie a program to do this .
but how can i automate it, i.e., include it in the script file. Once all the above three process generate output datasets successfully run the pipe_delimited.sas program that generated the desired output.

Let me know if you have further questions.
Thank you.
RB

[ChrisNZ]

If you used a sas session rather than a script file, you could use MPCONNECT to drive and control your parallel processes, and run your final routine upon completion of all 3 parallel processes.

If you want to use a script, you probably have to launch the 4th process at the same time as the first ones, and have the 4th process wait and monitor the completion of the others before starting its own processing.

If you are unsure how to detect completion, maybe create a dummy completion table/file/environment variable in the other processes that you can delete from the last one?

[lu]

Hi

You can use also
SYSTASK Statement: UNIX

http://support.sas.com/onlinedoc/913/getDoc/en/hostunx.hlp/a001215125.htm

this statment work only on Unix/Linux hosts.

Have a nice day :),
lu

[rsb7678]

Firstly let me thank you.

The script is set in such a way that it runs automatically as soon as there is a new data feed. This is the reason i cannot remove fromt he script.

Can you please explain with example how can I run the fourth program after the successfull completion of program 1, 2 and 3 automatically.

[ChrisNZ]

How about you run

nohup sas9 /pgms_rx/process1_v2.sas &
nohup sas9 /pgms_rx/process2_v2.sas &
nohup sas9 /pgms_rx/process3_v2.sas &
nohup sas9 /pgms_rx/process4_v2.sas &

and process4_v2.sas waits for the others to finish?

[rsb7678]

Hi Graffeuille ,

When I have an & after the program name that means the jobs can run parallelly. i need a logic using which SAS / UNIX can control and automate a multiple SAS programs process.

Thanks
RB

[ChrisNZ]

Yes, that's the point:
- start programs 1 to 4,
- when programs 1 to 3 complete, they raise a flag (create a file, set an environment variable, send a named pipe message, whatever).
- program 4 waits for the 3 flags to be there, and then starts its processing.