Have little knowledge on Submitting background jobs in Unix.
We have a script that run automatically as soon as the data gets updated.
I parallel process a job into three parallel process, see below. Each is independent of the other, uses different input files but follows the same logic in generating the output datasets.
I would like to run another sas program that convers the output of all these three processes into a single pipe delimited zipped file. I did wrtie a program to do this .
but how can i automate it, i.e., include it in the script file. Once all the above three process generate output datasets successfully run the pipe_delimited.sas program that generated the desired output.
Let me know if you have further questions.
If you used a sas session rather than a script file, you could use MPCONNECT to drive and control your parallel processes, and run your final routine upon completion of all 3 parallel processes.
If you want to use a script, you probably have to launch the 4th process at the same time as the first ones, and have the 4th process wait and monitor the completion of the others before starting its own processing.
If you are unsure how to detect completion, maybe create a dummy completion table/file/environment variable in the other processes that you can delete from the last one?
Yes, that's the point:
- start programs 1 to 4,
- when programs 1 to 3 complete, they raise a flag (create a file, set an environment variable, send a named pipe message, whatever).
- program 4 waits for the 3 flags to be there, and then starts its processing.