- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content
Posted 05-21-2009 02:37 PM
(950 views)
Hi
Have little knowledge on Submitting background jobs in Unix.
We have a script that run automatically as soon as the data gets updated.
I parallel process a job into three parallel process, see below. Each is independent of the other, uses different input files but follows the same logic in generating the output datasets.
script_file:
nohup sas9 /pgms_rx/process1_v2.sas &
nohup sas9 /pgms_rx/process2_v2.sas &
nohup sas9 /pgms_rx/process3_v2.sas &
I would like to run another sas program that convers the output of all these three processes into a single pipe delimited zipped file. I did wrtie a program to do this .
but how can i automate it, i.e., include it in the script file. Once all the above three process generate output datasets successfully run the pipe_delimited.sas program that generated the desired output.
Let me know if you have further questions.
Thank you.
RB
Have little knowledge on Submitting background jobs in Unix.
We have a script that run automatically as soon as the data gets updated.
I parallel process a job into three parallel process, see below. Each is independent of the other, uses different input files but follows the same logic in generating the output datasets.
script_file:
nohup sas9 /pgms_rx/process1_v2.sas &
nohup sas9 /pgms_rx/process2_v2.sas &
nohup sas9 /pgms_rx/process3_v2.sas &
I would like to run another sas program that convers the output of all these three processes into a single pipe delimited zipped file. I did wrtie a program to do this .
but how can i automate it, i.e., include it in the script file. Once all the above three process generate output datasets successfully run the pipe_delimited.sas program that generated the desired output.
Let me know if you have further questions.
Thank you.
RB
1 REPLY 1
- Mark as New
- Bookmark
- Subscribe
- Mute
- RSS Feed
- Permalink
- Report Inappropriate Content
A few techniques come to mind, however I found a technical conference paper at the SAS support http://support.sas.com/ website which discusses parallel processing. And for your successor SAS program, you can use a technique that detects the presence (or lack) of "gate" file(s), such that when all are present you proceed with your follow-on SAS program execution.
Or, maybe your enterprise already has a job scheduler specifically for this purpose - maybe ask around.
Regardless, I'm sure you can find a suitable tool/technique with SAS - take advantage of the support website and its resources.
Scott Barry
SBBWorks, Inc.
Parallel Processing on the Cheap: Using Unix Pipes to Run SAS® Programs in Parallel
Ted Conway, Ted Conway Consulting, Inc., Chicago, IL
http://www2.sas.com/proceedings/sugi28/114-28.pdf
Or, maybe your enterprise already has a job scheduler specifically for this purpose - maybe ask around.
Regardless, I'm sure you can find a suitable tool/technique with SAS - take advantage of the support website and its resources.
Scott Barry
SBBWorks, Inc.
Parallel Processing on the Cheap: Using Unix Pipes to Run SAS® Programs in Parallel
Ted Conway, Ted Conway Consulting, Inc., Chicago, IL
http://www2.sas.com/proceedings/sugi28/114-28.pdf