I have 5 queries in 5 different sas files that takes 5 hrs app. while running sequentially using %include in main sas code. I'm planning to run all of them in parallel so that they will complete in 3 hrs.
I tried using PIPE and X command by calling the shell scripts which are calling these five jobs in each, but is there any way to run them in parallel in main code itself?
I have successfully used SAS/CONNECT for this purpose. If you have this licensed then this could be an option. The trick is to use the WAIT = NO option when submitting the child jobs as this then allows the controlling SAS job to continue without waiting for the submitted child SAS job/session to finish.
For example running two parallel jobs from a controlling SAS session:
Please note that session1 and session2 could be defined as additional SAS sessions on the same computer you are working on or they could refer to a remote computer. The choice is yours.
There is the problem of not knowing when the child sessions finish their work so you can continue on afterwards with more processing. This can be solved by getting the child sessions to write a special file or dataset when they end. Your controlling SAS session can periodically check for these special files and only continue after they are created.
to support asynchronos remote processing (like sas mpConnect) SAS/Connect provides features like WAITFOR
Lifted from the on-line help and documentation [pre]/* Wait for both tasks to complete. */
waitfor _all_ task1 task2;[/pre] which comes in SAS/Connect Using Compute Services Example 5: Using MP CONNECT and the WAITFOR Statement
That collection of examples can be found at http://support.sas.com/documentation/cdl/en/connref/61908/HTML/default/a000585202.htm .