10-07-2013 06:49 AM
How to reliable schedule 10 large .egp files that run in sequence and are dependent on each other's result?
The default option in EG using a Windows-scheduler-based VBS script on my workstation is not an option.
Also running .BAT script is a no-go.
Converting the .egp files to .sas and importing+scheduling them as an Integration Studio job is a long path to go for... and maintain on the long run.
10-07-2013 08:16 AM
You export the EG projects to SAS code (.sas files). Then you write a script which simply batch submits each of these .sas files as a child process - starting the child processes synchronous.
eg. a very simple form of a .bat file
...sas.bat -sysin 'prog1.sas'
....sas.bat -sysin 'prog2.sas'
This would execute one .sas program after another. Of course in real life you also want to add some error handling and check after each .sas the return code and only continue execution if the program was successful.
You can also write such a script using SAS. Lookup "systask()" together with options "xwait" and "xsync" and statement "waitfor".
If you're having DIS and LSF: You could also create one DI job per EG project. If you don't want to import the .sas file into DIS then you could also simply have a single user written transformation in this DIS job with nothing else than a %include to your .sas file. And then you schedule these DIS jobs as you would with any other.
And what I forgot to say:
I assume your statement that vbs and .bat is no option is mainly due to the fact that you want to run the scripts on the server without involvement of your workstation. So for both the .bat and the SAS script option with "systask()" you would use the OS scheduler (Task Scheduler, Crontab) and start the master script time triggered.
10-07-2013 08:20 AM
You can also write your own script to chain together these EGP files, and then schedule a single script. See this page for links to blogs, a paper, and example scripts: