Grid option set in DIS and EG

[LukeL]

Hi all... I'm trying to better configure our grid environment by making better use of queues, options and so on.

We have a 4 nodes grid, in 9.4M1 with LSF 8.01, with a grid launched WS. Authentications are all set as host authentication.

Actually we have, among others, 2 grid option sets, defined for developers:

- the first one is for Data Integration Studio, and define all jobs launched using queue DEV

- the second one is for SAS EG, doing the same

First problem: the DIS option set is correctly applied. The EG one is not. Programs launched by devs in EG always use the default queue, and not the DEV one.

Second problem: in DIS, when defining a loop that should execute in parallell, we need to set the grid option set to use. We would like to able to say to the platform that if we are on DIS, so devs are launching the job, it should use the DEV queue, but if the job is scheduled and run in batch, should use the BATCH queue. Is this possible? Actually, the main job use the BATCH queue as definened in the job props in MC, but the jobs inside the loop goes on the DEV queue.

Third problem/question: instead of defining for each scheduled job in the MC the queue to use, is there a way to define the default queue to use for batch server? (setting the default queue for the LSF is not an option, since we have many SAS stack sharing the same architecture and the same LSF environment... since we use host authentication, a solution per os-user could be also good, if not possible via some option)

Thanks in advance 🙂

[AndreasWindisch]

Hello Luke,

we are running also a Grid but with different version (SAS 9.4 M3 to M5) and LSF 9.1.3.

1. How Do you set the queue definition in the grid options set? I use here queue=”<target queue>” in the Grid options field.
2. It’s a question how you run the batch jobs. We are using the SASGSUB for batch jobs with the ability to control most of the options. Which scheduler will you use, Plattform Process Manager?
3. There is an option for making SAS Applications gridCapable if they are not by default, maybe this is an option?
http://support.sas.com/documentation/cdl/en/gridref/67371/HTML/default/viewer.htm#n1inymfs0b7go2n147...

Also you could have a look to the ‘esub’ feature of LSF. It is described in ‘Administering Platform LSF’ under the topic ‘External Job Submission and Execution Controls’. It is an external script (bash) which can change a job’s Options. In your case you can switch the queues or something else based on the user or something else. We use it for the things not are possible under the standards (e.g. Connection Services, debug feature) for direct them to the grid. It is very handy option but could be a potential performance bottleneck if the script run’s long time.

Best regards,
Andreas