Submitting jobs in Queues using SAS Enterprise GUIDE

[Madhan_cog1]

Hi All,

We have installed and configured SAS 9.4 M5 Grid Version in a Linux environment.

We have configured four queues for job submission 

1.Normal

2.Priority

3.Big

4.Big Priority.

We are trying a submit a job with priority queue through EG.

By default EG Job runs in normal queue.

We have set the options based on few support papers 

https://support.sas.com/resources/papers/proceedings16/11100-2016.pdf

But still EG job always runs on normal by default.

Could you please suggest here.

Some screenshots attached in the document.

Thanks,

Madhan M.

[Anand_V]

Quick check: You have added users to EGforPowerusers configuration in SMC, right? I don't see that in the snaps attached.

[Madhan_cog1]

Yes its added

Thanks,

Madhan M

[Anand_V]

Are you able to submit jobs via bsub to these queues using cmd?
Also, would you be able to share the Q configuration?

[Madhan_cog1]

Yes Anand .

We were able to submit jobs to these queues using cmd.

Am attaching the queues file.

Thanks,

[gwootton]

When a grid-launched Workspace Server is requested from the Object Spawner, the spawner performs a check for any grid options sets associated with the requesting user and application.

In your example the jobs are not being submitted by the Object Spawner but by Enterprise Guide directly. You can see this based on the job name "EG*" instead of SAS Enterprise Guide.

This means you are not using grid-launched Workspace Servers but the grid functionality built in to Enterprise Guide, which does not use Grid Options Sets or specify a queue.

Switching your Workspace Servers to grid-launched and turning off the Use grid if available option in Enterprise Guide would allow the options set to be used, and prevent duplicate SAS sessions (the client Workspace Server session Enterprise Guide is using to start the grid sessions would not be required).

Configure SAS Enterprise Guide for Grid Computing

Using SAS Grid Manager or SAS Grid Manager for Platform for Server Load Balancing

[doug_sas]

You do not need the IsGridCapable setting - SAS Enterprise Guide is already grid capable (you should see it as a application when creating a grid options set without this setting).

Your pictures do not show who you have selected to be able to use the option set in your doc. An option set mapping has an application (SAS Enterprise Guide), an options set (EGforPowerusers), and user/groups that this options set mapping should be applied to. Do you have the metadata ID of the user you want to use the options set selected? When you signed into EG, did the credentials for that user map to the metadata ID of the user in the options set mapping?

Example:

1) I have a logical workspace server setup to do grid-launched workspace server load balancing for application server context 'SASApp'

2) I have a logical grid server with an options set mapping for 'SAS Enterprise Guide' to the options set 'EGforPowerusers' for the metadata ID 'Doug' for the application server context 'SASApp'.

3) When I log into EG, I use the OS user ID 'doug_sas'.

4) I have a User record in SAS Management Console's User Manager for the metadata user 'Doug' that has an account for the DefaultAuth domain with a user ID of 'doug_sas' so that the OS user 'doug_sas' is known in metadata as 'Doug'

When EG goes to launch a workspace server from the SASApp server context, the object spawner logs into the metadata server with the OS credentials and gets the grid information for the logical grid server. The grid information should see that the request is for 'SAS Enterprise Guide' grid application for the metadata user 'Doug' for the application server context 'SASApp' and use the queue from the EGforPowerusers options set.

[Anand_V]

I agree with Greg. Workspace servers should be enabled as grid launched. Also the queue doesn't have users property. As I understand this isn't required for LSF admin accounts but for other users I think it is required.

Did you try to submit a job using normal user and it worked?

USERS
A list of users who are allowed to submit jobs to this queue. LSF administrators can submit jobs to the queue even if they are not listed here.
User group names have a slash (/) added at the end of the group name. Use the bugroup command to see information about user groups.

If the fairshare scheduling policy is enabled, users and LSF administrators cannot submit jobs to the queue unless they also have a share assignment

https://www.ibm.com/support/knowledgecenter/SSETD4_9.1.3/lsf_command_ref/bqueues.1.html

[Madhan_cog1]

Hi All,

Thanks for the response.

As per the instructions we have converted the workspace servers as grid launched.

The jobs in EG are running now always in priority queue.

We have set the properties in extended attributes as mentioned in the documents and papers.

 we have used the values Ignore,NoForce in the extended attributes for EGGridPolicy and AOMGridPolicy. But still the job is running as Grid launched server.

We want the user to select the 'grid options if available' from SAS EG.

Kindly suggest.

Thanks,

Madhan M.

[doug_sas]

Back in 9.2 and 9.3 there were no grid-launched workspace servers as there are in 9.4. To simulate grid-launched servers back then, the spawner-launched workspace server would spawn a grid-launched CONNECT server (a.k.a., grid server), and all code submitted to the workspace server was wrapped in a RSUBMIT and sent to the CONNECT server. To make the workspace server spawn a grid server for processing, EG added the 'Use grid if available' option.

With the advent of true grid-launched workspace servers in 9.4, the EG 'Use grid if available' is no longer needed since the workspace server itself is launched in the grid. If you use the EG 'Use grid if available' in conjunction with a grid-launched workspace server results in two grid jobs - one for the workspace server and one for the spawner CONNECT server.

So in short, always use grid-launched workspace servers and do not use the EG 'Use grid if available' or 'Initialize Grid (if available)' options. Later versions of EG will detect that grid-launched workspace servers are being used and will disable the 'Use grid if available' or 'Initialize Grid (if available)' options; doing automatically what adding the EGGridPolicy/AMOGridPolicy set to Ignore does manually.

[anguraj1]

Hi Doug,

Thanks for the response. But when we use the EGGridPolicy/AMOGridPolicy to Ignore/NoForce option in SAS Grid server properties will that no take effect ? because we have defined set of users (for ex: User A ) defined in the Logical grid server for SAS Grid power users capability to run the jobs from EG with "high priority queue".
When ever the user submits the job thru SAS EG it is going to "high priority queue" only not on Normal queue, we need to avoid that based on the extended attributes "EGGridPolicy" set to ignore.
Thanks,
Anguraj Selvaraj

[doug_sas]

I see that I am not making this clear.

There are two ways to run SAS code through EG on a SAS server launched in the grid. The old method (spawner launched workspace server spawning a grid-launched CONNECT server) is controlled by EG since it is EG that adds code to the workspace server initialization to launch the grid-server. The new method (grid-launched workspace servers) is controlled by the object spawner and is independent of settings in EG. Options set mappings are applied by the object spawner for grid-launched workspace servers.

To sum it up, you do not need to do anything in EG to get the benefit of grid-launched workspace servers and options set mappings. In fact, if you are using EG with grid-launched workspace servers, you should set the EGGridPolicy/AMOGridPolicy to 'Ignore' so that EG does not try to auto-submit code to the newly launched workspace server to attach to a grid-launched CONNECT server.

[Madhan_cog1]

Hi Doug,

Thanks for the response.

How can a user define the queues if they run from SAS EG.

Thanks,

Madhan M.

[doug_sas]

You do not want users to define the queue they run in; you want the SAS administrator to define which queue the user runs in.

If users could define the queue they run in, all users would chose the highest priority queue since they think their work is the most important. 😁

Remember, the queue just determines the launching of the workspace server. Individual code submits do not start a new workspace server, they just send new code to the same workspace server.

[gwootton]

This documentation talks more about the EGGridPolicy and AMOGridPolicy settings:

Using SAS Enterprise Guide and SAS Add-In for Microsoft Office with a SAS Grid

Essentially, as @doug_sas mentions, when using grid-launched workspace servers you would want to set these to "Ignore" to prevent the client application (EG or AMO) from launching a second session to the grid from the workspace server which is already running as a grid job. In this scenario the grid-launched workspace server uses grid options sets specified in SAS Management Console to apply changes like queue selection based on the user and application requesting the workspace server.

If you were trying to launch child grid jobs from the grid-launched workspace server (for parallel execution for example), you can use GRDSVC_ENABLE to start child grid job sessions and connect to them using SAS/CONNECT like EG and AMO did behind the scenes when "use grid if available" was in play. With that function you can specify job options like queue or manually reference a grid option set.

GRDSVC_ENABLE Function