topic Re: How to find the Inactive Grid node programmatically in SAS Enterprise Guide

How to find the Inactive Grid node programmatically

yanamadala85 — Wed, 28 Feb 2018 16:29:34 GMT

I am using RSUBMIT method to distribute the jobs parallel. I have scenario where I am not able to find the solution.

1) I submitted the RSUBMIT blocks and distributed the jobs to different grids

2) due to some reason middle of the programs execution one of the grid node is unavailable/failed

Is there any way to find the unavailable/unresponsive/ Idle grid node?

Re: How to find the Inactive Grid node programmatically

JuanS_OCS — Thu, 01 Mar 2018 08:10:49 GMT

Hello @yanamadala85,

yours is a very good question. I think your answer is in checking the status of the Connect Spawner and the Object Spawner in every node of your grid.

UI-wise, RTM or Grid Monitor will help you.

Now, command-line-wise, you could go for one of these options:

- check the logs of the spawners

- check the status of the services with ego commands

- Run a Gridtest_fast.sas or Gridtest.sas before executing your program https://support.sas.com/rnd/scalability/grid/download.html however I think this might be a bit overkilling if you plan to run it before every program.

Nevertheless, I think that besides the checks, you rather would configure High Availability (HA) in your grid environment, precisely to ensure that,if any grid or service goes down, the service will start in other machine for the time being. On this way, your programs should not fail.

For all of that and more, you can check:

https://support.sas.com/rnd/scalability/grid/HA/gridha.html

Please note that most of those ways are for the administrators of your environment to execute. I mean, a programmer/user should not really be concerned about availability of the environment. If technical problems arise, you open a support ticket and other team would take care of it in your behalf. 🙂

Re: How to find the Inactive Grid node programmatically

yanamadala85 — Thu, 01 Mar 2018 08:31:46 GMT

Hi JuanS_OCS,

Thanks for the reply.

We are not using the EG UI for submitting the SAS jobs. In this Scenario I can not event configure the (HA) and I can't check the logs of the spawners.

I have another question, Is there any way to find the allocated memory and available free memory a server (Grid node) grammatically or by commands?.

Re: How to find the Inactive Grid node programmatically

JuanS_OCS — Thu, 01 Mar 2018 08:39:32 GMT

Hi @yanamadala85,

a couple of comments:

As mentioned, all of that is for your administrators
Are you capable / authorised to run X commands?

Re: How to find the Inactive Grid node programmatically

yanamadala85 — Thu, 01 Mar 2018 09:07:31 GMT

I am not authorized to run the commands. I asked to know that if any thing available or not? .

Mainly I am looking programmatic way to know the find the inactive grid server and available Free memory of the server.

Re: How to find the Inactive Grid node programmatically

JuanS_OCS — Thu, 01 Mar 2018 09:59:59 GMT

I don't think you can, without running X commands or running commands directly on your server's shell interface.

To run X commands is a high level risk, hence the reason because it is normally disabled. It is normally enabled only for few high level users or administrators.

All above make my advise to stand: please align with your SAS or system administrators. Open communication with them and ensure they understand your challenge, which is, in the end and probably, the challenge of your company business.

Re: How to find the Inactive Grid node programmatically

yanamadala85 — Thu, 01 Mar 2018 10:36:45 GMT

Thank you JuanS_OCS. I also felt the same.

I have general question, Any way to submit the jobs only to particular grid node. We have four Grid nodes, I want to submit the rsubmit blocks to only one Grid node.

Re: How to find the Inactive Grid node programmatically

yanamadala85 — Sat, 03 Mar 2018 04:24:51 GMT

I have a question regarding looping of resubmit blocks.

I want to submit 30 programs parallel.

Which one will be the best way to do rsubmits.
A. Writting 30 individual rsubmits
B. Looping rsubmits

%macro loop;
Preprocessing like enabling sas sessions on grid and autosignon option and
etc;
Do i= 1 to n;
Rsubmit task&i wait=no;
Some sas program....;
End rsubmt;
Waitfor _all_;
Some sas program;
Signoff _all_;
%mend;
%loop;

Or

Rsubmit task1 wait=no;
Some program;
End rsbmit;
....

Rsubmit task30 wait=no;
Some program;
End rsbmit;

Re: How to find the Inactive Grid node programmatically

Patrick — Sun, 04 Mar 2018 02:38:16 GMT

@yanamadala85

"I have general question, Any way to submit the jobs only to particular grid node."

Setting up workload balancing is a SAS Admin task. I don't know why you would only want to use a single node but yes, that can be done by defining a queue which only hits a single node. Setting this up is a SAS Admin task so discuss your requirement with this person at your site.

For execution of 30 jobs in parallel: First of all the LSF queue you're using must allow for 30 jobs in parallel else whatever you set-up won't execute as parallel as you believe. Secondly: If possible use LSF to define your flows and define parallelism and job dependencies there.