topic Re: lsf schedular in SAS Procedures

lsf schedular

SAS_1001 — Tue, 22 Apr 2014 14:20:07 GMT

Hello,

I'd like to schedula a SAS program in lsf, could you pl. guide me thorugh the process start-2-end. Let's say I'd like to deploy a SAS job abc.sas in lsf through process manager. How can I do that.

Thanks & appreciated.

Regards,

Paul

Re: lsf schedular

LinusH — Wed, 23 Apr 2014 11:26:07 GMT

I think that the general idea of integrating LSF and SAS, is to use the interfaces between those products. Which means that scheduling of jobs and flows are done via the Schedule Manager plugin in Managment Console.

How to do it this way, there is a pretty straight forward description in the on line documentation.

There are also some papers in the subject, just search support.sas.com.

Re: lsf schedular

Quentin — Wed, 23 Apr 2014 16:08:36 GMT

Gotta say, I still don't find the scheduling process to be straight forward. At least not if you're an old-school batch job SAS programmer, used to just scheduling batch jobs via cron.

I think this the relevant doc for 9.3 is: http://support.sas.com/documentation/cdl/en/scheduleug/64110/HTML/default/viewer.htm#titlepage.htm

Once I have a .sas file ready for scheduling as a batch job, here are the hoops I job through. Which I post in full hope that someone will point out how bad my process is. : )

1. In DI Studio I create a DI job which includes a single node of user-generated code. That code is an %INCLUDE statement which %includes my .sas program.

2. In DIS I deploy the job for scheduling.

3. In SAS Management Console Schedule Manager I create a flow which consists of the deployed job.

4. In SMC I schedule the flow.

phew.

Surely feels like there should be an easier way to say "Hey SAS Server, please run this .sas file daily at midnight." And likely there is.

Of course there are benefits to having job info in the metadata, even if the end the job is simply a simple command to batch submit a program.

If you get into scheduling with LSF, one great tip I picked up at NESUG (sigh) a couple years back is to install Flow Manager. Flow Manager is an LSF app, but SAS provides a stripped down version of it. It's proved very helpful to see what jobs are scheduled, which are running, history, etc. Particularly in a few of cases where I swear I unscheduled a flow (using SMC) and deleted the flow, but somehow it remained scheduled on the LSF side. Deleting the flow via Flow Manager did the trick.

-Q.

Re: lsf schedular

SAS_1001 — Wed, 23 Apr 2014 17:18:49 GMT

Thanks for the response. I was able to schedule a job in LSF but getting the following error. When running outside lsf thorugh EG or Base SAS, not getting these errors.

ERROR: Insufficient authorization to access PIPE.

ERROR: Error in the FILENAME statement.

Any advise would be greatly helpful.

Regards,

-P

Re: lsf schedular

Quentin — Wed, 23 Apr 2014 18:04:56 GMT

You get that error somewhere in your SAS code where you are trying to pipe OS commands, right?

Most likely NOXCMD is turned on for your batch server.

See eg:

https://platformadmin.com/blogs/paul/2011/06/noxcmd/

Can beg your admin to turn it off, or look for a solution that doesn't rely on OS commands.

Re: lsf schedular

SAS_1001 — Wed, 23 Apr 2014 19:00:38 GMT

Thanks for the response.

For some jobs, they are running fine, but some jobs they are failing due to this pipe error ? Do you think we need to assign any option the programs ?

Appreciaed feedback !

-P

Re: lsf schedular

Quentin — Wed, 23 Apr 2014 19:04:43 GMT

Maybe show more of the log from when you get this error.

If the issue is NOXCMD, it will be either on or off for the batch server used for scheduled jobs. So can't change this by setting an option in the program.

%put %sysfunc(getoption(XCMD)) ; should show the value of the option.

Re: lsf schedular

SAS_1001 — Wed, 23 Apr 2014 19:53:51 GMT

The value is being shown here is after running the above %put statement:

XCMD

How can I change to noxcmd ?

Re: lsf schedular

SAS_1001 — Wed, 23 Apr 2014 20:00:07 GMT

Quenten,

I'm scheduling the jobs through schedule manager and checking the flow from process manager. Jobs are running good in EG but in process manager, they get stuck up. No idea whats going on ?

Re: lsf schedular

Quentin — Wed, 23 Apr 2014 20:10:22 GMT

XCMD is what you want.

Did you run the %PUT statement in the job you scheduled? Scheduled jobs run on a different server (batch server) than code that you run in EG. So it's possilbe XCMD is allowed on the server where your EG session runs, but not on the batch server.

You have a SAS log file from the scheduled job that shows an error, right? Suggest you show that log. (key parts)

Re: lsf schedular

SAS_1001 — Wed, 23 Apr 2014 21:04:02 GMT

That's the issue. When I'm running through lsf I see the following in the log:

22 %put %sysfunc(getoption(XCMD)) ;

NOXCMD

How can I turn it to XCMD ? Pl. advise ! I guess this is the problem that I've been experiencing !

Re: lsf schedular

LinusH — Wed, 23 Apr 2014 21:19:24 GMT

Add -xcmd in the \Config\Lev1\SASApp\BatchServer\sasv9_usermods.cfg file.

Re: lsf schedular

SAS_1001 — Wed, 23 Apr 2014 21:37:22 GMT

Thanks LinusH,

Do I need to bounch the servers ? I ran the code after modifying sasv9_usermods.cfg but still seeing

NOXCMD !

Re: lsf schedular

SAS_1001 — Wed, 23 Apr 2014 22:32:15 GMT

Which admin (SAS or Windows). Where he has to turned this option off ?

Thanks for the input .

Re: lsf schedular

Quentin — Thu, 24 Apr 2014 00:04:38 GMT

SAS Admin. XCMD is an option that needs to be turned on in the config file for the batch server. LinusH gave the location of the default config file. If you have write access to that (would be surprising if you're not the SAS Admin), give it a shot. Conceivable that you would need to bounce the server. Regardless, whoever administers your SAS server should know how to set such SAS options. So I will stick with my earlier recommendation, call your SAS admin and say "pretty please can you turn on XCMD for the batch server?"

And if they say no (because they are mean, or because they don't understand how to turn it on while keeping stuff secure), then look at your code and see if you can change it to get the same result (you haven't told us *why* you're piping OS commands), with native SAS code. In fact, many folks would say even if XCMD is enabled, there are benefits to getting you job done with SAS code (portability etc).

Chris H has a nice post about XCMD:

http://blogs.sas.com/content/sasdummy/2012/08/31/the-case-for-xcmd-privileges-in-sas-enterprise-guide/

Re: lsf schedular

Patrick — Thu, 24 Apr 2014 00:21:25 GMT

LSF jobs run in batch so you shouldn't have to restart the server. Investigate what batch command LSF actually uses. This should show you which config files get executed and where you would have to change from NOXCMD to XCMD.

Alternatively: You could add -XCMD directly to the command line (done via SMC). I believe this would supersede most settings done somewhere else.

Re: lsf schedular

SAS_1001 — Thu, 24 Apr 2014 03:49:26 GMT

I added -xcmd command directly into the job's properties in SMC and it worked successfully with no issues !

Thanks everybody for the help extended here.

Best

-P

Re: lsf schedular

SAS_1001 — Thu, 24 Apr 2014 18:41:53 GMT

Just wondering what's the best method to notify programmers about job's fail and abend or even success through email and text messages from lsf ?