Architecting, installing and maintaining your SAS environment

Scheduling SAS Flows

Accepted Solution Solved
Reply
Occasional Contributor
Posts: 18
Accepted Solution

Scheduling SAS Flows

Hi All,

 

I would like to know what are the various 3rd party tools (apart from LSF) available for scheduling and monitoring of SAS Flows?

 

The client is looking to replace LSF and wants to know what are the various options available.

 

 

Regards,

Ankit.


Accepted Solutions
Solution
‎10-05-2015 09:31 AM
Super Contributor
Posts: 408

Re: Scheduling SAS Flows

Hi Ankit,

 

Actually any decent scheduling tool would be possible. I have seen OPC, TSM, Ctrl-M and a couple of others being used. Also, SAS offers the Operating System Services scheduling, which is quite capable but not really an enterprise class solution.

 

What I see now is that SAS is gearing towards Hadoop (Yarn) for orchestration of the workload. Very interesting development. Maybe even at the expense of LSF in the long run since it is now an IBM product. But that's just me saying.

 

Just bear in mind that none of the alternatives offer the same amount of out-of-the-box integration with SAS and the metadata as LSF. You must yourself bridge the gap between the scheduling metadata and your solution of choice. This is not an easy task. Therefore I have seen many companies live with LSF alongside their strategic choice.

 

Alternatively one can skip the scheduling metadata altogether and just deploy code and create schedules in whatever product calling SAS (preferably the logical batch server) for execution of this code.

 

It would be interesting to learn why your customer wants to move away from LSF. That would also help in answering this question. For which in my opinion there is no single best answer.

 

- Jan

View solution in original post


All Replies
Solution
‎10-05-2015 09:31 AM
Super Contributor
Posts: 408

Re: Scheduling SAS Flows

Hi Ankit,

 

Actually any decent scheduling tool would be possible. I have seen OPC, TSM, Ctrl-M and a couple of others being used. Also, SAS offers the Operating System Services scheduling, which is quite capable but not really an enterprise class solution.

 

What I see now is that SAS is gearing towards Hadoop (Yarn) for orchestration of the workload. Very interesting development. Maybe even at the expense of LSF in the long run since it is now an IBM product. But that's just me saying.

 

Just bear in mind that none of the alternatives offer the same amount of out-of-the-box integration with SAS and the metadata as LSF. You must yourself bridge the gap between the scheduling metadata and your solution of choice. This is not an easy task. Therefore I have seen many companies live with LSF alongside their strategic choice.

 

Alternatively one can skip the scheduling metadata altogether and just deploy code and create schedules in whatever product calling SAS (preferably the logical batch server) for execution of this code.

 

It would be interesting to learn why your customer wants to move away from LSF. That would also help in answering this question. For which in my opinion there is no single best answer.

 

- Jan

Occasional Contributor
Posts: 18

Re: Scheduling SAS Flows

Hi Jan,

 

Many thanks for your response.

 

I completely agree with you when you say LSF has the best integration with SAS.

 

I have recently joined the client and it seems they have upgraded to LSF 9.1 recently and are using it along with SAS 9.3

 

Since the LSF upgrade they have been facing weird issues (I am currently trying to understand those and resolve them), but they claim that tracks with IBM have revealed most issues are at LSF side.

 

Having said that client wants to look at other options while we resolve the currently ones being faced with LSF.

 

Regards,

Ankit.

Valued Guide
Posts: 3,208

Re: Scheduling SAS Flows

Ankit, your post is not very clear on the cause of the issues.

LSF is rather a complicated product/tool to understand (steep learing curve) once through it is more easy as all schedulers are sharing those same principal of operations (POP). 
When it are LSF problems those issues should be able to find at IBM LSF support/knowledgecenter It is rather easy to make a mess of this when that is not understood. SAS is trying to replace the common LSF interface by a SAS SMC approach. There must be basic LSF commands behind those like bsub.

Some error as basic Unix compliancy.

http://support.sas.com/kb/48/937.html 

 

The RTM is an dedicated one http://support.sas.com/resources/papers/proceedings12/370-2012.pdf

9.4 stille there is are the basics 9.4 rtm sas grid

What you are signalling is a frustration in the completion of the migration by ...?... and the service by SAS to solve those issues. 

---->-- ja karman --<-----
Occasional Contributor
Posts: 18

Re: Scheduling SAS Flows

Hi Ja Karman,

 

Thanks for sharing the articles. I agree to the fact LSF is one of the best scheduling tool to go along with SAS.

 

As I mentioned before I have just started with the client and am more on the front to understand the issues being faced and the root cause against the same.

 

Since the client has requested for a provision of alternate tools that can be considered hence I raised it out here asking of other well know and good to use tools.

 

This question is raised only to know what alternates are available for scheduling and monitoring jobs/flows.

 

Regards,

Ankit.

Valued Guide
Posts: 3,208

Re: Scheduling SAS Flows

Ankit, the only other scheduling approaches I have seen are based on classic batch processing (runnin the .sas code) on a single machine. With those having dedicated exploitation teams devoted to their tools and their singularity of "running the production".    

---->-- ja karman --<-----
Super User
Posts: 3,108

Re: Scheduling SAS Flows

Do your "weird" issues include changing job flow properties but these are not showing when you re-run the job? We too use LSF 9.1 and this appears to be one of the "features" with this version.

 

The workaround is to delete any deployed jobs then redeploy them in SMC, and also refresh scheduling server properties.

 

We tracked the problem to SAS and they said it would be fixed in a later release. 

☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 6 replies
  • 583 views
  • 1 like
  • 4 in conversation