Architecting, installing and maintaining your SAS environment

Some problems of scheduling

Accepted Solution Solved
Reply
Contributor
Posts: 28
Accepted Solution

Some problems of scheduling

I have a problem about scheduling.Now,I have deployed a SAS server and wanted to use clients to communicate with the server.I want to schedule different jobs from different clients.For example,the first  job is waited until the second is done but the submitted time of first job is earlier than the second one's.


Accepted Solutions
Solution
‎11-10-2014 08:10 AM
Trusted Advisor
Posts: 1,141

Re: Some problems of scheduling

In order to schedule you have different options, please choose the one that it fits more into your needs:

- SAS provides an out-of-the-box Scheduler server that you can configure within your SAS Management Console. By default, this Scheduler server will schedule on your OS task scheduler (windows task scheduler, cron, etc), with simple options.

     In this case, if you want to implement triggers and advanced scheduling options, you would want to implement them with batch code yourself.

- SAS also provides their own job scheduler, called Process Manager, and can also run with LSF. This will include new options into your SAS Management Console and a couple of new programs on your server (the Flow manager and calendar editor).

     This option have an extra cost on license and additional software will be installed and configured on your server and clients. And will include special scheduling features and triggers (file and time events, and conditionals AND/OR, etc).

- You can also include the calls to your SAS programs inside your company's scheduler (many options), or implement a new one of your choice.

     This will provide you the option to integrate your SAS processes with other processes in your company and you can make use of the same kind of triggers. Even though this will not be integrated in your SAS Management Console and you will still need to do some manual work, it is a possible option to consider.

- You have also the "hardcore" option. You can implement the calls to SAS, the triggers and everything and do the manual scheduling on your OS! Smiley Wink Hope you won't chose this option, but, still, it is very plausible and you will have full control.

I hope I am not missing any option... as you can see, may options, with pros and cons for your and your business purposes.

If you need some additional information and support, please share, or also you can contact your local SAS Support office. I am sure they will be happy to feed you with the information you need.

View solution in original post


All Replies
Trusted Advisor
Posts: 1,141

Re: Some problems of scheduling

Hi,

I am unsure about your point with the clients and the server on the deployment.

In other hand, maybe there is something that can help you with the scheduling of the jobs.

One option that you can try is to make Job1 to wait until a file appears in a single location. Then, Job1 will be waiting until the file in created in the location.

What you need to do with this option is: to make sure Job2 creates the file at the end of its execution. And to make sure Job1 deletes the file, also, along the execution (I would advice to do it just when Job1 starts to execute. In this way, you don;t need to be concerned about multiple Job1 execution, or dependencies with other jobs).

There are other options, but I hope this one can help you with the basics.

Contributor
Posts: 28

Re: Some problems of scheduling

I mean that I have one server and many clients(users) which are supported to different people.Every client(user) can connect to the server to submit their codes and then receive results.As a administer,I want that  their jobs are executed under my order which I design.Is there a tool like SAS Management Console helping me to design the order?

Trusted Advisor
Posts: 1,141

Re: Some problems of scheduling

Yes, there is. Do you have licensed Process Manager and/or LSF?

If not, maybe you would want to use one of your company's schedulers to run SAS code and implement triggers like the one I commented before

Super User
Posts: 6,926

Re: Some problems of scheduling

What scheduling system does your organisation use?

I recommend expanding that into your SAS environment.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Solution
‎11-10-2014 08:10 AM
Trusted Advisor
Posts: 1,141

Re: Some problems of scheduling

In order to schedule you have different options, please choose the one that it fits more into your needs:

- SAS provides an out-of-the-box Scheduler server that you can configure within your SAS Management Console. By default, this Scheduler server will schedule on your OS task scheduler (windows task scheduler, cron, etc), with simple options.

     In this case, if you want to implement triggers and advanced scheduling options, you would want to implement them with batch code yourself.

- SAS also provides their own job scheduler, called Process Manager, and can also run with LSF. This will include new options into your SAS Management Console and a couple of new programs on your server (the Flow manager and calendar editor).

     This option have an extra cost on license and additional software will be installed and configured on your server and clients. And will include special scheduling features and triggers (file and time events, and conditionals AND/OR, etc).

- You can also include the calls to your SAS programs inside your company's scheduler (many options), or implement a new one of your choice.

     This will provide you the option to integrate your SAS processes with other processes in your company and you can make use of the same kind of triggers. Even though this will not be integrated in your SAS Management Console and you will still need to do some manual work, it is a possible option to consider.

- You have also the "hardcore" option. You can implement the calls to SAS, the triggers and everything and do the manual scheduling on your OS! Smiley Wink Hope you won't chose this option, but, still, it is very plausible and you will have full control.

I hope I am not missing any option... as you can see, may options, with pros and cons for your and your business purposes.

If you need some additional information and support, please share, or also you can contact your local SAS Support office. I am sure they will be happy to feed you with the information you need.

Contributor
Posts: 28

Re: Some problems of scheduling

Thank you so much for your detailed answer.Smiley Wink

Contributor
Posts: 28

Re: Some problems of scheduling

Hi,What a pity!My company didn't buy the license of Process Manager.I wanna try the OS task scheduler what you commented.Do you have some study documents about that?

Another question.In my company,the data was stored by every day.If I want analyze one mouth data,I would read 30 related data files.As a administer,I want to limit the number of files while reading data(15 data files once,one mouth data need 2 times to read ) .I don't know whether SAS tools like SAS Management Console can make this limit.

Super User
Posts: 6,926

Re: Some problems of scheduling

Doesn't your company have a scheduling system? Or are you the only one who runs jobs regularly?

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Trusted Advisor
Posts: 1,141

Re: Some problems of scheduling

Kurt, I suppose jackhost was expecting to be able to try the schedule from the SAS MC.

jackhost, remember that you have different option for the scheduling. Process Manager is only of the tool available for you.

Btw: if you want to test simple scheduling from SAS MC, you don need Process Manager, just your Operating System services registered on your metadata (see under Server Manager), and make use of the Schedule Manager plug-in in the metadata.

Here I write for you some documentation from the SAS technical support:

Scheduling in SAS(R) 9.3

https://support.sas.com/rnd/scalability/platform/PSS6.1/pm8.1_using.pdf

http://support.sas.com/documentation/cdl/en/scheduleug/64110/PDF/default/scheduleug.pdf

An intresting overview that I quickly found googling: http://scorpiosoftware.blogspot.nl/2011/04/platform-suite-for-sas-devtestprod-tips.html

For your second question, I would advise to open a new thread so it will have more visibility to all the SAS community Smiley Wink

Contributor
Posts: 28

Re: Some problems of scheduling

No,For now,I am the only person to run jobs .our aim is to build a platform for the whole company to analyze our data which stored in one place.every user can read and analyze this data.If there are 10 or more users to read at the same time,my leader is worry about the performance of  SAS server.So we want to make the users read data by our order.

Trusted Advisor
Posts: 1,141

Re: Some problems of scheduling

I am not sure if there is an specific option for what you are asking for. I will check it.

What I can tell right now is that you can have 2 approaches:

1- Instruct your users to work on the way you require for performance/best practices purposes (I think it is preferable)

2- If you cannot trust your users behaviour, you can always play with permission in the Operating System and in the metadata. Setting the visibility on or off a couple of times per day. But I strongly believe that is not the best way to work.

And you can have always an overview of the performance with the log and some options like fullstimer.

Super User
Posts: 6,926

Re: Some problems of scheduling

And, if the users use fixed reports on changing data, consider to run the reports in batch and supply the results via webpage, mail, etc.

---------------------------------------------------------------------------------------------
Maxims of Maximally Efficient SAS Programmers
Valued Guide
Posts: 3,208

Re: Some problems of scheduling

Just a correction Juan....   SAS does not have his own scheduler LSF SAS(R) 9.4 Intelligence Platform: Installation and Configuration Guide

Scheduling in SAS(R) 9.4 (SMC) Scheduling in SAS(R) 9.4 (DI). There is just a nice interface build between those tools.

Full scheduling is designed with all kind of events where jobs get triggered on and the machine and grid is load balanced with a prediction when jobs are ready.

This far beyond of scope of SAS and that is why you are seeing LSF (an IBM company) popping up that much for this kind of IT processing.

There are several more options.

- within java there is some timing event (seen at planned refresh reports midtier)

- With Eguide you can attach to the Windows tasks manager when get open (admin rights) and having a relative small environment (1-2 IT persons)

- Within SAS you can build own timed planning

  With Unix you can connect to Cron. With SAS Windows you could use: SAS(R) 9.4 Companion for Windows, Third Edition (wakeup)

  Of course only for relative small environments (1-2 IT persons)  the dependency on machines with hands on is high.

In the analytics field with mining one person can blow up any machine. At the same machine a several like 30 users can be active reporting analytics without causing heavy load. Only a thing to understand what is happing by who and why. This where WLS Cgroup prirority of load balancing at the OS level is importan.to understand and do so minimum work for stability. Ignoring this will let you run into trouble at some day.  

This is the reason why a present OS Scheduler on servers is often set as mandatory, having no choice to connect on those (external shared servers). It is a assurance of stability on server processes

---->-- ja karman --<-----
Trusted Advisor
Posts: 1,141

Re: Some problems of scheduling

Thanks for the correction Jaap... probably I missed to say that LSF and Process Manager are the officially supported 3rd party scheduling software (IBM develops this software), it can be purchased as separate product or included in some SAS Solutions, and if so, it would be included in the installation depot and license, includind an additional features to the Scheduler Manager plug-in in SAS Management Console, to be more exact and complete on the statement. Probably we can be even more complete.

My point with this remark is to say that I appreciate the correction/remark, hopefully jackhost also does. And my intention was to simplify the explanation and bring to the table only the specifics he required to know, including the option of the creative and business integration solutions Smiley Wink, with some documentation for his/the community reference, that probably will bring more concerns (as the installation of differences between operating systems!).

I like your extra link for the schedule of reports, as special suggestion from the main Scheduling document: Scheduling in SAS(R) 9.4

Even though it is just an additional scheduled "job"from the technical point of view, I think it is interesting to know it in front, for the business/functional listeners. I recommend to read that one.

☑ This topic is SOLVED.

Need further help from the community? Please ask a new question.

Discussion stats
  • 14 replies
  • 1208 views
  • 2 likes
  • 4 in conversation