10 jobs in the same time

[danielmitu]

Hello,

Can you help me with one issue?

In this moment, on our system, in flow manager, there are 6 jobs that can run in the same time.

How can I increase those up to 10 simultaneous jobs ?

SAS 9.4 M2, Windows Server 2012 R2.

Thanks

[JuanS_OCS]

Hello @danielmitu,

you would need to check the number of cpus/threads your system can handle, and modify the size of your queues.

Just one reference, if you look into SAS or LSF documentation, or even just google, you will find several more:

https://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0/lsf_admin/queue_restrict_job_size.html

[danielmitu]

Hello,

You are saying that if I have 6 cpus/cores I can have only 1 job/cpu ?

Thanks

[JuanS_OCS]

Well, let me explain: first of all, 1 core is not necessarily the same as 1 cpu. 1 core can handle 2 (real time) threads, hence it can count as 2 logical cpus. This is named as Hyper-threading.

Second, in real time, yes, a computer can handle only as many real time threads as cpus. In total, not only SAS. This means that also operating system and other services will be launching threads.

In user/logical time, the computer can (and does) handle thousands of threads, as better as it can, giving processor time (cpu) to the threads that need it. That is why user time for processes is longer than real time. Because processes have to wait.

And as such, the operating system has its own queues to distribute the threads along the cpus. LSF a Resource Manager that is just another layer of thread distribution that can provide better handling of  jobs as execution units, and it has its own queues.

As you see, the queue is even another (logical) layer, that helps the resources manager to better grant resources to jobs. And jobs that come afterwards, they need to wait.

You can modify the queue lengths (logical limits) in LSF, just beware of the physical limitations. 

[danielmitu]

And where I need to modify this limit ?

Thanks

[SASKiwi]

In this folder: SAS\Thirdparty\LSF\conf\lsbatch\sas_cluster\configdir you should find a file called lsb.hosts.

In the file you should find a section that looks like this. The MXJ column specifies the maximum number of jobs per host. Note in this example a maximum of 8 jobs per host is specified.

Spoiler

Begin Host
HOST_NAME MXJ r1m pg ls tmp DISPATCH_WINDOW # Keywords
#hostA () 3.5/4.5 15/ 12/15 0 () # Example
#hostB ! 3.5 15/18 12/ 0/ (5:19:00-1:8:30 20:00-8:30)
#hostC 1 3.5/5 18 15 () () # Example
#hostD ! () () () () () # Example
#hostE 4 () () () () () # Example
#SPARCIPC () 4.0/5.0 18 16 () () # Example
default 8 () () () () () # Example
End Host

[danielmitu]

Hello SASKiwi,

in my lsb.hosts file I have this:

Begin Host
HOST_NAME MXJ r1m pg ls tmp DISPATCH_WINDOW # Keywords
#hostA () 3.5/4.5 15/ 12/15 0 () # Example
#hostB ! 3.5 15/18 12/ 0/ (5:19:00-1:8:30 20:00-8:30)
#hostC 1 3.5/5 18 15 () () # Example
#hostD ! () () () () () # Example
#hostE 4 () () () () () # Example
#SPARCIPC () 4.0/5.0 18 16 () () # Example
default ! () () () () () # Example
End Host

What does "!" means ?

Thanks

[JuanS_OCS]

Hi @danielmitu,

https://www.ibm.com/support/knowledgecenter/en/SSETD4_9.1.2/lsf_config_ref/lsb.hosts.5.html

Use ! to make the number of job slots equal to the number of CPUs on a host.