topic Re: yarn queues for resource control on SAS Grid Manger for Hadoop in Multi tenant architecture in Architecture

yarn queues for resource control on SAS Grid Manger for Hadoop in Multi tenant architecture

Jagadeesh_core — Tue, 29 Mar 2016 21:36:45 GMT

HI ,

we have deployed SAS Grid Manger for Hadoop ( yarn as grid Provider ) and now we want create a queue in order to allocate resources for each business unit. ( Finance , Marketing etc ).

my question is where we need to create a queues in Yarn or sasgrid-policy.xml files . if we it is sasgrid-policy.xml then can we define different queues for Single GridApplication type like below .

<?xml version ="1.0" encoding="UTF-8" standalone="yes"?>
<GridPolicy defaultAppType="normal">
<GridApplicaitonType name="PROD">
<jobname>Finance-Job</jobname>
<priority>43</priority>
<memory>4096</memory>
<vcores>2</vcores>
<runlimit>120</runlimit>
<queue>Finance</queue>
<hosts>
<hostGroup>PROD_Servers</hostGroup>
</hosts>
<jobname>Marketing Job</jobname>
<priority>30</priority>
<memory>4096</memory>
<vcores>2</vcores>
<runlimit>120</runlimit>
<queue>Marketing</queue>
<hosts>
<hostGroup>PROD_Servers</hostGroup>
</hosts>
</GridApplicationType>
<HostGroup name="PROD_Servers">
<host>server1</host>
<host>server2</host>
</HostGroup>
</GridPolicy>

Thanks,

Jagadeesh Chandra K

Re: yarn queues for resource control on SAS Grid Manger for Hadoop in Multi tenant architecture

davidbass — Thu, 31 Mar 2016 15:21:54 GMT

Jagadeesh:

You have to use both Yarn and sasgrid-policy.xml. You first create the queues in Yarn that you want to use, and then you can direct jobs to the queues using the sasgrid-policy file. The policy file you posted looks OK, but you will still need to create the Finance and Marketing queues in Yarn.

The SAS doc for the policy file is here:

http://support.sas.com/documentation/cdl/en/gridref/67944/HTML/default/viewer.htm#n1xyc60t8ml0fyn12p4h5grycdqu.htm

David

Re: yarn queues for resource control on SAS Grid Manger for Hadoop in Multi tenant architecture

Jagadeesh_core — Fri, 08 Apr 2016 15:02:58 GMT

Thanks David. so we no need to create different app servers for each queue which we defined in SAS grid Policy ?

Re: yarn queues for resource control on SAS Grid Manger for Hadoop in Multi tenant architecture

davidbass — Wed, 13 Apr 2016 20:11:50 GMT

Right. You can create as many servers as needed, then specify which ones are to be used for each type of processing by using the <host> and <hostGroup> elements.