yarn queues for resource control on SAS Grid Manger for Hadoop in Multi tenant architecture

[Jagadeesh_core]

HI , 

we have deployed SAS Grid Manger for Hadoop ( yarn as grid Provider ) and now we want create a queue in order to allocate resources for each business unit. ( Finance , Marketing  etc  ). 

my question is where we need to create a queues in Yarn or sasgrid-policy.xml files . if we it is sasgrid-policy.xml then can we define  different queues for Single GridApplication type like below .

<?xml version ="1.0" encoding="UTF-8" standalone="yes"?>
<GridPolicy defaultAppType="normal">
<GridApplicaitonType name="PROD">
<jobname>Finance-Job</jobname>
<priority>43</priority>
<memory>4096</memory>
<vcores>2</vcores>
<runlimit>120</runlimit>
<queue>Finance</queue>
<hosts>
<hostGroup>PROD_Servers</hostGroup>
</hosts>
<jobname>Marketing Job</jobname>
<priority>30</priority>
<memory>4096</memory>
<vcores>2</vcores>
<runlimit>120</runlimit>
<queue>Marketing</queue>
<hosts>
<hostGroup>PROD_Servers</hostGroup>
</hosts>
</GridApplicationType>
<HostGroup name="PROD_Servers">
<host>server1</host>
<host>server2</host>
</HostGroup>
</GridPolicy>

Thanks,

Jagadeesh Chandra K

[davidbass]

Jagadeesh:

You have to use both Yarn and sasgrid-policy.xml.  You first create the queues in Yarn that you want to use, and then you can direct jobs to the queues using the sasgrid-policy file. The policy file you posted looks OK, but you will still need to create the Finance and Marketing queues in Yarn.

The SAS doc for the policy file is here:

http://support.sas.com/documentation/cdl/en/gridref/67944/HTML/default/viewer.htm#n1xyc60t8ml0fyn12p...

David

[Jagadeesh_core]

Thanks David. so we no need to create different app servers for each queue which we defined in SAS grid Policy ? 

[davidbass]

Right. You can create as many servers as needed, then specify which ones are to be used for each type of processing by using the <host> and <hostGroup> elements.