Architecting, installing and maintaining your SAS environment

yarn queues for resource control on SAS Grid Manger for Hadoop in Multi tenant architecture

Reply
New Contributor
Posts: 2

yarn queues for resource control on SAS Grid Manger for Hadoop in Multi tenant architecture

HI , 

 

we have deployed SAS Grid Manger for Hadoop ( yarn as grid Provider ) and now we want create a queue in order to allocate resources for each business unit. ( Finance , Marketing  etc  ). 

my question is where we need to create a queues in Yarn or sasgrid-policy.xml files . if we it is sasgrid-policy.xml then can we define  different queues for Single GridApplication type like below .

<?xml version ="1.0" encoding="UTF-8" standalone="yes"?>
<GridPolicy defaultAppType="normal">
<GridApplicaitonType name="PROD">
<jobname>Finance-Job</jobname>
<priority>43</priority>
<memory>4096</memory>
<vcores>2</vcores>
<runlimit>120</runlimit>
<queue>Finance</queue>
<hosts>
<hostGroup>PROD_Servers</hostGroup>
</hosts>
<jobname>Marketing Job</jobname>
<priority>30</priority>
<memory>4096</memory>
<vcores>2</vcores>
<runlimit>120</runlimit>
<queue>Marketing</queue>
<hosts>
<hostGroup>PROD_Servers</hostGroup>
</hosts>
</GridApplicationType>
<HostGroup name="PROD_Servers">
<host>server1</host>
<host>server2</host>
</HostGroup>
</GridPolicy>

 

Thanks,

Jagadeesh Chandra K

SAS Employee
Posts: 5

Re: yarn queues for resource control on SAS Grid Manger for Hadoop in Multi tenant architecture

Posted in reply to Jagadeesh_core

Jagadeesh:

 

You have to use both Yarn and sasgrid-policy.xml.  You first create the queues in Yarn that you want to use, and then you can direct jobs to the queues using the sasgrid-policy file. The policy file you posted looks OK, but you will still need to create the Finance and Marketing queues in Yarn.

 

The SAS doc for the policy file is here:

 

http://support.sas.com/documentation/cdl/en/gridref/67944/HTML/default/viewer.htm#n1xyc60t8ml0fyn12p...

 

David

New Contributor
Posts: 2

Re: yarn queues for resource control on SAS Grid Manger for Hadoop in Multi tenant architecture

Posted in reply to davidbass

Thanks David. so we no need to create different app servers for each queue which we defined in SAS grid Policy ? 

 

 

 

SAS Employee
Posts: 5

Re: yarn queues for resource control on SAS Grid Manger for Hadoop in Multi tenant architecture

Posted in reply to Jagadeesh_core

Right. You can create as many servers as needed, then specify which ones are to be used for each type of processing by using the <host> and <hostGroup> elements.

Ask a Question
Discussion stats
  • 3 replies
  • 295 views
  • 2 likes
  • 2 in conversation