[torqueusers] How to set queue's max nodes?

shixing paradisehit at gmail.com
Sun Jan 27 01:21:24 MST 2013


When I set the queue both with nodect and nodes like this:
*create queue team1*
*set queue team1 queue_type = Execution*
*set queue team1 resources_max.nodect = 2*
*set queue team1 resources_max.nodes = 2*
*set queue team1 keep_completed = 2592000*
*set queue team1 enabled = True*
*set queue team1 started = True*

I can also submit large jobs and all the nodes used by the running jobs
exceed the resources_max.nodect or resources_max.nodes (here are both 2).
I submit the jobs 4 times like this:
*echo "sleep 100" | qsub -l nodes=1:ppn=4 -q team1*
*
*
And qstat shows the command like this:
*1661.vkvm161057            STDIN            shubao.sx              0 R team1
         *
*1662.vkvm161057            STDIN            shubao.sx              0 R team1
         *
*1663.vkvm161057            STDIN            shubao.sx              0 R team1
         *
*1664.vkvm161057            STDIN            shubao.sx              0 R
team1 *

I use torque 4.1.3 and maui 3.3.1.

On Sun, Jan 27, 2013 at 3:11 AM, Gustavo Correa <gus at ldeo.columbia.edu>wrote:

> I misundertood Shixing's original question.
> I though he wanted to prevent each job to exceed a certain number of nodes,
> but what he wants to apply the limit to the sum of all jobs in the queue,
> correct?
>
> Burkhard is right.
> "nodect" intent seems to be to limit the number of nodes used by all jobs
> in a specific queue,
> whereas  AFAIK "nodes" limits the number of nodes each job can request
> when submitted to
> a specific queue, right?
>
> Things may have changed in recent versions, but "nodes" , with the
> interpretation above, works for me in Torque 2.4.11, with Maui 3.2.6p21.
>
> However, "nodect", with the interpretation above, doesn't work for me,
> as Shixing also noted, even if I add "ppn=8" to my qsub command,
> to request all processors in my nodes, and try to exhaust the available
> resources and hit the nodect limit.
> Maybe there is a way to implement what Shixing wants in Maui?
>
> Quoting the Torque Admin Guide, section 4.1.1, "Queues attributes":
>
> "nodes  integer Specifies the number of nodes "
> [Note, integer, not a string.  Not in this context at least.]
>
> "nodect integer         Sets the number of nodes available. By default,
> TORQUE will set the number of nodes available to the number of nodes listed
> in the $TORQUE_HOME/server_priv/nodes file. nodect can be set to be greater
> than or less than that number. Generally, it is used to set the node count
> higher than the number of physical nodes in the cluster."
>
>
> http://docs.adaptivecomputing.com/torque/4-1-3/help.htm#topics/4-serverPolicies/queueAttributes.htm
>
> Admittedly, the Guide wording is not very clear.
> It could include "on a per queue basis", "on a per job basis", or
> something the like,
> to clarify the context.
> The final sentence in "nodect" sounds a bit awkward.
> Does it work to set the node count *smaller* than the number
> of physical nodes?
> Does this depend on the scheduler configuration?  [pbs_sched, Maui, Moab]
>
> Somebody from Adaptive could clarify.
>
> Gus Correa
>
> On Jan 26, 2013, at 8:31 AM, Burkhard Bunk wrote:
>
> > Hi,
> >
> > I would support that.
> > The correct form, IMHO, is
> >
> > set queue myqueue resources_max.nodect = 10
> >
> > Regards,
> > Burkhard Bunk.
> > ----------------------------------------------------------------------
> > bunk at physik.hu-berlin.de      Physics Institute, Humboldt University
> > fax:    ++49-30 2093 7628     Newtonstr. 15
> > phone:  ++49-30 2093 7980     12489 Berlin, Germany
> > ----------------------------------------------------------------------
> >
> > On Sat, 26 Jan 2013, André Gemünd wrote:
> >
> >> Hi,
> >> I'm a bit surprised by that. Since when is nodes an integer in Torque?
> It used to be a string in earlier versions.
> >>
> >> Greetings
> >> André
> >>
> >> ----- Ursprüngliche Mail -----
> >>> Hi Shixing
> >>> Have you tried:
> >>> set queue myqueue resources_max.nodes = 10
> >>> More info:
> >>>
> http://docs.adaptivecomputing.com/torque/4-1-3/help.htm#topics/4-serverPolicies/queueAttributes.htm
> >>> under "Assigning queue resource limits".
> >>> I hope this helps,
> >>> Gus
> >>> On 01/25/2013 05:44 AM, André Gemünd wrote:
> >>> > I don't think that setting resources_available.nodect has any
> >>> > effect.
> >>> > Have you tried resources_max.nodect?
> >>> >
> >>> > Greetings
> >>> >
> >>> > ----- Ursprüngliche Mail -----
> >>> >>
> >>> >> Hi, all:
> >>> >> Recently I have set up a cluster with 200 nodes. And this cluster
> >>> >> is
> >>> >> designed for serving several team. And I want split the nodes to
> >>> >> some queues.
> >>> >> I have set the queue attr like this:
> >>> >> set queue team1 resources_available.nodect = 3
> >>> >>
> >>> >>
> >>> >> But when I submit a job applying nodes>3, it will also run
> >>> >> successfully. The submit command is :
> >>> >> echo "sleep 100" | qsub -l nodes=4 -q team1
> >>> >>
> >>> >>
> >>> >> So how can I set the max nodes for the queues?
> >>> >> --
> >>> >> Best wishes!
> >>> >> My Friend~
> >>> >> _______________________________________________
> >>> >> torqueusers mailing list
> >>> >> torqueusers at supercluster.org
> >>> >> http://www.supercluster.org/mailman/listinfo/torqueusers
> >>> >>
> >>> >
> >>> _______________________________________________
> >>> torqueusers mailing list
> >>> torqueusers at supercluster.org
> >>> http://www.supercluster.org/mailman/listinfo/torqueusers
> >>
> >> --
> >> André Gemünd
> >> Fraunhofer-Institute for Algorithms and Scientific Computing
> >> andre.gemuend at scai.fraunhofer.de
> >> Tel: +49 2241 14-2193
> >> /C=DE/O=Fraunhofer/OU=SCAI/OU=People/CN=Andre Gemuend
> >> _______________________________________________
> >> torqueusers mailing list
> >> torqueusers at supercluster.org
> >> http://www.supercluster.org/mailman/listinfo/torqueusers
> > _______________________________________________
> > torqueusers mailing list
> > torqueusers at supercluster.org
> > http://www.supercluster.org/mailman/listinfo/torqueusers
>
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers
>



-- 
Best wishes!
My Friend~
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.supercluster.org/pipermail/torqueusers/attachments/20130127/55e8aa8c/attachment-0001.html 


More information about the torqueusers mailing list