[torqueusers] Specifying nodes which can be used in array job

Sreedhar Manchu sm4082 at nyu.edu
Sun Feb 12 08:53:14 MST 2012

Hi Rainer,

Like Ken wrote it is possible with feature property. I use this feature heavily to place jobs on specific nodes. 

To add feature to nodes

for i in {0..5}; do qmgr -c "set node node0$i properties += arrays"; done

Here feature is arrays. You can replace that with whatever you like. 

Once you've done this you can get array jobs placed on these nodes by requesting this feature in qsub such as

>>> qsub the_script.sub -t 1-10 -l feature='arrays'

This would put your jobs on the nodes that have property arrays. In this case the nodes are 0 to 5.

In my case I wrote a qsub wrapper which goes through the pbs scripts and command line and adds this feature line such as #PBS -l feature=<feature name> to the script so that they are placed on right nodes. This comes very handy especially when you have nodes with diiferent amounts of memory under the same queue. 

If your scheduler is moab you can do really cool stuff using this feature property. 

Hope this helps. 


On 10-Feb-2012, at 2:49 AM, Rainer M Krug <R.M.Krug at gmail.com> wrote:

> Hash: SHA1
> On 09/02/12 23:39, Ken Nielson wrote:
>> ----- Original Message -----
>>> From: "Rainer M Krug" <R.M.Krug at gmail.com> To:
>>> torqueusers at supercluster.org Sent: Thursday, February 9, 2012
>>> 2:16:07 AM Subject: [torqueusers] Specifying nodes which can be
>>> used in array job
>>> Hi
>>> assuming I have cluster of 10 nodes (node01, ... node10), of
>>> which I am not the administrator.
>>> Some nodes are setup slightly different, so that a certain job
>>> only runs on nodes node01 to node05.
>>> So I would like to submit an array job and specify "only use the 
>>> node01, node02, node03, node04 or node05 to run the each
>>> individual job".
>>> How can I do that? I know that I can use -l to specify resource 
>>> requirements, but if I specify nodes=..., *each* job will
>>> allocate *all* nodes for the job, which is not what I want - each
>>> individual job should run on one of the nodes.
>>> so:
>>> qsub the_script.sub -t 1-10
>>> and how do I specify the nodes?
>>> Thanks,
>>> Rainer
>> Rainer,
>> Are there feature (properties) in the nodes files of those hosts
>> which would allow you to specify a feature on the qsub line?
> No - unfortunately not.
>> Ken
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> iEYEARECAAYFAk80zBgACgkQoYgNqgF2egri1wCfUUqDmOigKB8hCyCvt30pu5jZ
> kewAnjfVc6o7rIjFua0ukEBhkaNe5McS
> =nBnt
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.supercluster.org/pipermail/torqueusers/attachments/20120212/d6dc9fed/attachment.html 

More information about the torqueusers mailing list