[torqueusers] Specifying nodes which can be used in array job

[Gustavo Correa]

On Feb 15, 2012, at 11:38 AM, Rainer M Krug wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> OK - the feature is added, but I don't get it to only use the nodes
> with this feature set.
> 
> I have the following simple script named test.sub:
> 
> 
> #PBS -j oe
> #PBS -m abe
> #PBS -M Rainer at krugs.de
> #PBS -V
> #PBS -t 1-10
> #PBS -l feature='Rnodeff'
> echo "---------------------"
> 
> 
> and I submit it via
> 
> qsub test.sub
> 
> and it always runs, whatever I put into the feature line, even if the
> feature property "Rnodeff" does not exist. It always uses all nodes,
> even when I use "Rnode" which is set only for a subset of nodes.
> 
> 
> I even setup torque on my desktop and set the features to try it.
> 
> 

There is one way to do what I guess you want to do [i.e. launch jobs on nodes with different
properties, right?] by setting up specific queues for each node property.
You may need maui or moab for this, I am not sure if it works with pbs_sched.

Say, create a queue qRnodeff the usual way.
Then add something like this:

qmgr -c 'set queue qRnodeff resources_default.neednodes = Rnodeff'

where Rnodeff is a property of some of your nodes listed in the $Torque/pbs_server/nodes file.

On your maui.cfg add this line:
ENABLEMULTIREQJOBS   TRUE

Restart pbs_server and maui.

On your PBS job script just direct the job to the appropriate queue:
#PBS -q qRnodeff

no need for "#PBS -l Rnodeff"

Anyway, for these non-default configurations, it is worth reading the 
Torque and Maui Admin guides:
http://www.adaptivecomputing.com/resources/docs/

I hope this helps.

Gus Correa

 
> Any ideas?
> 
> Rainer
> 
> qsub
> On 13/02/12 09:37, Rainer M Krug wrote:
>> Thanks a lot - this definitely helps. I will get in contact with
>> our admin to add the features to the nodes.
>> 
>> Cheers,
>> 
>> Rainer
>> 
>> On 12/02/12 16:53, Sreedhar Manchu wrote:
>>> Hi Rainer,
>> 
>>> Like Ken wrote it is possible with feature property. I use this 
>>> feature heavily to place jobs on specific nodes.
>> 
>>> To add feature to nodes
>> 
>>> for i in {0..5}; do qmgr -c "set node node0$i properties += 
>>> arrays"; done
>> 
>>> Here feature is arrays. You can replace that with whatever you 
>>> like.
>> 
>>> Once you've done this you can get array jobs placed on these
>>> nodes by requesting this feature in qsub such as
>> 
>>>>>> qsub the_script.sub -t 1-10 -l feature='arrays'
>> 
>>> This would put your jobs on the nodes that have property arrays.
>>> In this case the nodes are 0 to 5.
>> 
>>> In my case I wrote a qsub wrapper which goes through the pbs 
>>> scripts and command line and adds this feature line such as #PBS
>>> -l feature=<feature name> to the script so that they are placed
>>> on right nodes. This comes very handy especially when you have
>>> nodes with diiferent amounts of memory under the same queue.
>> 
>>> If your scheduler is moab you can do really cool stuff using this
>>> feature property.
>> 
>>> Hope this helps.
>> 
>>> Sreedhar.
>> 
>> 
>> 
>>> On 10-Feb-2012, at 2:49 AM, Rainer M Krug <R.M.Krug at gmail.com 
>>> <mailto:R.M.Krug at gmail.com>> wrote:
>> 
>>> On 09/02/12 23:39, Ken Nielson wrote:
>>>>>> 
>>>>>> 
>>>>>> ----- Original Message -----
>>>>>>> From: "Rainer M Krug" <R.M.Krug at gmail.com 
>>>>>>> <mailto:R.M.Krug at gmail.com>> To: 
>>>>>>> torqueusers at supercluster.org 
>>>>>>> <mailto:torqueusers at supercluster.org> Sent: Thursday, 
>>>>>>> February 9, 2012 2:16:07 AM Subject: [torqueusers] 
>>>>>>> Specifying nodes which can be used in array job
>>>>>>> 
>>>>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>>>>>> 
>>>>>>> Hi
>>>>>>> 
>>>>>>> assuming I have cluster of 10 nodes (node01, ...
>>>>>>> node10), of which I am not the administrator.
>>>>>>> 
>>>>>>> Some nodes are setup slightly different, so that a
>>>>>>> certain job only runs on nodes node01 to node05.
>>>>>>> 
>>>>>>> So I would like to submit an array job and specify "only 
>>>>>>> use the node01, node02, node03, node04 or node05 to run
>>>>>>> the each individual job".
>>>>>>> 
>>>>>>> How can I do that? I know that I can use -l to specify 
>>>>>>> resource requirements, but if I specify nodes=...,
>>>>>>> *each* job will allocate *all* nodes for the job, which
>>>>>>> is not what I want - each individual job should run on
>>>>>>> one of the nodes.
>>>>>>> 
>>>>>>> so:
>>>>>>> 
>>>>>>> qsub the_script.sub -t 1-10
>>>>>>> 
>>>>>>> and how do I specify the nodes?
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> 
>>>>>>> Rainer
>>>>>> 
>>>>>> Rainer,
>>>>>> 
>>>>>> Are there feature (properties) in the nodes files of those 
>>>>>> hosts which would allow you to specify a feature on the
>>>>>> qsub line?
>> 
>>> No - unfortunately not.
>> 
>>>>>> 
>>>>>> Ken
>> 
>>>> 
>>>> _______________________________________________ torqueusers 
>>>> mailing list torqueusers at supercluster.org 
>>>> <mailto:torqueusers at supercluster.org> 
>>>> http://www.supercluster.org/mailman/listinfo/torqueusers
>> 
>> 
>>> _______________________________________________ torqueusers
>>> mailing list torqueusers at supercluster.org 
>>> http://www.supercluster.org/mailman/listinfo/torqueusers
>> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> 
> iEYEARECAAYFAk8734IACgkQoYgNqgF2egoY3QCfVqXzOLwjZ+DLqgVQdMU0eLjr
> TVwAn0dQs8G0y+JTIc+lRtP/0oT7GCyo
> =vpZF
> -----END PGP SIGNATURE-----
> 
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers