[torqueusers] array job and resources

Sreedhar Manchu sm4082 at nyu.edu
Fri Mar 2 07:48:51 MST 2012

Hi Sergey,

Glad to hear that you found the solution. I also use node properties heavily to place right jobs on right nodes. It works great especially with moab. This feature comes handy when we have nodes with different memory sizes on cluster.


On Mar 2, 2012, at 3:31 AM, Sergey Bulk wrote:

> Hi, Sreedhar!
> Thank you for your responce. 
> It was important for me to understand that it is not
> the bug of an array job but its feature. 
> So it exists just for grouping jobs not for resource allocation.
> I've solved my current problem using properties of nodes.
> qsub -l nodes=bigmem mytask.sh   - is sending to the big memory node
> qsub -l anothertask.sh - is sending to any other node
> Good luck!
> Sergey
> 01 марта 2012, 19:00 от Sreedhar Manchu <sm4082 at nyu.edu>:
>> I don't think this is possible. In array of jobs each job is treated as a separate one from others. The only advantage with the arrays is that you don't have do qsub multiple times when you need to change very little in your script, let's say  changing input file from in1 to in2, etc.
>> What you saw is reasonable behavior as it put jobs one after another. If you really want each job to run on different node the only is to use for loop. I am not sure this can be done with arrays.
>> for i in {1..12}; do qsub -l nodes=node${i} <your script> ; done
>> Since ppn is not declared it is 1 by default. If you want node01 but not node1 you can use if block. Or may be seq can add zeros with some flag. Not sure about it.
>> You can use the same script without modifying since command line has precedence. Otherwise you can simply include just bash liners in the script.
>> Sreedhar.
>> --
>> Sent from my phone. Please excuse my brevity and any typos.
>> On Feb 29, 2012, at 5:17, Sergey Bulk <sergey_bulk at list.ru> wrote:
>>> I have torque 2.5.7-9.el6 from epel repo on SL6.
>>> I have 24-core nodes node01-node12.
>>> When requesting resources for an array job
>>> #!/bin/bash
>>> #PBS -t 1-12
>>> #PBS -l nodes=node01:ppn=1+node02:ppn=1+node03:ppn=1+node04:ppn=1+....+node12:ppn=1
>>> #PBS -d .
>>> for f in `seq 1 1000`;
>>> do
>>>   ps aux
>>> done;
>>> I would expect that each job in the array should occupy its own node,
>>> but, instead, -l option is for every job not for the whole array.
>>> So all jobs are running on the node01 because there is enough cores.
>>> What is the correct way to request resources for the whole array job
>>> rather then for the single job in the array?
>>> Thank you,
>>> SN
>>> _______________________________________________
>>> torqueusers mailing list
>>> torqueusers at supercluster.org
>>> http://www.supercluster.org/mailman/listinfo/torqueusers

Sreedhar Manchu
HPC Support Specialist
New York University
251 Mercer Street
New York, NY 10012-1110

More information about the torqueusers mailing list