[Mauiusers] Maui vs pbs_sched
Jan.Ploski at offis.de
Mon Sep 24 10:36:25 MDT 2007
mauiusers-bounces at supercluster.org schrieb am 09/24/2007 06:17:17 PM:
> Ok, I have been troubleshooting my issue, but with no solution.
> My job sumission script:
> #PBS -j oe
> #PBS -l nodes=5:ppn=1
> #PBS -N TestJob
> #PBS -q medium
> #PBS -o output.txt
> mpiexec --bynode /data/andrus/hello
> When I qsub a job with the default torque scheduler running on the
> head node it runs as expected on 5 different nodes.
> Whe I qsub a job with maui running instead, it only runs on one
> node. If I add -np 5 to the mpiexec line, it runs in 5 processes on
> the one node. If I try to specify some nodes, it usually aborts
> because I am requesting some nodes that are not in my allocated resource
> Grr. Why does my job run on only one node when using Maui? This I
> cannot figure out. Any help is certainly appreciated.
If you do qstat -f on the job, what is the output? Multiple processors
allocated on the same node or just one processor allocated? If it is the
former, then it's the expected behavior, referred to as "packing" in the
Maybe this will help you out, quoted from
"Maui is by default very liberal in its interpretation of
<NODECOUNT>:PPN=<X>. In its standard configuration, Maui interprets this
as 'give the job <NODECOUNT>*<X> tasks with AT LEAST <X> tasks per node'.
Set the JOBNODEMATCHPOLICY parameter to EXACTNODE to have Maui support
PBS's default allocation behavior of <NODECOUNT> nodes with exactly <X>
tasks per node."
(Personally, I find it a terrible idea that Maui configuration parameters
alter the semantics of the resource requirement specification. On the
other hand, I doubt if their meaning is precisely documented anywhere in
Dipl.-Inform. (FH) Jan Ploski
Escherweg 2 - 26121 Oldenburg - Germany
Fon: +49 441 9722 - 184 Fax: +49 441 9722 - 202
E-Mail: Jan.Ploski at offis.de - URL: http://www.offis.de
More information about the mauiusers