[Mauiusers] Maui vs pbs_sched

[Jan Ploski]

mauiusers-bounces at supercluster.org schrieb am 09/24/2007 06:17:17 PM:

> Ok, I have been troubleshooting my issue, but with no solution. 
> Facts: 
> My job sumission script: 
> ------------------------------------ 
> #!/bin/bash 
> #PBS -j oe 
> #PBS -l nodes=5:ppn=1 
> #PBS -N TestJob 
> #PBS -q medium 
> #PBS -o output.txt 
> mpiexec --bynode /data/andrus/hello 
> ------------------------------------ 
> When I qsub a job with the default torque scheduler running on the 
> head node it runs as expected on 5 different nodes. 
> Whe I qsub a job with maui running instead, it only runs on one 
> node. If I add -np 5 to the mpiexec line, it runs in 5 processes on 
> the one node. If I try to specify some nodes, it usually aborts 
> because I am requesting some nodes that are not in my allocated resource 
pool.
> Grr. Why does my job run on only one node when using Maui? This I 
> cannot figure out. Any help is certainly appreciated. 

Hello,

If you do qstat -f on the job, what is the output? Multiple processors 
allocated on the same node or just one processor allocated? If it is the 
former, then it's the expected behavior, referred to as "packing" in the 
documentation.

Maybe this will help you out, quoted from 
http://www.clusterresources.com/products/maui/docs/pbsintegration.shtml

"Maui is by default very liberal in its interpretation of 
<NODECOUNT>:PPN=<X>.  In its standard configuration, Maui interprets this 
as 'give the job <NODECOUNT>*<X> tasks with AT LEAST <X> tasks per node'. 
Set the JOBNODEMATCHPOLICY parameter to EXACTNODE to have Maui support 
PBS's default allocation behavior of <NODECOUNT> nodes with exactly <X> 
tasks per node."

(Personally, I find it a terrible idea that Maui configuration parameters 
alter the semantics of the resource requirement specification. On the 
other hand, I doubt if their meaning is precisely documented anywhere in 
PBS documentation.)

Regards,
Jan Ploski

--
Dipl.-Inform. (FH) Jan Ploski
OFFIS
Betriebliches Informationsmanagement
Escherweg 2  - 26121 Oldenburg - Germany
Fon: +49 441 9722 - 184 Fax: +49 441 9722 - 202
E-Mail: Jan.Ploski at offis.de - URL: http://www.offis.de