[torqueusers] PBS_NODEFILE order question

Jackie Scoggins jscoggins at lbl.gov
Tue Feb 17 18:54:43 MST 2009


I tried both of these and the bynode does do it but the user was  
complaining that if he has a 8 proc node and only need 2 cpus he felt  
that was wasting resources.  So I am not sure what is the right  
solution except to say that it can not be done in the manner in which  
he really wants. He can use 1/2 per node for what he wants and just be  
aware of how many processors they need at mpirun time to select ppn's  
correctly.


Thanks

Jackie

On Feb 17, 2009, at 5:44 PM, Brock Palen wrote:

> Try this is using Open MPI  1.2.x  series
>
> mpirun -bynode app
>
> -bynode places cpus like you have in your listing.
>
> You just have to be careful, that you always have 2 cpus per node,  
> or something similar.
>
> the tpn option and gemoetry option if using moab with toruque should  
> be a big help.
>
> For example:
>
> I want 2 nodes, 2 cpus per machine, I don't want 4 ever,  and I want  
> Open MPI to put rank 0 and 2 on node 1, and rank, 1 and 3 on node 2.
>
> #PBS -l nodes=2:ppn=2,tpn=2
>
> mpirun -bynode myapp
>
> Your not mixing threads with with MPI are you that you want that  
> kind of layout?
>
> Brock Palen
> www.umich.edu/~brockp
> Center for Advanced Computing
> brockp at umich.edu
> (734)936-1985
>
>
>
> On Feb 17, 2009, at 8:38 PM, Garrick Staples wrote:
>
>> On Tue, Feb 17, 2009 at 04:43:04PM -0800, Jackie Scoggins alleged:
>>> torque - version 2.1.8
>>> maui version 3.2.6p19
>>>
>>> Is there a way to make the PBS_NODEFILE order the nodes as follows:
>>>
>>> n0000
>>> n0001
>>> n0000
>>> n0001
>>> n0000
>>> n0001
>>
>> Nope.
>>
>>
>>> I have tried chopping up the PBS_NODEFILE and using a different
>>> hostfile for mpirun but I get an orted error from mpirun because it
>>> does not look like it is selecting the nodes in the order in which I
>>> specify in my hostfile.
>>
>> Probably best for whatever tool to have options for whatever sort  
>> orders you
>> like.  I'm sure this would fit nicely into openmpi.
>>
>> -- 
>> Garrick Staples, GNU/Linux HPCC SysAdmin
>> University of Southern California
>>
>> See the Prop 8 Dishonor Roll at http://www.californiansagainsthate.com/
>>
>> _______________________________________________
>> torqueusers mailing list
>> torqueusers at supercluster.org
>> http://www.supercluster.org/mailman/listinfo/torqueusers
>
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers



More information about the torqueusers mailing list