[Mauiusers] MPI / IB setup

Romaric David david at unistra.fr
Mon Mar 15 14:34:13 MDT 2010


Jacques Foury a écrit :
> Hello all.
> 
> 
> we have here a setup of Infiniband nodes, grouped around IB switches.
> 
> I wonder if it's possible to have MPI jobs run on a set of nodes that 
> are connected on the *same* IB switch.
> 
> I thought about special attributes for the nodes, but it is by no way 
> dynamical.
> 
> 
> 
> Let me be more precise :
> - we have 64 nodes, connected by groups of 4 nodes per chassis.
> - we have 4 "leaves" IB switches, all receiving 16 nodes (4 chassis) IB 
> plugs.
> ... and 2 more spine IB switches.
> 
> 
> We want the jobs fitting on 16 nodes to run on a set of nodes all 
> connected to the same leaf switch.
> 
> Does anyone have already configured such a setup ?
> 
> Thanks !
> 

Hello,

I used the FEATURES of Maui do to so. Each node has a FEATURE corresponding to 
its leaf switch. For instance :
nodecfg[node10] FEATURES=switchA




I then ask MAUI to choose nodes on a given feature :

NODESETLIST  switchA switchB switchC
NODESETPOLICY    ONEOF
NODESETATTRIBUTE FEATURE


It may be a little too restrictive for what you need, but hope this helps.


--
---------------------------------------------------------
   R. David - david at unistra.fr
   Responsable du département Expertise pour la Recherche
   UdS / Direction Informatique
   Tel. : 03 68 85 45 48  (Fax 45 47)
---------------------------------------------------------


More information about the mauiusers mailing list