[torquedev] Infiniband and sockets

Joshua Bernstein jbernstein at penguincomputing.com
Thu Feb 4 15:47:42 MST 2010


On Feb 4, 2010, at 8:31 AM, Garrick Staples wrote:

> On Thu, Feb 04, 2010 at 09:11:20AM -0700, Ken Nielson alleged:
>> We have been dealing with systems that are using infiniband and
>> ethernet. RFC 4392 defines an IP over infiniband standard which I am
>> guessing is being used by most sites. As far as DNS name resolution  
>> and
>> IP addressing are concerned is there anything special that needs to  
>> be
>> accounted for when using sockets, TCP/IP and infiniband?
>
> Torque doesn't kow or care about Infiniband. It's just the problem  
> of multiple
> networks. Forward and reverse resolution must be carefully set up.

IPoIB is used during IB configurations. But to Garricks point TORQUE  
shouldn't care about it and trying to run pbs_mom based on IPoIB  
hostnames generally yields no performance gains. Further still,  
Garricks point about making sure hostname resolution is properly  
configured is absolutely paramount. In Scyld ClusterWare, we  
automatically assigned internally resolvable hostnames tied to the   
Infiniband IP addresses. For example, if the Ethernet NIC has a  
configured hostname of, say node0, then Scyld automatically generates  
a hostname resolution for node0-infiniband. This is of course user  
configurable thus the resolution could also be node0-ib.

It sounds like your fishing for more detail however, so if you have  
specific IB questions, feel free to ping me offline.

-Joshua Bernstein
Senior Software Engineer
Penguin Computing


More information about the torquedev mailing list