[torquedev] Infiniband and sockets
jbernstein at penguincomputing.com
Thu Feb 4 15:47:42 MST 2010
On Feb 4, 2010, at 8:31 AM, Garrick Staples wrote:
> On Thu, Feb 04, 2010 at 09:11:20AM -0700, Ken Nielson alleged:
>> We have been dealing with systems that are using infiniband and
>> ethernet. RFC 4392 defines an IP over infiniband standard which I am
>> guessing is being used by most sites. As far as DNS name resolution
>> IP addressing are concerned is there anything special that needs to
>> accounted for when using sockets, TCP/IP and infiniband?
> Torque doesn't kow or care about Infiniband. It's just the problem
> of multiple
> networks. Forward and reverse resolution must be carefully set up.
IPoIB is used during IB configurations. But to Garricks point TORQUE
shouldn't care about it and trying to run pbs_mom based on IPoIB
hostnames generally yields no performance gains. Further still,
Garricks point about making sure hostname resolution is properly
configured is absolutely paramount. In Scyld ClusterWare, we
automatically assigned internally resolvable hostnames tied to the
Infiniband IP addresses. For example, if the Ethernet NIC has a
configured hostname of, say node0, then Scyld automatically generates
a hostname resolution for node0-infiniband. This is of course user
configurable thus the resolution could also be node0-ib.
It sounds like your fishing for more detail however, so if you have
specific IB questions, feel free to ping me offline.
Senior Software Engineer
More information about the torquedev