[torquedev] IP version-agnostic Address Representation

David Beer dbeer at adaptivecomputing.com
Thu Aug 25 10:22:54 MDT 2011



----- Original Message -----
> Ken Nielson wrote:
> > ----- Original Message -----
> >   
> >> From: "Donald Neal" <dmneal at wand.net.nz>
> >> To: "Torque Developers mailing list" <torquedev at supercluster.org>
> >> Sent: Thursday, August 11, 2011 6:05:59 PM
> >> Subject: [torquedev] IP version-agnostic Address Representation
> >> There are a range of cases in Torque where an IP address is
> >> represented
> >> by a 32-bit object. This is something of a problem where the IP
> >> address
> >> may actually be 128 bits long.
> >>
> >> I see two distinct cases here. In one the 32-bit object is being
> >> used
> >> as
> >> a key (as with AvlNode). In this case I propose initially to take
> >> the
> >> lower-order 32 bits of an IPv6 address and keep going as now. This
> >> is
> >> simple and effiicient, but would definitely give rise to
> >> collisions
> >> which will confuse people in future.
> >>
> >> So the key does in future need to change. My inclination is to use
> >> a
> >> struct created for the purpose containing two uint64_t's. But
> >> there
> >> are
> >> definitely other ways of doing this.
> >>
> >> In the other case, the address is not used as a key (as in struct
> >> pbsnode, say). That makes it viable to use either a struct
> >> containing
> >> the minimum fields necessary or a sockaddr*. I'm inclined towards
> >> the
> >> latter on the grounds that keeping down the number of struct types
> >> in
> >> use makes life simpler, But again there are clearly other ways of
> >> doing
> >> this.
> >>
> >> Does anyone have a view on this? Does anyone have any other reason
> >> to
> >> use a package like gmp, which I don't see as needed for this
> >> purpose
> >> alone?
> >>
> >> - Donald Neal
> >>
> >>     
> >
> > The current scheme TORQUE uses to identify trusted hosts is to use
> > the host name and IPv4 ip address. The 32-bit IP address is used
> > as a key to store trusted addresses in an AVL tree. The convenient
> > thing about using this scheme is that connections are verified
> > without the need to read any data from the stream. With the
> > 128-bit IPv6 address the AVL tree is broken. We could change the
> > AVL-tree to take a 128 bit key and then pad the IPv4 addresses. (I
> > am just brain-storming).
> >
> > Another thought is to have the server assign a random number to
> > each MOM and then distribute those numbers across the cluster.
> > This would require a change in the protocol. It also requires a
> > reading of the incoming stream to get the number.
> >
> > Just some thoughts.
> >
> > Ken
> >   
>  From the (limited) discussion, I'm inclined to conclude
> 
> i) Noone has another use for gmp, so general-purpose large objects
> are
> not the answer.
> ii)) Replacing the 32-bit key in AvlNode with a struct of two
> uint64_t's
> is as good an idea as anything else.
> 
> So life looks to be simpler if all AvlNodes have a 128-bit key in
> place
> of a 32-bit one. This becomes more complex if the same node
> identifier
> is used both with an AvlNode and with a resizable_array.
> 
> Is there a simple explanation anywhere of what the resizable_array
> struct is intended to be used for? This is a question about design
> philosophy rather than existing code.
> 

The resizable_array struct is being used in many places to replace linked lists. This struct actually has the capability of maintaining a fixed order, so in a way it is still a linked list that resides in an array.

The struct is used (in 4.0) to store jobs, nodes, queues, and tasks. As far as my intention for its use, I wanted to use it as an easy way for storing objects in arrays where the number of objects could change. I have tried to make it into a flexible and widely usable struct. Is there anything more specific you'd like to know about it?

David


More information about the torquedev mailing list