[torquedev] IP version-agnostic Address Representation

Ken Nielson knielson at adaptivecomputing.com
Thu Aug 25 10:49:04 MDT 2011


David,

I think you may have the wrong Ken on your mailing list for this. You may mean Ken Baldwin. Let me know if I am wrong.

Ken

----- Original Message -----
> From: "David Beer" <dbeer at adaptivecomputing.com>
> To: "Torque Developers mailing list" <torquedev at supercluster.org>
> Sent: Thursday, August 25, 2011 10:22:54 AM
> Subject: Re: [torquedev] IP version-agnostic Address Representation
> 
> 
> 
> ----- Original Message -----
> > Ken Nielson wrote:
> > > ----- Original Message -----
> > >   
> > >> From: "Donald Neal" <dmneal at wand.net.nz>
> > >> To: "Torque Developers mailing list"
> > >> <torquedev at supercluster.org>
> > >> Sent: Thursday, August 11, 2011 6:05:59 PM
> > >> Subject: [torquedev] IP version-agnostic Address Representation
> > >> There are a range of cases in Torque where an IP address is
> > >> represented
> > >> by a 32-bit object. This is something of a problem where the IP
> > >> address
> > >> may actually be 128 bits long.
> > >>
> > >> I see two distinct cases here. In one the 32-bit object is being
> > >> used
> > >> as
> > >> a key (as with AvlNode). In this case I propose initially to
> > >> take
> > >> the
> > >> lower-order 32 bits of an IPv6 address and keep going as now.
> > >> This
> > >> is
> > >> simple and effiicient, but would definitely give rise to
> > >> collisions
> > >> which will confuse people in future.
> > >>
> > >> So the key does in future need to change. My inclination is to
> > >> use
> > >> a
> > >> struct created for the purpose containing two uint64_t's. But
> > >> there
> > >> are
> > >> definitely other ways of doing this.
> > >>
> > >> In the other case, the address is not used as a key (as in
> > >> struct
> > >> pbsnode, say). That makes it viable to use either a struct
> > >> containing
> > >> the minimum fields necessary or a sockaddr*. I'm inclined
> > >> towards
> > >> the
> > >> latter on the grounds that keeping down the number of struct
> > >> types
> > >> in
> > >> use makes life simpler, But again there are clearly other ways
> > >> of
> > >> doing
> > >> this.
> > >>
> > >> Does anyone have a view on this? Does anyone have any other
> > >> reason
> > >> to
> > >> use a package like gmp, which I don't see as needed for this
> > >> purpose
> > >> alone?
> > >>
> > >> - Donald Neal
> > >>
> > >>     
> > >
> > > The current scheme TORQUE uses to identify trusted hosts is to
> > > use
> > > the host name and IPv4 ip address. The 32-bit IP address is used
> > > as a key to store trusted addresses in an AVL tree. The
> > > convenient
> > > thing about using this scheme is that connections are verified
> > > without the need to read any data from the stream. With the
> > > 128-bit IPv6 address the AVL tree is broken. We could change the
> > > AVL-tree to take a 128 bit key and then pad the IPv4 addresses.
> > > (I
> > > am just brain-storming).
> > >
> > > Another thought is to have the server assign a random number to
> > > each MOM and then distribute those numbers across the cluster.
> > > This would require a change in the protocol. It also requires a
> > > reading of the incoming stream to get the number.
> > >
> > > Just some thoughts.
> > >
> > > Ken
> > >   
> >  From the (limited) discussion, I'm inclined to conclude
> > 
> > i) Noone has another use for gmp, so general-purpose large objects
> > are
> > not the answer.
> > ii)) Replacing the 32-bit key in AvlNode with a struct of two
> > uint64_t's
> > is as good an idea as anything else.
> > 
> > So life looks to be simpler if all AvlNodes have a 128-bit key in
> > place
> > of a 32-bit one. This becomes more complex if the same node
> > identifier
> > is used both with an AvlNode and with a resizable_array.
> > 
> > Is there a simple explanation anywhere of what the resizable_array
> > struct is intended to be used for? This is a question about design
> > philosophy rather than existing code.
> > 
> 
> The resizable_array struct is being used in many places to replace
> linked lists. This struct actually has the capability of maintaining
> a fixed order, so in a way it is still a linked list that resides in
> an array.
> 
> The struct is used (in 4.0) to store jobs, nodes, queues, and tasks.
> As far as my intention for its use, I wanted to use it as an easy
> way for storing objects in arrays where the number of objects could
> change. I have tried to make it into a flexible and widely usable
> struct. Is there anything more specific you'd like to know about it?
> 
> David
> _______________________________________________
> torquedev mailing list
> torquedev at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torquedev
> 


More information about the torquedev mailing list