[torqueusers] Upgrade from 2.1.*

Ken Nielson knielson at adaptivecomputing.com
Tue Aug 17 20:16:55 MDT 2010


I second that. Yes the current method is the fastest. 

For serverdb there are some advantages to going with the XML. First it gives a portable format to move between systems. It also gives a human readable file that can be easily read and debugged if there are problems. The file is also relatively small. Most of the time the information in the file is in memory anyway so disk access is really not an issue.

So what advantage would we have by moving job files to XML?

Human readable. More portable, although I don't know how much utility there is in that. How often are job files manually copied to a new system. Because they are human readable they provide more information for debugging. Upgrading would be easier because we are not tied to a data size. Anything else?

It looks like right now job files are 5-6K in size. That seems kind of large. I guess all of that empty attribute stuff. We would reduce that with an XML file. I am curious now of how much of an impact the parsing routine would have over what we currently use?

Ken




----- Original Message -----
From: "Garrick Staples" <garrick at usc.edu>
To: "Torque Users Mailing List" <torqueusers at supercluster.org>
Sent: Tuesday, August 17, 2010 6:54:36 PM
Subject: Re: [torqueusers] Upgrade from 2.1.*

I'll see your top-post with my own top-post!

What would be the point of a database? Databases are great for reads, but noone
has stated any need for this data for anything. The only use I know of is when
the daemon starts.

The current raw struct to disk to the fastest possible code with the simpliest
possible code. The only downside is changing disk formats. Amiright?


On Tue, Aug 17, 2010 at 08:46:53PM -0400, Glen Beane alleged:
> 
> Sorry for the top post (on my phone), but what about storing the server and job info in an SQLite (or similar) database? 
> 
> 
> On Aug 17, 2010, at 5:57 PM, Ken Nielson <knielson at adaptivecomputing.com> wrote:
> 
> > On 08/17/2010 03:02 PM, Ken Nielson wrote:
> >> 
> >> On 08/17/2010 03:01 PM, Garrick Staples wrote:
> >>> 
> >>> On Tue, Aug 17, 2010 at 02:59:19PM -0600, Ken Nielson alleged:
> >>>   
> >>>> On 08/17/2010 12:14 PM, Glen Beane wrote:
> >>>>     
> >>>>> that would be good.  Although I'd really love to get rid of just
> >>>>> dumping the struct right to disk and have something that can be
> >>>>> extended without jumping through hoops to maintain compatibility.
> >>>>> 
> >>>>>    
> >>>>>       
> >>>> Did I hear XML?
> >>>> 
> >>>> serverdb is now an XML file in trunk. How about job files as well?
> >>>>     
> >>> How badly was performance affected with the XML change?
> >>> 
> >>>   
> >>> 
> >>> _______________________________________________
> >>> torqueusers mailing list
> >>> torqueusers at supercluster.org
> >>> http://www.supercluster.org/mailman/listinfo/torqueusers
> >>>   
> >> We really need to measure it. But it does not seem to have a noticeable impact on the systems where the trunk code is currently running.
> >> 
> >> Ken
> > 
> > Confession time. 
> > 
> > It is in the 3.0 code base but it has not yet been moved to trunk. However, because the information of serverdb is cached
> > the hit on performance is negligible.
> > 
> > We will be moving the code into trunk soon as well.
> > 
> > Ken
> > _______________________________________________
> > torqueusers mailing list
> > torqueusers at supercluster.org
> > http://www.supercluster.org/mailman/listinfo/torqueusers

> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers


-- 
Garrick Staples, GNU/Linux HPCC SysAdmin
University of Southern California

Life is Good!

_______________________________________________
torqueusers mailing list
torqueusers at supercluster.org
http://www.supercluster.org/mailman/listinfo/torqueusers


More information about the torqueusers mailing list