[torqueusers] Upgrade from 2.1.*

[Garrick Staples]

On Tue, Aug 17, 2010 at 04:30:00PM -0700, Joshua Bernstein alleged:
> 
> 
> Garrick Staples wrote:
> > On Tue, Aug 17, 2010 at 04:18:58PM -0700, Joshua Bernstein alleged:
> >>
> >> Ken Nielson wrote:
> >>> On 08/17/2010 12:14 PM, Glen Beane wrote:
> >>>> that would be good.  Although I'd really love to get rid of just
> >>>> dumping the struct right to disk and have something that can be
> >>>> extended without jumping through hoops to maintain compatibility.
> >>>>
> >>>>    
> >>> Did I hear XML?
> >>>
> >>> serverdb is now an XML file in trunk. How about job files as well?
> >> No. I'm not in favor of doing something like this. Imagine a third party 
> >> application, say like PBSQuery having now to parse XML!
> > 
> > PBSQuery reads the job files? Why would it do that? 
> 
> No, it doesn't read the job files themselves, but it does query the 
> server for information about jobs. I always figured then, in turn 
> pbs_server would have to read through its own job files to return the 
> results.

No, pbs_server only scribbles it down to disk to survive restarts of the
system/daemon.


> >> It might work well for the small systems you guys mess around on, but 
> >> I'd really like us all to consider what happens when hundreds of 
> >> thousands of jobs are present, as even with the current implementation 
> >> things are uberslow sometimes.
> > 
> > Obviously one doesn't just throw XML out there in the wild polluting the place.
> > One releases the associated tools and libraries to read/write the XML.
> > 
> > This data is only ever read when pbs_server starts. Read performance isn't
> > important. We only care about write performance.
> 
> What if pbs_server can't keep all of the job information in memory? 

It must. Or I suppose, the system runs out of ram.

> Write performance would seem to be limited to the performance of the 
> file system.

Either the filesystem for filesystem operations, or the hardware for data
operations.

-- 
Garrick Staples, GNU/Linux HPCC SysAdmin
University of Southern California

Life is Good!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://www.supercluster.org/pipermail/torqueusers/attachments/20100817/2205580e/attachment.bin