[torqueusers] accounting

[Naveed Near-Ansari]

I think if you specify in torque.cfg to use a qsub filter it will use it
even when submitting through qsub.

torque.cfg:

SUBMITFILTER    /opt/torque/bin/qsub_filter

I have not attempted exiting on certain conditions using this since i
only use it to add in default wallclock times when not set by the user,
but presumably you could do your checking here.

Naveed

On Tue, 2009-06-23 at 20:44 -0400, Gus Correa wrote:
> Hi Jon, Troy, list
> 
> Besides Troy's suggestions:
> 
> Ole Holm Nielsen wrote a suite of scripts to do Torque/PBS accounting
> reports:
> 
> ftp://ftp.fysik.dtu.dk/pub/Torque/pbsacct-1.4.6.tar.gz
> 
> I never tried it,
> but would guess a prologue script may be able to do #2 also,
> say, by checking if the project number is on a list of projects,
> and returning an abort exit code (1),
> if the project is not valid:
> 
> http://www.clusterresources.com/torquedocs21/a.gprologueepilogue.shtml
> 
> However, this may reject the job only when it starts,
> not right after qsub, like Troy's "qsub filter" would do.
> In other words, the prologue solution defers the job deletion to run time.
> 
> My doubt is how to enforce the use of the "qsub filter",
> instead of the actual qsub command.
> I suppose the user could always write the full path to qsub, instead of 
> the filter alias.
> 
> You could use both solutions also,
> if you are an overzealous sys admin. :)
> 
> I hope this helps,
> Gus Correa
> ---------------------------------------------------------------------
> Gustavo Correa
> Lamont-Doherty Earth Observatory - Columbia University
> Palisades, NY, 10964-8000 - USA
> ---------------------------------------------------------------------
> 
> Troy Baer wrote:
> > On Tue, 2009-06-23 at 21:09 +0200, Jon Tegner wrote:
> >> We need to monitor the usage of our cluster resources, and we need to 
> >> now how much time is spent by different projects (a user can work on 
> >> several projects). So what I need to do is the following:
> >>
> >> 1. Each user should supply a project number when submitting a job.
> >> 2. If the project number is not a valid one the job should be rejected.
> >> 3. We need to extract the relevant information from the 
> >> server_priv/accounting/2009xxxx-file.
> >>
> >> I can do 1 and 3 if I let all nodes have "project-properties", but I 
> >> haven't managed to find out how to achieve 2.
> >>
> >> Can this be done in torque? And surely there is a better approach than 
> >> this one?
> > 
> > One way to handle #1 and #2 is to write a submit filter that checks for
> > a valid project code rejects the user's job if none is found.
> > 
> > As far as #3, there are lots of examples of code out there of how to
> > parse and analyze the TORQUE accounting records.  For instance, PBS
> > Tools [1] includes a set of Perl scripts that parse the accounting logs
> > and stuff them into a MySQL database.  It all depends on what you want
> > to do with it, and it seems that every site is slightly different in
> > that regard.
> > 
> > [1] http://www.osc.edu/~troy/pbstools/pbstools.tar.gz
> > 
> > 	--Troy
> 
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers
>