[torqueusers] accounting

Gus Correa gus at ldeo.columbia.edu
Wed Jun 24 16:17:09 MDT 2009


Hi Naveed, list

Naveed, thank you very much!
That is very useful information I didn't know about.
David Singleton just sent yet another suggestion,
which you may have seen.

Gus Correa

Naveed Near-Ansari wrote:
> I think if you specify in torque.cfg to use a qsub filter it will use it
> even when submitting through qsub.
> 
> torque.cfg:
> 
> SUBMITFILTER    /opt/torque/bin/qsub_filter
> 
> I have not attempted exiting on certain conditions using this since i
> only use it to add in default wallclock times when not set by the user,
> but presumably you could do your checking here.
> 
> Naveed
> 
> On Tue, 2009-06-23 at 20:44 -0400, Gus Correa wrote:
>> Hi Jon, Troy, list
>>
>> Besides Troy's suggestions:
>>
>> Ole Holm Nielsen wrote a suite of scripts to do Torque/PBS accounting
>> reports:
>>
>> ftp://ftp.fysik.dtu.dk/pub/Torque/pbsacct-1.4.6.tar.gz
>>
>> I never tried it,
>> but would guess a prologue script may be able to do #2 also,
>> say, by checking if the project number is on a list of projects,
>> and returning an abort exit code (1),
>> if the project is not valid:
>>
>> http://www.clusterresources.com/torquedocs21/a.gprologueepilogue.shtml
>>
>> However, this may reject the job only when it starts,
>> not right after qsub, like Troy's "qsub filter" would do.
>> In other words, the prologue solution defers the job deletion to run time.
>>
>> My doubt is how to enforce the use of the "qsub filter",
>> instead of the actual qsub command.
>> I suppose the user could always write the full path to qsub, instead of 
>> the filter alias.
>>
>> You could use both solutions also,
>> if you are an overzealous sys admin. :)
>>
>> I hope this helps,
>> Gus Correa
>> ---------------------------------------------------------------------
>> Gustavo Correa
>> Lamont-Doherty Earth Observatory - Columbia University
>> Palisades, NY, 10964-8000 - USA
>> ---------------------------------------------------------------------
>>
>> Troy Baer wrote:
>>> On Tue, 2009-06-23 at 21:09 +0200, Jon Tegner wrote:
>>>> We need to monitor the usage of our cluster resources, and we need to 
>>>> now how much time is spent by different projects (a user can work on 
>>>> several projects). So what I need to do is the following:
>>>>
>>>> 1. Each user should supply a project number when submitting a job.
>>>> 2. If the project number is not a valid one the job should be rejected.
>>>> 3. We need to extract the relevant information from the 
>>>> server_priv/accounting/2009xxxx-file.
>>>>
>>>> I can do 1 and 3 if I let all nodes have "project-properties", but I 
>>>> haven't managed to find out how to achieve 2.
>>>>
>>>> Can this be done in torque? And surely there is a better approach than 
>>>> this one?
>>> One way to handle #1 and #2 is to write a submit filter that checks for
>>> a valid project code rejects the user's job if none is found.
>>>
>>> As far as #3, there are lots of examples of code out there of how to
>>> parse and analyze the TORQUE accounting records.  For instance, PBS
>>> Tools [1] includes a set of Perl scripts that parse the accounting logs
>>> and stuff them into a MySQL database.  It all depends on what you want
>>> to do with it, and it seems that every site is slightly different in
>>> that regard.
>>>
>>> [1] http://www.osc.edu/~troy/pbstools/pbstools.tar.gz
>>>
>>> 	--Troy
>> _______________________________________________
>> torqueusers mailing list
>> torqueusers at supercluster.org
>> http://www.supercluster.org/mailman/listinfo/torqueusers
>>



More information about the torqueusers mailing list