[gold-users] Questions about statistics (gstatement/glsjob)

Mehmet Soysal mehmet.soysal at kit.edu
Thu Aug 12 02:27:56 MDT 2010


Hi,
first of all, i wanna apologize for my email.
A lot of typo in that mail (and missing words), i coded whole day perl 
and after that it is hard to concentrate for a clean mail :).

Thanks for your detailed explanations. I understand that Gold is almost 
a whole booking system. But i was asking myself if there i not such way, 
that i can show
which jobs ended in a month, with a little bit summarization which 
projects used how much cpu time.

Im already using the perl API to do this things for me. Jobs are charged 
with detailed Data how much core/nodes are used and correct start- and 
enddates.

As far as i know, our batchsystem (slurm) is not able to communicate 
directly with Gold, like others (torque), thats the reason why we fill 
Gold with filtered logs.

I think Gold is a great piece of software and the Perl API is not hard 
to understand,
it is written very cleanly (and it is hard to find cleanly written perl 
Code). But i think some examples (or tutorials) would be great and help 
other users, especially using goldsh.

Thx for your help.
M.Soysal

> Hi M.Soysal,
>
> A couple of things:
>
> First, CreationTime represents when a job record was first created or
> when an import event actually happened, while StartTime and EndTime
> represent when a job started or ended respectively. It is not
> infrequently the case that some schedulers or resource managers do not
> supply the start or endtime of the job in the reservation or charge so
> these fields are sometimes not filled in.
>
> Assuming you are just looking at when jobs ran, and further assuming
> that you set proper startTime and endTime values for the jobs when you
> imported them, then you should be able to come up with an appropriate
> query when querying jobs to get the job listing that you want.
>
> Secondly, Gold maintains a consistent historical state of the balance
> and it does not allow you to arbitrarily change past balance states.
> Just as in a real bank, the debits and credits are registered. If a
> mistake is detected, this is corrected through a refund, or an
> appropriate present action. You cannot just go back and change past
> states thereby invalidating and reformulating previous balance sheets.
> If a wrong amount was charged yesterday, you can correct it today by a
> refund or deposit, but you cannot change yesterday's balance. Same for
> the importing of new job data. This will generally not affect the
> balance. If you decide to correlate these with charges, the charges will
> hit present balances, not previous balances.
>
> The gstatement command looks back at previous states of the account
> balance. It uses CreationTime on the Transactions to filter out what
> balance affecting actions occurred during the specified time period.
> Gstatement is really looking at changes to the state of the account
> balances over that time. Gold will not provide you a way to import the
> jobs in such a way that impacts the balances at a previous time. If you
> want to try to do this on your own you will have to accept whatever
> inconsistencies it may create in the journals.
>
> For your immediate question, you would have to either write a client
> using the perl gold api to do what you want or use the goldsh prompt,
> something like:
>
>
> Job Query EndTime>='2010-06-01' EndTime<'2010-07-01'
> Show:="GroupBy(Project),Sum(Charge)"
>
> I hope this helps,
>
> Scott
>
> Mehmet Soysal wrote:
>    
>> Hi,
>> since a few weeks we are using Gold now and i have some questions about
>> how to get some statistics from Gold.
>>
>> I wrote some script to fill Gold with all our old logs,
>> everything looks good and works great so far.
>>
>> But struggling about "CreationTime" and Job EndTime
>>
>> In my case i trying to get a summarization for example:
>> How many job finished in a month (incl. used cpu time)
>>      
>
>    
>> and maybe which projects consumed how many cpu hours in this month and
>> so on ..
>>
>> gstatement command looks right for this, but it is using the
>> Creationtime from  allocations. But if i fill Gold with old logs they
>> all have actual dates for creationtime.
>>
>> With glsjob i can tell which jobs i want:
>> glsjob  -s "2009-04-01 00:00:00" -e "2009-04-30 23:59:59"
>> but there i no kind of summarize for users/projects/machines ........
>>
>>
>> I added some options and a little bit for glsjob which does this now
>> for me, but im wondering if there is an already working solution for
>> this.
>>
>> MfG
>> M.Soysal
>>
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> gold-users mailing list
>> gold-users at supercluster.org
>> http://www.supercluster.org/mailman/listinfo/gold-users
>>
>>      
> _______________________________________________
> gold-users mailing list
> gold-users at supercluster.org
> http://www.supercluster.org/mailman/listinfo/gold-users
>    


-- 
---------------------------------------+--------------------------------------
Mehmet Soysal                          | Tel   : +49 721 608-6347
Karlsruhe Institut of Technology (KIT) | Email : mehmet.soysal at kit.edu
Steinbuch Centre for Computing (SCC)   |
Zirkel 2                               |
D-76128 Karlsruhe                      |
---------------------------------------+--------------------------------------


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6349 bytes
Desc: S/MIME Cryptographic Signature
Url : http://www.supercluster.org/pipermail/gold-users/attachments/20100812/36cc5c11/attachment.bin 


More information about the gold-users mailing list