[gold-users] listing jobs charged to a specific allocation

[Scott Jackson]

Kevin Van Workum wrote:
> On Mon, Mar 8, 2010 at 7:18 PM, Scott Jackson
> <scottmo at adaptivecomputing.com> wrote:
>   
>> Kevin,
>>
>> Kevin Van Workum wrote:
>>     
>>> I'm trying to create a command that will list jobs that have been
>>> charged to a specific allocation. I realize I can use "glstxn -i Id -O
>>> Job", but that lists the transactions.
>>>       
>> Well, it is not that simple because a job can have any number of charges
>> against it and each charge can be to one or more allocations. So, it is not
>> possible to directly relate a one-to-one answer to this question. You
>> basically are going to have to look at this in terms of the transactions,
>> though this can certainly be narrowed down to just Charges.
>>
>> Ohh, I see -- you know that, you just want any job listed that has ever had
>> a charge against a particular allocation...
>>
>> goldsh Transaction Query Show:=JobId,Amount Action==Charge Allocation==26
>>
>> or
>>
>> glstxn -A Charge -i 26 --show=JobId
>>
>> If you want the job info for these guys, you'll have to use goldsh to do a
>> join:
>>
>> goldsh Job,Transaction Query
>> Show:=Job.Id,Job.JobId,Job.User,Job.Project,Job.Machine,Job.Processors,Job.WallDuration,Job.Charge
>> Transaction.Action==Charge Transaction.Allocation==26
>> Transaction.JobId==Job.JobId
>>
>> Id  JobId        User    Project   Machine Processors WallDuration Charge
>> --- ------------ ------- --------- ------- ---------- ------------ ------
>> 32  2            root    catchall  colony             1               123
>> 36  root.8       root    catchall  keko    1          465               1
>> 37  root.9       root    catchall  keko    1          523               1
>> 40  root.12      root    catchall  keko    1          362               1
>> 43  RHEL3ASVM.15 root    catchall  keko    1          58              111
>> 45  RHEL3ASVM.17 root    catchall  keko    1          17              111
>> 51  80           scottmo catchall  keko    1          277             277
>>     
>
> Cool. I didn't know you could do that. Unfortunately, gold seems to
> choke on this. The same command as above just hangs. Then I have to
> restart postgresql to get things back to normal.
>
>   

Hmmmm..... maybe your Transaction table (and Job table) are ginormous. I 
can't think of any other reason for the hang. Just for curiosity's sake, 
how long have you waited for it to return? I'm sorry if I've asked 
before, but do you have the indexes in place? How many rows are in your 
Transaction table? Job table. We are doing a join on it, so this may not 
be practical. If they are too large and postgres does not have memory 
for the join, we may need to take the iterative approach. It would be 
interesting to try to do a direct join in Postgres to do the similar 
join query in the psql prompt. You can get the sql that gold is trying 
to enact in the goldd.log if you want a good starting point.

Scott



>>> I want to get the list of jobs.
>>> I could first use glstxn, then use glsjob for each job found, but is
>>> there a more efficient way. Maybe something like:
>>>
>>>  my $request = new Gold::Request(object => "Job", action => "Query");
>>>  Gold::Client::buildSupplements($request);
>>>  $request->setCondition("AllocationId", "123");
>>>
>>>       
>> To do this in the perl api, something like (I'm winging it here, you will
>> have to tweak this a tad):
>>
>> my $request = new Gold::Request(
>>  objects => [
>>           new Gold::Object(name => "Job"),
>>           new Gold::Object(name => "Transaction")
>>             ], action => "Query");
>>
>> $request->setSelection("Id", "", "Job");
>> $request->setSelection("JobId", "", "Job");
>> $request->setSelection("User", "", "Job");
>> ...
>> $request->setCondition("Action", "Charge", "", "", "", "Transaction");
>> $request->setCondition("Allocation", "26", "", "", "", "Transaction");
>> $request->setCondition("JobId", "JobId", "", "", "", "Transaction", "Job");
>>
>>
>>
>> See perldoc for Request, Condition, Selection, Object, etc.
>>
>>
>>
>>     
>>> --
>>> Kevin Van Workum, PhD
>>> Sabalcore Computing Inc.
>>> Run your code on 500 processors.
>>> Sign up for a free trial account.
>>> www.sabalcore.com
>>> 877-492-8027 ext. 11
>>> _______________________________________________
>>> gold-users mailing list
>>> gold-users at supercluster.org
>>> http://www.supercluster.org/mailman/listinfo/gold-users
>>>
>>>       
>>     
>
>
>
>