[gold-users] Newbie question: GUI and list of jobs

Tolis Sisiaridis tolis.sisiaridis at oerc.ox.ac.uk
Thu Nov 22 03:26:43 MST 2007


Thanks,
like you said it was the Start/Endtime fields that were causing this.
I modified the syntax of my 'charge' scripts to extract times from PBS pro,
convert them to GOLD format ( YYYY-MM-DD mm:hh) and use them.
Now the GUI prints everything just fine. It is also possible to query 
past jobs
with blank time fields from the GUI  if i leave the Prefilter text boxes 
blank.
  
Tolis




Scott Jackson wrote:
> I apologize if you have already got past this and if I am answering old
> email that may be no longer relevant to you but I got way behind around
> supercomputing.
>
> Now that I see your log file, I recall that the GUI was written to base
> its query on StartTime and EndTime. At a prior time, Moab was not
> sending StartTime and EndTime of the job to Gold so these fields were
> remaining blank.
>
> (glsjob was not imposing any StartTime and EndTime criteria, so it
> listed all jobs, but the GUI asked for StartTime>-infinity and
> EndTime<infinity or some such permutation and this did not match any
> rows because StartTime and EndTime were NULL).
>
> If you want to use the gui, you will have to ensure either that your
> jobs are populating the StartTime or EndTime fields, or change the GUI
> to not impose the date condition when nothing is selected. The latter
> may be a bit hard to do because all of these fields are generated in a
> dynamic way, and it may be a bit elaborate to architect the exception.
> The supported solution would be the former, make sure that StartTime and
> EndTime are being properly populated in the Gold job records.
>
> If you are using Moab, then it is possible that you are using an older
> version, before we corrected Moab to populate the fields correctly. If
> you are using something else, let me know what it is, and I can
> recommend the syntax modification needed in the Job Charge request.
>
> You may also want to go back and populate the jobs that have already
> been charged. This CAN be determined in most cases by examining moab
> event files, or by extracting the CreationTime field for the Reserve and
> Charge transactions in the Transaction log. There are also other
> elaborate means at your disposal, though all of them will require some
> scripting work to parse out the times and update the Job table.
>
> Thanks,
>
> Scott
>
>
>
> On Wed, 2007-11-07 at 14:13 +0000, Tolis Sisiaridis wrote:
>   
>> Tim Robinson wrote:
>>     
>>> Hi Tolis
>>> What do the logs say? You should see the SQL query that is being
>>> performed...
>>>
>>> Tim
>>>
>>> On Wed, 2007-11-07 at 11:54 +0000, Tolis Sisiaridis wrote:
>>>   
>>>       
>>>> Just installed GOLD 2.4.1.2 and i am testing integration with PBS PRO.
>>>> Have tried both SQLITE and POSTGRESQL and so far i am very pleased with the results. A small problem though: The GUI would not list jobs no matter what i use for 'Prefilter' options. Everything else on the GUI
>>>> seems to be working well and 'glsjob' lists jobs normally.
>>>> Anyone else with the same issue?
>>>>
>>>> Tolis
>>>>
>>>>
>>>>  
>>>>
>>>>
>>>> _______________________________________________
>>>> gold-users mailing list
>>>> gold-users at supercluster.org
>>>> http://www.supercluster.org/mailman/listinfo/gold-users
>>>>     
>>>>         
>> When i run a query for all jobs (- infinity, infinity) the daemon log 
>> ($GOLD_HOME/log/goldd.log) shows:
>>
>> 2007-11-07 13:55:53.761 INFO  main::  New Connection Received
>> 2007-11-07 13:55:53.763 TRACE Gold::Message::new  invoked with 
>> arguments: (connection => IO::Socket::INET=GLOB(0x60000000017f5b30))
>> 2007-11-07 13:55:53.764 TRACE Gold::Message::receiveChunk  invoked with 
>> arguments: ()
>> 2007-11-07 13:55:53.765 DEBUG Gold::Message::receiveChunk  Read message 
>> header (POST /SSSRMAP3 HTTP/1.1
>> Content-Type: text/xml; charset="utf-8"
>> Transfer-Encoding: chunked
>>
>> ).
>> 2007-11-07 13:55:53.765 INFO  Gold::Message::receiveChunk  Read message 
>> payload (462, <?xml version="1.0" encoding="UTF-8"?>
>> <Envelope><Body><Request action="Query" 
>> actor="gold"><Object>Job</Object><Where name="StartTime" 
>> op="GE">0</Where><Where name="EndTime" op="LT">2147483647</Where><Option 
>> name="Limit">1000</Option></Request></Body><Signature><DigestValue>73tVCxqOqMyFVY0hvtFTp25FwBs=</DigestValue><SignatureValue>zTZ4b/zxkAgM1rtHRUznUEkB+0Y=</SignatureValue><SecurityToken 
>> type="Password" name="gold"></SecurityToken></Signature></Envelope>
>> ).
>> 2007-11-07 13:55:53.766 TRACE Gold::Message::unmarshallChunk  invoked 
>> with arguments: (<?xml version="1.0" encoding="UTF-8"?>
>> <Envelope><Body><Request action="Query" 
>> actor="gold"><Object>Job</Object><Where name="StartTime" 
>> op="GE">0</Where><Where name="EndTime" op="LT">2147483647</Where><Option 
>> name="Limit">1000</Option></Request></Body><Signature><DigestValue>73tVCxqOqMyFVY0hvtFTp25FwBs=</DigestValue><SignatureValue>zTZ4b/zxkAgM1rtHRUznUEkB+0Y=</SignatureValue><SecurityToken 
>> type="Password" name="gold"></SecurityToken></Signature></Envelope>
>> )
>> 2007-11-07 13:55:53.768 TRACE Gold::Chunk::authenticate  invoked with 
>> arguments: (XML::LibXML::Document=SCALAR(0x600000000051f6a0))
>> 2007-11-07 13:55:53.769 TRACE Gold::Cache::getPasswordProperty  invoked 
>> with arguments: (gold, Password)
>> 2007-11-07 13:55:53.774 DEBUG Gold::Chunk::authenticate  The security 
>> token type is (Password).
>> 2007-11-07 13:55:53.775 DEBUG Gold::Chunk::authenticate  The 
>> canonicalized body text is (<Body><Request action="Query" 
>> actor="gold"><Object>Job</Object><Where name="StartTime" 
>> op="GE">0</Where><Where name="EndTime" op="LT">2147483647</Where><Option 
>> name="Limit">1000</Option></Request></Body>).
>> 2007-11-07 13:55:53.775 DEBUG Gold::Chunk::authenticate  The 
>> base64-encoded message digest is (73tVCxqOqMyFVY0hvtFTp25FwBs=).
>> 2007-11-07 13:55:53.776 DEBUG Gold::Chunk::authenticate  The 
>> base64-encoded MAC is (zTZ4b/zxkAgM1rtHRUznUEkB+0Y=).
>> 2007-11-07 13:55:53.787 TRACE Gold::Request::new  invoked with arguments: ()
>> 2007-11-07 13:55:53.789 DEBUG Gold::Message::unmarshallChunk  Extracted 
>> the request ((Query, gold, [(Job, , )], [], [], [(StartTime, 0, GE, , , 
>> , ), (EndTime, 2147483647, LT, , , , )], [(Limit, 1000, )], [], 0, 0, 
>> 100000, )).
>> 2007-11-07 13:55:53.790 DEBUG Gold::Message::receiveChunk  Read message 
>> tail (0).
>> 2007-11-07 13:55:53.790 TRACE Gold::Proxy::new  invoked with arguments: 
>> (request => Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.791 TRACE Gold::Database::new  invoked with 
>> arguments: ()
>> 2007-11-07 13:55:53.804 TRACE Gold::Database::nextId  invoked with 
>> arguments: (Request)
>> 2007-11-07 13:55:53.808 TRACE Gold::Database::nextId  generated new id 
>> 891 for Request
>> 2007-11-07 13:55:53.809 TRACE Gold::Proxy::prepare  invoked with 
>> arguments: (Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.833 TRACE Gold::Cache::objectExists  invoked with 
>> arguments: (Job)
>> 2007-11-07 13:55:53.834 TRACE Gold::Cache::actionExists  invoked with 
>> arguments: (Job, Query)
>> 2007-11-07 13:55:53.835 TRACE Gold::Proxy::authorize  invoked with 
>> arguments: (Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.836 TRACE Gold::Cache::listUserRoles  invoked with 
>> arguments: (gold)
>> 2007-11-07 13:55:53.837 TRACE Gold::Cache::listUserRoles  invoked with 
>> arguments: (ANY)
>> 2007-11-07 13:55:53.838 TRACE Gold::Cache::getObjectProperty  invoked 
>> with arguments: (Job, Association)
>> 2007-11-07 13:55:53.839 TRACE Gold::Cache::listRoleActionInstances  
>> invoked with arguments: (SystemAdmin, Job, Query, ANY)
>> 2007-11-07 13:55:53.839 TRACE Gold::Proxy::prepareOptions  invoked with 
>> arguments: (Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.840 DEBUG Gold::Proxy::prepareOptions  Adding cooked 
>> option: (Limit, 1000, )
>> 2007-11-07 13:55:53.840 TRACE Gold::Proxy::prepareSelections  invoked 
>> with arguments: (Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.841 TRACE Gold::Database::select  invoked with 
>> arguments: (object => Attribute, options => ARRAY(0x6000000001ac1d80), 
>> selections => ARRAY(0x6000000001ac1b00), chunkNum => 0, conditions => 
>> ARRAY(0x600000000116fb00))
>> 2007-11-07 13:55:53.841 TRACE Gold::Database::buildWhere  invoked with 
>> arguments: (objects => ARRAY(0x6000000000847b70), conditions => 
>> ARRAY(0x6000000000847cd0))
>> 2007-11-07 13:55:53.842 DEBUG Gold::Database::select  SQL Query: SELECT 
>> g_name FROM g_attribute WHERE ( g_object='Job' AND g_hidden!='True' ) 
>> AND g_deleted!='True' ORDER BY g_sequence
>> 2007-11-07 13:55:53.845 DEBUG Gold::Database::select  SQL Results: Id
>> JobId
>> User
>> Project
>> Machine
>> Charge
>> Queue
>> Type
>> Stage
>> QualityOfService
>> Nodes
>> Processors
>> Executable
>> Application
>> StartTime
>> EndTime
>> WallDuration
>> QuoteId
>> Description
>> 2007-11-07 13:55:53.846 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (Id)
>> 2007-11-07 13:55:53.846 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (JobId)
>> 2007-11-07 13:55:53.846 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (User)
>> 2007-11-07 13:55:53.847 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (Project)
>> 2007-11-07 13:55:53.847 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (Machine)
>> 2007-11-07 13:55:53.848 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (Charge)
>> 2007-11-07 13:55:53.848 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (Queue)
>> 2007-11-07 13:55:53.848 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (Type)
>> 2007-11-07 13:55:53.849 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (Stage)
>> 2007-11-07 13:55:53.849 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (QualityOfService)
>> 2007-11-07 13:55:53.850 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (Nodes)
>> 2007-11-07 13:55:53.850 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (Processors)
>> 2007-11-07 13:55:53.850 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (Executable)
>> 2007-11-07 13:55:53.851 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (Application)
>> 2007-11-07 13:55:53.851 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (StartTime)
>> 2007-11-07 13:55:53.851 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (EndTime)
>> 2007-11-07 13:55:53.852 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (WallDuration)
>> 2007-11-07 13:55:53.852 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (QuoteId)
>> 2007-11-07 13:55:53.853 DEBUG Gold::Proxy::prepareSelections  Adding 
>> cooked selection: (Description)
>> 2007-11-07 13:55:53.853 TRACE Gold::Proxy::prepareAssignments  invoked 
>> with arguments: (Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.853 TRACE Gold::Cache::listAttributes  invoked with 
>> arguments: (Job)
>> 2007-11-07 13:55:53.855 TRACE Gold::Proxy::prepareConditions  invoked 
>> with arguments: (Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.855 TRACE Gold::Cache::attributeExists  invoked with 
>> arguments: (Job, StartTime)
>> 2007-11-07 13:55:53.856 TRACE Gold::Cache::getAttributeProperty  invoked 
>> with arguments: (Job, StartTime, DataType)
>> 2007-11-07 13:55:53.858 DEBUG Gold::Proxy::prepareConditions  Adding 
>> cooked condition: (StartTime, 0, GE, , 0, , )
>> 2007-11-07 13:55:53.859 TRACE Gold::Cache::attributeExists  invoked with 
>> arguments: (Job, EndTime)
>> 2007-11-07 13:55:53.860 TRACE Gold::Cache::getAttributeProperty  invoked 
>> with arguments: (Job, EndTime, DataType)
>> 2007-11-07 13:55:53.861 DEBUG Gold::Proxy::prepareConditions  Adding 
>> cooked condition: (EndTime, 2147483647, LT, , 0, , )
>> 2007-11-07 13:55:53.862 DEBUG Gold::Proxy::prepare  Cooked request: 
>> (Query, gold, [(Job, , )], [(Id, , , ), (JobId, , , ), (User, , , ), 
>> (Project, , , ), (Machine, , , ), (Charge, , , ), (Queue, , , ), (Type, 
>> , , ), (Stage, , , ), (QualityOfService, , , ), (Nodes, , , ), 
>> (Processors, , , ), (Executable, , , ), (Application, , , ), (StartTime, 
>> , , ), (EndTime, , , ), (WallDuration, , , ), (QuoteId, , , ), 
>> (Description, , , )], [], [(StartTime, 0, GE, , , , ), (EndTime, 
>> 2147483647, LT, , , , )], [(Limit, 1000, )], [], 0, 0, 100000, 
>> Gold::Database=HASH(0x60000000018006e0))
>> 2007-11-07 13:55:53.863 TRACE Gold::Proxy::execute  invoked with 
>> arguments: ()
>> 2007-11-07 13:55:53.863 TRACE Gold::Bank::execute  invoked with 
>> arguments: (Gold::Proxy=HASH(0x6000000001800800))
>> 2007-11-07 13:55:53.864 DEBUG Gold::Bank::query  invoked with arguments: 
>> (Gold::Request=HASH(0x6000000000a14280), 891)
>> 2007-11-07 13:55:53.864 DEBUG Gold::Base::query  invoked with arguments: 
>> (Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.865 TRACE Gold::Database::select  invoked with 
>> arguments: (chunkSize => 100000, options => ARRAY(0x60000000010d9480), 
>> selections => ARRAY(0x60000000010d93e0), objects => 
>> ARRAY(0x60000000010d92a0), chunkNum => 1, conditions => 
>> ARRAY(0x60000000010d9430))
>> 2007-11-07 13:55:53.888 TRACE Gold::Database::buildWhere  invoked with 
>> arguments: (objects => ARRAY(0x6000000000847b70), conditions => 
>> ARRAY(0x6000000000847cd0))
>> 2007-11-07 13:55:53.888 DEBUG Gold::Database::select  SQL Query: SELECT 
>> g_id,g_job_id,g_user,g_project,g_machine,g_charge,g_queue,g_type,g_stage,g_quality_of_service,g_nodes,g_processors,g_executable,g_application,g_start_time,g_end_time,g_wall_duration,g_quote_id,g_description 
>> FROM g_job WHERE ( g_start_time>='0' AND g_end_time<'2147483647' ) AND 
>> g_deleted!='True' LIMIT 1000
>> 2007-11-07 13:55:53.891 DEBUG Gold::Database::select  SQL Results:
>> 2007-11-07 13:55:53.896 TRACE Gold::Response::new  invoked with 
>> arguments: ()
>> 2007-11-07 13:55:53.896 TRACE Gold::Response::success  invoked with 
>> arguments: (0, ARRAY(0x60000000010d9540))
>> 2007-11-07 13:55:53.907 TRACE Gold::Reply::new  invoked with arguments: 
>> (connection => IO::Socket::INET=GLOB(0x60000000017f5b30))
>> 2007-11-07 13:55:53.908 TRACE Gold::Reply::sendChunk  invoked with 
>> arguments: (Gold::Chunk=HASH(0x6000000001ab9910))
>> 2007-11-07 13:55:53.908 TRACE Gold::Reply::marshallChunk  invoked with 
>> arguments: (Gold::Chunk=HASH(0x6000000001ab9910))
>> 2007-11-07 13:55:53.909 DEBUG Gold::Reply::sendChunk  Writing reply 
>> header (HTTP/1.1 200 OK
>> Content-Type: text/xml; charset="utf-8"
>> Transfer-Encoding: chunked
>> ).
>> 2007-11-07 13:55:53.910 INFO  Gold::Reply::sendChunk  Writing reply 
>> payload (179, <?xml version="1.0" encoding="UTF-8"?>
>> <Envelope><Body><Response 
>> actor="gold"><Status><Value>Success</Value><Code>000</Code></Status><Count>0</Count></Response></Body></Envelope>
>> ).
>>
>>
>> I will try to run the query on the database and see if anything comes out.
>>
>>
>>
>> _______________________________________________
>> gold-users mailing list
>> gold-users at supercluster.org
>> http://www.supercluster.org/mailman/listinfo/gold-users
>>     
>
>   



More information about the gold-users mailing list