[gold-users] Newbie question: GUI and list of jobs

Scott Jackson scott at clusterresources.com
Wed Nov 21 13:17:17 MST 2007


I apologize if you have already got past this and if I am answering old
email that may be no longer relevant to you but I got way behind around
supercomputing.

Now that I see your log file, I recall that the GUI was written to base
its query on StartTime and EndTime. At a prior time, Moab was not
sending StartTime and EndTime of the job to Gold so these fields were
remaining blank.

(glsjob was not imposing any StartTime and EndTime criteria, so it
listed all jobs, but the GUI asked for StartTime>-infinity and
EndTime<infinity or some such permutation and this did not match any
rows because StartTime and EndTime were NULL).

If you want to use the gui, you will have to ensure either that your
jobs are populating the StartTime or EndTime fields, or change the GUI
to not impose the date condition when nothing is selected. The latter
may be a bit hard to do because all of these fields are generated in a
dynamic way, and it may be a bit elaborate to architect the exception.
The supported solution would be the former, make sure that StartTime and
EndTime are being properly populated in the Gold job records.

If you are using Moab, then it is possible that you are using an older
version, before we corrected Moab to populate the fields correctly. If
you are using something else, let me know what it is, and I can
recommend the syntax modification needed in the Job Charge request.

You may also want to go back and populate the jobs that have already
been charged. This CAN be determined in most cases by examining moab
event files, or by extracting the CreationTime field for the Reserve and
Charge transactions in the Transaction log. There are also other
elaborate means at your disposal, though all of them will require some
scripting work to parse out the times and update the Job table.

Thanks,

Scott



On Wed, 2007-11-07 at 14:13 +0000, Tolis Sisiaridis wrote:
> Tim Robinson wrote:
> > Hi Tolis
> > What do the logs say? You should see the SQL query that is being
> > performed...
> >
> > Tim
> >
> > On Wed, 2007-11-07 at 11:54 +0000, Tolis Sisiaridis wrote:
> >   
> >> Just installed GOLD 2.4.1.2 and i am testing integration with PBS PRO.
> >> Have tried both SQLITE and POSTGRESQL and so far i am very pleased with the results. A small problem though: The GUI would not list jobs no matter what i use for 'Prefilter' options. Everything else on the GUI
> >> seems to be working well and 'glsjob' lists jobs normally.
> >> Anyone else with the same issue?
> >>
> >> Tolis
> >>
> >>
> >>  
> >>
> >>
> >> _______________________________________________
> >> gold-users mailing list
> >> gold-users at supercluster.org
> >> http://www.supercluster.org/mailman/listinfo/gold-users
> >>     
> 
> 
> When i run a query for all jobs (- infinity, infinity) the daemon log 
> ($GOLD_HOME/log/goldd.log) shows:
> 
> 2007-11-07 13:55:53.761 INFO  main::  New Connection Received
> 2007-11-07 13:55:53.763 TRACE Gold::Message::new  invoked with 
> arguments: (connection => IO::Socket::INET=GLOB(0x60000000017f5b30))
> 2007-11-07 13:55:53.764 TRACE Gold::Message::receiveChunk  invoked with 
> arguments: ()
> 2007-11-07 13:55:53.765 DEBUG Gold::Message::receiveChunk  Read message 
> header (POST /SSSRMAP3 HTTP/1.1
> Content-Type: text/xml; charset="utf-8"
> Transfer-Encoding: chunked
> 
> ).
> 2007-11-07 13:55:53.765 INFO  Gold::Message::receiveChunk  Read message 
> payload (462, <?xml version="1.0" encoding="UTF-8"?>
> <Envelope><Body><Request action="Query" 
> actor="gold"><Object>Job</Object><Where name="StartTime" 
> op="GE">0</Where><Where name="EndTime" op="LT">2147483647</Where><Option 
> name="Limit">1000</Option></Request></Body><Signature><DigestValue>73tVCxqOqMyFVY0hvtFTp25FwBs=</DigestValue><SignatureValue>zTZ4b/zxkAgM1rtHRUznUEkB+0Y=</SignatureValue><SecurityToken 
> type="Password" name="gold"></SecurityToken></Signature></Envelope>
> ).
> 2007-11-07 13:55:53.766 TRACE Gold::Message::unmarshallChunk  invoked 
> with arguments: (<?xml version="1.0" encoding="UTF-8"?>
> <Envelope><Body><Request action="Query" 
> actor="gold"><Object>Job</Object><Where name="StartTime" 
> op="GE">0</Where><Where name="EndTime" op="LT">2147483647</Where><Option 
> name="Limit">1000</Option></Request></Body><Signature><DigestValue>73tVCxqOqMyFVY0hvtFTp25FwBs=</DigestValue><SignatureValue>zTZ4b/zxkAgM1rtHRUznUEkB+0Y=</SignatureValue><SecurityToken 
> type="Password" name="gold"></SecurityToken></Signature></Envelope>
> )
> 2007-11-07 13:55:53.768 TRACE Gold::Chunk::authenticate  invoked with 
> arguments: (XML::LibXML::Document=SCALAR(0x600000000051f6a0))
> 2007-11-07 13:55:53.769 TRACE Gold::Cache::getPasswordProperty  invoked 
> with arguments: (gold, Password)
> 2007-11-07 13:55:53.774 DEBUG Gold::Chunk::authenticate  The security 
> token type is (Password).
> 2007-11-07 13:55:53.775 DEBUG Gold::Chunk::authenticate  The 
> canonicalized body text is (<Body><Request action="Query" 
> actor="gold"><Object>Job</Object><Where name="StartTime" 
> op="GE">0</Where><Where name="EndTime" op="LT">2147483647</Where><Option 
> name="Limit">1000</Option></Request></Body>).
> 2007-11-07 13:55:53.775 DEBUG Gold::Chunk::authenticate  The 
> base64-encoded message digest is (73tVCxqOqMyFVY0hvtFTp25FwBs=).
> 2007-11-07 13:55:53.776 DEBUG Gold::Chunk::authenticate  The 
> base64-encoded MAC is (zTZ4b/zxkAgM1rtHRUznUEkB+0Y=).
> 2007-11-07 13:55:53.787 TRACE Gold::Request::new  invoked with arguments: ()
> 2007-11-07 13:55:53.789 DEBUG Gold::Message::unmarshallChunk  Extracted 
> the request ((Query, gold, [(Job, , )], [], [], [(StartTime, 0, GE, , , 
> , ), (EndTime, 2147483647, LT, , , , )], [(Limit, 1000, )], [], 0, 0, 
> 100000, )).
> 2007-11-07 13:55:53.790 DEBUG Gold::Message::receiveChunk  Read message 
> tail (0).
> 2007-11-07 13:55:53.790 TRACE Gold::Proxy::new  invoked with arguments: 
> (request => Gold::Request=HASH(0x6000000000a14280))
> 2007-11-07 13:55:53.791 TRACE Gold::Database::new  invoked with 
> arguments: ()
> 2007-11-07 13:55:53.804 TRACE Gold::Database::nextId  invoked with 
> arguments: (Request)
> 2007-11-07 13:55:53.808 TRACE Gold::Database::nextId  generated new id 
> 891 for Request
> 2007-11-07 13:55:53.809 TRACE Gold::Proxy::prepare  invoked with 
> arguments: (Gold::Request=HASH(0x6000000000a14280))
> 2007-11-07 13:55:53.833 TRACE Gold::Cache::objectExists  invoked with 
> arguments: (Job)
> 2007-11-07 13:55:53.834 TRACE Gold::Cache::actionExists  invoked with 
> arguments: (Job, Query)
> 2007-11-07 13:55:53.835 TRACE Gold::Proxy::authorize  invoked with 
> arguments: (Gold::Request=HASH(0x6000000000a14280))
> 2007-11-07 13:55:53.836 TRACE Gold::Cache::listUserRoles  invoked with 
> arguments: (gold)
> 2007-11-07 13:55:53.837 TRACE Gold::Cache::listUserRoles  invoked with 
> arguments: (ANY)
> 2007-11-07 13:55:53.838 TRACE Gold::Cache::getObjectProperty  invoked 
> with arguments: (Job, Association)
> 2007-11-07 13:55:53.839 TRACE Gold::Cache::listRoleActionInstances  
> invoked with arguments: (SystemAdmin, Job, Query, ANY)
> 2007-11-07 13:55:53.839 TRACE Gold::Proxy::prepareOptions  invoked with 
> arguments: (Gold::Request=HASH(0x6000000000a14280))
> 2007-11-07 13:55:53.840 DEBUG Gold::Proxy::prepareOptions  Adding cooked 
> option: (Limit, 1000, )
> 2007-11-07 13:55:53.840 TRACE Gold::Proxy::prepareSelections  invoked 
> with arguments: (Gold::Request=HASH(0x6000000000a14280))
> 2007-11-07 13:55:53.841 TRACE Gold::Database::select  invoked with 
> arguments: (object => Attribute, options => ARRAY(0x6000000001ac1d80), 
> selections => ARRAY(0x6000000001ac1b00), chunkNum => 0, conditions => 
> ARRAY(0x600000000116fb00))
> 2007-11-07 13:55:53.841 TRACE Gold::Database::buildWhere  invoked with 
> arguments: (objects => ARRAY(0x6000000000847b70), conditions => 
> ARRAY(0x6000000000847cd0))
> 2007-11-07 13:55:53.842 DEBUG Gold::Database::select  SQL Query: SELECT 
> g_name FROM g_attribute WHERE ( g_object='Job' AND g_hidden!='True' ) 
> AND g_deleted!='True' ORDER BY g_sequence
> 2007-11-07 13:55:53.845 DEBUG Gold::Database::select  SQL Results: Id
> JobId
> User
> Project
> Machine
> Charge
> Queue
> Type
> Stage
> QualityOfService
> Nodes
> Processors
> Executable
> Application
> StartTime
> EndTime
> WallDuration
> QuoteId
> Description
> 2007-11-07 13:55:53.846 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (Id)
> 2007-11-07 13:55:53.846 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (JobId)
> 2007-11-07 13:55:53.846 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (User)
> 2007-11-07 13:55:53.847 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (Project)
> 2007-11-07 13:55:53.847 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (Machine)
> 2007-11-07 13:55:53.848 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (Charge)
> 2007-11-07 13:55:53.848 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (Queue)
> 2007-11-07 13:55:53.848 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (Type)
> 2007-11-07 13:55:53.849 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (Stage)
> 2007-11-07 13:55:53.849 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (QualityOfService)
> 2007-11-07 13:55:53.850 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (Nodes)
> 2007-11-07 13:55:53.850 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (Processors)
> 2007-11-07 13:55:53.850 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (Executable)
> 2007-11-07 13:55:53.851 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (Application)
> 2007-11-07 13:55:53.851 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (StartTime)
> 2007-11-07 13:55:53.851 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (EndTime)
> 2007-11-07 13:55:53.852 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (WallDuration)
> 2007-11-07 13:55:53.852 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (QuoteId)
> 2007-11-07 13:55:53.853 DEBUG Gold::Proxy::prepareSelections  Adding 
> cooked selection: (Description)
> 2007-11-07 13:55:53.853 TRACE Gold::Proxy::prepareAssignments  invoked 
> with arguments: (Gold::Request=HASH(0x6000000000a14280))
> 2007-11-07 13:55:53.853 TRACE Gold::Cache::listAttributes  invoked with 
> arguments: (Job)
> 2007-11-07 13:55:53.855 TRACE Gold::Proxy::prepareConditions  invoked 
> with arguments: (Gold::Request=HASH(0x6000000000a14280))
> 2007-11-07 13:55:53.855 TRACE Gold::Cache::attributeExists  invoked with 
> arguments: (Job, StartTime)
> 2007-11-07 13:55:53.856 TRACE Gold::Cache::getAttributeProperty  invoked 
> with arguments: (Job, StartTime, DataType)
> 2007-11-07 13:55:53.858 DEBUG Gold::Proxy::prepareConditions  Adding 
> cooked condition: (StartTime, 0, GE, , 0, , )
> 2007-11-07 13:55:53.859 TRACE Gold::Cache::attributeExists  invoked with 
> arguments: (Job, EndTime)
> 2007-11-07 13:55:53.860 TRACE Gold::Cache::getAttributeProperty  invoked 
> with arguments: (Job, EndTime, DataType)
> 2007-11-07 13:55:53.861 DEBUG Gold::Proxy::prepareConditions  Adding 
> cooked condition: (EndTime, 2147483647, LT, , 0, , )
> 2007-11-07 13:55:53.862 DEBUG Gold::Proxy::prepare  Cooked request: 
> (Query, gold, [(Job, , )], [(Id, , , ), (JobId, , , ), (User, , , ), 
> (Project, , , ), (Machine, , , ), (Charge, , , ), (Queue, , , ), (Type, 
> , , ), (Stage, , , ), (QualityOfService, , , ), (Nodes, , , ), 
> (Processors, , , ), (Executable, , , ), (Application, , , ), (StartTime, 
> , , ), (EndTime, , , ), (WallDuration, , , ), (QuoteId, , , ), 
> (Description, , , )], [], [(StartTime, 0, GE, , , , ), (EndTime, 
> 2147483647, LT, , , , )], [(Limit, 1000, )], [], 0, 0, 100000, 
> Gold::Database=HASH(0x60000000018006e0))
> 2007-11-07 13:55:53.863 TRACE Gold::Proxy::execute  invoked with 
> arguments: ()
> 2007-11-07 13:55:53.863 TRACE Gold::Bank::execute  invoked with 
> arguments: (Gold::Proxy=HASH(0x6000000001800800))
> 2007-11-07 13:55:53.864 DEBUG Gold::Bank::query  invoked with arguments: 
> (Gold::Request=HASH(0x6000000000a14280), 891)
> 2007-11-07 13:55:53.864 DEBUG Gold::Base::query  invoked with arguments: 
> (Gold::Request=HASH(0x6000000000a14280))
> 2007-11-07 13:55:53.865 TRACE Gold::Database::select  invoked with 
> arguments: (chunkSize => 100000, options => ARRAY(0x60000000010d9480), 
> selections => ARRAY(0x60000000010d93e0), objects => 
> ARRAY(0x60000000010d92a0), chunkNum => 1, conditions => 
> ARRAY(0x60000000010d9430))
> 2007-11-07 13:55:53.888 TRACE Gold::Database::buildWhere  invoked with 
> arguments: (objects => ARRAY(0x6000000000847b70), conditions => 
> ARRAY(0x6000000000847cd0))
> 2007-11-07 13:55:53.888 DEBUG Gold::Database::select  SQL Query: SELECT 
> g_id,g_job_id,g_user,g_project,g_machine,g_charge,g_queue,g_type,g_stage,g_quality_of_service,g_nodes,g_processors,g_executable,g_application,g_start_time,g_end_time,g_wall_duration,g_quote_id,g_description 
> FROM g_job WHERE ( g_start_time>='0' AND g_end_time<'2147483647' ) AND 
> g_deleted!='True' LIMIT 1000
> 2007-11-07 13:55:53.891 DEBUG Gold::Database::select  SQL Results:
> 2007-11-07 13:55:53.896 TRACE Gold::Response::new  invoked with 
> arguments: ()
> 2007-11-07 13:55:53.896 TRACE Gold::Response::success  invoked with 
> arguments: (0, ARRAY(0x60000000010d9540))
> 2007-11-07 13:55:53.907 TRACE Gold::Reply::new  invoked with arguments: 
> (connection => IO::Socket::INET=GLOB(0x60000000017f5b30))
> 2007-11-07 13:55:53.908 TRACE Gold::Reply::sendChunk  invoked with 
> arguments: (Gold::Chunk=HASH(0x6000000001ab9910))
> 2007-11-07 13:55:53.908 TRACE Gold::Reply::marshallChunk  invoked with 
> arguments: (Gold::Chunk=HASH(0x6000000001ab9910))
> 2007-11-07 13:55:53.909 DEBUG Gold::Reply::sendChunk  Writing reply 
> header (HTTP/1.1 200 OK
> Content-Type: text/xml; charset="utf-8"
> Transfer-Encoding: chunked
> ).
> 2007-11-07 13:55:53.910 INFO  Gold::Reply::sendChunk  Writing reply 
> payload (179, <?xml version="1.0" encoding="UTF-8"?>
> <Envelope><Body><Response 
> actor="gold"><Status><Value>Success</Value><Code>000</Code></Status><Count>0</Count></Response></Body></Envelope>
> ).
> 
> 
> I will try to run the query on the database and see if anything comes out.
> 
> 
> 
> _______________________________________________
> gold-users mailing list
> gold-users at supercluster.org
> http://www.supercluster.org/mailman/listinfo/gold-users



More information about the gold-users mailing list