[gold-users] Newbie question: GUI and list of jobs
Tolis Sisiaridis
tolis.sisiaridis at oerc.ox.ac.uk
Thu Nov 22 03:26:43 MST 2007
Thanks,
like you said it was the Start/Endtime fields that were causing this.
I modified the syntax of my 'charge' scripts to extract times from PBS pro,
convert them to GOLD format ( YYYY-MM-DD mm:hh) and use them.
Now the GUI prints everything just fine. It is also possible to query
past jobs
with blank time fields from the GUI if i leave the Prefilter text boxes
blank.
Tolis
Scott Jackson wrote:
> I apologize if you have already got past this and if I am answering old
> email that may be no longer relevant to you but I got way behind around
> supercomputing.
>
> Now that I see your log file, I recall that the GUI was written to base
> its query on StartTime and EndTime. At a prior time, Moab was not
> sending StartTime and EndTime of the job to Gold so these fields were
> remaining blank.
>
> (glsjob was not imposing any StartTime and EndTime criteria, so it
> listed all jobs, but the GUI asked for StartTime>-infinity and
> EndTime<infinity or some such permutation and this did not match any
> rows because StartTime and EndTime were NULL).
>
> If you want to use the gui, you will have to ensure either that your
> jobs are populating the StartTime or EndTime fields, or change the GUI
> to not impose the date condition when nothing is selected. The latter
> may be a bit hard to do because all of these fields are generated in a
> dynamic way, and it may be a bit elaborate to architect the exception.
> The supported solution would be the former, make sure that StartTime and
> EndTime are being properly populated in the Gold job records.
>
> If you are using Moab, then it is possible that you are using an older
> version, before we corrected Moab to populate the fields correctly. If
> you are using something else, let me know what it is, and I can
> recommend the syntax modification needed in the Job Charge request.
>
> You may also want to go back and populate the jobs that have already
> been charged. This CAN be determined in most cases by examining moab
> event files, or by extracting the CreationTime field for the Reserve and
> Charge transactions in the Transaction log. There are also other
> elaborate means at your disposal, though all of them will require some
> scripting work to parse out the times and update the Job table.
>
> Thanks,
>
> Scott
>
>
>
> On Wed, 2007-11-07 at 14:13 +0000, Tolis Sisiaridis wrote:
>
>> Tim Robinson wrote:
>>
>>> Hi Tolis
>>> What do the logs say? You should see the SQL query that is being
>>> performed...
>>>
>>> Tim
>>>
>>> On Wed, 2007-11-07 at 11:54 +0000, Tolis Sisiaridis wrote:
>>>
>>>
>>>> Just installed GOLD 2.4.1.2 and i am testing integration with PBS PRO.
>>>> Have tried both SQLITE and POSTGRESQL and so far i am very pleased with the results. A small problem though: The GUI would not list jobs no matter what i use for 'Prefilter' options. Everything else on the GUI
>>>> seems to be working well and 'glsjob' lists jobs normally.
>>>> Anyone else with the same issue?
>>>>
>>>> Tolis
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> gold-users mailing list
>>>> gold-users at supercluster.org
>>>> http://www.supercluster.org/mailman/listinfo/gold-users
>>>>
>>>>
>> When i run a query for all jobs (- infinity, infinity) the daemon log
>> ($GOLD_HOME/log/goldd.log) shows:
>>
>> 2007-11-07 13:55:53.761 INFO main:: New Connection Received
>> 2007-11-07 13:55:53.763 TRACE Gold::Message::new invoked with
>> arguments: (connection => IO::Socket::INET=GLOB(0x60000000017f5b30))
>> 2007-11-07 13:55:53.764 TRACE Gold::Message::receiveChunk invoked with
>> arguments: ()
>> 2007-11-07 13:55:53.765 DEBUG Gold::Message::receiveChunk Read message
>> header (POST /SSSRMAP3 HTTP/1.1
>> Content-Type: text/xml; charset="utf-8"
>> Transfer-Encoding: chunked
>>
>> ).
>> 2007-11-07 13:55:53.765 INFO Gold::Message::receiveChunk Read message
>> payload (462, <?xml version="1.0" encoding="UTF-8"?>
>> <Envelope><Body><Request action="Query"
>> actor="gold"><Object>Job</Object><Where name="StartTime"
>> op="GE">0</Where><Where name="EndTime" op="LT">2147483647</Where><Option
>> name="Limit">1000</Option></Request></Body><Signature><DigestValue>73tVCxqOqMyFVY0hvtFTp25FwBs=</DigestValue><SignatureValue>zTZ4b/zxkAgM1rtHRUznUEkB+0Y=</SignatureValue><SecurityToken
>> type="Password" name="gold"></SecurityToken></Signature></Envelope>
>> ).
>> 2007-11-07 13:55:53.766 TRACE Gold::Message::unmarshallChunk invoked
>> with arguments: (<?xml version="1.0" encoding="UTF-8"?>
>> <Envelope><Body><Request action="Query"
>> actor="gold"><Object>Job</Object><Where name="StartTime"
>> op="GE">0</Where><Where name="EndTime" op="LT">2147483647</Where><Option
>> name="Limit">1000</Option></Request></Body><Signature><DigestValue>73tVCxqOqMyFVY0hvtFTp25FwBs=</DigestValue><SignatureValue>zTZ4b/zxkAgM1rtHRUznUEkB+0Y=</SignatureValue><SecurityToken
>> type="Password" name="gold"></SecurityToken></Signature></Envelope>
>> )
>> 2007-11-07 13:55:53.768 TRACE Gold::Chunk::authenticate invoked with
>> arguments: (XML::LibXML::Document=SCALAR(0x600000000051f6a0))
>> 2007-11-07 13:55:53.769 TRACE Gold::Cache::getPasswordProperty invoked
>> with arguments: (gold, Password)
>> 2007-11-07 13:55:53.774 DEBUG Gold::Chunk::authenticate The security
>> token type is (Password).
>> 2007-11-07 13:55:53.775 DEBUG Gold::Chunk::authenticate The
>> canonicalized body text is (<Body><Request action="Query"
>> actor="gold"><Object>Job</Object><Where name="StartTime"
>> op="GE">0</Where><Where name="EndTime" op="LT">2147483647</Where><Option
>> name="Limit">1000</Option></Request></Body>).
>> 2007-11-07 13:55:53.775 DEBUG Gold::Chunk::authenticate The
>> base64-encoded message digest is (73tVCxqOqMyFVY0hvtFTp25FwBs=).
>> 2007-11-07 13:55:53.776 DEBUG Gold::Chunk::authenticate The
>> base64-encoded MAC is (zTZ4b/zxkAgM1rtHRUznUEkB+0Y=).
>> 2007-11-07 13:55:53.787 TRACE Gold::Request::new invoked with arguments: ()
>> 2007-11-07 13:55:53.789 DEBUG Gold::Message::unmarshallChunk Extracted
>> the request ((Query, gold, [(Job, , )], [], [], [(StartTime, 0, GE, , ,
>> , ), (EndTime, 2147483647, LT, , , , )], [(Limit, 1000, )], [], 0, 0,
>> 100000, )).
>> 2007-11-07 13:55:53.790 DEBUG Gold::Message::receiveChunk Read message
>> tail (0).
>> 2007-11-07 13:55:53.790 TRACE Gold::Proxy::new invoked with arguments:
>> (request => Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.791 TRACE Gold::Database::new invoked with
>> arguments: ()
>> 2007-11-07 13:55:53.804 TRACE Gold::Database::nextId invoked with
>> arguments: (Request)
>> 2007-11-07 13:55:53.808 TRACE Gold::Database::nextId generated new id
>> 891 for Request
>> 2007-11-07 13:55:53.809 TRACE Gold::Proxy::prepare invoked with
>> arguments: (Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.833 TRACE Gold::Cache::objectExists invoked with
>> arguments: (Job)
>> 2007-11-07 13:55:53.834 TRACE Gold::Cache::actionExists invoked with
>> arguments: (Job, Query)
>> 2007-11-07 13:55:53.835 TRACE Gold::Proxy::authorize invoked with
>> arguments: (Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.836 TRACE Gold::Cache::listUserRoles invoked with
>> arguments: (gold)
>> 2007-11-07 13:55:53.837 TRACE Gold::Cache::listUserRoles invoked with
>> arguments: (ANY)
>> 2007-11-07 13:55:53.838 TRACE Gold::Cache::getObjectProperty invoked
>> with arguments: (Job, Association)
>> 2007-11-07 13:55:53.839 TRACE Gold::Cache::listRoleActionInstances
>> invoked with arguments: (SystemAdmin, Job, Query, ANY)
>> 2007-11-07 13:55:53.839 TRACE Gold::Proxy::prepareOptions invoked with
>> arguments: (Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.840 DEBUG Gold::Proxy::prepareOptions Adding cooked
>> option: (Limit, 1000, )
>> 2007-11-07 13:55:53.840 TRACE Gold::Proxy::prepareSelections invoked
>> with arguments: (Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.841 TRACE Gold::Database::select invoked with
>> arguments: (object => Attribute, options => ARRAY(0x6000000001ac1d80),
>> selections => ARRAY(0x6000000001ac1b00), chunkNum => 0, conditions =>
>> ARRAY(0x600000000116fb00))
>> 2007-11-07 13:55:53.841 TRACE Gold::Database::buildWhere invoked with
>> arguments: (objects => ARRAY(0x6000000000847b70), conditions =>
>> ARRAY(0x6000000000847cd0))
>> 2007-11-07 13:55:53.842 DEBUG Gold::Database::select SQL Query: SELECT
>> g_name FROM g_attribute WHERE ( g_object='Job' AND g_hidden!='True' )
>> AND g_deleted!='True' ORDER BY g_sequence
>> 2007-11-07 13:55:53.845 DEBUG Gold::Database::select SQL Results: Id
>> JobId
>> User
>> Project
>> Machine
>> Charge
>> Queue
>> Type
>> Stage
>> QualityOfService
>> Nodes
>> Processors
>> Executable
>> Application
>> StartTime
>> EndTime
>> WallDuration
>> QuoteId
>> Description
>> 2007-11-07 13:55:53.846 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (Id)
>> 2007-11-07 13:55:53.846 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (JobId)
>> 2007-11-07 13:55:53.846 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (User)
>> 2007-11-07 13:55:53.847 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (Project)
>> 2007-11-07 13:55:53.847 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (Machine)
>> 2007-11-07 13:55:53.848 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (Charge)
>> 2007-11-07 13:55:53.848 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (Queue)
>> 2007-11-07 13:55:53.848 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (Type)
>> 2007-11-07 13:55:53.849 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (Stage)
>> 2007-11-07 13:55:53.849 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (QualityOfService)
>> 2007-11-07 13:55:53.850 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (Nodes)
>> 2007-11-07 13:55:53.850 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (Processors)
>> 2007-11-07 13:55:53.850 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (Executable)
>> 2007-11-07 13:55:53.851 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (Application)
>> 2007-11-07 13:55:53.851 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (StartTime)
>> 2007-11-07 13:55:53.851 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (EndTime)
>> 2007-11-07 13:55:53.852 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (WallDuration)
>> 2007-11-07 13:55:53.852 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (QuoteId)
>> 2007-11-07 13:55:53.853 DEBUG Gold::Proxy::prepareSelections Adding
>> cooked selection: (Description)
>> 2007-11-07 13:55:53.853 TRACE Gold::Proxy::prepareAssignments invoked
>> with arguments: (Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.853 TRACE Gold::Cache::listAttributes invoked with
>> arguments: (Job)
>> 2007-11-07 13:55:53.855 TRACE Gold::Proxy::prepareConditions invoked
>> with arguments: (Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.855 TRACE Gold::Cache::attributeExists invoked with
>> arguments: (Job, StartTime)
>> 2007-11-07 13:55:53.856 TRACE Gold::Cache::getAttributeProperty invoked
>> with arguments: (Job, StartTime, DataType)
>> 2007-11-07 13:55:53.858 DEBUG Gold::Proxy::prepareConditions Adding
>> cooked condition: (StartTime, 0, GE, , 0, , )
>> 2007-11-07 13:55:53.859 TRACE Gold::Cache::attributeExists invoked with
>> arguments: (Job, EndTime)
>> 2007-11-07 13:55:53.860 TRACE Gold::Cache::getAttributeProperty invoked
>> with arguments: (Job, EndTime, DataType)
>> 2007-11-07 13:55:53.861 DEBUG Gold::Proxy::prepareConditions Adding
>> cooked condition: (EndTime, 2147483647, LT, , 0, , )
>> 2007-11-07 13:55:53.862 DEBUG Gold::Proxy::prepare Cooked request:
>> (Query, gold, [(Job, , )], [(Id, , , ), (JobId, , , ), (User, , , ),
>> (Project, , , ), (Machine, , , ), (Charge, , , ), (Queue, , , ), (Type,
>> , , ), (Stage, , , ), (QualityOfService, , , ), (Nodes, , , ),
>> (Processors, , , ), (Executable, , , ), (Application, , , ), (StartTime,
>> , , ), (EndTime, , , ), (WallDuration, , , ), (QuoteId, , , ),
>> (Description, , , )], [], [(StartTime, 0, GE, , , , ), (EndTime,
>> 2147483647, LT, , , , )], [(Limit, 1000, )], [], 0, 0, 100000,
>> Gold::Database=HASH(0x60000000018006e0))
>> 2007-11-07 13:55:53.863 TRACE Gold::Proxy::execute invoked with
>> arguments: ()
>> 2007-11-07 13:55:53.863 TRACE Gold::Bank::execute invoked with
>> arguments: (Gold::Proxy=HASH(0x6000000001800800))
>> 2007-11-07 13:55:53.864 DEBUG Gold::Bank::query invoked with arguments:
>> (Gold::Request=HASH(0x6000000000a14280), 891)
>> 2007-11-07 13:55:53.864 DEBUG Gold::Base::query invoked with arguments:
>> (Gold::Request=HASH(0x6000000000a14280))
>> 2007-11-07 13:55:53.865 TRACE Gold::Database::select invoked with
>> arguments: (chunkSize => 100000, options => ARRAY(0x60000000010d9480),
>> selections => ARRAY(0x60000000010d93e0), objects =>
>> ARRAY(0x60000000010d92a0), chunkNum => 1, conditions =>
>> ARRAY(0x60000000010d9430))
>> 2007-11-07 13:55:53.888 TRACE Gold::Database::buildWhere invoked with
>> arguments: (objects => ARRAY(0x6000000000847b70), conditions =>
>> ARRAY(0x6000000000847cd0))
>> 2007-11-07 13:55:53.888 DEBUG Gold::Database::select SQL Query: SELECT
>> g_id,g_job_id,g_user,g_project,g_machine,g_charge,g_queue,g_type,g_stage,g_quality_of_service,g_nodes,g_processors,g_executable,g_application,g_start_time,g_end_time,g_wall_duration,g_quote_id,g_description
>> FROM g_job WHERE ( g_start_time>='0' AND g_end_time<'2147483647' ) AND
>> g_deleted!='True' LIMIT 1000
>> 2007-11-07 13:55:53.891 DEBUG Gold::Database::select SQL Results:
>> 2007-11-07 13:55:53.896 TRACE Gold::Response::new invoked with
>> arguments: ()
>> 2007-11-07 13:55:53.896 TRACE Gold::Response::success invoked with
>> arguments: (0, ARRAY(0x60000000010d9540))
>> 2007-11-07 13:55:53.907 TRACE Gold::Reply::new invoked with arguments:
>> (connection => IO::Socket::INET=GLOB(0x60000000017f5b30))
>> 2007-11-07 13:55:53.908 TRACE Gold::Reply::sendChunk invoked with
>> arguments: (Gold::Chunk=HASH(0x6000000001ab9910))
>> 2007-11-07 13:55:53.908 TRACE Gold::Reply::marshallChunk invoked with
>> arguments: (Gold::Chunk=HASH(0x6000000001ab9910))
>> 2007-11-07 13:55:53.909 DEBUG Gold::Reply::sendChunk Writing reply
>> header (HTTP/1.1 200 OK
>> Content-Type: text/xml; charset="utf-8"
>> Transfer-Encoding: chunked
>> ).
>> 2007-11-07 13:55:53.910 INFO Gold::Reply::sendChunk Writing reply
>> payload (179, <?xml version="1.0" encoding="UTF-8"?>
>> <Envelope><Body><Response
>> actor="gold"><Status><Value>Success</Value><Code>000</Code></Status><Count>0</Count></Response></Body></Envelope>
>> ).
>>
>>
>> I will try to run the query on the database and see if anything comes out.
>>
>>
>>
>> _______________________________________________
>> gold-users mailing list
>> gold-users at supercluster.org
>> http://www.supercluster.org/mailman/listinfo/gold-users
>>
>
>
More information about the gold-users
mailing list