[gold-users] gcharge issued twice via Torque's epilogue script

Kevin Van Workum vanw at sabalcore.com
Tue Dec 22 12:53:31 MST 2009


On Tue, Dec 22, 2009 at 2:51 PM, Kevin Van Workum <vanw at sabalcore.com> wrote:
> On Tue, Dec 22, 2009 at 12:46 PM, Wojciech Turek <wjt27 at cam.ac.uk> wrote:
>>
>> What about glstxn -J <job id> ? You could use this command in your epilogue script to check if charge transaction was made for the particular jobid.
>
> Yes, I tried that, but surprisingly that doesn't always work. It
> appears that gcharge is implemented asynchronously, at least wrt
> glstxn. Here's a simplified snippet of my epilogue script (perl) and
> glstxn output for a job that still got charged twice.
>
> #!/usr/bin/perl
>
> open LG, "glstxn -J $jobid|";

Before you mention it, I'm actually using "glstxn --quite -J $jobid|" here.

> @buf = <LG>;
> close LG;
>
> if(@buf == 0) {
>    system("gcharge $args");
> } else {
>    print STDERR "$jobid has already been charged ", @buf+0, " times\n";
> }
>
>
> # glstxn -J 230465.jman --show JobId,Id,CreationTime
> JobId       Id     CreationTime
> ----------- ------ -------------------
> 230465.jman 847102 2009-12-22 14:35:32
> 230465.jman 847107 2009-12-22 14:35:32
>
>
>>
>> Cheers
>>
>> Wojciech
>>
>> 2009/12/22 Scott Jackson <scottmo at adaptivecomputing.com>
>>>
>>> Kevin,
>>>
>>> No, I'm sorry. There is not. Gold will charge for a job as many times as
>>> it is called. There are provisions for incremental charging where it all
>>> goes against the same job instance, and if not, it considers them
>>> separate jobs with the same jobid. All I can think of is that you could
>>> write a wrapper script that looks up the jobid and if it has already
>>> been charged that same day, ignores the second charge.
>>>
>>> I assume you have a ticket open with the Torque support queue on this.
>>>
>>> Scott
>>>
>>>
>>> Kevin Van Workum wrote:
>>> > I use Torque's epilogue script to issue the gcharge command after a
>>> > job completes. However, it occasionally happens that the epilogue
>>> > script runs twice for a given job. This happens when Torque sends a
>>> > sigkill a few seconds after the initial sigterm is sent. Though I'd
>>> > like to prevent the script from running twice, I haven't had much
>>> > success. So, I'm now searching for a solution though gold.
>>> >
>>> > Is there a way to have gold ignore duplicate charges for the same JobId?
>>> >
>>> > --
>>> > Kevin Van Workum, PhD
>>> > Sabalcore Computing Inc.
>>> > Run your code on 500 processors.
>>> > Sign up for a free trial account.
>>> > www.sabalcore.com <http://www.sabalcore.com>
>>> > 877-492-8027 ext. 11
>>> > ------------------------------------------------------------------------
>>> >
>>> > _______________________________________________
>>> > gold-users mailing list
>>> > gold-users at supercluster.org
>>> > http://www.supercluster.org/mailman/listinfo/gold-users
>>> >
>>>
>>> _______________________________________________
>>> gold-users mailing list
>>> gold-users at supercluster.org
>>> http://www.supercluster.org/mailman/listinfo/gold-users
>>
>>
>>
>> --
>> --
>> Wojciech Turek
>>
>> Assistant System Manager
>>
>> High Performance Computing Service
>> University of Cambridge
>> Email: wjt27 at cam.ac.uk
>> Tel: (+)44 1223 763517
>
>
>
> --
> Kevin Van Workum, PhD
> Sabalcore Computing Inc.
> Run your code on 500 processors.
> Sign up for a free trial account.
> www.sabalcore.com
> 877-492-8027 ext. 11
>



-- 
Kevin Van Workum, PhD
Sabalcore Computing Inc.
Run your code on 500 processors.
Sign up for a free trial account.
www.sabalcore.com
877-492-8027 ext. 11


More information about the gold-users mailing list