[torqueusers] Interacting with Torque/Maui from Perl scripts

Anthony Ferrari ferraria at gmail.com
Wed Aug 17 02:33:44 MDT 2011


Hi all,

Basically, I would like to launch a job on a cluster with specific resources
and dependencies, wait for the job to complete or fail, check that all has
gone OK, then do something with the generated files ...

To launch jobs, I am using a customized version of PBS-Client-0.09 package
written by Ka-Wai Mak.
I also saw that packages named PBS-0.03 by Todd Merritt and perl-PBS by
Garrick Staples exist but they do not seem maintained or seem outdated
(please correct me if I am wrong).
What do you personally use ?

My problem here is more after the job submission, to keep 'control' of the
job.
Because with the PBS-Client module, once the job is submitted, Torque does
what it has to do and that's it. It does not give any information in return
to my Perl script.

I was thinking about several solutions to achieve this but I do not find
them very smart ... like :
 - using qstat | grep '^myjobid' in a while loop (with open3 perl function)
 - waiting for the output & error files to be written where I expect them
(hum...)
 - use an epilogue script in the job to do something useful...

But there may exist brighter answers to this question. Perhaps with some
modules I don't know about.
Do you have any advice ?


And actually this is to know when the job has finished but how do you check
that the job completed and did not crash ?
Do you always parse the created logs ?
Is there a way to propagate the exit code of the program launched with qsub
to the perl script running on the master node ?


Thank you very much.

Cheers,
Anthony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.supercluster.org/pipermail/torqueusers/attachments/20110817/aedae3e5/attachment.html 


More information about the torqueusers mailing list