[torquedev] Option to make qsub block until job completes ?

Gareth.Williams at csiro.au Gareth.Williams at csiro.au
Wed Aug 31 01:32:58 MDT 2011


> Hi folks,
> 
> I've been asked by some folks locally who are developing some
> pipelines if there is an option to make qsub block until the
> (standard non-interactive batch) job submitted completes.
> 
> It sounds like a reasonable idea and they believe this is
> already in PBSPro.
> 
> They've also asked if the exit status of the qsub in that
> mode can be the exit status of the last program (as logged
> as Exit_status in the Torque logs).  I can see that part
> causing issues given that qsub already has its own exit
> statuses..
> 
> Thoughts ?
> 
> All the best,
> Chris

I've thought a few times about writing a script to do something like:
qwait $jobid
which could form part of a simple wrapper to do what is being requested.
I envisage polling until the job is finished (handling qstat errors nicely) and grabbing the complete job info from tracejob.  It is a bit daunting to try and make this really robust.

Probably others have already written such a thing and not worried too much about handling all the corner cases.  They *might* be willing to share...

It would indeed be a useful in building workflows, though care should be taken to not overly burden the pbs_server so I'd be reluctant to widely recommend such a solution without say an extra service in between to protect the pbs_server.  Then it just gets more complicated. 

Please can others contribute their thoughts too?

Regards,

Gareth

> - --
>     Christopher Samuel - Senior Systems Administrator
>  VLSCI - Victorian Life Sciences Computation Initiative
>  Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545
>          http://www.vlsci.unimelb.edu.au/
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> 
> iEYEARECAAYFAk5dlAsACgkQO2KABBYQAh/jAACcDuTYVxikBFsLC3HvEc/CfLZ4
> S90An0L14AUeDUTaAQ1sp3ZIgTkioZGJ
> =ObSY
> -----END PGP SIGNATURE-----
> _______________________________________________
> torquedev mailing list
> torquedev at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torquedev


More information about the torquedev mailing list