[torqueusers] Running multiple jobs and claiming memory

Axel Kohlmeyer akohlmey at cmm.chem.upenn.edu
Tue Sep 7 09:16:21 MDT 2010


On Mon, Sep 6, 2010 at 1:23 PM, Bryan Herman <hermab53 at gmail.com> wrote:
> Dear torque users,

dear bryan,

>    I would like to submit 30 jobs, one right after another. Each job takes
> up 10GB of RAM.  I would like to use torque to run each of these jobs and
> when each job is executed, claim 10 GB of memory.  Since my nodes only have
> 16 GB of memory, I would only expect one file to be running on each node.  I
> tried using the mem keyword, but it allows all of the jobs to be sent of the
> same node.  Please let me know how to configure the job script for this
> effort.

the easiest way is probably to use

-l nodes=1:ppn=#

with # being the number of processors per node that are configured.
this way you'd reserve a full node for your job and no other job can
enter. please also keep in mind that if your job requires 10GB resident
memory, it will typically also require some extra storage for i/o caching
and other things to run smoothly, and that the OS will claim some of the
16GB physical for the rest of the running processes, so squeezing in
additional small jobs might actually have a significant negative effect
on the performance, particularly if those are I/O heavy jobs. so reserving
a full node seems a good choice to me.

the second, option would be to take note of the job id and
then submit the second job with
-W depend=afterok:<jobid>
that will start the second job only after the first has completed
successfully and so on for other jobs.

HTH,
   axel.

>
> Thanks,
>
> Bryan
>
> _______________________________________________
> torqueusers mailing list
> torqueusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/torqueusers
>
>



-- 
Dr. Axel Kohlmeyer    akohlmey at gmail.com
http://sites.google.com/site/akohlmey/

Institute for Computational Molecular Science
Temple University, Philadelphia PA, USA.


More information about the torqueusers mailing list