[torqueusers] Re: [torquedev] First Torque impressions on Altix

Michel Béland michel.beland at rqchp.qc.ca
Wed Jan 7 15:29:09 MST 2009

Last night, I wrote:

> Some testing showed that by specifying -lnodes=1:ppn=n together with
> -lncpus=n, I would get a correct cpuset and a correct qstat output. This
> works with pbs_sched, but Maui thinks that I need n*n cpus!

For some reason, it started to work this morning. The only thing I did
was restarting the server, as far as I can tell. Maui does not multiply
the processor count of -lnodes=1:ppn=n and -lncpus=n...

> Another problem when one does not use -lncpus is that queue limits on
> ncpus are not obeyed anymore. How can I specify queue limits so that a
> job with -lnodes=1:ppn=n cannot run in this queue if n is larger than 32
> processors? I tried with nodect, but it does not work better.

By using both -lncpus and -lnodes, I can now use resources_max.ncpus.

One another problem with Torque on Altix is that it ignores boot
cpusets. It tries to run jobs in them, which fails. To make Torque work
with boot cpusets, we changed the mem_exclusive and cpu_exclusive
options of the boot cpuset to 0 and submitted a dummy job having the
same size as the boot cpuset but sleeping for 10000 hours. Other jobs
will then run elsewhere.

Michel Béland, analyste en calcul scientifique
michel.beland at rqchp.qc.ca
bureau S-250, pavillon Roger-Gaudry (principal), Université de Montréal
téléphone   : 514 343-6111 poste 3892     télécopieur : 514 343-2155
RQCHP (Réseau québécois de calcul de haute performance)  www.rqchp.qc.ca

More information about the torquedev mailing list