[torqueusers] specifying nodes for MPI jobs on small cluster

Michel Béland michel.beland at calculquebec.ca
Thu Feb 7 14:17:11 MST 2013


Andrew Dawson wrote:

> From the torque documentation, it seems to say I can do:
>
> #PBS -l nodes=8
>
> and this will be interpreted as 8 CPUs rather than 8 physical nodes. 
> This is what I want. Unfortunately I get the error message at 
> submission time saying there are not enough resources to fulfill this 
> request, even though there are 33 CPUs in the system. If on my system I do
>
> #PBS -l nodes=5
>
> then my MPI job gets sent to 5 CPUs, not necessarily on the same 
> physical node, which is great and exactly what I want. I would 
> therefore expect this to work for larger numbers but it seems that at 
> submission time the request is checked against the number of physical 
> nodes rather than virtual processors, meaning I cannot do this! It is 
> quite frustrating.
>
> Please ask if there is further clarification I can make.

According to

http://docs.adaptivecomputing.com/torque/Content/topics/11-troubleshooting/faq.htm#qsubNotAllow

you have to set resources_available.nodect for the server and the queue 
in qmgr.

I think that you also have to be careful with Maui/Moab configuration. 
You should make sure that you have

JOBNODEMATCHPOLICY EXACTPROC


in maui.cfg or moab.cfg.



-- 
Michel Béland, analyste en calcul scientifique
michel.beland at calculquebec.ca
bureau S-250, pavillon Roger-Gaudry (principal), Université de Montréal
téléphone : 514 343-6111 poste 3892     télécopieur : 514 343-2155
Calcul Québec (www.calculquebec.ca)
Calcul Canada (calculcanada.ca)



More information about the torqueusers mailing list