[torqueusers] specifying nodes for MPI jobs on small cluster

Glen Beane glen.beane at gmail.com
Thu Feb 7 09:27:05 MST 2013


On Thu, Feb 7, 2013 at 4:10 AM, Andrew Dawson <dawson at atm.ox.ac.uk> wrote:
> Hi all,
>
> I'm configuring a recent torque/maui installation and I'm having trouble
> with submitting MPI jobs. I would like for MPI jobs to specify the number of
> processors they require and have those come from any available physical
> machine, the users shouldn't need to specify processors per node etc.
>
> The torque manual says that the nodes option is mapped to virtual
> processors, so for example:
>
>     #PBS -l nodes=8
>
> should request 8 virtual processors. The problem I'm having is that our
> cluster currently has only 5 physical machines (nodes), and setting nodes to
> anything greater than 5 gives the error:
>
>     qsub: Job exceeds queue resource limits MSG=cannot locate feasible nodes
> (nodes file is empty or all systems are busy)
>
> I'm confused by this, we have 33 virtual processors available across the 5
> nodes (4 8-core machines and one single core) so my interpretation of the
> manual is that I should be able to request 8 nodes, since these should be
> understood as virtual processors? Am I doing something wrong?
>
> I tried setting
>
> #PBS -l procs=8
>
> but that doesn't seem to do anything, MPI stops due to having only 1 worker
> available (single core allocated to the job).

I don't think Maui correctly interprets procs.  You would need Moab.


More information about the torqueusers mailing list