[torqueusers] LAM-MPI won't boot with torque-1.2.0p6

Fabio Carvalho fabio.carv at gmail.com
Thu Sep 15 11:11:57 MDT 2005

You can use mpiexec to start your tasks and use the -pernode option. 

Clifton Kirby wrote: 

The user is running parallel code but his single process requires more
memory than two processes can address per node. (Make sense?)  I do not want
to make them exclusive permanently just for the duration of the job.

I want to keep the node definition in the $PBS_HOME/server_priv/nodes file
as follows,
node01 np=2
node02 np=2
node03 np=2
node04 np=2
node05 np=2
node06 np=2

When the user submits a job as "qsub -I -l nodes=3:ppn=1" his job will run
as expected with 3 nodes allocated using 1 process per node and the job runs
fine.  When he submits the same job again when the first job is still
running PBS will see that the node still has one processor left and will
dump the job on the same set of nodes and there will not be enough memory
for both jobs to run.  I would like to see the second job go onto a whole
set of new nodes.

The qstat -n command will show,

Job ID          Username Queue    Jobname    SessID NDS TSK Memory Time  S
--------------- -------- -------- ---------- ------ --- --- ------ ----- - -
28451.mach5c.ma <http://28451.mach5c.ma> joe	  long     job1         
--  3  --    --  10000 R 00:10

28452.mach5c.ma <http://28452.mach5c.ma> joe	  long     job2         
--  3  --    --  10000 R 00:03

- Cliff

-----Original Message-----
From: Piotr Siwczak [mailto:psiwczak at man.poznan.pl]
Sent: Thursday, September 15, 2005 8:29 AM
To: Clifton Kirby
Cc: Torqueusers (E-mail)
Subject: Re: [torqueusers] Exclusive Nodes

You could probably define the node as np=1 in pbs_server's config file
called "nodes".

This would allow only one job to run at a time. Anyway I don't know if I
get it right: do You want to assign jobs exclusively only to a given job,
or to make them "exclusive" permanently (only one job at a time regardles
of the user running it).


  Piotr Siwczak <psiwczak at man.poznan.pl>
  System Administrator

  Poznan Supercomputing and Networking Center
  Supercomputing Department

  (www.eu-egee.org <http://www.eu-egee.org> <piotr.siwczak at cern.ch>)

On Thu, 15 Sep 2005, Clifton Kirby wrote:

On page 4 of the PBS administrators guide there is mention of "Exclusive
Nodes".  How would you define them in the server nodes file?
I have a situation where my user only wants one CPU from a dual CPU system
but wants exclusive access to the system to avoid a second job running on
the system since one CPU is still available from PBS.  The PBS directive


"nodes=2:ppn=1" and he does not want two processes on the machine as in
"nodes=2:ppn=2".  Is there a mechanism for qsub or a PBS directive to


for what he is doing or can I simply define the nodes as "Exclusive"?


torqueusers mailing list
torqueusers at supercluster.org

No virus found in this incoming message.
Checked by AVG Anti-Virus.
Version: 7.0.344 / Virus Database: 267.11.0/103 - Release Date: 9/15/2005

torqueusers mailing list
torqueusers at supercluster.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.supercluster.org/pipermail/torqueusers/attachments/20050915/5319587f/attachment.html

More information about the torqueusers mailing list