[torqueusers] Resources_List.ncpus & resources_assigned.ncpus meaning

Garrick Staples garrick at usc.edu
Thu Sep 1 20:11:17 MDT 2005


On Mon, Aug 29, 2005 at 12:31:23AM +0200, Emir Imamagic alleged:
> Hi,
> 
> I'm using Maui 3.2.6p13 and torque_1.2.0p4 on dual processor cluster. I've 
> tried to find exact meaning of ncpus field but didn't find consistent 
> definiton.
> 
> Does it mean:
> - total amount of cpus allocated to a job, or
> - amount of cpus allocated to a job on a single node?

I went searching the code and found it closer to the later.  First, it
means absolutely nothing to TORQUE; it is just a value that is passed
along to the scheduler.  Second, in maui it defines the size of a
"task".

A "task" in maui is smallest indivisible set of resources that are
assigned to jobs.  So asking for "ncpus=4" means you want 1 chunk of 4
processors.  Since tasks are indivisible, it means that all cpus must be
on the same node.


> Furthermore, test in practice shown that none of these definition are 
> correct. I adjusted nodes $PBS_HOME/server_priv/nodes so the nodes have 4 
> cpus, and submitted two jobs:
> - first demanding nodes=1:ppn=3, and
> - second demanding nodes=1:ppn=2.
> 
> Command qstat -f reported for both of them:
> ...
> Resource_List.ncpus = 1
> Resource_List.nodect = 1

I think ncpus is being inherited from your queue config in this case?

Keep in mind that Resource_List is initially populated by pbs_server by
combining the job request and the queue config.  Then maui may or may
not adjust the individual values.

It is probably more useful to look at 'checkjob'.  For example,
requesting 'ncpus=2' gives me:

Dedicated Resources Per Task: PROCS: 2
NodeCount: 1


> Command qstat -fQ reported that 2 cpus are occupied by the jobs in that 
> queue:
> ...
> resources_assigned.ncpus = 2
> resources_assigned.nodect = 2
> ...

That's for the queue, not for individual jobs.  And again, I think you
are only getting seeing ncpus because you have
resources_{assigned,max,min}.ncpus in the queue.



> I also tried setting ncpus explicity and the the values reported 
> correctly, but I only got one cpu allocated (in PBS_NODEFILE). 
> Furthermore, I found in previous mails, that this is not a preffered way 
> of allocating nodes for clusters and that one should use -l nodes=*:ppn=* 
> instead.

Correct, any value for ncpus > 1 gives you 1 "thing".


> Is this normal behaviour?
> 
> If yes, is there some way to get number of cpus allocated to the jobs in 
> a queue (beside from calling qstat -f and parsing nodes=*:ppn=* and 
> queue=* for each job)?

Not that I can find.

-- 
Garrick Staples, Linux/HPCC Administrator
University of Southern California
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://www.supercluster.org/pipermail/torqueusers/attachments/20050901/88cc0baf/attachment.bin


More information about the torqueusers mailing list