[torqueusers] [Mauiusers] Assigning Nodes to specific nodes
brianm at usc.edu
Thu Oct 13 09:34:39 MDT 2011
On Thu, 2011-10-13 at 11:08 +0000, Jonathan Smale wrote:
> set queue firstgen resources_default.neednodes = 1stgennodes
> set nodes compute-0-0 properties = 1stgennodes
Not sure if you were just trying to be brief or if it was simply a typo,
but here you have said you set the node compute-0-0's properties to
'1stgennodes', and later you have it as 'firstgennodes'. If there is a
difference, then that could be the problem.
> I've found a fair few previous emails about this and have followed their solution without sucess. I can submit the jobs to the queues but they remain in a queued state, qstat -f gives:-
> [root at che-hydra /]# qstat -f 48691
> Job Id: 48691.che-hydra.bham.ac.uk
> Job_Name = allenega_p2000_g300_r2.cmd
> Job_Owner = jsmale at che-hydra.bham.ac.uk
> resources_used.cput = 529:06:32
> resources_used.mem = 16872kb
> resources_used.vmem = 245908kb
> resources_used.walltime = 529:11:31
> job_state = R
I do believe that 'job_state = R' means it is running from torque's
perspective... Someone correct me if I'm wrong, please.
> queue = default
Please note the queue it is in, default, *not* firstgen, this is due to
the definition of the default queue, below I'll comment more ...
> checking job 48691
> State: Running
> Creds: user:jsmale group:worth class:default qos:DEFAULT
According to Maui, the job is running as well, in class 'default'
> [root at che-hydra]# qmgr -c "p s"
> # Create queues and set their attributes.
> # Create and define queue default
> create queue default
> set queue default queue_type = Execution
> set queue default Priority = 100
> set queue default resources_default.nodes = 1
> set queue default enabled = True
> set queue default started = True
So, I think that here is the root of your problem. You have configured
the first queue that a job will be set to (in the server config below)
is the 'default' queue, but this queue is an execution queue which means
that jobs are going to 'run' in this queue. What I believe you want this
queue to be is a routing queue, with route_destinations of the queues
you want your jobs to run in. For example, my default queue
configuration is thus:
create queue default
set queue default queue_type = Route
set queue default max_queuable = 2000
set queue default resources_default.nodes = 1
set queue default resources_default.walltime = 00:30:00
set queue default route_destinations = quick
set queue default route_destinations += long
set queue default route_destinations += large_long
set queue default route_destinations += large_route
set queue default route_destinations += main_route
set queue default route_waiting_jobs = True
set queue default route_lifetime = 600
set queue default enabled = True
set queue default started = True
Please take note that torque will run the job in the FIRST place that it
*can* run in (based on neednodes, resources_min, resources_max), so you
should also order your queues appropriately.
> # Set server attributes.
> set server scheduling = True
> set server acl_host_enable = False
> set server acl_hosts = che-hydra.bham.ac.uk
> set server default_queue = default
^^^ this is the important line that sets the first queue destination.
All jobs that do not specify a queue name will go through this named
I hope this helps,
University of Southern California
More information about the torqueusers