A.2 Case Study: Semi-Partitioned Heterogeneous Cluster Dedicated to Parallel and Time-sharing Serial Jobs
A site possessing a mixture of uniprocessor and dual
processor nodes desires to dedicate a subset of nodes to time-sharing serial
jobs, a subset to parallel batch jobs, and provide a set of nodes to be
used as 'overflow'.
Group A: 16 uniprocessor Linux based nodes, each
with 128 MB of RAM and 1 GB local scratch space
Resource Manager: OpenPBS 2.3
range in size from 1 to 32 processors with approximately the following
quartile job frequency distribution
Job Length: jobs range in length from 1 to 24 hours
Job Owners: job are submitted from 6 major groups consisting of a total of about 50 users
During prime time hours, the majority of jobs submitted are smaller, short
running development jobs where users are testing out new code and new data
sets. The owners of these jobs are often unable to proceed with their
work until a job they have submitted completes. Many of these jobs
are interactive in nature. Throughout the day, large, longer running
production workload is also submitted but these jobs do not have comparable
turnaround time pressure.
Constraints: (Must do)
Nodes in Group A must run only parallel jobs.
Nodes in Group B must only run serial jobs, with up to 4 serial jobs per
node. Nodes in Group C must not be used unless a job cannot locate
Goals: (Should do)
The scheduler should attempt to intelligently load
balance the timesharing nodes.
As in Case Study 1, The network topology is flat and and nodes are homogeneous within each group. The only tricky part of this configuration is the 'overflow' group. The easiest configuration is to create two PBS queues, serial and parallel, with appropriate min and max node counts as desired. By default, Maui interprets the PBS 'exclusive hostlist' queue attribute as constraining jobs in the queue to run only on the nodes contained in the hostlist. We can take advantage of this behavior to assign nodes in Group A and Group C to the queue 'parallel' while the nodes in Group B and Group C are assigned to the queue 'serial' (The same can be done with classes if using Loadleveler) Maui will incorporate this queue information when making scheduling decisions.
The next step is to make the scheduler use the 'overflow' nodes of group C only as a last resort. This can be accomplished using a negative affinity standing reservation. This configuration will tell the scheduler that these nodes can be used, but should only be used if it cannot find compute resources elsewhere.
The final step, load balancing, is accomplished in
two parts. First, the nodes in group B must be configured to allow
up to 4 serial jobs to run at a time. This is best accomplished using
the PBS 'virtual nodes' feature. To load balance, simply select
the CPULOAD allocation
algorithm in Maui. This algorithm will instruct Maui to schedule
the job based on which node has the most available, unused idle CPU time.
This site requires both resource manager and scheduler configuration.
The following Maui configuration would be needed.
# allow SMP node sharing
PBS qmgr config
set queue parallel resources_min.nodecount=2
PBS 'server_priv/nodes' file
|© 2001-2010 Adaptive Computing Enterprises, Inc.|