[torqueusers] cpusets: Controlling the "mems" allocation for jobs

Michael Sternberg sternberg at anl.gov
Wed Mar 27 23:53:43 MDT 2013


David,

Oh my - I overlooked that option in the post-2.x qsub man page. Yes, "qsub -n" does exactly what I asked for, for PBS_NODEFILE, cpus, and mems. Examples below.

Thank you very much!


Michael


# testing on nodes with 2 CPUs x 6 cores:

echo 'uniq -c $PBS_NODEFILE' | qsub -q debug -l nodes=1:ppn=5

	login6 pbs_mom: LOG_INFO::create_job_cpuset, creating cpuset for job 307774.sched1.carboncluster: 5 cpus (0-4), 1 mems (0)

echo 'uniq -c $PBS_NODEFILE' | qsub -q debug -l nodes=1:ppn=5 -n

	login6 pbs_mom: LOG_INFO::create_job_cpuset, creating cpuset for job 307775.sched1.carboncluster: 12 cpus (0-11), 2 mems (0,1)


Same for multiple node requests:

echo 'uniq -c $PBS_NODEFILE' | qsub -q debug -l nodes=1:ppn=5+1:ppn=7

	login6 pbs_mom: LOG_INFO::create_job_cpuset, creating cpuset for job 307776.sched1.carboncluster: 5 cpus (0-4), 1 mems (0)

	login5 pbs_mom: LOG_INFO::im_join_job_as_sister, about to create cpuset for job 307776.sched1.carboncluster. 
	login5 pbs_mom: LOG_INFO::create_job_cpuset, creating cpuset for job 307776.sched1.carboncluster: 7 cpus (0-6), 2 mems (0,1)

echo 'uniq -c $PBS_NODEFILE' | qsub -q debug -l nodes=1:ppn=5+1:ppn=7 -n

	login6 pbs_mom: LOG_INFO::create_job_cpuset, creating cpuset for job 307777.sched1.carboncluster: 12 cpus (0-11), 2 mems (0,1)

	login5 pbs_mom: LOG_INFO::im_join_job_as_sister, about to create cpuset for job 307777.sched1.carboncluster. 
	login5 pbs_mom: LOG_INFO::create_job_cpuset, creating cpuset for job 307777.sched1.carboncluster: 12 cpus (0-11), 2 mems (0,1)


On Mar 27, 2013, at 23:44 , David Beer <dbeer at adaptivecomputing.com> wrote:

> Michael, 
> 
> Have you tried submitting with -n on your qsub line? This is meant to grant exclusive access and I believe it expands the cpuset for the job.  
> 
> David
> 
> Michael Sternberg <sternberg at anl.gov> wrote:
>> Is there a way to control the "mems" component of cpusets in user-space by parameters to qsub?



More information about the torqueusers mailing list