Moab Workload Manager®

Synopsis

msub  [-a datetime][-A account][-c interval][-C directive_prefix][-d path]
      [-e path][-h][-j join][-k keep][-l resourcelist][-m mailoptions]
      [-M user_list][-N name][-o path][-p priority][-q destination][-r]
      [-S pathlist][-u userlist][-v variablelist][-V][-W additionalattributes]
      [-z][script]

Overview

msub allows users to submit jobs directly to Moab.  When a job is submitted directly to a resource manager (like TORQUE), it is constrained to run on only those nodes that the resource manager is directly monitoring.  In many instances, a site may be controlling multiple resource managers.  When a job is submitted to Moab rather than to a specific resource manager, it is not constrained as to what nodes it be be executed on.  msub can accept command line arguments (with the same syntax as qsub), job scripts (in either PBS or LoadLeveler syntax), or the SSS Job XML specification.

Submitted jobs can then be viewed and controlled via the mjobctl command.

Access

Format

[[[[CC]YY]MM]DD]hhmm[.SS]

---

> msub -a 12041300 cmd.pbs

<ACCOUNT NAME>

---

msub -A research cmd.pbs

[n|s|c|c=<minutes>]

---

n - No Checkpoint is to be performed
s - Checkpointing is to be performed only when the server executing the job is shutdown
c - Checkpoint is to be performed at the default minimum time for the server executing the job
c=<minutes> - Checkpoint is to be performed at an interval of minutes

msub -c=12 cmd.pbs

'<PREFIX NAME>'

First known prefix
(#PBS, #@, #BSUB, #!, 
#MOAB, #MSUB)

It is best to submit with single quotes. '#PBS'
An empty prefix will cause Moab to not search for any prefix. -C ''
Command line arguments have precedence over script arguments.
Custom prefixes can be used with the -C flag. -C '#MYPREFIX'
Custom directive prefixes must use PBS syntax.
If the -C flag is not given, Moab will take the first default prefix found.
Once a directive is found, others are ignored.

msub -C '#MYPREFIX' cmd.pbs

#MYPREFIX -l walltime=5:00:00 (in cmd.pbs)

<path>

$HOME

msub -d /home/test/job12 cmd.pbs

[<hostname>:]<path>

$SUBMISSIONDIR/$JOBNAME.e$JOBID

msub -e test12/stderr.txt

N/A

---

msub -h cmd.ll

[oe|n]

n (not merged)

msub -j oe cmd.sh

[e|o|eo|oe|n]

n (not retained)

msub -k oe

<STRING>

---

> msub -l nodes=32:ppn=2,pmem=1800mb,walltime=3600 cmd.sh

[[n]|[a][b][e]]

a

> msub -m be cmd.sh

<user>[@<host>][,<user>[@<host>],...]

$JOBOWNER

msub -M jon@node01,bill@node01,jill@node02 cmd.sh

<string>

msub -N chemjob3 cmd.sh

[<hostname>:]<path>

$SUBMISSIONDIR/$JOBNAME.o$JOBID

msub -o test12/stdout.txt

<INTEGER> (between -1024 and 0)

0

msub -p 25 cmd.sh

[<queue>][@<server>]

[<DEFAULT>]

msub -q priority cmd.sh

[y|n]

y

msub -r n cmd.sh

<path>[@<host>][,<path>[@<host>],...]

$SHELL

msub -S /bin/bash

<user>[@<host>[,<user>[@<host>],...]

UID of msub command

msub -u bill@node01 cmd.sh

<string>[,<string>,...]

---

mrsvctl -v DEBUG cmd.sh

N/A

N/A

msub -V cmd.sh

<string>

---

msub -W x=GRES:matlab:1 cmd.sh

N/A

N/A

msub -z cmd.sh

Job Script

   The msub command supports job scripts written in any one of the following languages:

Language	Notes
PBS/TORQUE Job Submission Language	---
LoadLeveler Job Submission Language	Use the INSTANTSTAGE parameter as only a subset of the command file keywords are interpreted by Moab.
SSS XML Job Object Specification	---
LSF Job Submission Language	enabled in Moab 4.2.4 and higher

/etc/msubrc

-W x=GRES:matlab:2

Usage Notes

   msub is designed to be as flexible as possible, allowing users accustomed to PBS, LSF, or LoadLeveler syntax, to continue submitting jobs as they normally would.  It is not recommended that different styles be mixed together in the same msub command.

   When only one resource manager is configured inside of Moab, all jobs are immediately staged to the only resource manager available.  However, when multiple resource managers are configured Moab will determine which resource manager can run the job soonest.  Once this has been determined, Moab will stage the job to the resource manager.

   It is possible to have Moab take a "best effort" approach at submission time using the forward flag.  When this flag is specified, Moab will do a quick check and make an intelligent guess as to which resource manager can run the job soonest and then immediately stage the job.

   Moab can be configured to instantly stage a job to the underlying resource manager (like TORQUE/LOADLEVELER) through the parameter INSTANTSTAGE.  When set inside moab.cfg, Moab will migrate the job instantly to an appropriate resource manager.  Once migrated, Moab will destroy all knowledge of the job and refresh itself based on the informat ion given to it from the underlying resource manager.

   In most instances Moab can determine what syntax style the job belongs to (PBS or LoadLeveler); if Moab is unable to make a guess, it will default the style to whatever resource manager was configured at compile time.  If LoadLeveler and PBS were both compiled then LoadLeveler takes precedence.

   Moab can translate a subset of job attributes from one syntax to another.  It is therefore possible to submit a PBS style job to a LoadLeveler resource manager, and vice versa, though not all job attributes will be translated.

Example 1

> msub -l nodes=3:ppn=2,walltime=1:00:00,pmem=100kb script2.pbs.cmd

4364.orion 

See Also

• mjobctl command to view, modify, and cancel jobs.
  
• checkjob command to view detailed information about the job.
  
• mshow command to view all jobs in the queue.
• DEFAULTSUBMITLANGUAGE parameter