[torqueusers] set automatically a mem max ressources when the user submits a job with a nodes/cpu param

giggzounet giggzounet at gmail.com
Fri Sep 24 09:11:33 MDT 2010

Le 24/09/2010 16:52, Gabe Turner a écrit :
> On Fri, Sep 24, 2010 at 10:10:38AM +0200, giggzounet wrote:
>>> On Thu, 2010-09-23 at 11:59 +0200, giggzounet wrote:
>> Could you, please, send me a more complicated example of filter script
>> for Torque ?
>> Because I don't understand the input of the filter ? is it the name of
>> the job file or direct the data ???
> The submission script is just passed to the filter on stdin, hence the
> 'while read i' in the example bash script.  'read()' just reads a line on
> standard input.  Then within the while in the example, the line is just
> echoed to stdout.  See the sh man page, or read(1p) if your distribution of
> choice has it.
> So, the example script:
> echo "#PBS -l mem=16MB"
> while read i
>   do
>     echo $i
>   done
> Effectively adds the line '#PBS -l mem=16MB' to the top of the submission
> script.  It should be noted that one _must_ write to stdout anything that
> they want to appear in the script that is submitted.  If you exit() from
> the script before writing everything the script needs to stdout, you will
> not get everything that was in the submitted script.
> HTH,
> Gabe


at the moment, my first version is:


# add default memory constraints to all requests
# that did not specify it in user's script or on command line


# default memory allocation per core (unit mb):

# init variable:

while read i
# test on the existence of a memory allocation request
  MEM_LINE_LOC=`echo $i | $CMD_GREP "^#PBS" | $CMD_GREP -c "mem"`
# test on the existence of a nodes allocation request
  NODES_LINE=`echo $i | $CMD_GREP "^#PBS" | $CMD_GREP -c "nodes"`
  if [ $NODES_LINE -ge 1 ]; then
# compute the number of requested cores:
      NB_NODES=`echo $i | $CMD_GREP "^#PBS" | $CMD_GREP "nodes" |
$CMD_AWK -F'nodes=' '{ print $2 }' | $CMD_CUT -d: -f1`
      NB_PPN=`echo $i | $CMD_GREP "^#PBS" | $CMD_GREP "nodes" | $CMD_AWK
-F'ppn=' '{ print $2 }' | $CMD_CUT -d, -f1`
      NB_CORES=$(($NB_NODES * $NB_PPN))
  echo $i >> /tmp/torque_filter_$$.tmp

# if the MEM_LINE is null, there is no memory allocation request, so add
if [ $MEM_LINE -lt 1 ]; then
# set a default memory allocation:
    echo '#PBS -l mem='$MEM_ALLOC_TOT'mb'

# send the original job
$CMD_CAT /tmp/torque_filter_$$.tmp

# remove the temporary file
$CMD_RM /tmp/torque_filter_$$.tmp

it works. but it is quite tricky...

Best regards,

More information about the torqueusers mailing list