[torqueusers] ANNOUNCE: Torque accounting scripts pbsacct version 1.4.6

Ole Holm Nielsen Ole.H.Nielsen at fysik.dtu.dk
Fri Mar 6 07:24:58 MST 2009


An updated version of the simple Torque accounting scripts pbsacct version 1.4.6
is now available at ftp://ftp.fysik.dtu.dk/pub/Torque/pbsacct-1.4.6.tar.gz.

These tools may be used to extract accounting information from Torque's
accounting file directory and present an overview.  A script for monthly
summaries is included.  Please see the README file for features.

The CHANGELOG file for version 1.4.6:

pbsjobhist:
         NEW SCRIPT: Produce a Torque/PBS job history from the accounting records.
         This may be useful for tracking job histories of individual jobs, or
         those jobs belonging to specific users and groups.
         Usage:
            cat [accounting-files] | pbsjobhist
         Accounting files are usually in /var/spool/torque/server_priv/accounting/.
         Please make sure that the accounting-files covers the entire period 
that you're
         interested in, since some jobs will usually fall outside this period.
         Output: One line per job containing:
            job-id username group list-of-event-timestamps
         The job events are mainly: Q (queued), S (started), E (ended).  Additional
         event types are listed in the script. If available, the CPU-time, 
Wallclock-time
         and Queue-wait-time are printed.
pbsacct:
         * Fixed wrong column used in the sort command (now uses SORTCOLUMN).
         * Introduced a variable MAUI_NODEACCESSPOLICY with default value "SHARED"
         for those sites that account by entire nodes in stead of CPU cores used,
         see the explanation under pbsjobs below.
         * Make a check for zero accounting lines in input files.
pbsreportmonth:
         Configure SUBSCRIBERS and SYSTEMNAME for your local environment.
pbsnodect:
         * Calculate number of days from first to last job.
         Previously we just counted the number of accounting files, but on
         systems with very few jobs there may be gaps in the daily accounting
         files.
         * Assume a default np=1 in the Torque nodes file if np= is absent.
         * Introduced a variable MAUI_NODEACCESSPOLICY with default value "SHARED"
         for those sites that account by entire nodes in stead of CPU cores used,
         see the explanation under pbsjobs below.
         * Make a check for zero accounting lines in input files.
pbsjobs:
        * Calculate number of CPU cores used from the exec_host record, in stead of
        the Resource_List.nodes used previously.
        * Introduced an optional argument -S (for "Single").
        This should be used if you use the MAUI scheduler's single
        job/task/user per node features SINGLEUSER/SINGLEJOB/SINGLETASK, see
        http://www.clusterresources.com/products/maui/docs/5.3nodeaccess.shtml.
        In this case a job is charged for the use of an ENTIRE NODE in stead of
        the default, which is the actual number of processors used within each
        node. Requested by Niels Carl Hansen <recnh at cscaa.dk>.
        We get the node's #CPU cores from the Torque "nodes" file.
        NOTE: One caveat is required: If the system was changed after the accounting
        record was made, a nodename may have disappeared from the "nodes"
        file, and hence we have lost all memory of this node's #CPU cores.
        In this case we only account for the actual #CPU cores used.
        Therefore the accounting script should NOT be rerun after changes have
        been made to the nodes in the system, otherwise the results may become
        inaccurate !
        * Make a check for pbs_server logfiles in stead of accounting files.


-- 
Ole Holm Nielsen
Department of Physics, Technical University of Denmark


More information about the torqueusers mailing list