[an error occurred while processing this directive] [an error occurred while processing this directive] [an error occurred while processing this directive]
Maui provides extensive accounting facilities which allow resource usage to be tracked by resources (i.e., compute nodes), jobs, users, and other objects. The accounting facilities may be used in conjunction with and correlated with the accounting provided by the resource manager.
Maui maintains a large number of statistics and provides several commands to allow easy access to and helpful consolidation of this information. These statistics are of three primary types:
188.8.131.52 Job Accounting
As each job completes, Maui records a complete job trace containing information about who ran the job, when and where it ran, what resources it requested, what it actually utilized, and other pieces of key information. A complete description of each of the job accounting data fields can be found within section 16.3 Workload Traces of the admin manual.
The Moab Cluster ManagerTM can generate reports, graphs and charts for accounting and usage statistics.
184.108.40.206 Resource Accounting
220.127.116.11 Credential Accounting
In addition to the showstats command, the showgrid command also obtains its information from the in memory stats and checkpoint file. This command display a processor-time based matrix of scheduling performance for a wide variety of metrics. Information such as backfill effectiveness or average job queue time can be determined on a job size/duration basis. See the showgrid command documentation for more information.
Historical usage information can be obtained for a specific timeframe, class of jobs, and/or portion of resources using the profiler command. This command operates on the detailed job trace information recorded at the completion of each job. These traces are stored in the directory pointed to by the STATDIR parameter which defaults to $(MAUIHOMEDIR)/stats. Within this directory, statistics files are maintained using the format WWW_MMM_DD_YYYY (i.e, Mon_Jul_16_2001) with jobs traces being recorded in the file associated with the day the job completed. Each job trace is white space delimited flat text and may be viewed directly with any text reader.
When profiling statistics, stat files covering the time frame of interest should be aggregated into a single file. This file can be passed to the profiler command along with a number of configuration flags controlling what data should be processed and how it should be display. Command line flags allow specification of constraints such as earliest start date, or latest completion date. Flags can also be used to evaluate only jobs associated with specific users, groups, accounts, or QOS's. Further, it is possible to specify that only jobs run on certain nodes be processed. Because the trace files are flat text, simple UNIX text processing tools such as awk, sed, or grep can be used to create more elaborate filters should they be needed.
The output of the profiler command provides extensive detailed information about what jobs ran and what level of scheduling service they received. The profiler command documentation should be consulted for more information.
The Moab Cluster ManagerTM graphically organizes the fairshare values by credential for easy navigation and provides a GUI to specify the Decay Factor, Depth, Interval Length and Usage Metric.
Shared vs Dedicated
SMP nodes are often used to run jobs which do not use all available resources on that node. How Maui handles these unused resources is controlled by the parameter NODEACCESSPOLICY. If this parameter is set to SHARED, Maui will allow tasks of other jobs to use the resources. If this parameter is set to DEDICATED, Maui will mark these resources unavailable for use by other jobs.
[an error occurred while processing this directive] [an error occurred while processing this directive]