Moab Workload Manager®

Appendix H: Interfacing with Moab (APIs)

Moab provides numerous interfaces allowing it to monitor and manage most services and resources. It also possesses flexible interfaces to allow it to interact with peer services and applications as both a broker and an information service. This appendix is designed to provide a general overview and links to more detailed interface documentation.

• H.1 Moab Query and Control APIs
  • Allow external portals and services to obtain information about compute resources, workload, and usage statistics.
• H.2 Resource Management Interfaces
  • Allow Moab to monitor, schedule, and control services and resources.
• H.3 Identity and Credential Management Interfaces
  • Allow monitoring and active management of user configuration, credentials, policies, and usage information.
• H.4 Accounting and Event Interfaces
  • Allow import/export of accounting and event information to external entities.
• H.5 Grid Services API
  • Provide and use information, data, job, and resource management services in a distributed environment.
• H.6 Discovery/Directory Services
• H.7 Job Submission and Management Interface
  • Query resource availability, submit, modify, and manage jobs, and query the status of active and completed jobs.

Moab interfaces to systems providing various services and using various protocols. This appendix is designed to assist users who want to enable Moab in new environments using one of the existing interfaces. It does not cover the steps required to create a new interface.

H.1 Query and Control APIs

The Moab Cluster and Grid Suites provide a (Moab) workload manager server that supports a broad array of client services. These services can either be directly accessed via Moab client commands or by way of a number of APIs. Which approach is best will depend on the particular use case.

• Java API
• C API
• CLI/XML API

H.1.1 Java API

The Moab-API is a Java library that allows Java developers to easily communicate with and interact with the Moab Workload Manager. The API handles the details of creating a secure connection, reporting information about workload and resources, and creating basic commands.

New users are encouraged to view the Moab Java API Quick Start Guide for details on interfacing with this API. The complete Javadocs are also available for review.

H.1.2 C API

The Moab C API provides access to all Moab information and services. This API consists of both easy to use high-level calls and powerful and flexible low-level calls.

High-Level C API Calls

Example 1 - Interfacing to an Explicitly Specified Moab Server

#include <stdio.h>
#include <stdlib.h>
#include "mapi.h"

int main(

  int   ArgC,
  char *ArgV[],
  char *EnvP[])

  {
  void *C;

  char  EMsg[1024];
  char *OBuf;

  char JobID[32];

  char *JobDescription = "<Job><Arguments>600</Arguments>
<Executable>/bin/sleep</Executable><InitialWorkingDirectory>/home/wightman</InitialWorkingDirectory>\

<NodeCount>1</NodeCount><GroupId>wightman</GroupId><UserId>wightman</UserId>\

<Requested></Requested><Processors>1</Processors></Job>";

  if (MCCInitialize("sn-master03",14522,&C) < 0)
    {
    fprintf(stderr,"cannot initialize interface\n");

    exit(-1);
    }
 
  if (MCCJobSubmit(&C,JobDescription,&OBuf,EMsg) < 0)
    {
    fprintf(stderr,"Error: '%s'\n",
      EMsg);

    exit(-1);
    }

  fprintf(stdout,"Results: Job '%s' successfully submitted\n",
    OBuf);

  snprintf(JobID,sizeof(JobID),"%s",OBuf);

  free(OBuf);

  OBuf = NULL;

  if (MCCJobQuery(&C,JobID,&OBuf,EMsg) < 0)
    {
    fprintf(stderr,"Error: '%s'\n",
      EMsg);

    exit(-1);
    }

  fprintf(stdout,"Results: '%s'\n",
    OBuf);

  free(OBuf);

  OBuf = NULL;

  if (MCCJobCancel(&C,JobID,EMsg) < 0)
    {
    fprintf(stderr,"ERROR: '%s'\n",
      EMsg);

    exit(-1);
    }

  fprintf(stdout,"Job '%s' successfully canceled\n",
    JobID); 

  MCCFree(NULL);

  return(0);
  }  /* END main() */

Example 2 - Interface to Monitor Remaining Time w/in a Batch Job

#include <stdio.h>
#include <stdlib.h>
#include "mapi.h"

int main(

  int   ArgC,
  char *ArgV[],
  char *EnvP[])

  {
  mulong Time;

  /* from inside a batch job, API will auto allocate the interface and
     discover the appropriate jobid */

  /* monitor time, clean-up and exit if less then 300 seconds remain */

  while (1)
    {
    MCCJobGetRemainingTime(NULL,NULL,&Time,NULL);

    if (Time < 300)
      exit(0);
      
    sleep(1);
    }

  /*NOTREACHED*/

  exit(1);
  }

gcc client.c -I/opt/moab-5.1.0/include/ -L/opt/moab-5.1.0/lib/ -lcmoab -lmoab -lpthread -lm -lmcom -lminit -o client

Low-Level C API

The Moab low-level C API allows direct access to all Moab client services. This API uses the MCCExecute routine and takes as an argument an XML request string. This interface provides a high-speed, low-latency interface directly into the server and provides very broad and flexible cluster, workload, and resource control. This routine creates a direct socket connection to provides the following services:

• socket creation
• client authentication
• server discovery via built-in, configfile, and environment settings
• command request and response encryption
• marshalling and de-marshalling of requests and data

Requests are sent using native XML request strings and, depending on the command used, responses are returned in unformatted native XML or pretty-print human-readable format. The commands that do not return XML can be configured to return XML with the --format=xml option. The commands that do not natively return XML are:

• setspri
• setres
• releaseres
• runjob
• checkjob
• showconfig
• mdiag
• mjobctl

Use of Cluster Resources Consulting Services is recommended for organizations using this interface.

H.1.3 CLI (Command Line Interface) XML API

All Moab client commands can report results in XML format to allow the information to be easily integrated into peer services, portals, databases, and other applications. To request that a client command report its output in XML, specify the --format=xml flag as in the following example:

> showq --format=xml

<Data>
<Object>queue</Object>
<cluster LocalActiveNodes="1" LocalAllocProcs="1" LocalIdleNodes="0" LocalIdleProcs="3" LocalUpNodes="1" 
  LocalUpProcs="4" RemoteActiveNodes="0" RemoteAllocProcs="0" RemoteIdleNodes="0" RemoteIdleProcs="0" 
  RemoteUpNodes="0" RemoteUpProcs="0" time="1128451812"></cluster>
<queue count="1" option="active">
<job AWDuration="11672" EEDuration="1128451812" Group="[DEFAULT]" JobID="Moab.2" MasterHost="cw2" PAL="2" 
  QOS="bug3" ReqAWDuration="54000" ReqNodes="1" ReqProcs="1" RsvStartTime="1128451812" RunPriority="0" 
  StartPriority="1" StartTime="1128451812" StatPSDed="11886.580000" StatPSUtl="11886.580000" State="Running" 
  SubmissionTime="1128451812" SuspendDuration="0" User="smith"></job>
</queue>
<queue count="1" option="eligible">
<job EEDuration="1128451812" Group="jacksond" JobID="customer.35" QOS="bug" ReqAWDuration="3600" 
  ReqProcs="1" StartPriority="1" StartTime="0" State="Idle" SubmissionTime="1128451812" SuspendDuration="0" 
  User="johnson"></job>
<queue><queue count="0" option="blocked"></queue>
</Data>

Common Query/Control Services

• jobs
  • query status - mdiag -j (XML details)
  • submit - msub (XML format)
  • cancel - mjobctl -c
• nodes
  • query status - mdiag -n (XML details)
  • create resource reservation - mrsvctl -c
  • destroy resource reservation - mrsvctl -r

H.2 Resource Management Interfaces

• HTTP
• LDAP
• SQL
• script/flat file
• Resource Manager Specific Interfaces - LSF, SGE, TORQUE, PBSPro, Loadleveler, and so forth

Using the resource manager interfaces, Moab can do the following:

• monitor resources (compute host, network, storage, and software license based resources)
  • load configuration, architecture, and feature information
  • load state, utilization, and workload information
  • load policy and ownership information
• manage resources
• dynamically reconfigure and reprovision resource hardware (processors, memory, etc.)
• dynamically reconfigure and reprovision resource software (operating system, application software, filesystem mounts, etc.)
• dynamically reconfigure and reprovision resource security (VPN's, VLAN's, host security, etc.)
• monitor workload (batch jobs, interactive jobs, persistent services, dynamic services, distributed services)
  • load state, resource requirement, and required environment information
  • load user, group, and credential information
  • load utilization, resource allocation, and policy information
• manage workload
  • migrate jobs from one resource to another (intra-cluster and inter-cluster)
  • modify jobs for translation and optimization purposes
  • suspend, resume, checkpoint, restart, and cancel jobs
• query cluster policies and configuration

H.3 Identity and Credential Management Interfaces

Moab's identity and credential management interfaces allow Moab to exchange credential and user configuration, access, policy, and usage information.

• Identity Manager
• Allocation Manager
• Moab Workload Manager for Grids

H.4 Accounting Interfaces

Moab accounting interfaces allow Moab to export local utilization statistics, events, and accounting information to site specific scripts.

• Accounting Interface

H.5 Grid Interfaces

Moab provides interfaces to allow interaction with various grid brokers and services. These interfaces allow Moab to provide services as well as utilize services.

Services Utilized

• Information Services (import and utilize information service data in making scheduling decisions)
• Job Migration
• Data Migration
• Credential Mapping
• Security and Delegation

See Moab Workload Manager for Grids for more information on utilized services.

Services Provided

• Information Services (provide resource, workload, and credential information)
• Job Migration
• Data Migration
• Credential Mapping

See Moab Workload Manager for Grids for more information on provided services.

H.6 Discovery/Directory Services

Moab can import and export key event information regarding workload, cluster resources, cluster and grid services, and other components of hardware and software infrastructure.

By default, these clients communicate with the scheduler using the U.S. Department of Energy (DOE) Scalable Systems Software socket and wire protocols. These protocols are largely HTML- and XML-based, using PKI, 3DES, MD5, challenge, and other security protocols and are documented within the SSS project pages.

As part of this initiative, the scheduler/client protocol has been extended to support multiple socket level protocol standards in communicating with its clients and peer services. These include SingleUseTCP, SSS-HALF, and HTTP. The client socket protocol can be specified by setting the MCSOCKETPROTOCOL parameter to SUTCP, SSS-HALF, or HTTP. Further protocols are being defined and standardized over time and backwards compatibility will be maintained. Documentation on the SSS-HALF implementation can be found within the DOE's SSS Project Notebooks.

H.7 Job Submission and Management Interface

Moab provides interfaces to enable the following services:

• Resource Availability Query
  • Determine quantity, state, and configuration of configured resources (idle, busy, and down nodes)
  • Determine quantity and configuration of all available resources (idle nodes)
  • Determine resources available subject now and in the future for potential job
  • Determine best target cluster destination for potential job
  • Determine largest/longest job which could start immediately
  • Determine estimated start time for potential job
  • Determine earliest guaranteed start time for potential job
• Reserve Resources
  • Reserve specific resources for desired time frame
• Submit Job (XML format)
  • Submit job to specific cluster
  • Submit job to global job queue
• Manage Job
  • Hold job
  • Adjust job priority
  • Modify job executable, args, data requirements, job dependencies, duration, hostcount, or other attributes
  • Suspend/resume job
  • Checkpoint/requeue job
  • Cancel job
  • Migrate job
  • Adjust job quality of service (QoS)
• Query Job
  • Determine job state, utilization, or output results for idle, active, or completed job
  • Determine estimated start time
  • Determine guaranteed start time