Moab Workload Manager® for Grids

17.8 Reservations in the Grid

17.8.1 Shared Resources

In some environments, globally shared resources may need to be managed to guarantee the full environment required by a particular job. Resources such as networks, storage systems, and license managers may be used only by batch workload but this workload may be distributed among multiple independent clusters. Consequently, the jobs from one cluster may utilize resources required by jobs from another. Without a method of coordinating the needs of the various cluster schedulers, resource reservations will not be respected by other clusters and will be of only limited value.

Using the centralized model, Moab allows the importing and exporting of reservations from one peer server to another. With this capability, a destination peer can be set up for the shared resource to act as a clearinghouse for other Moab cluster schedulers. This destination peer Moab server reports configured and available resource state and in essence possesses a global view of resource reservations for all clusters for the associated resource.

To allow the destination peer to export reservation information, the RMCFG lines for all client resource managers must include the flag RSVEXPORT. On the cluster scheduler side, each Moab should be configured with a resource manager interface to the destination peer and include both the RSVEXPORT and RSVIMPORT flags. For a non-client resource manager, the RSVEXPORT indicates that the local Moab server should push information about newly created reservations to the remote system, while the RSVIMPORT flag indicates that the local Moab server should import and locally enforce reservations detected on the destination peer server.

Example: A Multi-Cluster Shared License Manager

An academic organization has four clusters which share a single common license manager. Because the license manager is shared, no single cluster has exclusive control over it and so no single cluster can guarantee that needed software licenses will be available when a particular job starts. What is needed is a way for each of the cluster schedulers to coordinate their software license needs.

Using Moab peer services, this can be accomplished with the addition of a single destination Moab peer running on the license manager host with reservation export enabled. To coordinate software license requirements, each cluster Moab server will need to define a resource manager interface to the new peer and enable both reservation import and export for this interface. With this configuration, each cluster's Moab server will, in essence, register its software license reservations and import the reservations of all other clusters. With the destination peer's reservation map imported, each local cluster scheduler can then make local scheduling decisions which respect the license requirements of the other cluster schedulers.

SCHEDCFG[cluster1] MODE=NORMAL SERVER=cluster1:42334
RMCFG[base]        TYPE=PBS
RMCFG[license]     TYPE=MOAB SERVER=license1:42559 FLAGS=rsvimport,rsvexport
...

SCHEDCFG[license] MODE=NORMAL SERVER=cluster1:42559
RMCFG[lm]         TYPE=NATIVE CLUSTERQUERYURL=/usr/local/tools/flexlmquery.pl
RMCFG[cluster1]   TYPE=MOAB FLAGS=client,rsvexport
...