7.1.2 Administrative Reservations
Administrative reservations behave much like standing reservations but are generally created to address non-periodic, one-time issues. All administrative reservations are created using the mrsvctl -c (or setres) command and are persistent until they expire or are removed using the mrsvctl -r (or releaseres) command.
Reservations can be labeled and annotated using comments allowing other administrators, local users, portals and other services to obtain more detailed information regarding the reservations. Naming and annotations are configured using the -n and -D options of the mrsvctl command respectively as in the following example:
> mrsvctl -c -D 'testing infiniband performance' -n nettest -h 'r:agt[15-245]'
You can set up reservation profiles to avoid manually and repetitively inputting standard reservation attributes. Profiles can specify reservation names, descriptions, ACLs, durations, hostlists, triggers, flags, and other aspects that are commonly used. With a reservation profile defined, a new administrative reservation can be created that uses this profile by specifying the -P flag as in the following example:
RSVPROFILE[mtn1] TRIGGER=AType=exec,Action="/tmp/trigger1.sh",EType=start RSVPROFILE[mtn1] USERLIST=steve,marym RSVPROFILE[mtn1] HOSTEXP="r:50-250"
> mrsvctl -c -P mtn1 -s 12:00:00_10/03 -d 2:00:00
Example 2: Non-Blocking System Reservations with Scheduler Pause
RSVPROFILE[pause] TRIGGER=atype=exec,etype=start,action="/opt/moab/bin/mschedctl -p" RSVPROFILE[pause] TRIGGER=atype=exec,etype=cancel,action="/opt/moab/bin/mschedctl -r" RSVPROFILE[pause] TRIGGER=atype=exec,etype=end,action="/opt/moab/bin/mschedctl -r"
> mrsvctl -c -P pause -s 12:00:00_10/03 -d 2:00:00
Any reservation causes some negative impact on cluster performance as it further limits the scheduler's ability to optimize scheduling decisions. You can mitigate this impact by using flexible ACLs and triggers.
In particular, a maintenance reservation can be configured to reduce its effective reservation shadow by allowing overlap with checkpointable/preemptible jobs until the time the reservation becomes active. This can be done using a series of triggers that perform the following actions:
The following example highlights one possible configuration:
RSVPROFILE[adm1] JOBATTRLIST=PREEMPTEE RSVPROFILE[adm1] DESCRIPTION="regular system maintenance" RSVPROFILE[adm1] TRIGGER=EType=start,Offset=-300,AType=internal,Action="rsv:-:modify:acl:jattr-=PREEMPTEE" RSVPROFILE[adm1] TRIGGER=EType=start,Offset=-240,AType=jobpreempt,Action="checkpoint" RSVPROFILE[adm1] TRIGGER=EType=start,Offset=-60,AType=jobpreempt,Action="cancel"
> mrsvctl -c -P adm1 -s 12:00:00_10/03 -d 8:00:00 -h ALL
This reservation reserves all nodes in the cluster for a period of eight hours. Five minutes before the reservation starts, the reservation is modified to remove access to new preemptible jobs. Four minutes before the reservation starts, preemptible jobs that overlap the reservation are checkpointed. One minute before the reservation, all remaining jobs that overlap the reservation are canceled.
Searches Moab documentation only
|© 2001-2010 Adaptive Computing Enterprises, Inc.|