[Mauiusers] detecting and scheduling vmem

[Brian Christiansen]

Roy,

Looking in MPBSI.c, it appears that nodememovercommitfactor is already 
implemented to work with swap, except that in in _MPBSIGetSSSStatus the 
setting of N->CRes.Swap is after the overcommitting work is done. How 
does the following patch work for you?

Thanks,
Brian

--- src/moab/MPBSI.c   
+++ src/moab/MPBSI.c   
@@ -6263,6 +6263,11 @@
       }  /* END else if (!strcmp(Name,"message")) */
     }    /* END while (ptr != NULL) */
 
+  /* NOTE:  PBS totmem = swap + RAM */
+
+  if (TotMem > 0)
+    N->CRes.Swap = TotMem;
+
   if (MSched.NodeMemOverCommitFactor > 0.0)
     {
     /* NOTE:  both real memory and swap overcommitted */
@@ -6276,11 +6281,6 @@
     /* memory factor not applied to node load */
     }
 
-  /* NOTE:  PBS totmem = swap + RAM */
-
-  if (TotMem > 0)
-    N->CRes.Swap = TotMem;
-
   return(SUCCESS);
   }  /* END __MPBSIGetSSSStatus() */


Gareth.Williams at csiro.au wrote:
> I passed your email on to CR in my open moab ticket.  This seems like a straightforward and good idea to me.
>
> thanks,
>
> Gareth
> ________________________________________
> From: Roy Dragseth [roy.dragseth at cc.uit.no]
> Sent: Tuesday, 24 February 2009 9:37 PM
> To: mauiusers at supercluster.org
> Subject: Re: [Mauiusers] detecting and scheduling vmem
>
> On Friday 20 February 2009 07:08:42 Gareth.Williams at csiro.au wrote:
>   
>> Hi All,
>>
>> I have been working with CR to get a patch in maui and moab to improve
>> treatment of vmem.  The patch is in maui-3.2.6p21-snap.1234905291 and will
>> be in upcoming moab releases.  Many thanks to Brian Christiansen at CR.
>>
>> With the patch, maui/moab correctly detects a node's available vmem as
>> being the total physical memory plus swap space.  This can then be
>> scheduled/allocated by requesting vmem and job's virtual memory allocation
>> can be limited on a per process basis and periodically measured (and action
>> taken or overuse) on a per job basis.
>>
>> I think this makes scheduling on vmem a good option.  It's a more
>> predictable quantity than mem/pmem.
>>     
>
> This is great, but is it possible to get
>
>     TotMem = (int)(TotMem * MSched.NodeMemOverCommitFactor);
>
> at line 6384?  This will make the NODEMEMOVERCOMMITFACTOR work both for mem
> and swap as earlier.
>
> We want to use a slight overcommit to allow a user to specify -lpvmem=2gb
> instead of -lpvmem=2000mb on a machine with 2GB RAM per core.  Some of the ram
> isn't available to userspace, so using 2gb will underutilize the cores on a
> node and users aren't really good at guessing the memory consumption anyway.
>
> r.
>
>
>
>
> _______________________________________________
> mauiusers mailing list
> mauiusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/mauiusers
>