[Mauiusers] Maui/Torque and Fairshare with secondary Unix groups

Bas van der Vlies basv at sara.nl
Mon Jan 7 08:55:59 MST 2008


I have made a patch for maui version:
  - 3.2.6p20-snap.1182974819

The patch is different then that what i suggested. It does not make use of 
the getgrouplist function nor it uses the FSGROUPS keyword. When the 
configuration file is parsed. We know the FairShare groups. When a user 
submits a job maui will now find out if it is a member of one of this 
groups. If yes then set the group credentials to this group and it will 
ignore then the PBS egroup variable.

Maui did not parse the PBS group_list parameter:
  qsub -W group_list=<value>

This variable has the highest priority and will set the group credential to 
  the value specified. We at SARA want to ignore this parameter, because we 
do not want an user to specify its own group and bypass the FairShare 
targets. So we added an option to ignore this parameter:
  - IGNPBSGROUPLIST          TRUE

This patch also fix some debug problems that i had when the debug level is 
set to another value then the default one, eg:
  - LOGLEVEL 9

Maui has a lot of keywords and the parsing of the keywords are only be 
shown if a certain loglevel is set. Somebody thought it is nice that the 
maui keyword table is sorted alphabetically. This is not correct because 
now only the keywords after the LOG... definitions are shown. This patch 
moves the LOG.. keywords definition to the beginning of keyword table and 
fix this problem.

patch is attached

regards,


Bas van der Vlies wrote:
> Happy new year everyone,
> 
>   I had some time to examine the source of maui and found two ways to
> implement this functionality in maui. So we are not depended anymore
> on the user primary group for Fairshare or specifying some
> commandline parameters for qsub.
> 
> Here are the two solutions:
>   1) Introduce a new parameter: FSGROUPS, eg:
>      FSGROUPS one two three
> 
>     When a job is submitted:
>      a) check if the user is a member of one of this groups
>      b) if yes then set the credentials to the matching group.
> 
> 2) When a job is submitted:
>   a) get all the groups where the user is a member of with the
> "getgrouplist" function. I do not know if this function is available
> on all platforms.
>   b) Then check if for one of the user groups a fairshare target is set
>   c) if yes then set the credentials to this group.
> 
> 
> Option 2 has more freedom but i do not know if all OS'es support the
> getgrouplist function.
> 
> I want to know which solution the community thinks is the best one?
> 
> Regards
> 
> On Dec 10, 2007, at 3:05 PM, Bas van der Vlies wrote:
> 
>> In our old setup the primary user group was the same as the
>> fairshare group
>> in Maui. so we did not have any problems with the Fairshare (FS)
>> setup.
>>
>> We are now switching to a new User administration system and use
>> the Linux
>> setup. Every user has as primary group his own group (user: bas,
>> group: bas).
>>
>> I have read some docs and the problem is that torque only sent the
>> primary
>> group to Maui. There are several setup's possible:
>>   * write a torque submit filter that adds -W or -A, But to my
>> knowledge
>> the  submit filter does not work for batch jobs that are submitted
>> via qsub -I
>>
>>   * In maui:
>>      USERCFG[bas] QDEF=SARA
>>      QOSCFG[SARA] FSTARGET=80.0
>>   But this is not very scalable. Every time a new user is added we
>> have to
>>   restart Maui and generate a line as above.
>>
>>   * Are there other solutions possible?
>>
>> What i am thinking about is to patch Maui, like the way Torque has
>> done it
>> with the acl_group_sloppy paramter or patch Torque to send also the
>> secondary groups of user.
>>
>> regards,
>>
>>
>> --
>> ********************************************************************
>> *                                                                  *
>> *  Bas van der Vlies                     e-mail: basv at sara.nl      *
>> *  SARA - Academic Computing Services    phone:  +31 20 592 8012   *
>> *  Kruislaan 415                         fax:    +31 20 6683167    *
>> *  1098 SJ Amsterdam                                               *
>> *                                                                  *
>> ********************************************************************
>> _______________________________________________
>> mauiusers mailing list
>> mauiusers at supercluster.org
>> http://www.supercluster.org/mailman/listinfo/mauiusers
> 
> --
> Bas van der Vlies
> basv at sara.nl
> 
> 
> 
> _______________________________________________
> mauiusers mailing list
> mauiusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/mauiusers


-- 
--
********************************************************************
*                                                                  *
*  Bas van der Vlies                     e-mail: basv at sara.nl      *
*  SARA - Academic Computing Services    phone:  +31 20 592 8012   *
*  Kruislaan 415                         fax:    +31 20 6683167    *
*  1098 SJ Amsterdam                                               *
*                                                                  *
********************************************************************
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fsgroup.patch
Type: text/x-patch
Size: 10745 bytes
Desc: not available
Url : http://www.supercluster.org/pipermail/mauiusers/attachments/20080107/1a765fc0/fsgroup.bin


More information about the mauiusers mailing list