[Mauiusers] Maui/Torque and Fairshare with secondary Unix groups]

Bas van der Vlies basv at sara.nl
Sun Apr 27 12:28:21 MDT 2008


On Apr 24, 2008, at 11:19 PM, Brian Christiansen wrote:

> We have the patch implemented but haven't checked it in yet. Before
> checking it in, can you provide a patch that turns on the FS groups
> behavior with a parameter rather than having it be the default  
> behavior?
>

Brian,

    My first thought was also to define a parameter to enable it.   
After that i realized that a user can only have a fair share for one  
group.   If cluster resources want it a parameter for it i can create  
it,  What should be the name of the parameter?

FSSECONDARYGROUPS TRUE


> Douglas Wightman wrote:
>>
>> --------------------------------------------------------------------- 
>> ---
>>
>> Subject:
>> Re: [Mauiusers] Maui/Torque and Fairshare with secondary Unix groups
>> From:
>> Bas van der Vlies <basv at sara.nl>
>> Date:
>> Mon, 7 Jan 2008 16:55:59 +0100
>> To:
>> "mauiusers at supercluster.org" <mauiusers at supercluster.org>
>>
>> To:
>> "mauiusers at supercluster.org" <mauiusers at supercluster.org>
>>
>>
>> I have made a patch for maui version:
>>  - 3.2.6p20-snap.1182974819
>>
>> The patch is different then that what i suggested. It does not make
>> use of the getgrouplist function nor it uses the FSGROUPS keyword.
>> When the configuration file is parsed. We know the FairShare groups.
>> When a user submits a job maui will now find out if it is a member of
>> one of this groups. If yes then set the group credentials to this
>> group and it will ignore then the PBS egroup variable.
>>
>> Maui did not parse the PBS group_list parameter:
>>  qsub -W group_list=<value>
>>
>> This variable has the highest priority and will set the group
>> credential to  the value specified. We at SARA want to ignore this
>> parameter, because we do not want an user to specify its own group  
>> and
>> bypass the FairShare targets. So we added an option to ignore this
>> parameter:
>>  - IGNPBSGROUPLIST          TRUE
>>
>> This patch also fix some debug problems that i had when the debug
>> level is set to another value then the default one, eg:
>>  - LOGLEVEL 9
>>
>> Maui has a lot of keywords and the parsing of the keywords are  
>> only be
>> shown if a certain loglevel is set. Somebody thought it is nice that
>> the maui keyword table is sorted alphabetically. This is not correct
>> because now only the keywords after the LOG... definitions are shown.
>> This patch moves the LOG.. keywords definition to the beginning of
>> keyword table and fix this problem.
>>
>> patch is attached
>>
>> regards,
>>
>>
>> Bas van der Vlies wrote:
>>> Happy new year everyone,
>>>
>>>   I had some time to examine the source of maui and found two  
>>> ways to
>>> implement this functionality in maui. So we are not depended anymore
>>> on the user primary group for Fairshare or specifying some
>>> commandline parameters for qsub.
>>>
>>> Here are the two solutions:
>>>   1) Introduce a new parameter: FSGROUPS, eg:
>>>      FSGROUPS one two three
>>>
>>>     When a job is submitted:
>>>      a) check if the user is a member of one of this groups
>>>      b) if yes then set the credentials to the matching group.
>>>
>>> 2) When a job is submitted:
>>>   a) get all the groups where the user is a member of with the
>>> "getgrouplist" function. I do not know if this function is available
>>> on all platforms.
>>>   b) Then check if for one of the user groups a fairshare target  
>>> is set
>>>   c) if yes then set the credentials to this group.
>>>
>>>
>>> Option 2 has more freedom but i do not know if all OS'es support the
>>> getgrouplist function.
>>>
>>> I want to know which solution the community thinks is the best one?
>>>
>>> Regards
>>>
>>> On Dec 10, 2007, at 3:05 PM, Bas van der Vlies wrote:
>>>
>>>> In our old setup the primary user group was the same as the
>>>> fairshare group
>>>> in Maui. so we did not have any problems with the Fairshare (FS)
>>>> setup.
>>>>
>>>> We are now switching to a new User administration system and use
>>>> the Linux
>>>> setup. Every user has as primary group his own group (user: bas,
>>>> group: bas).
>>>>
>>>> I have read some docs and the problem is that torque only sent the
>>>> primary
>>>> group to Maui. There are several setup's possible:
>>>>   * write a torque submit filter that adds -W or -A, But to my
>>>> knowledge
>>>> the  submit filter does not work for batch jobs that are submitted
>>>> via qsub -I
>>>>
>>>>   * In maui:
>>>>      USERCFG[bas] QDEF=SARA
>>>>      QOSCFG[SARA] FSTARGET=80.0
>>>>   But this is not very scalable. Every time a new user is added we
>>>> have to
>>>>   restart Maui and generate a line as above.
>>>>
>>>>   * Are there other solutions possible?
>>>>
>>>> What i am thinking about is to patch Maui, like the way Torque has
>>>> done it
>>>> with the acl_group_sloppy paramter or patch Torque to send also the
>>>> secondary groups of user.
>>>>
>>>> regards,
>>>>
>>>>
>>>> --
>>>> ******************************************************************* 
>>>> *
>>>> *                                                                   
>>>> *
>>>> *  Bas van der Vlies                     e-mail:  
>>>> basv at sara.nl      *
>>>> *  SARA - Academic Computing Services    phone:  +31 20 592  
>>>> 8012   *
>>>> *  Kruislaan 415                         fax:    +31 20  
>>>> 6683167    *
>>>> *  1098 SJ  
>>>> Amsterdam                                               *
>>>> *                                                                   
>>>> *
>>>> ******************************************************************* 
>>>> *
>>>> _______________________________________________
>>>> mauiusers mailing list
>>>> mauiusers at supercluster.org
>>>> http://www.supercluster.org/mailman/listinfo/mauiusers
>>>
>>> --
>>> Bas van der Vlies
>>> basv at sara.nl
>>>
>>>
>>>
>>> _______________________________________________
>>> mauiusers mailing list
>>> mauiusers at supercluster.org
>>> http://www.supercluster.org/mailman/listinfo/mauiusers
>>
>>
>> --------------------------------------------------------------------- 
>> ---
>>
>> _______________________________________________
>> mauiusers mailing list
>> mauiusers at supercluster.org
>> http://www.supercluster.org/mailman/listinfo/mauiusers
> _______________________________________________
> mauiusers mailing list
> mauiusers at supercluster.org
> http://www.supercluster.org/mailman/listinfo/mauiusers

--
Bas van der Vlies
basv at sara.nl





More information about the mauiusers mailing list