[gold-users] Default projects for users

Steve Crusan scrusan at ur.rochester.edu
Wed Sep 29 16:23:46 MDT 2010


Hmmm, we don't use Moab right now, and I'm not sure if Maui actually
supports the IDCFG configuration. I'm just going to restart maui until I get
a submit filter in place, as well as figure out how this is going to work
being submitted from bluegene's loadleveler also...ugh


Also, one other strange thing I noticed the accounts. Some of our users run
job arrays (supported in our version of torque), but I've been seeing some
odd default_project behavior:
Job Charge 494941[1] rglor_group jng6 -14.98 2010-09-29 15:40:04
Job Charge 494941[2] federal_reserve jng6 -16.38 2010-09-29 16:02:42
Job Charge 494941[3] federal_reserve jng6 -18.96 2010-09-29 16:38:31
Job Charge 494941[4] federal_reserve jng6 -17.25 2010-09-29 16:43:33
Job Charge 494941[5] federal_reserve jng6 -17.45 2010-09-29 17:18:58
Job Charge 494941[6] federal_reserve jng6 -18.02 2010-09-29 17:33:37
Job Charge 494941[8] federal_reserve jng6 -17.30 2010-09-29 17:49:51
Job Charge 494941[7] federal_reserve jng6 -18.17 2010-09-29 17:52:49
Job Charge 494941[9] federal_reserve jng6 -15.13 2010-09-29 17:56:14


Basically, the first element in his job array gets the proper default
account, but the rest do not.

I appreciate the help.


On 9/27/10 7:06 PM, "Scott Jackson" <scottmo at adaptivecomputing.com> wrote:

> Yea, I think so. I think both Moab and Maui do a lookup into Gold for a list
> of default accounts (projects) when they startup --- and I don't think they
> resynchronize after that. I believe that Moab can handle this better through
> the IDCFG files where you can specify the refresh period.

Scott


-----
> Original Message -----
From: "Steve Crusan" <scrusan at ur.rochester.edu>
To:
> "Gold Users Mailing List" <gold-users at supercluster.org>
Sent: Monday,
> September 27, 2010 8:21:26 AM
Subject: Re: [gold-users] Default projects for
> users

Scott,


    I think I figured out the issue. When I've restarted the
> Maui daemon,
the changes to the default project are honored. Does Maui have
> some sort of
internal user name/account caching system?

    If restarting
> Maui is indeed the 'fix' is there are better way to do
this? Even though Maui
> becomes fully functional after a minute or so, and
job submission works, I
> wish there would be a better way to honor
project/group changes. Something
> along the lines of nscd -i group...


    Thanks!


On 9/23/10 1:40 PM, "Scott
> Jackson" <scottmo at adaptivecomputing.com> wrote:

> Steve,

I'm missing
> something here. Perhaps Maui is running a quote request
> that I am not asking
> for which is failing (but I didn't think Maui did this,
> just Moab). Please
> just send me your entire goldd.log after reproducing the
> problem and let me
> know the jobid. I will sift through it and see what I can
> determine.

scp it
> to guest at clusterresources.com: (password:
> guest)

Scott


----- Original
> Message -----
From: "Steve Crusan"
> <scrusan at ur.rochester.edu>
To: "Gold
> Users Mailing List"
> <gold-users at supercluster.org>
Sent: Thursday, September
> 23, 2010 11:17:24
> AM
Subject: Re: [gold-users] Default projects for
> users

Scott,


Here are the
> results from the commands:

    gbalance -u
> vchaban -p federal_reserve -m
> bhsn-int.bluehive.crc.private
Id Name
> Amount        Reserved
> Balance       CreditLimit
Available     
--
> --------------- -------------
> --------- -------------
> -----------
------------- 
2  federal_reserve
> 3599895302723 121500000
> 3599773802723     3600000
3599777402723 


gbalance -u
> vchaban
Id Name
> Amount        Reserved  Balance
> CreditLimit
Available     
--
> --------------- ------------- ---------
> -------------
> -----------
------------- 
2  federal_reserve 3599895302723
> 121500000
> 3599773802723     3600000
3599777402723 
77 oprezhdo_group
> 35871153
> 0      35871153           0
35871153 


glsaccount -u
> vchaban
Id Name
> Amount        Projects        Users
> Machines
Description    
--
> --------------- ------------- ---------------
> -------
> --------
-------------- 
2  federal_reserve 3599895302723
> federal_reserve
> ANY     ANY
Auto-generated 
77 oprezhdo_group       35871153
> oprezhdo_group
> MEMBERS ANY
Auto-generated 




I've attached the gold
> response envelope
> (it's long). I could only find it
after grepping for the
> latest jobID the
> vchaban user ran.

Thanks for the help.


~Steve

On 9/23/10
> 11:45 AM,
> "Scott Jackson" <scottmo at adaptivecomputing.com> wrote:

> Great
> Steve,

That
> helps. Can I also get the corresponding response from Gold
>
> (Response
> Envelope).

Also, can you show me the result of:

gbalance -u
>
> vchaban -p
> federal_reserve -m bhsn-int.bluehive.crc.private

(after fixing
> any
>
> typos:)

and gbalance -u vchaban
and glsaccount -u
>
>
> vchaban

Thanks,

Scott


----- Original Message -----
From: "Steve
> Crusan"
>
> <scrusan at ur.rochester.edu>
To: "Gold Users Mailing List"
>
>
> <gold-users at supercluster.org>
Sent: Thursday, September 23, 2010 9:32:37
>
>
> AM
Subject: Re: [gold-users] Default projects for users

Scott,

    I've
>
>
> attached the requested information. This is all I could find before
the
> SQL
>
> queries started.

    Thanks!


~Steve

On 9/22/10 7:40 PM, "Scott
> Jackson"
>
> <scottmo at adaptivecomputing.com> wrote:

> Steve,

Per my
> understanding, Moab
>
> will not use the fallback account until
> after it has
> tried an initial
>
> attempt at making a reservation against the
> user's
> normal
> credentials.

What
> I would need to see is the Request Envelope
>
> and the
> Response Envelope from
> the goldd.log for the initial reservation
>
> attempt
> from Moab that fails (and
> which spurs Moab to make the second
>
> reservation
> attempt against the fallback
> account).

Scott


-----
> Original
> Message
> -----
From: "Steve Crusan"
>
> <scrusan at ur.rochester.edu>
To: "Gold Users
>
> Mailing List"
>
> <gold-users at supercluster.org>
Sent: Wednesday, September 22,
>
> 2010
> 2:50:36
> PM
Subject: Re: [gold-users] Default projects for
>
> users

Scott,


> This is
> the query which is causing the project/account
> to
> be set to
> the
reserve
> account (see the >>>):

SELECT g_account.g_id
> AS
>
>
> g_id,g_account_project.g_name AS
g_name,g_account_user.g_name
> AS
>
>
> g_name,g_account_machine.g_name AS g_name
FROM g_account,
> g_account_user,
>
>
> g_account_project, g_account_machine,
g_allocation WHERE
> (
>
>
> g_allocation.g_account=g_account.g_id
>
>
>
> AND
g_account_project.g_account=g_account.g_id
>
>
>
> AND
g_account_user.g_account=g_account.g_id
>
>
>
> AND
g_account_machine.g_account=g_account.g_id
>
>
>
> AND
g_allocation.g_call_type='Normal'
>
>
>
> AND
g_allocation.g_start_time<='1285184623'
>
>
>
> AND
g_allocation.g_end_time>'1285184623' AND  ( (
>
>
>
> (
g_account_project.g_name='ANY' AND g_account_project.g_access='True' )
>
>
>
> OR
>>>>( g_account_project.g_name='reserve'
>
>
>
> AND
g_account_project.g_access='True' ) )  AND  ( (
>
>
>
> g_account_user.g_name='ANY'
AND g_account_user.g_access='True' )  OR  (
>
>
>
> g_account_user.g_name='vchaban'
AND g_account_user.g_access='True' )  OR
>
> (
>
> g_account_user.g_name='MEMBERS'
AND g_account_user.g_access='True' ) )
>
> AND
> (
> (
g_account_machine.g_name='ANY' AND
>
> g_account_machine.g_access='True' )
> OR
(
>
>
> g_account_machine.g_name='bhsn-int.bluehive.crc.private'
>
>
>
> AND
g_account_machine.g_access='True' )  OR
>
>
>
> (
g_account_machine.g_name='MEMBERS' AND g_account_machine.g_access='True'
>
> )
>
> )
)  ) AND g_account.g_deleted!='True' AND
>
>
>
> g_account_user.g_deleted!='True'
AND g_account_project.g_deleted!='True'
>
>
>
> AND
g_account_machine.g_deleted!='True' AND
>
>
>
> g_allocation.g_deleted!='True';



This seems to me that the when the new
>
>
> user
> is being properly added to an
account. I can find the user in the
>
>
>
> g_project_user SQL table, as well as the
g_user table, and in both of them
>
>
> the
> default project is set correctly.


On 9/22/10 3:42 PM, "Scott
>
>
> Jackson"
> <scottmo at adaptivecomputing.com> wrote:

> OK, now everything
>
> seems
> pretty
> self-consistent now. Let me try to understand
> the problem
>
> better.
> After you
> made your modifications are you still having the
>
>
> problem?

You
>
> said:

Whenever I run a
> job (We don¹t have our users run
>
> use
> the ­A pbs
>
> directive), the user¹s job is
> always charged to the
>
> fallback
> account
>
> (reserve). Other users that I have
> created in the
>
> past via a large
>
> import
> script correctly use their default
> project,
>
> and not the
> Œfallback¹
>
> account.

If you are still seeing a problem,
>
> please review the
> goldd.log so
>
> you can see the reservation request
> that
> Moab is trying to
> make with Gold
> and
> let me know the error or
> problem
> that Gold is returning
> that is causing
> Moab
> to use the
> fallback
>
> account.

Thanks,

Scott




----- Original
> Message
>
> -----
From: "Steve
>
> Crusan" <scrusan at ur.rochester.edu>
To: "Gold
> Users
> Mailing
> List"
>
> <gold-users at supercluster.org>
Sent: Wednesday, September
>
> 22, 2010
>
> 1:31:50
> PM
Subject: Re: [gold-users] Default projects for
> users

Hi
>
> Scott,


> Yes,
> I noticed the issue with the MEMBERS user in
> the project,
>
> which
>
> was
incorrect. Had to fix the script for that, but
> I've removed
>
> that.


>
> Also, here is the answer to your first
> question:

$ glsaccount
> -p
>
>
> oprezhdo_group
Id Name           Amount
> Projects       Users
> Machines
>
>
> Description
-- -------------- --------
> -------------- -------
> --------
>
>
> --------------
77 oprezhdo_group
> 35886449 oprezhdo_group
> MEMBERS ANY
>
>
> Auto-generated

Now, when I
> created this account, I actually
> created the
>
>
> project w/
> the
--createAccount=yes option.

Also, I've fixed
> the
>
>
> 'space'(actually
> a formatting issue) in the glsproject
-p
>
> oprezhdo_group:
$
>
> glsproject
> -p oprezhdo_group --show Name,Users
Name
>
>
> Users
--------------
>
> ------------------------------
oprezhdo_group
>
>
>
>
> oprezhdo,vchaban_guest,vchaban


When I do an ls on the user:
$
> glsuser
>
>
>
> vchaban --show Name,Active,DefaultProject
Name
> Active
>
>
>
> DefaultProject
------- ------ --------------
vchaban
> True
>
>
>
> oprezhdo_group


Everything in these commands is obv backed up
> when using
>
>
> the
> goldsh shell
for ProjectUser, Project, etc. I've
> recreated the user
>
> and
> such,
> and made
sure to explicitly add the user
> to the project, as
> well
> as
> activate
> them.





On 9/22/10 3:17 PM,
> "Scott Jackson"
>
>
> <scottmo at adaptivecomputing.com>
> wrote:

> Steve,

I
> would bet that the
>
>
> problem has more to do with the way the
> Account
>
> was created than the
>
>
> Project.

First of all, MEMBERS should not be
> in
> the
> Project User
> list.
>
> This does not mean anything here. The list of
>
> users in the
>
> project are
> the
> very definition of MEMBERS, which can be
> used
> in the
>
> AccountProject
> field
> for the associated accounts.

I
> notice that
> user10
> does
> not have a
> default
> project. It kind of
> sounded like you wanted
>
> him to.

I also
> see
> that there
> is a space
> between MEMBERS,user1 and
> the
> user10 in the
>
> glsproject. This
> suggest
> either user10 was inputted
> somehow
> with a space
> in
> front of its
> name
> (could it have been quoted
> with an
> additional
> space?), or it
> was
>
> inputted as a machine member and
> not a user
> member,
> or it is a
>
>
> transcription/cut-paste error in the
> email. If it is one
> of
> the first
> two,
> you
> probably will get undesired
> behavior.

Can you do
>
>
> glsaccount -p
> oprezhdo_group
> so we can see how
> the accounts'
>
> AccountUsers
> are
>
> defined?

Thanks,

Scott


-----
> Original Message
>
> -----
From: "Steve
>
> Crusan"
>
> <scrusan at ur.rochester.edu>
To: "Gold
> Users
> Mailing List"
>
>
>
> <gold-users at supercluster.org>
Sent: Wednesday,
> September
> 22, 2010
> 12:26:56
>
>
> PM
Subject: [gold-users] Default
> projects for
> users


Default
> projects for
>
>
> users Hi All,

Having some
> strange
> occurances with new
> users I add to
>
>
> systems. I have setup a
> fallback
> account called
> Œreserve¹ in gold. Each
> user
>
> that I add to
> the system is
> given a
> default project group. When I
> list the
>
> user,
> it shows the
> default
> group properly:

$ glsuser -u user10
>
Name
>
> Active
> CommonName
>
> PhoneNumber EmailAddress DefaultProject
>
> Description
>
------- ------
>
>
> ---------- ----------- ------------
>
> --------------
> ------------
user10
>
> True
> bme_group User Account


Now,
> if
> I list the
> project, the user is
>
> also there:
>
[scrusan_admin at gold
>
> netbackup]$ glsproject
> -p
>
> oprezhdo_group
Name Active
> Users Machines
>
> Description
--------------
>
>
> ------
>
> --------------------------------------
> --------
>
>
>
>
> ---------------------------------------------
bme_group True
>
>
>
> MEMBERS,user1,
>
> user10 cluster Project/account for bme_group LDAP
>
>
> group
>


Whenever I run a
>
> job (We don¹t have our users run use the ­A
>
>
> pbs
> directive), the user¹s job
> is
> always charged to the fallback
>
>
> account
> (reserve). Other users that I
> have
> created in the past via a
>
>
> large import
> script correctly use their
> default
> project, and not the
>
>
> Œfallback¹
> account.

Anyone have similar
> problems? This
> is only
>
> seeming
> to happen
> with newer accounts. My thinking is
> that these new
>
>
> users
> aren¹t Œproper¹
> members of the respective projects,
> which then
> map
> to
>
> the accounts
> themselves, but that does seem to be the
> case,
> at
> least from
> the
> commands
> I¹ve run. That would explain the
> behavior,
>
> because if I try
> to use
> the ­A
> flag from a job submission
> for a group I
> am
> not a part of,
> I get the
>
> fallback
>
> account...



----------------------
>
Steve
> Crusan
System
>
>
>
> Administrator
Center for Research Computing
>
University of
>
>
> Rochester
>
>
https://www.crc.rochester.edu/
>
>
>

__________________________
> 
> 
> _____________________
gold-users mailing
>
>
>
>
>
> list
gold-users at supercluster.org
http://www.supercluster.org/mailman/listinfo/
> 
> 
>
>
>
> 
> 
>
> gold-users
_______________________________________________
gold-users
>
>
>
>
> mailing
>
>
> 
> 
>
> list
gold-users at supercluster.org
http://www.supercluster.org/mailman/listinfo/
> 
> 
>
>
>
> gold-users




----------------------
Steve Crusan
System
>
>
>
>
> Administrator
Center for Research Computing
University of
>
>
>
>
> Rochester
https://www.crc.rochester.edu/

____________________________________
> 
> 
>
>
> ___________
gold-users mailing
>
>
>
>
> list
gold-users at supercluster.org
http://www.supercluster.org/mailman/listinfo/
> 
> 
>
>
> 
> 
> gold-users
_______________________________________________
gold-users
>
>
>
> mailing
>
> 
> 
>
> list
gold-users at supercluster.org
http://www.supercluster.org/mailman/listinfo/
> 
> 
>
>
> gold-users




----------------------
Steve Crusan
System
>
>
>
> Administrator
Center for Research Computing
University of
>
>
>
> Rochester
https://www.crc.rochester.edu/

____________________________________
> 
> 
>
> ___________
gold-users mailing
>
>
>
> list
gold-users at supercluster.org
http://www.supercluster.org/mailman/listinfo/
> 
> 
>
> 
> gold-users
_______________________________________________
gold-users
>
>
> mailing
> 
> 
> 
> list
gold-users at supercluster.org
http://www.supercluster.org/mailman/listinfo/
> 
> 
>
> gold-users




----------------------
Steve Crusan
System
>
>
> Administrator
Center for Research Computing
University of
>
>
> Rochester
https://www.crc.rochester.edu/


___________________________________
> 
> 
> ____________
gold-users mailing
>
>
> list
gold-users at supercluster.org
http://www.supercluster.org/mailman/listinfo/
> 
> 
> gold-users
_______________________________________________
gold-users
>
> mailing
> 
> 
> list
gold-users at supercluster.org
http://www.supercluster.org/mailman/listinfo/
> 
> 
> gold-users




----------------------
Steve Crusan
System
>
> Administrator
Center for Research Computing
University of
>
> Rochester
https://www.crc.rochester.edu/


___________________________________
> 
> ____________
gold-users mailing
>
> list
gold-users at supercluster.org
http://www.supercluster.org/mailman/listinfo/
> 
> gold-users
_______________________________________________
gold-users
> mailing
> 
> list
gold-users at supercluster.org
http://www.supercluster.org/mailman/listinfo/
> 
> gold-users




----------------------
Steve Crusan
System
> Administrator
Center for Research Computing
University of
> Rochester
https://www.crc.rochester.edu/

____________________________________
> ___________
gold-users mailing
> list
gold-users at supercluster.org
http://www.supercluster.org/mailman/listinfo/
> gold-users
_______________________________________________
gold-users mailing
> list
gold-users at supercluster.org
http://www.supercluster.org/mailman/listinfo/
> gold-users




----------------------
Steve Crusan
System Administrator
Center for Research Computing
University of Rochester
https://www.crc.rochester.edu/



More information about the gold-users mailing list