[Mauiusers] maui limits? looking for experience

[Arnau Bria]

> Idle limits are mostly good.  This mostly limits the number of each

> > users jobs that maui will consider in any scheduling cycle so it make
> > the scheduling cycle shorter/faster.  It also limits the priority
> > accumulated by queued jobs and alleviates 'queue stuffing'. I'd
> > recommend idle limits given that maui does not contain a better
> > facility to handle such issues.
>
> Yep, a short queue reduces maui stress. I completely agree that.
> Seting a limit of 100 jobs per user  leaves a 1k idle queue in normal
> behaviour, when many user are running jobs. That's the limit I'd use.
>
>
> But, as I've never tried this before, let me ask how maui will behave
> in this situation:
>
>
> if the farm is 70%, and I have only two users who have submited jobs
> (user A and B). User A has much more priority than user B, so let's say
> that the 30% must be filled with 25% of jobs from user A and 5% jobs
> from user B, if I have 1000 jobs in queue (500 from A and 500 from B)
> IDLE queue will contain 100 jobs of each user, so each scheduling
> cycle is going to schedule 200 jobs, is maui going to fill up the farm
> respecting our policies (25/5)? or is it going to start 100 jobs from
> each user on each scheduling cycle filling up the farm 15% and 15%?
>


I've worked in an example:

QoS Target:

 QOS
-------------
lhcatlas*        --  47.42     (user atprd)
magic*           --   3.80      (user maprd)

I set a limit of 5 jobs per user. I use only one node with 10/20/40 "cpus".
80 jobs each user.

np=10)

# qstat |grep R|grep -c map
5
# qstat |grep R|grep -c at
5

QOS
-------------
lhcatlas         50.00  47.42    50.00
magic*           50.00   3.80    50.00

np=20)
# qmgr -c "s n pbs01-test.pic.es np=20"
# qstat |grep R|grep -c map
10
# qstat |grep R|grep -c at
10
QOS
-------------
lhcatlas         50.00  47.42    50.00
magic*           50.00   3.80    50.00

np=40)
# qmgr -c "s n pbs01-test.pic.es np=40"
# qstat |grep R|grep -c ma
20
# qstat |grep R|grep -c at
20
QOS
-------------
lhcatlas         50.00  47.42    50.00   50.00   50.00
magic*           50.00   3.80    50.00   50.00   50.00


Without the limit FS is respected.

Maybe my conf is wrong, but this behaviour is why I'm afraid of limitng idle
queue.

Cheers,
Arnau
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.supercluster.org/pipermail/mauiusers/attachments/20110930/0f2b485b/attachment.html