[gold-users] Gold system getting bogged down
Stijn De Weirdt
stijn.deweirdt at ugent.be
Fri Dec 4 10:28:44 MST 2009
> What can be done to the post processing process, gold daemon or the
> gold database to speed up the processing of these gcharge records? I
> wish these gcharge commands would take less than a second. Maybe they
> have always processed in more than 1 second, maybe even more but when
> we had 1000-2000 or less jobs is wasn’t really noticed.
you can postprocess the gcharge commands in parallel. gold should be ok
with a few threads (but not to many), depending on the hardware that
gold runs on.
but more importntly is speeding up the individual gcharge commands.
best things to do first is to restart gold daemon in debug logging mode,
execute a few queries and look at the timings in the logs where all that
time is spend. you should be able to clearly distinguish the remote
actions (gcharge stuff, between end of goldd interaction and beginning
of a new one); perl stuff (goldd that is not a DB query) and DB queries.
not much can be doen on the first two (except maybe using recent perl
version, eg when you are using some rhel5 version, make sure the perl is
to date (5.3 came with a slow perl)).
if much time is spend in the DB, tuning the DB is needed (set some cache
parameters correctly etc etc); we are using postgres, and i have 2 extra
indices for the schema that speed up the queries. (see an other mail i
post on this list)
> These records in Gold are being used to copy information to another
> postgres database using gold commands to collect the info and then
> subsequently these records are posted to an off site database.
what do you use for this. you can use slony for offsite replication,
which should be low on impact (once the bulk data is synced). we had a
test HA setup working using slony and gold, so that's fine i guess.
> With these three things going on (charge posting, processing charges
> into the other database, and sending the charges offsite) gold is
> doing quite a bit and seems to get bogged down with these requests.
> Also gold is getting queried at the job submission to make sure there
> is time available in the account. All this is putting pressure on
> gold and it seems to be slowing significantly (or noticeably I guess)
> under the pressure.
> Thanks for any help you can provide.
> Victor Hazlewood, CISSP
> Senior HPC Systems Analyst
> National Institute for Computational Science
> University of Tennessee
> gold-users mailing list
> gold-users at supercluster.org
More information about the gold-users