[gold-users] goldd speedup

Stijn De Weirdt stijn.deweirdt at ugent.be
Mon May 31 07:52:57 MDT 2010


hi all,

i'm preparing a new system to setup gold (and some other tools), and i
took a bit of time to look at improving gold(d) speed.

first i (re)wrote a stress test tool (in attachement). it's a copy/paste
of gquote,gcharge and greserve and some simple code that quotes,reserves
and charges in a mixed/random way. (usage: ./stress_gold.pl <start>
<end>). maybe this could be useful for others or even in cluded in gold
(after a bit of a cleanup ;)

then i looked at goldd for further improvements. i started with
attaching strace to a running goldd to see what could be done there.
what i found was that still a whole number of stat/open/read syscalls
are executed.
one set is due to autoload/DESTROY when the forked child gets killed (no
clean db connection closing?). 
another one was related to UTF8. some digging later revealed that UTF8
in perl uses AUTOLOAD, so when the child is forked in goldd, this stuff
isn't loaded yet (so no COW benefits).

by adding the attached patch to goldd, this is all loaded before the
fork.

for people with perl 5.8.8 (version i tested), you also need a small
patch to make the SWASH caching work properly (fix "borrowed" from perl
5.10.0) (file to patch is eg /usr/lib/perl5/5.8.8/utf8_heavy.pl)

(you need to restart goldd to see any effect)


some timings i performed below:
timings are from stress_gold.pl 1 100

fix means fix applied
load means that at the same time 4 other stress_gold.pl processes were
running (to create some background noise)

the fix gives a rather easy 10% speedup. it's not much, but it's almost
free ;)

# no fix + load
charge avg_t : 0.615790704081633
charge max_t : 1.023987
charge min_t : 0.293908
quote avg_t : 0.619676540816326
quote max_t : 1.330601
quote min_t : 0.265317
reserve avg_t : 0.552087673469388
reserve max_t : 1.084353
reserve min_t : 0.275308

# fix + load
charge avg_t : 0.432632887755102
charge max_t : 0.669635
charge min_t : 0.223356
quote avg_t : 0.399230142857143
quote max_t : 0.613825
quote min_t : 0.208265
reserve avg_t : 0.398342734693877
reserve max_t : 0.793154
reserve min_t : 0.198987

# no fix + no load
charge avg_t : 0.159177602040816
charge max_t : 0.171299
charge min_t : 0.151309
quote avg_t : 0.152681765306122
quote max_t : 0.163476
quote min_t : 0.147303
reserve avg_t : 0.141935020408163
reserve max_t : 0.15936
reserve min_t : 0.135379


# fix + no load
charge avg_t : 0.127912255102041
charge max_t : 0.295243
charge min_t : 0.119437
quote avg_t : 0.113814785714286
quote max_t : 0.123436
quote min_t : 0.107484
reserve avg_t : 0.108878989795918
reserve max_t : 0.163271
reserve min_t : 0.103475




stijn

-- 
http://hasthelhcdestroyedtheearth.com/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: stress_gold.pl
Type: application/x-perl
Size: 13149 bytes
Desc: not available
Url : http://www.supercluster.org/pipermail/gold-users/attachments/20100531/e140c4ef/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: goldd_utf8_preload.patch
Type: text/x-patch
Size: 410 bytes
Desc: not available
Url : http://www.supercluster.org/pipermail/gold-users/attachments/20100531/e140c4ef/attachment-0001.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: utf8_heavy_cache_fix.patch
Type: text/x-patch
Size: 495 bytes
Desc: not available
Url : http://www.supercluster.org/pipermail/gold-users/attachments/20100531/e140c4ef/attachment-0002.bin 


More information about the gold-users mailing list