From: Eric Wong Date: 2014-08-04T01:10:49+00:00 Subject: [ruby-core:64182] Re: [ruby-trunk - Feature #10096] [PATCH] use khash for fstring and id_str tables ko1@atdot.net wrote: > Eric Wong wrote: > > frozen_strings and global_symbols.id_str hashes are two of the bigger > > hashes in Ruby. They are needlessly ordered and incurs malloc overhead > > in every st_table_entry. > > We will change global_symbols.id_str (simple array) soon. > So do not touch it. > > ID is now includes serial number. So use it. > But there are several issues to apply this simple approach. > For example, some IDs share same serial number. > But we (Nobu and I) discussed to solve them, and made up solutions. OK, I look forward to your changes. > > Use an unordered open-addressing table which incurs no additional malloc > > overhead besides the (larger) table itself. > > > > > Reduces "ruby -e exit" (w/RubyGems) by ~200K on eglibc malloc on > > amd64 Debian stable due to having fewer allocations > > Please write "from which size". > 100MB -> (100MB-200KB) is trivial. 300KB->100KB is great work. The results go from around ~9300KB -> ~9100KB or so. I think the variation is due to hash randomization, but there may be other factors. It used to be in the ~7300KB range a few months ago but I think GC changes increased usage of initial startup... > > I chose khash because it is flexible and has (IMHO) a good API. > > The API should also be familiar to mruby hackers, as mruby uses > > a version of khash. > > Did you check performance? > I know mruby uses it (and maybe more products). > So I assume it is sophisticated. How do you feel? Our benchmark suite doesn't reveal much: http://80x24.org/bmlog-20140803-231204.7455 Adding an unrealistic micro-bench shows st is faster: http://80x24.org/fstring_iter_st.patch http://80x24.org/fstring_iter_khash.patch $ FSTRING_ITER=1000000 /usr/bin/time ./ruby.st -e exit 1.30user 0.00system 0:01.30elapsed 99%CPU (0avgtext+0avgdata 9240maxresident)k 0inputs+0outputs (0major+1348minor)pagefaults 0swaps $ FSTRING_ITER=1000000 /usr/bin/time ./ruby.khash -e exit 1.60user 0.00system 0:01.61elapsed 99%CPU (0avgtext+0avgdata 9108maxresident)k 0inputs+0outputs (0major+1306minor)pagefaults 0swaps Memory usage is from fstring change only, id_str untouched. I don't think the performance is noticeable in real world, and the memory/pagefault reduction is not big, either. Your call :)