From: Xavier Shay <xavier-list@...>
Date: 2011-05-03T20:03:48+09:00
Subject: [ruby-core:35979] Re: require performance on 1.9

Hello,
I have more information, see below.

On 1/05/11 10:23 AM, Xavier Shay wrote:
> Hello,
> I am looking into the performance issues requiring files in 1.9.
> Here is a graph I am trying to fix:
> https://skitch.com/xaviershay/rh64d/tc-load-time
> https://gist.github.com/c8d0d422a9203e1fe492 (after creating 10000 files
> in the `f` dir)
>
> A commit from a few years back [1] adds some behaviour around
> loaded_feature_path. When disabled, the require time in my benchmark is
> halved [2]. There is no test covering this behaviour, and I can't quite
> figure out what it is doing.
>
> It is probably explained in ruby-dev:31932 but I can't read Japanese:
> http://markmail.org/message/d3ef53c63kx2xqkr
>
> Can anyone either translate the above message for me, or provide a test
> case that covers the expected behaviour?
>
> Cheers,
> Xavier
>
> P.S. This may or many not be related to this issue:
> http://redmine.ruby-lang.org/issues/show/3924
>
> [1]
> https://github.com/ruby/ruby/commit/bb1f1c57823758d2450d184ce7c85beb0d538bf0
>
>
> [2] add continue; to load.c:161 gives 3s for n=2500
>

I have tracked down the remaining performance loss from the above graph 
to load.c:152. On every require, StringValuePtr is called on every 
loaded feature. ruby18 does this also, but my hunch is this is more 
expensive in 1.9 perhaps because of encoding or something?

Can loaded_features ever shrink? Can a value in loaded_features change? 
Is there a sane way to cache this value? I tried keeping a really stupid 
cache of the return values in a char** but that idea crashed and burned.

In summary, there are two problems that are making ruby 1.9 requires so 
much slower than 1.8:
1) The loaded_feature_path function, which I cannot find a use for 
(load.c:158)
2) The repeated calling of StringValuePtr (load.c:152) for every 
loaded_feature on every require

With the above two code paths disabled, require time is back down to 
ruby18 levels, as per this graph*:
https://skitch.com/xaviershay/r7jid/tc-load-time

Any ideas for progressing on #2?

Cheers,
Xavier

* of course, disabling #2 is definitely invalid, this is just for sake 
of demonstration)