[ruby-core:70612] [Ruby trunk - Bug #11490] Allocation tracer sometimes attributes allocations to the wrong source file/line

Issue #11490 has been updated by Godfrey Chan.


Thinking about it more, it seems my original hunch is *probably* correct: w=
hen loading/parsing/evaluating the file, the VM has to allocate a large amo=
unt of objects (e.g. when it runs `def zomg; ...; end`, it needs to create =
the "immortal"symbol `:zomg`, among other things like `InstructionSequence`=
, etc). It then attributed the allocation of these objects to the closest R=
uby frame. Since the "raw" `Kernel#require` is implemented in C, whoever ca=
lls it will end up "getting the blame".

That is fine in an ideal world =E2=80=93 this would point to the line where=
 you `require`d the file with lots of code (e.g. the `require "nokogiri"` l=
ine), and that is at least somewhat useful (I can save X MB if I don't use =
nokogiri, and you can choose to make that tradeoff; although that is rarely=
 a real problem). Unfortunately, **all** Rubies (C Ruby) have at least one =
`Kernel#require` override in Ruby-land (in RubyGems), so in practice, this =
information is always going to point you to the "wrong" place. Most commonl=
y, it will be [this line](https://github.com/rubygems/rubygems/blob/ae1718f=
d2d0e238940c38d56e32c56bffe083894/lib/rubygems/core_ext/kernel_require.rb#L=
54) in RubyGems, but it could also be [this line](https://github.com/rails/=
rails/blob/cae2b5bb59212961c4a35c939381ebece48d1177/activesupport/lib/activ=
e_support/dependencies.rb#L300) in Active Support or a similar place in bun=
dler.

I'm not sure what's the best way to go about fixing this. Perhaps it should=
 not report any allocation source/line for these internal VM things (the sy=
mbols, instruction sequence, etc), report it as "(VM internal):0", or perha=
ps point it to the corresponding line in the source (`:zomg` is "allocated"=
 by the `def zomg` line).

Or perhaps it is just not a big deal and not worth the effort. If my explan=
ation is right, I can accept that this is technically "correct", but eviden=
tly people have tried to use this information to "slim down" their app and =
this has caused them to go down the wrong rabbit hole.

----------------------------------------
Bug #11490: Allocation tracer sometimes attributes allocations to the wrong=
 source file/line
https://bugs.ruby-lang.org/issues/11490#change-54018

* Author: Godfrey Chan
* Status: Open
* Priority: Normal
* Assignee:=20
* ruby -v:=20
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
See the reproduction script in https://gist.github.com/chancancode/dc175e70=
2c02cdfa5ffb

This was originally brought to my attention via https://github.com/skylight=
io/skylight-ruby/issues/36

As you can see in the Github ticket, we overwrote `Kernel#require` and was =
incorrectly blamed for allocating a lot of objects/memory. I checked the so=
urce and saw that the file/line info is filled by the [last Ruby-land contr=
ol frame](https://github.com/ruby/ruby/blob/7cf523c7db67c22ffc09b38a9c5bea0=
57f578db2/vm_trace.c#L752), so I was almost ready to accept this as an unfo=
rtunate artifact that cannot be fixed.

However, when I made the standalone reproduction script to test my theory, =
I noticed that it actually gets it right sometimes (the lines that reports =
`nokogiri.bundle:0` instead of `allocation_tracker_bug.rb:11`), so it seems=
 like something else is going on here.

By the way, since RubyGems overwrites `Kernel#require`, even without any ot=
her third-party libraries loaded, you can already observe allocation tracer=
 attributing a lot of allocations to RubyGem's `kernel_require.rb` out of t=
he box.

---Files--------------------------------
allocation_tracker_bug.rb (786 Bytes)


--=20
https://bugs.ruby-lang.org/