From: mk <noreply@...>
Date: 2022-11-29T15:00:52+00:00
Subject: [ruby-core:111065] [Ruby master Bug#19156] ObjectSpace.dump_all segfault during string inspection

Issue #19156 has been updated by mk (Matthias K�ppler).





I'm not sure this makes sense but: we know the address being referenced is somewhere (half-way) into a memory region that is a memory-mapped file using that Prometheus library's binary encoding.



This is the file:



```

xxd gauge_max_puma_0-0.db

00000000: 6800 0000 0000 0000 5100 0000 5b22 6465  h.......Q...["de

00000010: 706c 6f79 6d65 6e74 7322 2c22 6465 706c  ployments","depl

00000020: 6f79 6d65 6e74 7322 2c5b 2272 6576 6973  oyments",["revis

00000030: 696f 6e22 2c22 7665 7273 696f 6e22 5d2c  ion","version"],

00000040: 5b22 6335 3365 6336 3336 6435 3022 2c22  ["c53ec636d50","

00000050: 3135 2e37 2e30 2d70 7265 225d 5d20 2020  15.7.0-pre"]]   

00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................

00000070: 0000 0000 0000 0000 0000 0000 0000 0000  ................

...

00000ff0: 0000 0000 0000 0000 0000 0000 0000 0000  ................

```



We can see there is only a single sample in here, the application version (we track this into a metric).



The library zeroes these files out with a minimum size of 4KB. So somehow there is an object slot in MRI's `ObjectSpace` that shares an address with this empty mmap'ed space? That doesn't make any sense. Of course it would explain why it doesn't pass the test "are you a valid string", but how did we even get here to begin with?



`prometheus-client-mmap` also doesn't use `str_new_static`, but it does allocate Ruby strings in its C extension e.g. here: https://gitlab.com/gitlab-org/prometheus-client-mmap/-/blob/b4eae83927efd5132b9b2726f21d9a3ea49e9336/ext/fast_mmaped_file/fast_mmaped_file.c#L81



I'm not sure at this point whether this is just another red herring.



----------------------------------------

Bug #19156: ObjectSpace.dump_all segfault during string inspection

https://bugs.ruby-lang.org/issues/19156#change-100325



* Author: mk (Matthias K�ppler)

* Status: Open

* Priority: Normal

* ruby -v: ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux]

* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN

----------------------------------------

I am working on a feature that would allow our application to capture heap dumps during shutdown for later inspection.



These heap dumps are captured via `ObjectSpace.dump_all(output: io)`. While walking the object space, MRI occasionally segfaults while inspecting string objects in `search_nonascii` of `string.c`:



```

/usr/local/lib/ruby/3.0.0/objspace.rb:87: [BUG] Segmentation fault at 0x00007efee4201000

ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-linux]

...



-- Control frame information -----------------------------------------------

c:0053 p:---- s:0312 e:000311 CFUNC  :_dump_all

c:0052 p:0130 s:0305 e:000304 METHOD /usr/local/lib/ruby/3.0.0/objspace.rb:87

c:0051 p:0023 s:0295 e:000294 METHOD /home/git/gitlab/lib/gitlab/memory/reports/heap_dump.rb:26

...



-- C level backtrace information -------------------------------------------

/usr/local/lib/libruby.so.3.0(rb_print_backtrace+0x11) [0x7efee4ad0c5e] vm_dump.c:758

/usr/local/lib/libruby.so.3.0(rb_vm_bugreport) vm_dump.c:998

/usr/local/lib/libruby.so.3.0(rb_bug_for_fatal_signal+0xf8) [0x7efee48d0b08] error.c:787

/usr/local/lib/libruby.so.3.0(sigsegv+0x55) [0x7efee4a23db5] signal.c:963

/lib/x86_64-linux-gnu/libpthread.so.0(__restore_rt+0x0) [0x7efee4f12140] ../sysdeps/pthread/funlockfile.c:28

/usr/local/lib/libruby.so.3.0(search_nonascii+0x30) [0x7efee4a3ca60] string.c:552

/usr/local/lib/libruby.so.3.0(coderange_scan) string.c:585

/usr/local/lib/libruby.so.3.0(enc_coderange_scan+0x1b) [0x7efee4a3e28a] string.c:709

/usr/local/lib/libruby.so.3.0(rb_enc_str_coderange) string.c:727

/usr/local/lib/ruby/3.0.0/x86_64-linux/objspace.so(is_broken_string+0x8) [0x7efeced9c304] ../../internal/string.h:116

/usr/local/lib/ruby/3.0.0/x86_64-linux/objspace.so(dump_object) objspace_dump.c:388

/usr/local/lib/ruby/3.0.0/x86_64-linux/objspace.so(heap_i+0x39) [0x7efeced9caa9] objspace_dump.c:521

/usr/local/lib/libruby.so.3.0(objspace_each_objects_without_setup+0xaf) [0x7efee48e878f] gc.c:3232

/usr/local/lib/libruby.so.3.0(objspace_each_objects_protected+0x14) [0x7efee48e87c4] gc.c:3242

/usr/local/lib/libruby.so.3.0(rb_ensure+0x12a) [0x7efee48d96aa] eval.c:1162

/usr/local/lib/libruby.so.3.0(objspace_each_objects+0x28) [0x7efee48fb458] gc.c:3310

/usr/local/lib/libruby.so.3.0(rb_objspace_each_objects) gc.c:3298

/usr/local/lib/ruby/3.0.0/x86_64-linux/objspace.so(objspace_dump_all+0x88) [0x7efeced9b068] objspace_dump.c:616

...

```



Unfortunately I couldn't get my hands on that memory region to see which strings are causing this since this doesn't always happen.



I suspect this is also a problem with MRI master since the code looks unchanged from 3.0.4.







-- 

https://bugs.ruby-lang.org/

 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/