Re: Symbols overlap ordinary objects, especially on OS X (Was: Re: Problem with weak references on OS X 10.3)

On Wed, Feb 22, 2006 at 02:21:24PM +0900, Eric Hodel wrote:
> On Feb 5, 2006, at 5:05 AM, Mauricio Fernandez wrote:
> >I'd conjecture that the above works on Linux because glibc's malloc() always
> >returns 8-byte aligned memory addresses, which doesn't seem to be  
> >the case in
> >OSX:
> >
> > 0x1d421c % 8                                      # => 4
> 
> OS X's malloc aligns memory on 16 byte boundaries.  This problem is  
> not unique to OS X, you just need enough symbols.

I was being stupid. I forgot that sizeof(RVALUE) % 8 = 4, so the
alignment of the heap slots doesn't matter.

> >Another possibility would be that the address space for the data  
> >segment
> >used in OSX is lower than on Linux, so the SYM2ID matches an existent
> >symbol:
> 
> If you're close enough to the beginning of memory ObjectSpace#_id2ref  
> will pick a Symbol over the real object like you mention above:

On my system the values returned by malloc start around 0xb7d00000 (when using brk)
so I can hardly create enough symbols to force a collision, but I can get
a crash on Linux with the following:

batsman@tux-chan:~/mess/current$ cat symbols_overlap.rb
Objs = (1..10000).map{ Object.new }
STR = 'b!aaaaaaaa'

def UNSIGNED(i)
  # too lazy to do it properly
  (0..(8*i.size-2)).map{|n| i[n]}.inject([0,1]){|(s,n),x| [s + x * 2**n,n+1]}.first
end

def ID2ULONG(i)
  i < 0 ? UNSIGNED(i) : i
end

$min = Symbol.all_symbols.sort_by{|s| s.object_id}.first

def symbol_info
   min_id, max_id = [$min, $max].map{|s| ID2ULONG(s.object_id)}
   puts "last: #{$max.object_id} ===>  #{Objs.object_id}"
   puts "extrema:   id: 0x%010x      ----   id: 0x%010x" % [min_id, max_id]
   puts "       symbol:   %10p      ----      %10p" % [$min, $max]
   pending_before_crash = ((UNSIGNED(Objs.object_id) - max_id) / 2048)
   puts "About ~%d symbols expected before crash" % pending_before_crash
   pending_before_crash
end

def make_more_syms(num = 100000)
   $stdout.sync = true
   num.times do |n|
     print "\r%7d / %8d" % [n, num] if n % 10000 == 0
     STR.intern
     $max = STR.intern
     STR.succ!
   end
   puts
   puts "Created #{num} new symbols"
end


puts "Objects around 0x%x " % ID2ULONG(Objs.object_id)

make_more_syms(10001)

pending = symbol_info

(1..5).each do |i|
  make_more_syms((pending * 2**(-i)).to_i)
  symbol_info
end
batsman@tux-chan:~/mess/current$ ruby -v symbols_overlap.rb
ruby 1.8.4 (2005-12-24) [i686-linux]
Objects around 0xb7dbf2ec
  10000 /    10001
Created 10001 new symbols
last: 23144206 ===>  -605161098
extrema:   id: 0x000000210e      ----   id: 0x000161270e
       symbol:         :"!"      ----      :"b!aaaaaouq"
About ~1494873 symbols expected before crash
 740000 /   747436
Created 747436 new symbols
last: 1553893134 ===>  -605161098
extrema:   id: 0x000000210e      ----   id: 0x005c9e870e
       symbol:         :"!"      ----      :"b!aaabrcme"
About ~747437 symbols expected before crash
 370000 /   373718
Created 373718 new symbols
last: -1975699698 ===>  -605161098
extrema:   id: 0x000000210e      ----   id: 0x00147a6e1c
symbols_overlap.rb:18: [BUG] Segmentation fault
ruby 1.8.4 (2005-12-24) [i686-linux]

Aborted
batsman@tux-chan:~/mess/current$ ruby19 -v symbols_overlap.rb
ruby 1.9.0 (2006-01-14) [i686-linux]
Objects around 0xb7dafea0
  10000 /    10001
Created 10001 new symbols
last: 23246606 ===>  -605192368
extrema:   id: 0x000000210e      ----   id: 0x000162b70e
       symbol:         :"!"      ----      :"b!aaaaaouq"
About ~1494792 symbols expected before crash
 740000 /   747396
Created 747396 new symbols
last: 1553913614 ===>  -605192368
extrema:   id: 0x000000210e      ----   id: 0x005c9ed70e
       symbol:         :"!"      ----      :"b!aaabrckq"
About ~747396 symbols expected before crash
 370000 /   373698
Created 373698 new symbols
last: -1975720178 ===>  -605192368
extrema:   id: 0x000000210e      ----   id: 0x001479ce1c
symbols_overlap.rb:18: [BUG] Segmentation fault
ruby 1.9.0 (2006-01-14) [i686-linux]

Aborted



Some more info:

(gdb) set args symbols_overlap.rb
(gdb) run
Starting program: /home/batsman/usr/bin/ruby19 symbols_overlap.rb
Objects around 0xb7e11ea0
  10000 /    10001
Created 10001 new symbols
last: 23246606 ===>  -604991664
extrema:   id: 0x000000210e      ----   id: 0x000162b70e
       symbol:         :"!"      ----      :"b!aaaaaouq"
About ~1494988 symbols expected before crash
 740000 /   747494
Created 747494 new symbols
last: 1554114318 ===>  -604991664
extrema:   id: 0x000000210e      ----   id: 0x005ca1e70e
       symbol:         :"!"      ----      :"b!aaabrcok"
About ~747494 symbols expected before crash
 370000 /   373747
Created 373747 new symbols
last: -1975419122 ===>  -604991664
extrema:   id: 0x000000210e      ----   id: 0x001482fe1c

Program received signal SIGSEGV, Segmentation fault.
0x0808ccb4 in sym_inspect (sym=2319548174) at object.c:1053
1053        str = rb_str_new(0, strlen(name)+1);
(gdb) p name
$1 = 0x0
(gdb) where
#0  0x0808ccb4 in sym_inspect (sym=2319548174) at object.c:1053
#1  0x08060d0b in call_cfunc (func=0x808cc82 <sym_inspect>, recv=2319548174, len=0, argc=0, argv=0x0) at eval.c:5435
#2  0x080605b2 in rb_call0 (klass=3085058588, recv=2319548174, id=3185, oid=3185, argc=0, argv=0x0, body=0xb7e2417c,
    flags=8) at eval.c:5643
#3  0x0806162d in rb_call (klass=3085058588, recv=2319548174, mid=3185, argc=0, argv=0x0, scope=CALLING_FCALL)
    at eval.c:5816
#4  0x08061835 in rb_funcall (recv=2319548174, mid=3185, n=0) at eval.c:5916
#5  0x0808c439 in rb_inspect (obj=2319548174) at object.c:301
#6  0x080d0754 in rb_str_format (argc=3, argv=0xc6804fc, fmt=3084987368) at sprintf.c:414
#7  0x080d498a in rb_str_format_m (str=3084545100, arg=3084539600) at string.c:413
#8  0x08060d2a in call_cfunc (func=0x80d495a <rb_str_format_m>, recv=3084545100, len=1, argc=1, argv=0xbfadf730)
    at eval.c:5438
#9  0x080605b2 in rb_call0 (klass=3085051628, recv=3084545100, id=37, oid=37, argc=1, argv=0xbfadf730, body=0xb7e2250c,
    flags=0) at eval.c:5643
#10 0x0806162d in rb_call (klass=3085051628, recv=3084545100, mid=37, argc=1, argv=0xbfadf730, scope=CALLING_NORMAL)
    at eval.c:5816
#11 0x0805aba6 in rb_eval (self=3085056528, n=0xb7e12b70) at eval.c:3278
#12 0x0805ad0d in rb_eval (self=3085056528, n=0xb7e12d28) at eval.c:3288
#13 0x080609d9 in rb_call0 (klass=3085061408, recv=3085056528, id=10737, oid=10737, argc=0, argv=0x0, body=0xb7e12d28,
    flags=10) at eval.c:5724
#14 0x0806162d in rb_call (klass=3085061408, recv=3085056528, mid=10737, argc=0, argv=0x0, scope=CALLING_VCALL)
    at eval.c:5816
#15 0x0805ae8e in rb_eval (self=3085056528, n=0xb7e11f04) at eval.c:3299
#16 0x0805e86a in rb_yield_0 (val=5, self=3085056528, klass=0, flags=0, avalue=0) at eval.c:4821
#17 0x0805ec02 in rb_yield (val=5) at eval.c:4902
#18 0x080aaf91 in range_each (range=3084984468) at range.c:389
#19 0x08060d0b in call_cfunc (func=0x80aae93 <range_each>, recv=3084984468, len=0, argc=0, argv=0x0) at eval.c:5435
#20 0x080605b2 in rb_call0 (klass=3085021928, recv=3084984468, id=3905, oid=3905, argc=0, argv=0x0, body=0xb7e1b1e4,
    flags=0) at eval.c:5643
#21 0x0806162d in rb_call (klass=3085021928, recv=3084984468, mid=3905, argc=0, argv=0x0, scope=CALLING_NORMAL)
    at eval.c:5816
#22 0x0805aba6 in rb_eval (self=3085056528, n=0xb7e12080) at eval.c:3278
#23 0x08059982 in rb_eval (self=3085056528, n=0xb7e138e0) at eval.c:3002
#24 0x08055f58 in eval_node (self=3085056528, node=0xb7e138e0) at eval.c:1337
#25 0x0805655e in ruby_exec_internal () at eval.c:1515
#26 0x080565ac in ruby_exec () at eval.c:1531
#27 0x080565e9 in ruby_run () at eval.c:1547
#28 0x080543ce in main (argc=2, argv=0xbfae1684, envp=0xbfae1690) at main.c:43
(gdb) list object.c:1053
1048        VALUE str;
1049        char *name;
1050        ID id = SYM2ID(sym);
1051
1052        name = rb_id2name(id);
1053        str = rb_str_new(0, strlen(name)+1);
1054        RSTRING(str)->ptr[0] = ':';
1055        strcpy(RSTRING(str)->ptr+1, name);
1056        if (!rb_symname_p(name)) {
1057            str = rb_str_dump(str);



-- 
Mauricio Fernandez  -   http://eigenclass.org   -  non-trivial Ruby