[ruby-core:26578] [Backport #2326] 1.8.7 Segmentation fault

Issue #2326 has been updated by Kirk Haines.


In testing, the case which causes the error can be simplified to:

Time.now while true

In general, this segfault can be triggered by creating, in a tight loop, any object that has a Data_Make_Struct call in it's creation, and that has a free function passed into that call.

What appears to happen is that these objects are deferred when obj_free() is called during the gc_sweep():

      case T_DATA:
        if (DATA_PTR(obj)) {
            if ((long)RANY(obj)->as.data.dfree == -1) {
                RUBY_CRITICAL(free(DATA_PTR(obj)));
            }
            else if (RANY(obj)->as.data.dfree) {
                make_deferred(RANY(obj));
                return 1;
            }
        }

Consequently, they don't get removed from the heap before the heap fills up.  This wouldn't be a problem except that rb_newobj has the following code:

    if (ruby_gc_stress || !freelist) garbage_collect();

    obj = (VALUE)freelist;
    freelist = freelist->as.free.next;


The problem with this check is that it doesn't go far enough.  If freelist points to something, but freelist->as.free.next points to 0, then when the assignment to freelist runs, a segfault occurs.

The simple fix for this is to expand the check:

    if (ruby_gc_stress || !freelist || !freelist->as.free.next) garbage_collect();

This works, and it's the same fix on the 1.8.6 and 1.8.7 versions which are broken.  But is it the best way to fix this?  Feedback is appreciated.


Kirk Haines
----------------------------------------
http://redmine.ruby-lang.org/issues/show/2326

----------------------------------------
http://redmine.ruby-lang.org