From: Matthew Gaudet <magaudet@...>
Date: 2017-04-28T10:52:04-04:00
Subject: [ruby-core:80924] Ensuring a de-referenceable method definition

Hi Ruby-core: I was hoping someone familiar with
rb_callable_method_entry could help me figure out what the rules are
for de-referencing the contained rb_method_definition_struct 'def'.

I'm looking for a target instruction sequence to inline by peeking
inside of the CALL_CACHE. The pseudocode looks something like this:

```
CALL_CACHE cc = <load call cache>;
const rb_callable_method_entry_t *me = cc->me;
if(!me)
    {
    // abort
    }
  if (!me->def)
    {
    // abort
    }

switch (METHOD_ENTRY_VISI(cc->me))
    {
    case METHOD_VISI_PUBLIC:
       break; // OK for inlining.
    case METHOD_VISI_PRIVATE:
       if (ci->flag & VM_CALL_FCALL)
          break; // Ok for inlining
    default:
       //abort
    }

switch(me->def->type) // Crashes here de-referencing me->def
    {
    // ...
    }
```

I'm missing some sort of safety check that's required to ensure
`me->def` is a valid pointer, but I've yet to quite put it all
together.

I've tried piecing a check together METHOD_ENTRY_BASIC and
METHOD_ENTRY_COMPLEMENTED, with no real luck.

Empirically, what I _have_ found is that should me->flags have its low
bit set (if (me->flags & 1), me->def isn't pointing to allocated
memory, however, it's not clear to me what this is actually saying, as
I'm not sure what those bits of the flags word are actually being used
for.

Any advice on what the actual checks are that one would need to make to 
ensure I can look in `me->def`. I'm sure it's something simple, but I
have yet to be able to reverse engineer it, and probably would be better
to just ask.

Thanks so much,

-- Matthew



[1]: https://github.com/rubyomr-preview/ruby/issues/61



Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>