[ruby-core:18010] Re: Improving the metaprogramming facilities of Ruby.

Hi --

On Mon, 28 Jul 2008, Daniel Pittman wrote:

> At the moment Ruby has two distinct and parallel methods for managing
> instance variables[2], being:
>
>    @foo
>    instance_variable_get("@foo")
>
> I had, initially, expected that I could simply define a method,
> 'instance_variable_get', on my object which would allow me to override
> the standard ivar lookup process for our own.
>
> What I found, to my surprise, was that @foo acts directly on the ivar
> table, accessing or setting the instance variable directly.  The
> instance_variable_get method does the same, and there is no actual
> relationship between the two.

I remembering having a similar expectation with regard to Array#[] and
#[]= -- namely, thinking that if I overrode them, I could hook into
#push, #pop, and so forth. That doesn't happen mainly for optimization
reasons, in that particular case.

> So, after thinking about this and how I would have expected Ruby to
> handle the situation, I have two questions:
>
> First, is these a reason why these two operations are distinct?

I don't know the exact rationale but I've always assumed that
instance_variable_[gs]et exists simply to allow the operation to take
place outside of the object, and/or symbolically. I've never thought
of it as a hook on the basic variable assignment.

In fact, one question that occurs to me is: if you want to use this as
a hook, but also need the current functionality, what would you do? I
can imagine the usefulness of the hook, possibly, but I wouldn't want
to lose the simple symbolic assignment.

> Second, would it be acceptable to change Ruby (1.9, and ideally 1.8) so
> that these two paths /were/ uniform, at least in principal?

I would hope not 1.8. 1.8.7 is already further from 1.8.6 than any
other one right-hand digit change I can remember. I think it would be
better not to widen that gap, but to concentrate on the 1.9 stuff.
(Then again, I don't have any standing to say anything about that,
other than to give an opinion in passing, if that.)

>
> "the results ... are undefined":
>
>    Ruby code may do this, but the behaviour is not guaranteed by Ruby
>    in any fashion.  The code may behave as they expect, or it may cause
>    an exception, cause Ruby to malfunction, or any other result.
>
>    Implementers are encouraged to document the behaviour and limits of
>    their implementation, as well as to provide sensible results when
>    undefined actions are taken, such as raising an appropriate
>    exception.
>
> "conforming code":
>
>    Ruby code that is written within the defined behaviour of the core
>    system, rather than code relying on the behaviour of any particular
>    implementation of Ruby.
>
>
> The documentation below covers the range of Object instance_* methods
> associated with ivars; the quoted text is the Ruby 1.9 documentation,
> and my proposed additions are included.
>
> Editorial comments -- the "why" I added the text are in square brackets,
> such that:
>
>> This is Ruby 1.9 documentation.
>
> This is my modified text.  [This is my editorial comment on my text.]
>
>
> The modified method specifications:
>
>> obj.instance_variables    => array
>>
>> Returns an array of instance variable names for the receiver. Note
>> that simply defining an accessor does not create the corresponding
>> instance variable.
>
> The array returned from this method may not be modified by the caller;
> the results of doing so are undefined.
>
> [This text was added so that anyone overriding this method knows that
> they can safely return an internal array of names without risk of it
> being modified -- for performance reasons, mostly.]

There's a difference, though, between its being truly immutable, and
being mutable with no defined effect (which could mean disastrous
effect). I'm not sure but I don't think anyone is returning anything
but new array objects for introspective methods like
#instance_variables, #methods, and so forth.

>> obj.remove_instance_variable(symbol)    => obj
>>
>> Removes the named instance variable from <i>obj</i>, returning that
>> variable's value.
>
> This method may or may not be called during object destruction;
> conforming code should not rely on this to perform variable
> finalization.
>
> [Added so that implementers are guaranteed to be free of the burden of
> calling this during object destruction, and to ensure users know where
> they stand.]

Is this something that users would even have to know or worry about?

>> obj.instance_variable_defined?(symbol)    => true or false
>>
>> Returns <code>true</code> if the given instance variable is defined in
>> <i>obj</i>.
>
> If instance_variable_get or instance_variable_set are defined then it is
> the responsibility of the user to ensure that this method must also be
> return accurate results.  The results are undefined if this rule is not
> respected.
>
> [Added so that Ruby libraries can assume that this method and the
> get/set methods are symmetric, and to make it clear to the user that
> they are expected to preserve this behaviour.]

Is this likely to be an issue? Presumably the
instance_variable_set(sym,value) method would, whatever else it did,
set the instance variable to the value, and that the instance variable
would then be defined. What would be an example of how to violate this
expectation?

>> obj.instance_variable_get(symbol)    => obj
>>
>> Returns the value of the given instance variable, or nil if the
>> instance variable is not set.
>
> This method is called by Ruby when you write <code>@variable</code>
> within the scope of your object methods.  You can override this method
> to change the rules for instance variable lookup, as long as you abide
> by the specified rules.
>
> [Added to make it clear that the user /is/ permitted to override this
> method, and that this will result in *all* ivar references calling
> their method.]

I know this is the heart of your proposal. I still have the issue with
it that if it does change the rules for instance variable lookup, and
I still want a way that uses the old rules but does so symbolically, I
would have a problem.

>> The <code>@</code> part of the variable name should be included for
>> regular instance variables.  Throws a <code>NameError</code> exception
>> if the supplied symbol is not valid as an instance variable name.
>
> The default implementation of this method returns <code>nil</code> if
> the instance variable has not been set.  Conforming code is free to
> override this in user defined classes, but the results of overriding
> this method in system classes is not defined.
>
> [This is added to make it clear to the user that, for example, returning
> false rather than nil from Object.instance_variable_get would be a bad
> idea, and to allow the implementer and third party library authors to
> assume the standard behaviour in core classes.]

I can see some really serious problems here. Mainly, it would no
longer be possible to assume anything about instance_variable_[gs]et,
so basically it would not be usable. I know that it's sort of
"impolite" to use those methods from outside anyway, but it's always
been possible and it's always meant a certain thing. If it can now
mean anything, then you'd literally never want to do it.

That comes back to the issue with not wanting to lose the old
functionality in the interest of the new. What you're describing
sounds like methods that would be good candidates for being protected
-- and, indeed, I think they should be if they can be redefined. But
that leaves a gap in the language.

>> obj.instance_variable_set(symbol, obj)    => obj
>>
>> Sets the instance variable names by <i>symbol</i> to <i>object</i>,
>> thereby frustrating the efforts of the class's author to attempt to
>> provide proper encapsulation.
>
> [This is the original text, and is probably not terribly appropriate
> after the changes here, but I preserve it for the moment.]
>
>> The variable did not have to exist prior to this call.
>
> [The proceeding should be replaced with:]
>
> This method is called to set the initial, and all subsequent, values of
> the instance variable.  The variable will not exist prior to setting the
> initial value.
>
> Ruby will call this method whenever an instance variable is set,
> including for <code>@variable = value</code> expressions within the
> scope of your object.
>
> [Added to make it clear this is always called by Ruby to set a
> variable.]
>
> Conforming code may override this method, provided the specified
> behaviour is implemented.  However, the results of overriding this
> method on core Ruby classes is undefined.

The problem, I think, is that it's really undefined, in exactly that
sense, in any class. The reason it's undefined in the core classes is
that if you override, say, Range#instance_variable_set, you don't
necessarily know what the implementer has done with instance variables
of ranges, so you don't know what you might be breaking. But the same
is true of any class where someone from outside the class overrides
the method for the class or one of its instances: the implementer of
that class may have been depending on the original behavior, or some
prior override.

Again, I think this all points to the appropriateness of making these
methods protected if they do what you're envisioning.

> [Added to make it clear what the user can, and can't, do.]
>
> Care should be taken to avoid recursion in this method; the results of
> directly setting an instance variable within this method are undefined,
> and you should call your superclass instance_variable_set method
> instead.

Does that mean that the setting of the instance variable would not
automatically happen? That seems odd to me. Also, if you have to
explicitly call 'super', then you're not really doing an override; at
most, you're chaining some behavior in. (Which is often a good
approach, but in this case it would limit you.)

In sum, my concerns include:

   1. losing the current, very useful behavior of simple get/set
      operations performed symbolically;
   2. the effectively "undefined" behavior of doing these overrides
      for any class (not just core classes) whose internals you don't
      control (since the class's implementer may be depending on a
      different behavior;
   3. the issue of whether or not instance_variable_set(sym,value)
      might actually not even set the instance variable, which raises
      questions of how the overrides relate to their 'super' versions,
      and therefore how much one would really be able to do with them.


David

-- 
Rails training from David A. Black and Ruby Power and Light:
  *  Advancing With Rails    August 18-21    Edison, NJ
  * Co-taught by D.A. Black and Erik Kastner
See http://www.rubypal.com for details and updates!