[ruby-core:49883] [ruby-trunk - Feature #4085] Refinements and nested methods

Issue #4085 has been updated by headius (Charles Nutter).


Escaping the "should we or shouldn't we" question for a bit, I thought of an alternative implementation, building off ko1's idea.

ko1's suggestion, as I understand it, was to add a flag to the method table (or method entry) of a refined class/method as a trigger for the call site to search refinements. While writing my blog post, I started to type the sentence "the methods defined in the refinement do not actually go on the class in question", and then I realized: why not?

Currently, Ruby implementations structure the method table as a simple map from names to method bodies. If instead the method table was a map from names to collections of methods, we could use that to choose the appropriate method for a given context.

So, for code like this:

module X
  refine String do
    def upcase; downcase; end
  end
end

String's method table would contain an entry like this:

{:upcase => {
  :default => <builtin upcase>,
  X => <upcase patched>}
}

Method lookup would then proceed as normal in all situations. The result of lookup would be a table mapping refinements to methods with a default entry if the method is defined directly on String.

After lookup, call sites would know there's potentially refinements active for the given method. The calling scope (or parent scopes) would have references to individual refinements, and if there were an entry for one of them it would be used.

This still requires access to the caller scope, of course, to understand what refinements are active. However, because refinement changes would invalidate the String class directly (since they actually modify the method table), the method (refined or otherwise) could be cached as normal. The caller's scope never changes (statically determined at compile time), so it does not participate in invalidation.

This also works for refinements added to classes after the initial run through. If we cache the default downcase method from String, and then the refinement is updated to add downcase, we would see that as an invalidation event for String's method table. Future calls would then re-cache and pick up the change.

This also feels a bit more OO-friendly to me. Rather than storing patches on separate structures sprinkled around memory, we store the patches directly on the refined class, only using the module containing the refinements as a key. The methods *do* live on String, but depending on the *namespace* they're looked up from we *select* the appropriate implementation. It's basically just double-dispatch at that point, with the selector being the calling context.

It also makes available an interesting possibility for #method and friends: return all methods. So...

using X

String.instance_methods(:upcase) # => {:default => <builtin upcase>, X => <new upcase>}

Note that this is "instance_methods", plural, to avoid breaking instance_method and to make it explicit that we're asking for all implementations of a given method. This allows accessing the original method even if refinements are active, and still also allows searching for the refined method active in the current scope.

I admit I am a bit reluctant to suggest this, because I still have concerns about the feature itself. But it would be possible for call sites to only need a reference to their calling scope (determined at parse time) to implement dynamic refinements without severe impact to normal code. Dynamic refinements, as in module_eval, would work by simply invalidating the call sites they contain. This could be done actively, walking all call sites and resetting them. This could also be done by invalidating the classes refined. An example in pseudo-code:

def X.module_eval_refined(&block)
  unless block.using? self
    refinements.each_key {|cls| cls.touch } # invalidate all refined classes
    block.using(self)
  end
  module_eval &block
end

This is obviously not a thread-safe mechanism. An alternative that invalidates the block's call sites (this would require more work and be more expensive at invalidation time, but less globally-damaging):

def X.module_eval_refined(&block)
  unless block.using? self
    block.invalidate
    block.using(self)
  end
  module_eval &block
end

Proc#using would either mutate the block's already-present scope (permanently adding the refinement) or duplicate the block and its scope and tweak it (more expensive, of course).

---

In any case, I would really like more time for this dialog to continue. If we push refinements into Ruby in their current form, we're not giving adequate time to flesh out the edge cases. If we push a partial implementation now, we may be making a future implementation harder and we would not be protecting ourselves from mistakes. I want to work with you to find a definition and implementation of refinements that meets requirements without punishing future Rubyists.

I also must apologize for not joining the dialog sooner. This bug was filed in 2010, and the current refinements implementation was pushed to master a few months ago. We should have started discussing a long time ago.
----------------------------------------
Feature #4085: Refinements and nested methods
https://bugs.ruby-lang.org/issues/4085#change-33640

Author: shugo (Shugo Maeda)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version: 2.0.0


=begin
 As I said at RubyConf 2010, I'd like to propose a new features called
 "Refinements."
 
 Refinements are similar to Classboxes.  However, Refinements doesn't
 support local rebinding as mentioned later.  In this sense,
 Refinements might be more similar to selector namespaces, but I'm not
 sure because I have never seen any implementation of selector
 namespaces.
 
 In Refinements, a Ruby module is used as a namespace (or classbox) for
 class extensions.  Such class extensions are called refinements.  For
 example, the following module refines Fixnum.
 
   module MathN
     refine Fixnum do
       def /(other) quo(other) end
     end
   end
 
 Module#refine(klass) takes one argument, which is a class to be
 extended.  Module#refine also takes a block, where additional or
 overriding methods of klass can be defined.  In this example, MathN
 refines Fixnum so that 1 / 2 returns a rational number (1/2) instead
 of an integer 0.
 
 This refinement can be enabled by the method using.
 
   class Foo
     using MathN
 
     def foo
       p 1 / 2
     end
   end
 
   f = Foo.new
   f.foo #=> (1/2)
   p 1 / 2
 
 In this example, the refinement in MathN is enabled in the definition
 of Foo.  The effective scope of the refinement is the innermost class,
 module, or method where using is called; however the refinement is not
 enabled before the call of using.  If there is no such class, module,
 or method, then the effective scope is the file where using is called.
 Note that refinements are pseudo-lexically scoped.  For example,
 foo.baz prints not "FooExt#bar" but "Foo#bar" in the following code:
 
   class Foo
     def bar
       puts "Foo#bar"
     end
 
     def baz
       bar
     end
   end
 
   module FooExt
     refine Foo do
       def bar
         puts "FooExt#bar"
       end
     end
   end
 
   module Quux
     using FooExt
 
     foo = Foo.new
     foo.bar  # => FooExt#bar
     foo.baz  # => Foo#bar
   end
 
 Refinements are also enabled in reopened definitions of classes using
 refinements and definitions of their subclasses, so they are
 *pseudo*-lexically scoped.
 
   class Foo
     using MathN
   end
 
   class Foo
     # MathN is enabled in a reopened definition.
     p 1 / 2  #=> (1/2)
   end
 
   class Bar < Foo
     # MathN is enabled in a subclass definition.
     p 1 / 2  #=> (1/2)
   end
 
 If a module or class is using refinements, they are enabled in
 module_eval, class_eval, and instance_eval if the receiver is the
 class or module, or an instance of the class.
 
   module A
     using MathN
   end
   class B
     using MathN
   end
   MathN.module_eval do
     p 1 / 2  #=> (1/2)
   end
   A.module_eval do
     p 1 / 2  #=> (1/2)
   end
   B.class_eval do
     p 1 / 2  #=> (1/2)
   end
   B.new.instance_eval do
     p 1 / 2  #=> (1/2)
   end
 
 Besides refinements, I'd like to propose new behavior of nested methods.
 Currently, the scope of a nested method is not closed in the outer method.
 
   def foo
     def bar
       puts "bar"
     end
     bar
   end
   foo  #=> bar
   bar  #=> bar
 
 In Ruby, there are no functions, but only methods.  So there are no
 right places where nested methods are defined.  However, if
 refinements are introduced, a refinement enabled only in the outer
 method would be the right place.  For example, the above code is
 almost equivalent to the following code:
 
   def foo
     klass = self.class
     m = Module.new {
       refine klass do
         def bar
           puts "bar"
         end
       end
     }
     using m
     bar
   end
   foo  #=> bar
   bar  #=> NoMethodError
 
 The attached patch is based on SVN trunk r29837.
=end



-- 
http://bugs.ruby-lang.org/