[ruby-core:84578] [Ruby trunk Feature#14045] Lazy Proc allocation for block parameters

Issue #14045 has been updated by myronmarston (Myron Marston).


For those who are interested, the work around I've implemented in RSpec is [here](
https://github.com/rspec/rspec-core/pull/2497/commits/84670489bb4943a62e783bd65f96e4b55360b141):

```
From 84670489bb4943a62e783bd65f96e4b55360b141 Mon Sep 17 00:00:00 2001
From: Myron Marston <[e-mail]>
Date: Mon, 1 Jan 2018 12:22:16 -0800
Subject: [PATCH] Work around regressions introduced by lazy proc allocation.

This Ruby 2.5 feature is causing bugs with our hooks because
we depend upon `Hook#==` working properly for the same source
hook block. In Ruby 2.5 they introduced lazy proc allocation
which can result in two `Hook` instances which were previously
correctly considered to be equal to no longer be considered
equal. To work around this, we just need to invoke a method on
the proc before passing it along to other methods.

Note that there might be other bugs introduced by the Ruby 2.5
change, but this fixes the only test failures due to it, so this
is all we are changing for now.

Fixes #2488.
---
 lib/rspec/core/configuration.rb | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/lib/rspec/core/configuration.rb b/lib/rspec/core/configuration.rb
index 4152173f0..b02b803df 100644
--- a/lib/rspec/core/configuration.rb
+++ b/lib/rspec/core/configuration.rb
@@ -1808,6 +1808,12 @@ def before(scope=nil, *meta, &block)
         handle_suite_hook(scope, meta) do
           @before_suite_hooks << Hooks::BeforeHook.new(block, {})
         end || begin
+          # defeat Ruby 2.5 lazy proc allocation to ensure
+          # the methods below are passed the same proc instances
+          # so `Hook` equality is preserved. For more info, see:
+          # https://bugs.ruby-lang.org/issues/14045#note-5
+          block.__id__
+
           add_hook_to_existing_matching_groups(meta, scope) { |g| g.before(scope, *meta, &block) }
           super(scope, *meta, &block)
         end
@@ -1831,6 +1837,12 @@ def prepend_before(scope=nil, *meta, &block)
         handle_suite_hook(scope, meta) do
           @before_suite_hooks.unshift Hooks::BeforeHook.new(block, {})
         end || begin
+          # defeat Ruby 2.5 lazy proc allocation to ensure
+          # the methods below are passed the same proc instances
+          # so `Hook` equality is preserved. For more info, see:
+          # https://bugs.ruby-lang.org/issues/14045#note-5
+          block.__id__
+
           add_hook_to_existing_matching_groups(meta, scope) { |g| g.prepend_before(scope, *meta, &block) }
           super(scope, *meta, &block)
         end
@@ -1849,6 +1861,12 @@ def after(scope=nil, *meta, &block)
         handle_suite_hook(scope, meta) do
           @after_suite_hooks.unshift Hooks::AfterHook.new(block, {})
         end || begin
+          # defeat Ruby 2.5 lazy proc allocation to ensure
+          # the methods below are passed the same proc instances
+          # so `Hook` equality is preserved. For more info, see:
+          # https://bugs.ruby-lang.org/issues/14045#note-5
+          block.__id__
+
           add_hook_to_existing_matching_groups(meta, scope) { |g| g.after(scope, *meta, &block) }
           super(scope, *meta, &block)
         end
@@ -1872,6 +1890,12 @@ def append_after(scope=nil, *meta, &block)
         handle_suite_hook(scope, meta) do
           @after_suite_hooks << Hooks::AfterHook.new(block, {})
         end || begin
+          # defeat Ruby 2.5 lazy proc allocation to ensure
+          # the methods below are passed the same proc instances
+          # so `Hook` equality is preserved. For more info, see:
+          # https://bugs.ruby-lang.org/issues/14045#note-5
+          block.__id__
+
           add_hook_to_existing_matching_groups(meta, scope) { |g| g.append_after(scope, *meta, &block) }
           super(scope, *meta, &block)
         end
@@ -1881,6 +1905,12 @@ def append_after(scope=nil, *meta, &block)
       #
       # See {Hooks#around} for full `around` hook docs.
       def around(scope=nil, *meta, &block)
+        # defeat Ruby 2.5 lazy proc allocation to ensure
+        # the methods below are passed the same proc instances
+        # so `Hook` equality is preserved. For more info, see:
+        # https://bugs.ruby-lang.org/issues/14045#note-5
+        block.__id__
+
         add_hook_to_existing_matching_groups(meta, scope) { |g| g.around(scope, *meta, &block) }
         super(scope, *meta, &block)
       end
``

----------------------------------------
Feature #14045: Lazy Proc allocation for block parameters
https://bugs.ruby-lang.org/issues/14045#change-69118

* Author: ko1 (Koichi Sasada)
* Status: Closed
* Priority: Normal
* Assignee: ko1 (Koichi Sasada)
* Target version: 
----------------------------------------
# Background

If we need to pass given block, we need to capture by block parameter as a Proc object and pass it parameter as a block argument. Like that:

```
def block_yield
  yield
end

def block_pass &b
  # do something
  block_yield(&b)
end
```

There are no way to pass given blocks to other methods without using block parameters.

One problem of this technique is performance. `Proc` creation is one of heavyweight operation because we need to store all of local variables (represented by Env objects in MRI internal) to heap. If block parameter is declared as one of method parameter, we need to make a new `Proc` object for the block parameter.

# Proposal: Lazy Proc allocation for 

To avoid this overhead, I propose lazy Proc creation for block parameters.

Ideas:

* At the beginning of method, a block parameter is `nil`
* If block parameter is accessed, then create a `Proc` object by given block.
* If we pass the block parameter to other methods like `block_yield(&b)` then don't make a `Proc`, but pass given block information.

We don't optimize `b.call` type block invocations. If we call block with `b.call`, then create `Proc` object.We need to hack more because `Proc#call` is different from `yield` statement (especially they can change `$SAFE`).

# Evaluation

```
def iter_yield
  yield
end

def iter_pass &b
  iter_yield(&b)
end

def iter_yield_bp &b
  yield
end

def iter_call &b
  b.call
end

N = 10_000_000 # 10M

require 'benchmark'
Benchmark.bmbm(10){|x|
  x.report("yield"){
    N.times{
      iter_yield{}
    }
  }
  x.report("yield_bp"){
    N.times{
      iter_yield_bp{}
    }
  }
  x.report("yield_pass"){
    N.times{
      iter_pass{}
    }
  }
  x.report("send_pass"){
    N.times{
      send(:iter_pass){}
    }
  }
  x.report("call"){
    N.times{
      iter_call{}
    }
  }
}

__END__

ruby 2.5.0dev (2017-10-24 trunk 60392) [x86_64-linux]
                 user     system      total        real
yield        0.634891   0.000000   0.634891 (  0.634518)
yield_bp     2.770929   0.000008   2.770937 (  2.769743)
yield_pass   3.047114   0.000000   3.047114 (  3.046895)
send_pass    3.322597   0.000002   3.322599 (  3.323657)
call         3.144668   0.000000   3.144668 (  3.143812)

modified
                 user     system      total        real
yield        0.582620   0.000000   0.582620 (  0.582526)
yield_bp     0.731068   0.000000   0.731068 (  0.730315)
yield_pass   0.926866   0.000000   0.926866 (  0.926902)
send_pass    1.110110   0.000000   1.110110 (  1.109579)
call         2.891364   0.000000   2.891364 (  2.890716)
```

# Related work

To delegate the given block to other methods, Single `&` block parameter had been proposed (https://bugs.ruby-lang.org/issues/3447#note-18) (using like: `def foo(&); bar(&); end`). This idea is straightforward to represent `block passing`. Also we don't need to name a block parameter.

The advantage of this ticket proposal is we don't change any syntax. We can write compatible code for past versions.

Thanks,
Koichi



-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>