From: merch-redmine@...
Date: 2019-09-04T23:51:02+00:00
Subject: [ruby-core:94780] [Ruby master Feature#14183] "Real" keyword	argument

Issue #14183 has been updated by jeremyevans0 (Jeremy Evans).


Dan0042 (Daniel DeLorme) wrote:
> @jeremyevans0 First, thank you very much for taking your time to engage with me like this.
> 
> > Here, the intention is to pass the keyword arguments from one method as a positional hash to another method. This is one of the cases that currently breaks in 2.6, that will warn and break in 2.7, and that will be fixed in Ruby 3. With your approach, it will remain broken, since kw in foo will be implicitly converted to keyword arguments to bar.
> 
> I think in this case the 2.6 behavior is better. Because `:a=>1` is specified without braces, it should ideally remain a keyword all the way down to `bar`. It should not become a Hash in `foo` without explicit conversion.
> 
> To my eyes, the intention signaled via `bar(kw)` in your example would be to pass-through the keyword arguments. Because kw is not a Hash but KwHash (provisional name). In order to pass it as positional argument you would need to first convert it to Hash via `bar({**kw})` or such. The idea is that a KwHash can only be passed as a kwarg. No automatic conversion. Passing a KwHash `kw` to a method is strictly equivalent to `**kw`. After all the entire point of "Real keyword arguments" is to keep them distinct from the rest right? But syntax is not the only way to do that; this KwHash class would also be a way to achieve the same result. Even though `bar(kw)` may look like a positional argument, it's really a keyword argument, properly and fully differentiated from positional arguments via its class. I realize that's a fairly different interpretation than the current one but I believe it makes sense. Matz may [like](https://bugs.ruby-lang.org/issues/14183#note-45) syntactical separatio
 n but I think he would remain open to other possibilities.

I disagree that taking a keyword argument hash in one method and passing them as a positional argument to another method should force the argument to become keyword arguments in the other method.  Conceptually, once a method has been entered, there is no longer a distinction between positional arguments and keyword arguments, they are all just local variables at that point.  Your proposal attempts to introduce a distinction that does not and should not exist.

Here's a simple example showing undesired behavior with your approach:

```ruby
class A
  # Same as Kernel#p, except do nothing if :skip keyword is present
  def p(*args, skip: false)
    super(*args) unless skip
  end

  def foo(*args, **kw)
    # In debug mode, prints method name, positional arguments, and keyword arguments
    p(:foo, *args, kw) if $DEBUG

    # do something
  end
end

# No problems
A.new.foo(1, 2, a: 1)

$DEBUG = true

# ArgumentError! (unknown keyword: :a)
# Even though the only purpose was to print out the arguments for debugging
A.new.foo(1, 2, a: 1)
```

> 
> ```ruby
> args = [1, 2, hash]
> foo(*args) #=> args.last is Hash -> positional; warning in 2.7
> args = [1, 2, **hash]
> foo(*args) #=> args.last is KwHash -> keyword
> ```
> 
> I would even go as far as saying that with this KwHash, `bar(kw1, 2, kw3, 4)` must either raise an error or be equivalent to `bar(2, 4, **kw1, **kw3)`, otherwise the separation of keyword and positional arguments doesn't hold. I realize this is not backward compatible, but it's the kind of incompatibility I'm ok with because it fixes incorrect semantics (if it was originally a kwarg it shouldn't suddenly be a Hash).

`**hash` in arrays is not done for keyword argument purposes (after all, there are no arguments).  It is used to merge multiple hashes and literal keywords:

```ruby
[1, a: 2, **{b: 3}, **{c: 4}]
# => [1, {:a=>2, :b=>3, :c=>4}]
```

So you are trying to introduce an idea `**` in arrays as being for keywords arguments, when it has not been used for that in the past.  The introduction of such behavior would result in additional backward incompatibility.

> And on the receiver side, even if the kwarg is converted to positional argument because of your compatibility mode, it would still be a KwHash and behave as such unless *explicitly* converted to Hash. The positional/keyword separation is maintained even despite the compatibility mode.
> 
> I know that code speaks loudest so I would like to write a branch for this idea, but I'm too unfamiliar with the VM code. I wouldn't be able to write something in time to make it for review before the November code freeze. :-(

The keyword argument branch in this ticket was my first time working significantly in the VM code.  mame posted his initial patch on March 18.  I posted my initial patch based off his patch on March 25.  It's the start of September, there is still time to work on an actual proposal with code if you are passionate about this change.  The only person you need to convince is matz :) .

> > I understand that keyword argument separation is going to require updating some code.  It is not going to be fully backwards compatible for methods that accept keyword arguments.  The good news is that if you don't use keyword arguments in your methods, the behavior will remain backwards compatible.  Additionally, any cases that will break will be warned about in Ruby 2.7 so you can update the related code.
> 
> Updating some code in itself is not a problem at all. What makes me uncomfortable is that updating code in order to fix 2.7 warnings can result in code that is no longer compatible with 2.6. This now seems to be the only way to write correct forwarding code?
> 
> ```ruby
> if RUBY_VERSION.to_f <= 2.6
>   def method_missing(*a, &b)
>     @x.send(*a, &b)
>   end
> else
>   def method_missing(*a, **o, &b)
>     @x.send(*a, **o, &b)
>   end
> end
> ```

You do not need to have two separate definitions of `method_missing`, unless you want to be backwards compatible with 1.9 (which doesn't support `**` for keyword parameters).  You should always use `*a, **o, &b` when forwarding.  Example:

```ruby
class B
  def initialize(x) @x = x end
  def method_missing(*a, **o, &b)
    @x.send(*a, **o, &b)
  end
end

class C
  def initialize(x) @x = x end
  def method_missing(*a, &b)
    @x.send(*a, &b)
  end
end

class D
  def method_missing(*a, **o, &b)
    [*a, o, b]
  end
end

b = B.new(D.new)
c = C.new(D.new)

b.a == c.a                 # true
b.a(1) == c.a(1)           # true
b.a(a: 1) == c.a(a: 1)     # true, c.a warns in 2.7
b.a({a: 1}) == c.a({a: 1}) # true, both b.a and c.a warn in 2.7
b.a({a: 1}, **(;{})) == c.a({a: 1}, **(;{})) # true, c.a warns in 2.7
```

For the case where `b.a` warns, you'll need to make changes for Ruby 3 to get the same behavior.  However, you probably wouldn't want to make changes in this example, as it is pretty obvious you intend to pass a positional hash and not keywords.

> If it was unavoidable then I'd just say that's the cost of progress. But I'm convinced it's avoidable.
> 
> Please understand that I'm not clinging to old behavior just as a knee-jerk reaction to change. I've taken your earlier words to heart and spent several hours reading this entire thread carefully as well as related tickets, digesting and pondering the information. So I think I've reached a pretty decent understanding. The current changes are obviously great and fix a lot of problems. It's just that adding keyword separation via class in addition to syntax allows to keep better backward compatibility with **stricter** keyword/positional separation, while still fixing all the issues related to the previous implementation. I think that's worth serious consideration.

I understand what you want and why you want it.  You want `def m(*a, &b) n(*a, &b) end` to implicitly forward keyword arguments as keyword arguments, so you don't have to modify the related code.  Unfortunately, that's not possible when separating keyword arguments, and trying to work around it with separate classes causes more problems then it solves.  You will need to switch the code to: `def m(*a, **o, &b) n(*a, **o, &b) end`.

Your proposal does not necessarily keep better backwards compatibility.  By attempting to implicitly convert positional arguments to keyword parameters, it introduces new backwards compatibility issues.  Whether the backwards compatibility issues it introduces are better or worse than the behavior currently planned for Ruby 3 is subjective, but I think your proposal would make hurt backwards compatibility more than it helps.

Your proposal does not result in stricter keyword/positional separation.  It makes the separation less strict by using implicit conversion of positional argument to keyword argument.

Your proposal does not fix all of the issues with the previous implementation.  It enables you to not have to modify some code, at the expense of opening a pandora's box of possible issues, such as the example given above.

----------------------------------------
Feature #14183: "Real" keyword argument
https://bugs.ruby-lang.org/issues/14183#change-81398

* Author: mame (Yusuke Endoh)
* Status: Closed
* Priority: Normal
* Assignee: 
* Target version: Next Major
----------------------------------------
In RubyWorld Conference 2017 and RubyConf 2017, Matz officially said that Ruby 3.0 will have "real" keyword arguments.  AFAIK there is no ticket about it, so I'm creating this (based on my understanding).

In Ruby 2, the keyword argument is a normal argument that is a Hash object (whose keys are all symbols) and is passed as the last argument.  This design is chosen because of compatibility, but it is fairly complex, and has been a source of many corner cases where the behavior is not intuitive.  (Some related tickets: #8040, #8316, #9898, #10856, #11236, #11967, #12104, #12717, #12821, #13336, #13647, #14130)

In Ruby 3, a keyword argument will be completely separated from normal arguments.  (Like a block parameter that is also completely separated from normal arguments.)
This change will break compatibility; if you want to pass or accept keyword argument, you always need to use bare `sym: val` or double-splat `**` syntax:

```
# The following calls pass keyword arguments
foo(..., key: val)
foo(..., **hsh)
foo(..., key: val, **hsh)

# The following calls pass **normal** arguments
foo(..., {key: val})
foo(..., hsh)
foo(..., {key: val, **hsh})

# The following method definitions accept keyword argument
def foo(..., key: val)
end
def foo(..., **hsh)
end

# The following method definitions accept **normal** argument
def foo(..., hsh)
end
```

In other words, the following programs WILL NOT work:

```
# This will cause an ArgumentError because the method foo does not accept keyword argument
def foo(a, b, c, hsh)
  p hsh[:key]
end
foo(1, 2, 3, key: 42)

# The following will work; you need to use keyword rest operator explicitly
def foo(a, b, c, **hsh)
  p hsh[:key]
end
foo(1, 2, 3, key: 42)

# This will cause an ArgumentError because the method call does not pass keyword argument
def foo(a, b, c, key: 1)
end
h = {key: 42}
foo(1, 2, 3, h)

# The following will work; you need to use keyword rest operator explicitly
def foo(a, b, c, key: 1)
end
h = {key: 42}
foo(1, 2, 3, **h)
```

I think here is a transition path:

* Ruby 2.6 (or 2.7?) will output a warning when a normal argument is interpreted as keyword argument, or vice versa.
* Ruby 3.0 will use the new semantics.

---Files--------------------------------
vm_args.diff (4.19 KB)
vm_args_v2.diff (4.18 KB)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>