[ruby-core:32673] Re: Proposal for Optional Static Typing for Ruby

On 2 Oct 2010, at 01:24, Loren Segal wrote:
> On 9/30/2010 7:52 PM, Eleanor McHugh wrote:
>> It appears that this proposal would require the runtime to add a new layer to for checking the posited nominal type annotations which perforce could be invoked at any point during program execution and indeed possibly repeatedly during the lifetime of that execution. Bearing in mind that nominal type is not the primary determinant of object validity or code correctness in Ruby, and that there are already existing methods for confirming nominal type identity (more of that anon) it's not at all clear that type annotations on this model would be a noticeable improvement.
> 
> Syntactically they would be. Documentation-wise, they would be as well. Tooling-wise, they would aid in some *basic* (keyword: basic) code comprehension and analysis. IDE's, for instance, would have useful information to communicate back to the programmer, be it warnings or for code completion.

I don't use an IDE or code completion and therefore it's hard for me to really comment either way on this. I can see that if someone has an established working pattern then being able to leverage that could be a great advantage, and I certainly don't want to further any opinion just on the grounds that that working patterns and choice of tools are different to mine, but conversely I'm not really that keen to have code peppered with explicit meta-statements which are primarily of interest to tools. that brings back nasty memories of working with archaic assemblers and C compilers.

And yes I do appreciate that if I view the code containing annotations in the context of nominal typing then they do possess semantic information that might on occasion be a useful prompt to a human reader, but really if I'm going to use nominal typing constraints I expect them to be explicitly embodied in code using the mechanisms Ruby already provides so that they have both programmer value and runtime effect.

> Secondly I should point out that type-checking need not be implemented by the VM-- having the type annotations would give a Ruby developer a standard way in which to implement such type checking themselves, without resorting to ugly undocumentable DSLs as we've seen in this thread (ie. "def_with_types"). I think that's a big win in itself. We've recently seen Method#parameters get introduced into 1.9.2 from the nudging of real-world usage in merb/rails to route via given parameter names. Imagine what else could be done if, for instance, the route knew of the expected parameter *type* as well (again, without resorting to DSL)? I see plenty of room for improvement in this space.

But note that Method::parameters is itself a method call, not a syntax addition. And the best, surest way of determining the type of any Ruby object (method parameter or not) is to send it a message and see if it responds, much as the best way to determine if an orange really is orange is to shine light on it and measure the frequencies reflected back. In this regard it's essentially Lisp with both syntax _and_ physics

>> With Ruby you're free to play with the raw type-space in its full glory, and that introduces certain fundamental limitations on just how much you can definitively say about that type-space at any given point in time.
> 
> I still don't see an actual incompatibility here. Yes, there will be times when type annotations will be less useful, but I don't see the logical step in saying: "because type annotations are not always useful, they are *never* useful." Ruby certainly gives you lots of freedom, but as far as I've seen, most of the real world still uses that 80% of Ruby that does not bend any space-time continuums. Correct me if I'm wrong. Those annotations could be applied there with great benefit.

There is an implicit assumption in this argument about the runtime object environment in which particular code is executed. It is not a sound assumption if code has any external dependencies unless they guarantee that objects generated will always obey these nominal type annotations.

>> Annotations might well be useful markers in some circumstances, but just as in QM an observation causes the quantum world to collapse into classical reality so with Ruby you run the risk of collapsing back into a static nominal type system with all that entails.
> 
> I'm not sure I buy this comparison. Adding *optional* type annotations is unlikely to reduce Ruby's expressiveness down to that of Java unless you make *improper* use of the type annotation syntax (and iff you actually use type annotations at all). In a properly designed system, however, you're not actually losing any freedom, you're simply enforcing rules that would have otherwise been defined in business logic (type checking, object dispatch). For example, if you have method foo that takes param x and you call `x.reduce` inside foo, you're not "collapsing" any expressiveness reality by enforcing that x be of duck-type #reduce. The same idea would go for any code that would "raise unless type.is_a?(Foo)"-- which happens to be a lot of code. Whether or not the latter is a valid idiom is, as I said before, an opinionated/stylistic discussion for another thread.

If the coding styles which the Ruby Way encourages seem opinionated it's because they've evolved over a long period of time to suit the essential character of Ruby's type system. The same is true of the popular coding styles adopted amongst any language community.

Ruby already gives us all the tools we need to enforce nominal typing in those rare situations where it might be appropriate to a problem domain, which is a very different proposition than whether or not the resulting code fits nicely with a particular design style.

Incidentally asking object x if it responds to method y is not about enforcing type, any more than poking a bear with a stick is: it's about finding out whether or not that particular entity can and will respond to the message _at this instant in time_. An object may gain or loose responsiveness throughout the lifetime of a program execution and there's no reason in principle why any particular nominal type with which it is cognate will not do the same. Just consider Object::method_missing. This allows any object to respond to any message regardless of all other type considerations and to respond with an object of arbitrary type (leaving it up to the object sending that message to decide whether or not the resulting response is meaningful) or to raise a custom exception applicable at a higher level of scope than the sending object can be aware of.

>> I appreciate this is a very high-level analysis so I'll bring this down to earth.
>> 
>> In essence every Ruby object is first and foremost an instance of its own singleton class backed up by a superposition of classes, meta-classes and modules from which that singleton class draws its specific character. Any of these elements might be reloaded or redefined at any time using appropriate hooks in the runtime. Duck-typing allows us to conveniently overlook this fact by focusing on what usually matters to us when we're reasoning about our code: whether or not an object responds to the messages we're interested in sending it, and how to elegantly recover when in fact that message is inappropriate - either through runtime exception handling or by capturing the message with method_missing and performing some fallback operation.
>> 
>> In this model runtime type checking and method overloading can already be performed with Object::kind_of?, but the type guarantee is localised to a particular expression in the code and subsequent expressions may change the nominal type of the object in question. As such it's a brittle idiom because it captures only a snapshot of the full system state.
> 
> I'm not sure I see a problem with this either. Can you provide an actual example that shows why you would need full system state to do run-time type checking or run-time method dispatch involving method overloading? It doesn't seem to me that knowing the "full" system state is necessary.

In a given situation it may be possible to reduce either the number of dimensions of Ruby's type space which had a bearing, or at least to place limits on where along that particular axis the state would be relevant. However the trouble with languages in which type space transformations happen routinely throughout execution is that both of those operations can be impossible to replicate from a fixed and static perspective. This sounds suspiciously like the general algorithm for determining the correct bounding dimensions might be O(N.M) or possibly even O((N.M)^2) or higher - i.e. pathological for all except the most trivial examples.

The main tactic which would diminish this complexity would be pervasive use of type annotations, but then we're back to collapsing Ruby's type space into a static form which removes many (most?) of the advantages that Ruby code currently provides in expressive domain modelling compared to static languages _but without any of the performance advantages that static languages provide compared to Ruby_.

> As I see it, method dispatch would simply need to be updated so that before the VM passes a method call to an object, it searches the method best matching the given parameters (comparing the parameters' "kind_of?"/"respond_to?" against the given type annotations, be they class/mixins or duck-types). Note that this would be precisely what people do now when implementing overloading, except that it would be offloaded onto the VM, providing a little bit of performance (not that relevant), but more importantly, a much more eloquent syntax.

This all has a runtime dispatch cost. Admittedly the cost would be more serious for those cases where methods have been overloaded, but it would still impact on every method dispatch and hence slow down all Ruby programs to support an idiom that seems fundamentally at odds with the Ruby way of doing things. There may be optimisations which could reduce this cost such that it's close to that currently involved in method dispatch but it's unlikely it could be completely removed without some further additions to the language.

I think it's also important to remember that an explicit axiom of Martin's proposal was that it would involve no runtime changes to Ruby and that the annotations would only be of interest to the compilation phases of program interpretation.

> You're focusing a lot on compiler optimizations, and you only seem to point out that there are complications with such optimizations (not surprising), but you didn't really tackle any problems with the other benefits I listed. Personally, I see compiler optimization as the least worthwhile benefit here, and I wouldn't really mind taking that out of the list entirely.
> 
> To be clear, I wouldn't want Ruby to be less expressive OR lose the typing benefits it already has. If compiler optimizations introduced those realities, I would be against it too. However, as I mentioned, performance is not the only benefit of type annotations.


If I focused primarily on performance optimisations it's because the other points (as mentioned above) are either already addressable in one form or another with Ruby as it stands, or else could be addressed by making tools aware of the particular idioms involved.

In particular we can enforce isonominal types (types which at a given degree of resolution can be considered to be the same nominal type thanks to their location in a class or meta-class hierarchy) using Object::is_a? or Object::kind_of? so in cases where an incorrect isonominal type would be a domain error we already get a meaningful exception which can be rescued.

Any tool should be capable of spotting these statements where they're explicitly expressed in code (although that becomes a little more tricky where the constraint is introduced with an eval or a closure) and generating an appropriate comment in our documentation without needing Ruby to possess an explicit annotation syntax.

Working the other way around - having the annotation but not enforcing the constraint in runtime code - would give no demonstrable benefit and should probably be considered a defect to be rectified before code ships.

The method overloading issue is somewhat tangental to type annotations as that's an example of trying to fit a foreign idiom based on direct method invocation into a language which favours a message dispatch style. Since I started working with Ruby I've come across many ad hoc implementations of method overloading (and in my early code even cooked up a few myself) but this kind of tight coupling to nominal types really isn't necessary in structurally typed languages and is generally a code smell.

To summarise, I guess I just find the whole line of thought embodied in this modified proposal troubling. It makes assumptions about the tools used for development, presents an inaccurate view of Ruby's fundamental type system, and provides an artificial sense of security regarding the quality of code.

I fully concede that the mileage of others may vary on these point and that my caution may well be misplaced.


Ellie

Eleanor McHugh
Games With Brains
http://feyeleanor.tel
----
raise ArgumentError unless @reality.responds_to? :reason