[ruby-core:123380] [Ruby Bug#21618] Allow to use the build-in prism version to parse code

Issue #21618 has been updated by Eregon (Benoit Daloze).


Earlopain (Earlopain _) wrote in #note-7:
> I don't think parsing exactly (not just almost the same) as the ruby I am running as is an unreasonable request.

I think they are exactly the same, and if it's not the case that's a bug we can fix in Prism.

I think it is unreasonable to have a duplicate Ruby API just for this under another module (BTW it's quite a lot of classes and memory footprint, not great to have effectively 2 copies of Prism in the same process), + namespacing functions at the C level too (otherwise we get native symbol conflicts), that's a huge amount of complications for as far as I can see no useful difference with using `version: RUBY_VERSION`.
There is no possible way to share Prism node classes, as e.g. newer/older versions might have extra node classes.

> But prism is not patch level aware (that is fine) so even if you do `RUBY_VERSION` it is not perfect. 3.2.0 I think had a bug with nested anonymous block parameters that had a pretty big impact.

I think if there is a relevant difference between Ruby X.Y.Z and X.Y.(Z+1) then Prism could/should take that into account, since it already takes `RUBY_VERSION` as the input.
I would guess though the only differences are bug fixes, and then is it really useful to have the buggy behavior on the buggy Ruby, isn't it better to just have the bug-fixed behavior by ensuring to use a new enough `prism` gem?

BTW do you have a link for that bug?
That's before Prism became a default gem though, so of course it had less testing and usage back then.

> Now I am required to also bump prism with every patch version. If I run older patch versions, I would be have to run older prism versions from around the time my patch version was released.

Only if there is a such a bug and it matters for your usage and you don't use the prism default gem (since that would have the fix).
That's a lot of `if`, IOW I think it's so rare in practice it doesn't matter.
It's also a totally normal workflow to update a gem to get a bug fix.
In such a scenario it's also clearly better to use a newer prism >=A.B.C as a gem/bundle dependency than to get the broken behavior on Ruby 3.2.0 and the correct behavior on 3.2.1.

> > From the description it seems a (the?) main concern is that Prism.parse(code) etc use the latest version by default
> 
> Yeah, I am not a fan of that behaviour as well. I guess this is intential, similar to how the unversioned parser translator class also uses the latest version. But changing that was not really what I was asking for.

I think it's historical at this point, and maybe comes from Prism initially only having one version, but @kddnewton would know better.
My guess is changing the default here would fix more issues than cause incompatibilities.

> ----
> 
> To be clear, patch versions of ruby should never contain any big syntax changes. The impact of using any prism version that understands my major/minor version should be minimal in the majority of cases. But, I would prefer to just have something where I don't even have to worry about such things.
> 
> `Prism.parse("foo", version: RUBY_VERSION)` works good enough but differently to the real intention to the point where it makes me uncomfortable.

I think the solution there is, if there is a relevant difference between Ruby X.Y.Z and X.Y.(Z+1) then Prism should take that into account, and then it matches the intention exactly.

Another thought is if you really want to ensure you use the exact same Prism version as the one used by the interpreter, you could check that as long as that default version version is exposed somehow (e.g. it is though gemspecs). I don't think that's great though then, because it prevents other gems to use a newer Prism version (which would work just fine for your use case too).

----------------------------------------
Bug #21618: Allow to use the build-in prism version to parse code
https://bugs.ruby-lang.org/issues/21618#change-114752

* Author: Earlopain (Earlopain _)
* Status: Open
* Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
Prism is a multi-version parser, which is a great feature.

If one calls `Prism.parse("foo")`, it will always use the latest version prism knows about. This may or may not be the version that is currently executing. This is problematic when one wants to introspect code the same way that the currently running ruby version would.

Consider that some syntax will be disallowed in the next ruby version. I would have to specify the ruby version I want to parse as in order to not run into syntax errors: `Prism.parse("foo", version: RUBY_VERSION)`. But doing so is not feasable because `prism` is distributed out of sync with ruby itself. If the user already has prism in their lockfile, the user may run a prism version that doesn't yet know about lets say ruby 4.0 and thus raise. Similarly, it may parse as an older patch version that has subtle behaviour differences.

`ripper` does not have this issue because it is always tied to the ruby version and it is not distributed as a gem, so what the user has will always be exactly what ruby shipped with. I wish for a similar API that utilizes the prism version bundled with ruby itself.

Libraries like `rails` have moved from ripper to prism because of its superior developer experience but it risks breaking in unexpected ways with new `prism` versions that know about more recent syntax. `error_highlight` for example also suffers from the same defect.

It seems like prism currently has 34 methods that optionally take a version (per the rbi file). Many of these are trivial helper methods like `Prism.parse_success?` (does the parse result have any errors?). I would be happy with supporting only the minimal functions like `Prism.parse` and `Prism.parse_file`. Currently I don't have a use-case for any of the other methods. Pretty much just functions to mirror `RubyVM::InstructionSequence.compile_file` and `RubyVM::InstructionSequence.compile`.

It would be optimal if I would be able to transparently call `Prism.parse("foo", version: "current")` (or maybe even have an unspecified version mean the build-in prism version) but I am not sure how feasable that is since I'm pretty sure logic for this would have to live in ruby itself.

@kddnewton does this feature request make sense? Do you have any other ideas/inputs?



-- 
https://bugs.ruby-lang.org/
______________________________________________
 ruby-core mailing list -- [e-mail]
 To unsubscribe send an email to [e-mail]
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/