[ruby-core:116111] [Ruby master Bug#20159] Prism assertion failure even if it is executed with --parser=parse.y

Issue #20159 has been updated by mame (Yusuke Endoh).


@kddnewton I could reproduce the issue with no `RUBYOPT`. The error occurs because SyntaxSuggest uses `Prism.lex_compat`

```
$ cat a.rb
<<A+%
A

$ ruby -v a.rb
ruby 3.4.0dev (2024-01-09T08:14:55Z master 149373ce7f) [x86_64-linux]
a.rb:1: warning: possibly useless use of + in void context
ruby: prism/util/pm_newline_list.c:42: pm_newline_list_append: Assertion `list->size == 0 || newline_offset > list->offsets[list->size - 1]' failed.
Aborted

$ gdb --args ruby a.rb
...
(gdb) r
Starting program: /home/mame/work/ruby/ruby a.rb
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffde56f640 (LWP 55846)]
[New Thread 0x7fffdc5ef640 (LWP 55847)]
ruby: prism/util/pm_newline_list.c:42: pm_newline_list_append: Assertion `list->size == 0 || newline_offset > list->offsets[list->size - 1]' failed.

Thread 1 "ruby" received signal SIGABRT, Aborted.
__pthread_kill_implementation (no_tid=0, signo=6, threadid=140737349576640) at ./nptl/pthread_kill.c:44
44      ./nptl/pthread_kill.c: No such file or directory.
(gdb) p rb_backtrace()
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/syntax_suggest/core_ext.rb:40:in `detailed_message'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/syntax_suggest/api.rb:94:in `call'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/timeout.rb:196:in `timeout'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/timeout.rb:42:in `handle_timeout'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/timeout.rb:187:in `block in timeout'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/syntax_suggest/api.rb:96:in `block in call'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/syntax_suggest/api.rb:96:in `new'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/syntax_suggest/code_search.rb:61:in `initialize'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/syntax_suggest/code_search.rb:61:in `new'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/syntax_suggest/clean_document.rb:89:in `initialize'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/syntax_suggest/code_line.rb:31:in `from_source'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/syntax_suggest/code_line.rb:31:in `new'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/syntax_suggest/lex_all.rb:22:in `initialize'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/syntax_suggest/lex_all.rb:45:in `lex'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/prism.rb:47:in `lex_compat'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/prism/lex_compat.rb:610:in `result'
        from /home/mame/work/ruby/local/lib/ruby/3.4.0+0/prism/lex_compat.rb:610:in `lex'
$1 = void
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737349576640) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140737349576640) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140737349576640, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007ffff7beb476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007ffff7bd17f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x00007ffff7bd171b in __assert_fail_base (fmt=0x7ffff7d86130 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=0x5555559ea548 "list->size == 0 || newline_offset > list->offsets[list->size - 1]",
    file=0x5555559ea500 "prism/util/pm_newline_list.c", line=42, function=<optimized out>) at ./assert/assert.c:92
#6  0x00007ffff7be2e96 in __GI___assert_fail (
    assertion=assertion@entry=0x5555559ea548 "list->size == 0 || newline_offset > list->offsets[list->size - 1]",
    file=file@entry=0x5555559ea500 "prism/util/pm_newline_list.c", line=line@entry=42,
    function=function@entry=0x5555559ea5b0 <__PRETTY_FUNCTION__.1> "pm_newline_list_append") at ./assert/assert.c:101
#7  0x00005555557de7e6 in pm_newline_list_append (list=list@entry=0x7fffffffcdd8, cursor=<optimized out>) at prism/util/pm_newline_list.c:42
#8  0x00005555557eb307 in parser_lex (parser=0x7fffffffcbb0) at prism/prism.c:9057
#9  0x0000555555806d68 in parse_expression_infix (parser=0x7fffffffcbb0, node=0x5555566bdda0,
    previous_binding_power=PM_BINDING_POWER_STATEMENT, binding_power=39, accepts_command_call=<optimized out>) at prism/prism.c:16836
#10 0x00005555557fb94a in parse_expression (parser=parser@entry=0x7fffffffcbb0, binding_power=binding_power@entry=PM_BINDING_POWER_STATEMENT,
    accepts_command_call=accepts_command_call@entry=true, diag_id=diag_id@entry=PM_ERR_CANNOT_PARSE_EXPRESSION) at prism/prism.c:17166
#11 0x00005555557fbb55 in parse_statements (parser=parser@entry=0x7fffffffcbb0, context=context@entry=PM_CONTEXT_MAIN) at prism/prism.c:11045
#12 0x00005555557fbd5c in parse_program (parser=parser@entry=0x7fffffffcbb0) at prism/prism.c:17244
#13 0x000055555580b049 in pm_parse (parser=parser@entry=0x7fffffffcbb0) at prism/prism.c:17478
#14 0x0000555555990e07 in parse_lex_input (input=input@entry=0x7fffffffce80, options=options@entry=0x7fffffffcea0,
    return_nodes=return_nodes@entry=false) at prism/extension.c:525
#15 0x0000555555991b6a in lex (argc=<optimized out>, argv=<optimized out>, self=<optimized out>) at prism/extension.c:571
#16 0x000055555578f7f4 in vm_call_cfunc_with_frame_ (stack_bottom=<optimized out>, argv=<optimized out>, argc=<optimized out>,
    calling=<optimized out>, reg_cfp=<optimized out>, ec=<optimized out>) at /home/mame/work/ruby/vm_insnhelper.c:3490
#17 vm_call_cfunc_with_frame (calling=<optimized out>, reg_cfp=<optimized out>, ec=<optimized out>)
    at /home/mame/work/ruby/vm_insnhelper.c:3518
...
```

----------------------------------------
Bug #20159: Prism assertion failure even if it is executed with --parser=parse.y
https://bugs.ruby-lang.org/issues/20159#change-106106

* Author: tompng (tomoya ishida)
* Status: Open
* Priority: Normal
* Assignee: kddnewton (Kevin Newton)
* ruby -v: ruby 3.4.0dev (2024-01-07T16:23:58Z master 8b86d6f0c1) [x86_64-linux]
* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
Prism fails to parse this code with assertion failed. Reported at https://github.com/ruby/prism/issues/1616
~~~ruby
<<A+%
A
~~~
Ruby exits with assertion failed even if it's run with option `--parser=parse.y`

~~~
# ruby -v
ruby 3.4.0dev (2024-01-07T16:23:58Z master 8b86d6f0c1) [x86_64-linux]
# (echo "<<A+%"; echo "A") > a.rb
# ruby --parser=parse.y a.rb
ruby: prism/util/pm_newline_list.c:42: pm_newline_list_append: Assertion `list->size == 0 || newline_offset > list->offsets[list->size - 1]' failed.
qemu: uncaught target signal 6 (Aborted) - core dumped
Aborted
~~~

`eval` will correclty result in SyntaxError, `require` and `load` exits with assertion failed.
~~~ruby
# irb
irb(main):001> eval File.read('a.rb')
(irb):1:in `eval': (eval at (irb):1):1: unterminated string meets end of file (SyntaxError)
irb(main):002> load 'a.rb'
prism/util/pm_newline_list.c:42: pm_newline_list_append: Assertion `list->size == 0 || newline_offset > list->offsets[list->size - 1]' failed.
qemu: uncaught target signal 6 (Aborted) - core dumped
Aborted
~~~



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- [e-mail]
 To unsubscribe send an email to [e-mail]
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/