Can I try the error tolerance of Prism on CRuby?

[k-nasa]

Hello. I ran a program with a syntax error in Ruby 3.3, and it displayed a crash report from Prism.

class Foo
  def initialize(
    true
  end

$  ruby --parser=prism main.rb

ruby: warning: The compiler based on the Prism parser is currently experimental and compatibility with the compiler based on parse.y is not yet complete. Please report any issues you find on the `ruby/prism` issue tracker.
main.rb: [BUG] A pm_missing_node_t should not exist in prism's AST.
ruby 3.3.0 (2023-12-25 revision 5124f9ac75) +PRISM [arm64-darwin22]

-- Crash Report log information --------------------------------------------
   See Crash Report log file in one of the following locations:
     * ~/Library/Logs/DiagnosticReports
     * /Library/Logs/DiagnosticReports
   for more details.
Don't forget to include the above Crash Report log file in bug reports.

-- Control frame information -----------------------------------------------
c:0001 p:0000 s:0003 E:001dc0 DUMMY  [FINISH]


-- Threading information ---------------------------------------------------
Total ractor count: 1
Ruby thread count for this ractor: 1

-- C level backtrace information -------------------------------------------
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(rb_vm_bugreport+0xb4c) [0x1053eaae8]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(rb_bug_without_die+0xfc) [0x10522cf3c]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(rb_bug+0x1c) [0x1054dfa98]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(pm_compile_node.cold.8+0x0) [0x1054df374]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(pm_compile_node+0xa938) [0x1051dc41c]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(pm_compile_node+0x3a2c) [0x1051d5510]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(pm_compile_node+0xa418) [0x1051dbefc]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(rb_iseq_compile_prism_node+0xf8) [0x1051ca910]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(pm_iseq_new_with_opt+0xf8) [0x10528f050]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(pm_new_child_iseq+0x8c) [0x1051fa5e4]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(pm_compile_node+0x16c8) [0x1051d31ac]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(pm_compile_node+0x3a2c) [0x1051d5510]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(pm_compile_node+0xa418) [0x1051dbefc]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(rb_iseq_compile_prism_node+0xf8) [0x1051ca910]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(iseqw_s_compile_prism_compile+0xf0) [0x1052900f8]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(rb_iseq_new_main_prism+0xc8) [0x10528ffc8]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(ruby_process_options+0x1728) [0x105349288]
/Users/nasa/.rbenv/versions/3.3.0/lib/libruby.3.3.dylib(ruby_options+0xb4) [0x1052382f4]
/Users/nasa/.rbenv/versions/3.3.0/bin/ruby(main+0x64) [0x104b07f30]

-- Other runtime information -----------------------------------------------

* Loaded script: main.rb

* Loaded features:

    0 enumerator.so
    1 thread.rb
    2 fiber.so
    3 rational.so
    4 complex.so
    5 ruby2_keywords.rb
    6 /Users/nasa/.rbenv/versions/3.3.0/lib/ruby/3.3.0/arm64-darwin22/enc/encdb.bundle
    7 /Users/nasa/.rbenv/versions/3.3.0/lib/ruby/3.3.0/arm64-darwin22/enc/trans/transdb.bundle

I am interested in testing Prism's error tolerance and its ergonomically designed error messages. I would be grateful if you could let me know if these features are in a phase where they can be tested with CRuby.

I think Prism is fantastic. I am very pleased that such a parser has been developed.

[kddnewton]

Thanks for opening this @k-nasa — unfortunately we didn't get out error messages properly handled before 3.3 went out, so it still crashes. If you were to run that on Ruby master, it would say:

./miniruby: warning: The compiler based on the Prism parser is currently experimental and compatibility with the compiler based on parse.y is not yet complete. Please report any issues you find on the `ruby/prism` issue tracker.
test.rb: syntax errors found (SyntaxError)
  1 | class Foo
> 2 |   def initialize(
    |                  ^ expected a `)` to close the parameters
  3 |     true
  4 |   end
> 5 | 
    | ^ expected an `end` to close the `class` statement
    | ^ cannot parse the expression

I'm hoping to get it backported to Ruby 3.3.1, but at the moment you'll have to use master to get access to this.

[k-nasa]

@kddnewton

Thank you! I'm looking forward to the release of 3.3.1.
Could you tell me which version you ran?

I'm investigating the cause because it didn't work on the master.
ruby/ruby@7b0f6d6

It only hangs when I enable prism.
Looking at the crash report, it seems like the mutex lock might not be functioning properly.

~/lab/sandbox/ruby_sandobx - SIGHUP
:( % /Users/wantedly564/lab/oss/ruby/build/ruby --parser=prism main.rb
/Users/wantedly564/lab/oss/ruby/build/ruby: warning: The compiler based on the Prism parser is currently experimental and compatibility with the compiler based on parse.y is not yet complete. Please report any issues you find on the `ruby/prism` issue tracker.
^C[BUG] pthread_mutex_lock: Invalid argument (EINVAL)
ruby 3.4.0dev (2024-01-19T11:02:59Z master 7b0f6d6d94) +PRISM [arm64-darwin22]

-- Crash Report log information --------------------------------------------
   See Crash Report log file in one of the following locations:
     * ~/Library/Logs/DiagnosticReports
     * /Library/Logs/DiagnosticReports
   for more details.
Don't forget to include the above Crash Report log file in bug reports.

-- C level backtrace information -------------------------------------------
/Users/wantedly564/lab/oss/ruby/build/ruby(rb_vm_bugreport+0xb4c) [0x10458bbec] ../vm_dump.c:1151
/Users/wantedly564/lab/oss/ruby/build/ruby(rb_vm_bugreport) (null):0
/Users/wantedly564/lab/oss/ruby/build/ruby(rb_bug_without_die+0xfc) [0x1043ccdcc] ../error.c:1042
/Users/wantedly564/lab/oss/ruby/build/ruby(rb_bug+0x1c) [0x104683500] ../error.c:1050
/Users/wantedly564/lab/oss/ruby/build/ruby(rb_strerrno+0x0) [0x1043cd410] ../error.c:1082
/Users/wantedly564/lab/oss/ruby/build/ruby(rb_bug_errno) (null):0
/Users/wantedly564/lab/oss/ruby/build/ruby(timer_thread_func+0x0) [0x10453778c]
/Users/wantedly564/lab/oss/ruby/build/ruby(timer_thread_polling) ../thread_pthread_mn.c:890
/Users/wantedly564/lab/oss/ruby/build/ruby(timer_thread_func) ../thread_pthread.c:3026
/usr/lib/system/libsystem_pthread.dylib(_pthread_start+0x94) [0x1a7abffa8]

-- Other runtime information -----------------------------------------------

* Loaded script: main.rb

* Loaded features:

    0 enumerator.so
    1 thread.rb
    2 fiber.so
    3 rational.so
    4 complex.so
    5 ruby2_keywords.rb
    6 /Users/wantedly564/.rubies/ruby-master/lib/ruby/3.4.0+0/arm64-darwin22/enc/encdb.bundle
    7 /Users/wantedly564/.rubies/ruby-master/lib/ruby/3.4.0+0/arm64-darwin22/enc/trans/transdb.bundle

[k-nasa]

I have confirmed that it works in a Linux environment!
It might not work on macOS.

ubuntu@ip-172-31-0-76:~/ruby/build$ ./miniruby --parser=prism test.rb
./miniruby: warning: The compiler based on the Prism parser is currently experimental and compatibility with the compiler based on parse.y is not yet complete. Please report any issues you find on the `ruby/prism` issue tracker.
test.rb: syntax errors found (SyntaxError)
  1 | class Foo
> 2 |   def initialize(
    |                  ^ expected a `)` to close the parameters
  3 |     true
  4 |   end
> 5 |
    | ^ expected an `end` to close the `class` statement
    | ^ cannot parse the expression

[k-nasa]

I was mistaken. It seems that whether a parse error is displayed or not is determined by whether there is a blank line at the end of the file, not by differences in the execution environment

exists new line

~/lab/sandbox/ruby_sandobx
:) % bat main.rb
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: main.rb
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ class Foo
   2   │   def initialize(
   3   │     true
   4   │   end
   5   │
───────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

~/lab/sandbox/ruby_sandobx
:) % /Users/wantedly564/lab/oss/ruby/build/ruby --parser=prism main.rb
/Users/wantedly564/lab/oss/ruby/build/ruby: warning: The compiler based on the Prism parser is currently experimental and compatibility with the compiler based on parse.y is not yet complete. Please report any issues you find on the `ruby/prism` issue tracker.
main.rb: syntax errors found (SyntaxError)
  1 | class Foo
> 2 |   def initialize(
    |                  ^ expected a `)` to close the parameters
  3 |     true
  4 |   end
> 5 |
    | ^ expected an `end` to close the `class` statement
    | ^ cannot parse the expression

not exists new line

~/lab/sandbox/ruby_sandobx
:) % bat main.rb
───────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: main.rb
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ class Foo
   2   │   def initialize(
   3   │     true
   4   │   end
───────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

~/lab/sandbox/ruby_sandobx
:) % /Users/wantedly564/lab/oss/ruby/build/ruby --parser=prism main.rb
/Users/wantedly564/lab/oss/ruby/build/ruby: warning: The compiler based on the Prism parser is currently experimental and compatibility with the compiler based on parse.y is not yet complete. Please report any issues you find on the `ruby/prism` issue tracker.
SEGV received in BUS handler
zsh: abort      /Users/wantedly564/lab/oss/ruby/build/ruby --parser=prism main.rb

[kddnewton]

@k-nasa this issue should be fixed on master now. Are you still experiencing it?

[k-nasa]

@kddnewton thanks.
I confirmed fixed it.

~/lab/oss/ruby /build - master  - SIGHUP
:( % ./miniruby --parser=prism ./test.rb
./miniruby: warning: The compiler based on the Prism parser is currently experimental and compatibility with the compiler based on parse.y is not yet complete. Please report any issues you find on the `ruby/prism` issue tracker.
./test.rb: syntax errors found (SyntaxError)
  1 | class Foo
> 2 |   def initialize(
    |                  ^ expected a `)` to close the parameters
  3 |     true
  4 |   end
> 5 |     | ^ expected an `end` to close the `class` statement
    | ^ unexpected end of file, assuming it is closing the parent top level context