When formatting error messages, first try to format into a fixed
buffer instead of formatting once to get the length then a second
time to get the actual format.
https://github.com/ruby/prism/commit/557cffe23e
This is the output of
ruby tool/sync_default_gems.rb prism
git rm -r lib/prism.rb lib/prism
at https://github.com/ruby/prism/commit/0a5f9c9, with the new mapping:
lib/prism is now lib/ruby/prism, the namespace is rewritten to
Ruby::Prism, and lib/prism/prism.gemspec is no longer imported, so
prism is no longer installed as a default gem.
prism/srcs.mk.in (excluded from the sync) follows the template engine,
now Ruby::Prism::Template: it defines the Ruby module, which does not
exist in BASERUBY older than Ruby 4.0, before loading
templates/template.rb.
Ruby::Prism is the very parser that parsed the running code, no
matter which version of the prism gem is installed; the two can be
loaded side by side (the ruby binary is built with -fvisibility=hidden,
so the gem's pm_* and rb_cPrism* symbols do not collide with the
built-in ones).
[Bug #22379]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When we hit EOF and still have lex modes left, it means some content was unterminated.
Heredocs specifically have logic that needs to happen when the body finished lexing.
If we don't reset the mode back to how it was before, it will not continue lexing at the correct place.
Followup to https://github.com/ruby/prism/pull/3918.
We can't call into `parser_lex` since it resets token locations.
https://github.com/ruby/prism/commit/27c24fdc0d
This is more akin to how all the other ones work.
Instead I added most of this to the errors test like focusing
a few specifically and creating new expectations.
https://github.com/ruby/prism/commit/b21922b0a1
In the C API, we want to use slices instead of locations in the
AST. In this case a "slice" is effectively the same thing as the
location, expect it is represented using a 32-bit offset and a
32-bit length. This will cut down on half of the space of all of
the locations in the AST.
Note that from the Ruby/Java/JavaScript side, this is effectively
an invisible change. This only impacts the C/Rust side.
Mostly not having to list version-specific excludes when testing against ripper/parse.y
Also don't test new syntax additions against the parser gems. The version support
for them may (or may not) be expanded but we shouldn't bother while the ruby version
hasn't even released yet.
(ruby_parser translation is not versioned, so let as is for now)
I also removed excludes that have since been implemented by parse.y
https://github.com/ruby/prism/commit/e5a0221c37
This one has been on my mind for a while now.
Currently, there are only tests against the latest syntax version.
This changes the snapshot structure as follows:
* Snapshots at their current location are tested against all syntax versions
* Snapshots inside a version folder like "3.3" are tested against all versions starting from that version
* Snapshots inside a version folder like "3.3-4.2" are tested against all versions in the given range.
This makes sure that as new syntax is added, older versions still work as expected.
I also added a few tests for now valid syntax that should be invalid in older versions (and the other way around as well)
These tests run really fast. So even though it does 3x the work for these, I am still able to run the whole test suite in just 11 seconds.
https://github.com/ruby/prism/commit/5191b1aa68
If we hit an EOF token, and the character before the EOF is a newline,
we should make EOF token start at the previous newline. That way any
errors reported will occur on that line.
For example "foo(\n" should report an error on line 1 even though the
EOF technically occurs on line 2.
[Bug #20918]
https://bugs.ruby-lang.org/issues/20918https://github.com/ruby/prism/commit/60bc43de8e
In some cases Prism was either not raising an appropriate `void value
expression` error, or raising that error when the syntax is considered
valid.
To fix this Prism needs to check whether we have other clauses on the
`begin` rather than just returning `cast->statements`.
* If the `cast->statements` are null and the `cast->ensure_clause` is
not null, set the code to `cast->ensure_clause`
* else
* If there is a `cast->rescue_clause`
* Check if `cast->statements` are null and `cast->rescue_clause->statements`
are null, and return `NULL`
* Check if there is an `else_clause`, and set the node to
`cast->else_clause`.
* Otherwise return `cast->statements` as the node
* return `cast->statements` as the node
See tests for test cases. Note I took these directly from CRuby so if
desired I can delete them since the test will now pass. This only fixes
one test in the `test_parse` file, taking failures from 14 to 13.
This fixes `TestParse#test_void_value_in_rhs` and is related to
issue #2791.
https://github.com/ruby/prism/commit/398152b412
Kevin Newton