timed-out memory leak child
(https://github.com/ruby/test-unit-ruby-core/pull/37)
* Close stdin before terminating a timed-out memory leak child
With a sample range, assert_no_memory_leak keeps the child's stdin open,
and the child's ensure clause reads a "continue" line from it after each
sample. When the timeout fires while the child is running the sampled
code, EnvUtil.terminate sends SIGTERM, and the child's ensure clause
then blocks reading stdin, which was closed only after terminate
returned. SIGTERM cannot end the child, so terminate escalates to
SIGABRT and then waits for the aborted child without a time limit. On
ruby/ruby's YJIT macOS CI jobs the child sometimes did not exit after
the RUBY_ON_BUG core dump, and the test worker hung until the runner
killed it 20 minutes later. In other runs the test passed but left a
crash report for the child.
Close stdin first. The child's read then returns nil, and its ensure
clause breaks out of the sample loop. The break discards the
SignalException raised by SIGTERM, so the child exits with status 0. If
SIGTERM arrives before the child reaches the sample loop, the child
still dies from SIGTERM.
The existing timeout test now also checks that the child exited
normally or by SIGTERM.
https://github.com/ruby/test-unit-ruby-core/commit/ce7a92e724
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_for_fd in rbs 4.2.0 calls Dir.for_fd twice with the fd of a Dir it
then closes, and never closes the two Dirs Dir.for_fd returned. Both
still own that fd number. When GC later finalizes them, closedir()
closes whatever file has since been given the same number, and the
owner of that file fails on close with:
Errno::EBADF: Bad file descriptor @ fptr_finalize_flush
In test-bundled-gems this usually hits FileTestSingletonTest, which
runs right after Dir_test.rb and opens files in every test. Whether it
fails depends on GC timing, so it shows up as a flake.
https://github.com/ruby/rbs/pull/3191 fixes the test upstream, but no released rbs has it yet.
Skip the test until the bundled rbs includes that fix.
busybox lsof ignores the options and prints no header line, so the
FD/TYPE/NODE columns cannot be found and check_fd_leak crashed with
NoMethodError, which killed the test worker. Treat such lsof as
unavailable; leaked fds are still reported.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a new `Primitive.attr :always_inline`. Force inline ISEQs with this
attribute. This is helpful for e.g. `BasicObject#!=`, which is otherwise
megamorphic, and very small.
Other JITs also have a "always inline small methods" policy but I didn't
want to wade into that yet.
The current version is too dynamic for rdoc
and so C methods like `Ruby::Prism.lex` are not doccumented.
Also tweak `Ruby::Prism` introduction. Ruby makes breaking changes
in minor versions all the time, no need to call it out.
Gemspecs can assign local variables such as `files`, `file` and
`base`, which overwrote the ones of `load_gemspec`. For instance,
ipaddr.gemspec and net-http.gemspec assign `file`, and `loaded_from`
of their specs pointed to their Ruby files instead of the gemspecs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The files collected for a default gem are relative to its lib
directory, while the files listed by the gemspec itself are relative
to the gem root. `load_gemspec` prefixed `lib/` to any `.rb` entry
missing from the current directory, so entries of the gemspec itself
became `lib/lib/...`, and collected extension libraries such as
`io/console.bundle` were left without the prefix.
Also keep the collected files before evaluating the gemspec, since
the gemspec is evaluated in the binding of `load_gemspec` and may
overwrite `files`, as strscan.gemspec does.
https://bugs.ruby-lang.org/issues/22420
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since a1efb9ae6b (Promote prism to a bundled gem), the prism extension
is also built with gcov under .bundle/gems/prism-1.9.0/ext/prism. Its
generated node.c carries `#line` directives pointing at
"prism/templates/src/node.c.erb", a path relative to the ruby/ruby
tree layout. Resolved against the gem's build directory it becomes
.bundle/gems/prism-1.9.0/ext/prism/prism/templates/src/node.c.erb,
which does not exist.
genhtml in LCOV 2.0 (ubuntu-latest) aborts on a missing source file
even with `--ignore-errors source`, so `make lcov` has been failing on
every ruby/actions coverage run since then.
Bundled gems are not part of the interpreter sources, so drop
.bundle/* in run_lcov_remove, next to the existing test/* and
ext/-test-/* exclusions.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Ruby::Prism itself does not compile the program (prism_compile.c does,
and other implementations may not even compile an AST into bytecode).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
prism is no longer a default gem (the built-in parser is Ruby::Prism
now), so add the prism gem to gems/bundled_gems. `require "prism"`
keeps providing the gem, which can be any version. lib/bundled_gems.rb
warns when it is required under bundler without a Gemfile entry, like
other gems promoted to bundled gems.
The gem is pinned to the upstream revision that ruby/ruby is synced
with, because the tests of the released 1.9.0 compare with the parser
of the running Ruby, whose grammar has moved on in 4.1.
prism does not keep its templated files in git, so BundledGem.build
(used by both common.mk and defs/gmake.mk) generates them before
building the gem from the checkout. Since the gem built from the
revision has the same version as the released one, BundledGem.unpack
now clears the target directory first; otherwise files removed upstream
(ext/prism/api_pack.c) remain from the released gem and break the
build.
tool/test-bundled-gems.rb: several bundled gems (irb, rdoc, rbs,
typeprof, ...) require prism, and their tests spawn ruby with
--disable=gems, where the prism gem is not loadable anymore. Preset its
load paths in RUBYLIB, as is already done for debug.
[Bug #22379]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
prism will be built into the interpreter as a plain library under the
Ruby::Prism namespace, not as a default gem (the prism gem becomes a
bundled gem). Teach the sync tool about the new layout:
- lib/prism{,.rb} is mapped to lib/ruby/prism{,.rb}, and the gemspec is
no longer imported.
- prism_do_fixup rewrites the namespace of the synced files, so that the
upstream repository does not need to know about the renaming:
* Ruby: `Prism` -> `Ruby::Prism`, and `require`/`autoload` of
"prism/..." -> "ruby/prism/..." (require_relative is untouched)
* C: rb_define_module("Prism") -> rb_define_module_under(Ruby, "Prism")
The C rule applies to plain C files, and the Ruby rule to the others,
including the ERB templates and the template engine
(prism/templates/template.rb, i.e. Ruby::Prism::Template).
The rewrite is idempotent and is applied in both the full-copy sync
and the commit-by-commit sync (fixup_commit) used by GitHub Actions.
- check_prerelease_version skips prism, which has no gemspec in-tree.
Unit tests for the rewrite rules are added to
tool/test/test_sync_default_gems.rb.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`$(LIBRUBYARG_STATIC)` already ends with `$(MAINLIBS)`, so appending them again makes the Xcode 15+ linker warn about duplicate libraries. The shared ruby still needs `$(MAINLIBS)` [Bug #20072], so drop them only when `$(LIBRUBYARG)` is the static one. `PROGRAM_LIBS` is internal to the build and is not exported to rbconfig.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ranking tag names picked up ruby/ruby's own tags fetched into a clone, such as v4.0.2 in syntax_suggest, and skipped prereleases like RubyGems 4.1.0.beta1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The top-level depend declares paths of the master tree, so minimizing dependencies after syncing an older prism release raised and stopped `all release`. Marking the version as a prerelease is wrong there too, and could happen right after a release while latest.json still served the previous version.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The script body runs in the `-n` loop, so `info` kept the value from the previous gem when the current one has no revision.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Worker#kill and quit_workers both need the test process rather than the runruby.rb process on Windows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The KILL sent when Process.waitall times out in quit_workers also hit only the runruby.rb process on Windows, leaving the test process running. It now targets only the workers still running, since the pids of the test processes that have exited may already be reused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Windows, exec in runruby.rb spawns the worker as another process and waits for it, so killing the launched pid left the worker running with the inherited stderr open, and GitHub Actions waited on it until the step timed out. The worker now reports its own pid, which is also the process the debugger should dump.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Git for Windows 2.56.0 dies with "unable to access 'NUL': Invalid argument" when GIT_CONFIG_SYSTEM or GIT_CONFIG_GLOBAL is `NUL`, which stops the revision.h generation on mswin. `/dev/null` is the value git documents for them, and Git for Windows has accepted it since 2.32, the first version that reads them.
https://github.com/git-for-windows/git/issues/6449
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test-bundled-gems runs the tests with the test-unit gem, which loads EnvUtil through gems/lib but never goes through tool/lib/test/unit.rb, so the scale was ignored there. `--timeout-scale` still takes precedence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Extended Support for Solaris 9 ended in 2014. Solaris 10 and later have setenv and unsetenv, and libumem is always available, so the version specific fallbacks are no longer needed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EnvUtil passes RUBY_ON_BUG down to child processes, so one set for the whole test run acts on these two tests, which crash a child on purpose. The foreign store test runs only with RUBY_DEBUG. The other such tests already clear it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Like the other tests that crash a child on purpose, these leave a core on every run wherever the core size limit allows it, which buries real crashes among the collected dumps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A relative name was expanded against the current directory first, so a
build directory nested inside the source tree made generated files such
as ext/ripper/ripper.y look like source-tree files, and the extension's
own depend file was no longer found.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
In 7eab8b5d48 I wanted to avoid extra
work. Since we don't actually use any profile information in ZJIT for
compiling `opt_case_dispatch`, I decided not to call the profiling
instruction.
However, this caused a regression in `mail` and `rubocop`: this call to
the profiler is also our hook to the function lifecycle code, which
triggers a rewrite from the ZJIT version of the instruction back to the
normal bare opcode. My commit stopped rewriting back to the original
opcode when the ZJIT machinery should have.
Because in the interpreter the bare opcode path is much faster for
integer-only case/when dispatch, this regressed `mail` and `rubocop`.
We'll have to figure out why we are spending a lot of time in
interpreter case/when dispatch for those benchmarks (code size?), but
for now this fixes the regression.
Ruby 1.9.1 dropped DJGPP, Windows CE and VMS, NeXTSTEP, OpenStep and Rhapsody went in 2011, and Ruby 2.3 dropped OS/2, BeOS and Borland C. None of these names appears in RUBY_PLATFORM or host_os on a supported platform.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Timeout.timeout in Ruby 2.3 takes at most two arguments, so EnvUtil.timeout always raised ArgumentError there. EnvUtil.terminate reaches it through Debugger#dump on Windows when gdb is installed, which broke test_terminate_reaps_exited_child on windows-latest 2.3.
https://github.com/ruby/test-unit-ruby-core/commit/42d15562b2
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
We are not going to optimized switch-based dispatch with the hash table, at
least not yet, but we should let the interpreter profile the execution paths
before ZJIT runs them.
This should inject a lot more type information for the compiler. For example,
on rubyboy, we go from ~1080ms/iter to ~820ms/iter.
Later, we can make the dispatch faster too.
On Windows, Process.kill raises Errno::ESRCH for a child that has exited but has not been reaped, so terminate returned a stale $? and left the child to ZombieHunter. TestIO#test_write_32bit_boundary hit this on Windows 11-arm, where the child finished writing after its timeout expired and before it was killed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Regexp fixes are merged into ruby/ruby directly and the copies have diverged, so asking contributors to file them to k-takata/Onigmo only misleads them.
https://github.com/ruby/ruby/pull/19026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
