224 Commits
Author SHA1 Message Date
Hiroshi SHIBATAandClaude Opus 5.5 03ec621e90 Pick the pid to kill in one place
Worker#kill and quit_workers both need the test process rather than the runruby.rb process on Windows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 12:12:22 +09:00
Hiroshi SHIBATAandClaude Opus 5.5 162ce2fee8 Kill the test processes left after waiting for workers on Windows
The KILL sent when Process.waitall times out in quit_workers also hit only the runruby.rb process on Windows, leaving the test process running. It now targets only the workers still running, since the pids of the test processes that have exited may already be reused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 12:12:22 +09:00
Hiroshi SHIBATAandClaude Opus 5.5 16d7abf963 Kill the hung worker itself on Windows
On Windows, exec in runruby.rb spawns the worker as another process and waits for it, so killing the launched pid left the worker running with the inherited stderr open, and GitHub Actions waited on it until the step timed out. The worker now reports its own pid, which is also the process the debugger should dump.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 12:12:22 +09:00
Hiroshi SHIBATAandClaude Opus 5.5 fa6ffde024 Read RUBY_TEST_TIMEOUT_SCALE in EnvUtil
test-bundled-gems runs the tests with the test-unit gem, which loads EnvUtil through gems/lib but never goes through tool/lib/test/unit.rb, so the scale was ignored there. `--timeout-scale` still takes precedence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 06:21:58 +09:00
Peter Zhu 99d67b64a1 Add assert_in_ractor
This commit adds assert_in_ractor which runs code in a Ractor and allows
the use of assertions in the Ractor.
2026-10-01 20:07:00 +09:00
Nobuyoshi Nakada 9cbb4e67ef Disable repeat counts for parallel tests
Ignore `--repeat-count` during parallel tests because fresh workers
cannot expose state carried over between repetitions.
2026-09-24 21:51:13 +09:00
Nobuyoshi Nakada 28ae8099dc Preserve jobserver limits across repeated test runs
Keep the jobserver connection until all repetitions finish so that
later runs do not launch up to 256 workers without token limits.
2026-09-24 21:51:13 +09:00
Nobuyoshi Nakada 1f11ffaee4 Run GC in test workers before running suites
Run `GC.start` in the process executing tests because collecting the
parent heap does not reclaim objects in parallel workers.
2026-09-24 21:51:12 +09:00
Jeremy Evans 3ee17ab904 Make Test::Unit stop aliasing method in prepended module
Ruby will be emitting a deprecation warning for this shortly. Switch
to using super instead of an alias.
2026-09-17 18:29:51 -07:00
Hiroshi SHIBATAandClaude Opus 5 b6e284dcdf Keep the message of a Timeout::Error raised in a test
A subprocess that expires its timeout reports which assertion it was and
the output it did manage to produce.  Dropping that left nothing to go on
for the crash report timeouts on macOS CI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-11 11:59:54 +09:00
Hiroshi SHIBATA f7462782e3 Split -I option with File::PATH_SEPARATOR in test runner
The test runner split -Idirectory on a hardcoded ":", so on Windows an
absolute path such as -IV:/foo/lib was broken into "V" and "/foo/lib".
The tail only resolved by accident when the current drive matched.
File::PATH_SEPARATOR is the right delimiter, matching ruby -I itself.
This is the same class of bug as the test-syntax-suggest -I fix in
commit 3c7a4c174f, which replaced ":" with $(PATH_SEPARATOR) in
common.mk. Makefiles only pass relative paths, so this stayed latent.
2026-07-13 14:19:42 +09:00
Koichi SasadaandClaude Opus 4.8 6819e80a07 test/unit: optionally verify GC consistency after each test
Set RUBY_TEST_GC_VERIFY to call GC.verify_internal_consistency after each
test. An integer >= 2 samples once every N tests; any other non-empty value
verifies every test. Useful for pinning down which test corrupts the heap.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 06:55:37 +09:00
XrXr 5eb78494be Copyedit test-all -j crash message. Add hints about RUBY_CRASH_REPORT
The GitHub Actions part particularly is for ZJIT and YJIT jobs which
have a separate "Dump crash logs" fold. There doesn't seem to be a way
to ask for the web UI to unfold on loading of the logs.
2026-06-17 17:01:49 -04:00
Koichi Sasada 2ba2950d20 show loadavg if the test fails
to survey the following failure:

```
  1) Failure:
TestFile#test_stat [/tmp/ruby/src/trunk-repeat20/test/ruby/test_file.rb:412]:
Expected |1770056232.744032 - 1770056230.6465776| (2.097454309463501) to be <= 1.
```

BTW I found that `test/ruby/test_file.rb:412` `assert_in_delta atime, stat.atime.to_f, delta`
only fails (mtime, ctime doesn't fail on 100 days).
2026-02-04 12:40:59 +09:00
Nobuyoshi Nakada bfba65d8c1 Extract Test::JobServer module
A placeholder to handle GNU make jobserver option.
spec/default.mspec didn't handle the jobserver using a FIFO.
2025-12-19 13:36:42 +09:00
Nobuyoshi Nakada 04925c6608 Add --keep-repeating option
It directs the program to continue repeating the tests the specified
number of times, even if any tests fail along the way.
2025-06-15 23:59:46 +09:00
Nobuyoshi Nakada 36a04de9f0 Dump with debugger before killing stuck worker 2025-06-15 20:24:30 +09:00
Nobuyoshi Nakada f7af75d3d9 Show test task names longer 2025-03-06 16:58:13 +09:00
Naoto Ono 48b6c60969 [Misc #20661] Stop retrying tests in make test-all command by default (#11271)
[Misc #20661] Stop retrying tests in make test-all command by default
2025-02-27 06:12:02 +00:00
Hiroshi SHIBATA a450e9304d Extract Benchmark.measure on assert_cpu_usage_low 2025-01-10 13:27:16 +09:00
Nobuyoshi Nakada 14c72e96ae Remove an unnecessary blank line added by mistake [ci skip] 2024-09-10 17:19:02 +09:00
Nobuyoshi Nakada 2d12fbc4db Add predicates for platforms 2024-09-10 16:50:21 +09:00
Nobuyoshi Nakada e884d65094 Ignore blocks to Test::Unit::Assertions#pend silently
Although `pend` in test-unit gem expects the block does not pass if it
is given, our `pend` should ignore the block silently as same as it
just skips the test with no message by default.  Add an unused
parameter, to suppress the warning for the block to be ignored.
2024-09-04 14:49:25 +09:00
Hiroshi SHIBATA 64a005df3b make test-tool with Windows is broken. We should use :KILL with that platform.
```
[ 14/191] TestParallel::TestParallel#test_hungup = 11.02 s
  1) Failure:
TestParallel::TestParallel#test_hungup [C:/Users/hsbt/DevDrive/github.com/ruby/ruby/tool/test/testunit/test_parallel.rb:211]:
Expected /^Retrying hung up testcases\.+$/ to match "Run options: \n" +
"  --seed=35582\n" +
"  --ruby\n" +
"  \".\\\\ruby.exe -I../../ruby/lib -I.ext/x64-mswin64_140 -I. ../../ruby/tool/runruby.rb --extout=.ext -- --disable-gems\"\n" +
"  -j\n" +
"  t1\n" +
"  --worker-timeout=1\n" +
"\n" +
"# Running tests:\n" +
"\n" +
"[1/1] 50340=test4test_hungup.\n" +
"C:/Users/hsbt/DevDrive/github.com/ruby/ruby/tool/lib/test/unit.rb:418:in 'Process.kill': Invalid argument (Errno::EINVAL)\n" +
```
2024-08-22 11:29:48 +09:00
Yusuke Endoh 671f6e78b4 Use 20min for parallel test worker timeout 2024-08-07 10:17:43 +09:00
Yusuke Endoh fdfde6ab2e Use SIGSEGV to kill a parallel test worker (to dump backtrace) 2024-08-07 09:44:06 +09:00
Yusuke Endoh e62c2ce0e0 Wait a bit longer for worker quit 2024-08-07 03:54:47 +09:00
Yusuke Endoh 47c45d5c18 Add a log when SIGKILL is sent to a test parallel worker 2024-08-07 02:41:27 +09:00
Yusuke Endoh 2c35a37918 Fix a typo 2024-08-07 02:34:46 +09:00
Yusuke Endoh c0938fd24c Use 3min for test parallel worker timeout again 2024-08-06 21:11:41 +09:00
Yusuke Endoh 062f85e141 Dump all-thread backtraces when test parallel worker exceeds time limit 2024-08-06 21:11:41 +09:00
Yusuke Endoh 24d46a6781 Extend the default timeout of parallel testing to one hour 2024-08-06 16:59:24 +09:00
Yusuke Endoh 30be7fb85d Extend the default timeout of parallel testing 2024-08-06 15:00:12 +09:00
Naoto Ono 8ede84aa95 Move the file location of launchable.rb 2024-07-08 10:15:04 +09:00
Naoto Ono 5b78925455 Integrate Launchable into make btest 2024-07-08 10:15:04 +09:00
Alan Wu 0dcb0fdfcd Enable LeakChecker for RJIT previously disabled for MJIT (#10998)
RJIT doesn't spawn subprocesses so there should now be no need to
special case it.
2024-06-14 00:05:38 -07:00
Nobuyoshi Nakada aabe718e64 Show the caller location of assertion methods
Not only defined in `Test::Unit` or `CoreAssertions`, also show the
caller location of assertion methods defined in the current class or
ancestors.
2024-05-08 17:24:36 +09:00
Kevin Newton b4b39a6199 [PRISM] Enable test_methoddef_endless_command 2024-05-07 14:24:31 -04:00
Naoto Ono 56ecd8038a Launchable: Add lineNumber field 2024-04-03 16:17:30 +09:00
Naoto Ono 3d1d1435c1 Launchable: Refactor the logic of JsonStreamWriter 2024-04-02 21:55:32 +09:00
Nobuyoshi Nakada 114c0b71be Inform failures in parallel tests before retrying
Displays for each failure which test it actually occurred in.  The
output destination follows the --{stdout,stderr}-on-failure option.
2024-02-29 18:05:00 +09:00
Naoto Ono 3371936b6f Add Launchable into CI 2024-02-23 14:10:01 +01:00
Nobuyoshi Nakada 1fa4d15ca7 Add Test::Unit::TestCase#method_name
For the compatibility with test-unit gem.
2024-02-21 15:45:53 +09:00
Yusuke Endoh 9d1b000bd1 Show the method owner in backtraces
```
test.rb:1:in 'Object#toplevel_meth': unhandled exception
        from test.rb:4:in 'Foo.class_meth'
        from test.rb:6:in 'Foo#instance_meth'
        from test.rb:11:in 'singleton_meth'
        from test.rb:13:in '<main>'
```

[Feature #19117]
2024-02-15 19:11:58 +09:00
Naoto Ono a7ebe467a7 Add newline between end and def 2024-01-31 16:14:50 +09:00
Naoto Ono 45b53dd897 Add the ability to generate Launchable test reports 2024-01-31 16:14:50 +09:00
Takashi Kokubun c25705c267 Get rid of obsoleted RJITFirst
It was renamed from test_mjit, but we did not maintain it as test_rjit.
We test RJIT very differently.
2023-12-22 20:20:42 -08:00
Yusuke Endoh 5bf75c20a2 Refactor the settings of test-all out
test/runner.rb and tool/lib/test/unit/parallel.rb must use the same
settings. However, some settings were copied and pasted, while some were
added only to test/runner.rb.
This changeset creates tool/test/init.rb for all settings of test-unit,
which is loaded not only by test/runner.rb but also
tool/lib/test/unit/parallel.rb.

Background: the GEM_HOME environment variable was removed in
test/runner.rb, which prohibit `require "rake"` (note that rake is a
bundled gem). However the parallel mode didn't refrect this setting,
i.e., `require "rake"` was allowed.
This leads to an inconsistency, which actually affected a test test
defines s test class *only when* `require "rake"` is successful.
(test/rubygems/test_gem_package_task.rb)

https://github.com/ruby/ruby/actions/runs/6807729617/job/18511055636#step:8:1714
```
  /home/runner/work/ruby/ruby/src/tool/lib/test/unit.rb:729:in `const_get': uninitialized constant TestGemPackageTask (NameError)

                suites.map! {|r| ::Object.const_get(r[:testcase])}
                                         ^^^^^^^^^^
```
2023-11-15 19:27:10 +09:00
lukeg 7717684d16 tool test/unit/testcase: rename vars @passed, @@current
to @__passed__, @@__current__.
@passed is redefined in a few test suites, and this could lead to bugs.

Also rename @options (Runner#options) to @__runner_options__, which is
only used in make test-tool anyway.
2023-10-26 12:39:13 +09:00
Nobuyoshi Nakada 38b79b05fd Fallback job status to normal if no tty 2023-10-21 19:47:05 +09:00