Building the error message from a NULL pointer raised ArgumentError instead of IO::TimeoutError.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without pthread, mswin fell back to GETADDRINFO_IMPL 1, which ignores the timeout of Addrinfo.getaddrinfo and lets neither Timeout, Thread#kill nor Thread#raise stop the caller until getaddrinfo(3) returns. Start the resolver thread with _beginthreadex(3) so mswin takes GETADDRINFO_IMPL 2 as the pthread platforms do. MinGW stays on GETADDRINFO_IMPL 1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No caller reads the pthread_t, so raddrinfo_thread_create() no longer takes one, and a non-pthread implementation can sit behind it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since the fix for Bug #21441, ec->cfp is set to NULL before clearing
the VM stack so that thread_profile_frames() called from a signal
handler returns early instead of walking a stack that is no longer set
on the ec. But these are plain stores to distinct fields of the same
struct, so the compiler is free to reorder them.
With Apple clang 21 and GCC 16 on macOS arm64, and with clang 18 and
GCC 13 on Linux aarch64, both stores of rb_ec_set_vm_stack() were
combined into a single stp emitted before the store to ec->cfp
(vm_stack is at offset 0, vm_stack_size at 0x8 and cfp at 0x10):
stp xzr, xzr, [x0] ; ec->vm_stack = NULL, vm_stack_size = 0
str xzr, [x0, #0x10] ; ec->cfp = NULL
ret
So a signal in between would see a non-NULL ec->cfp with a NULL
ec->vm_stack. Stopping there in lldb when a thread terminates and
calling rb_profile_frames() segfaults in thread_profile_frames(), as
end_cfp is computed from the NULL ec->vm_stack and the loop walks past
the control frames. With this change, ec->cfp is already NULL at that
point and rb_profile_frames() returns 0.
Like in vm_push_frame(), use atomic_signal_fence() to keep the store to
ec->cfp first. With it, all these compilers emit the store to ec->cfp no
later than the stores clearing the stack. The rest of vm.o is unchanged
with Apple clang 21, GCC 16 and GCC 13 on aarch64. Clang 21 and 18:
stp xzr, xzr, [x0, #0x8] ; ec->vm_stack_size = 0, cfp = NULL
str xzr, [x0] ; ec->vm_stack = NULL
ret
GCC 16 and 13:
str xzr, [x0, #0x10] ; ec->cfp = NULL
stp xzr, xzr, [x0] ; ec->vm_stack = NULL, vm_stack_size = 0
ret
With GCC 13 on x86_64, the store to ec->cfp is already emitted first
without the fence, and vm.o is identical with and without it:
endbr64
movq $0x0, 0x10(%rdi) ; ec->cfp = NULL
movq $0x0, (%rdi) ; ec->vm_stack = NULL
movq $0x0, 0x8(%rdi) ; ec->vm_stack_size = 0
ret
[Bug #22477]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The converter can still hold readagain bytes from an earlier rbuf fill,
so fill_cbuf must not put all of them back into the current rbuf. Clamp
the putback to the bytes taken from the current fill; the rest stay in
the converter and are re-read first on the next conversion.
Co-authored-by: Percze Dániel <162882837+Flashmyname@users.noreply.github.com>
Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
A text character can case-fold to more bytes than remain in the
pattern, e.g. U+00DF folds to "ss" against /s/i. Such a character
cannot match, so stop there instead of comparing beyond the pattern.
Reported-by: Paul Ryan (https://hackerone.com/secpaul)
Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
rb_str_format always allocates its result with rb_str_buf_new, so the
result is a plain String even when the receiver is a subclass.
Fixes https://github.com/Shopify/ruby/issues/1085
Existing tests only read a local that eval wrote, which passes even if
the side exit resumed before the call. Pin the snapshot pc in an HIR
test and count how many times eval runs when the patch point fires.
rb_vm_empty_cc(), rb_vm_empty_cc_for_super() and rb_vm_uncached_cc()
shouldn't be real function calls. rb_iseq_mark_and_move was calling
these as real functions in a loop.
The assumption is that we're usually calling these methods on boot, and they don't
need CCs. CCs scale with the number of classes that use these methods in the app, so
if you have 10k classes and these classes use 3 of these methods on average, then that
saves 30k CC objects.
Methods on Module that are marked cold are:
public, protected, private, module_function, ruby2_keywords, alias_method, undef_method,
remove_method, attr_reader, attr_writer, attr_accessor, include, prepend.
If these methods ever get redefined, they get new method definitions that are not
marked uncacheable (CCs will get created).
In one of our apps (not the largest), we observed this difference after boot:
| method | before | after | delta |
|-------------------|-------:|------:|------------:|
| `include` | 4518 | 2 | -4516 |
| `alias_method` | 3921 | 0 | -3921 |
| `ruby2_keywords` | 3784 | 0 | -3784 |
| `private` | 3519 | 3 | -3516 |
| `attr_reader` | 2738 | 0 | -2738 |
| `attr_accessor` | 2288 | 0 | -2288 |
| `prepend` | 354 | 1 | -353 |
| `protected` | 303 | 0 | -303 |
| `public` | 302 | 0 | -302 |
| `undef_method` | 178 | 0 | -178 |
| `attr_writer` | 124 | 0 | -124 |
| `module_function` | 42 | 0 | -42 |
| `remove_method` | 24 | 0 | -24 |
| **total** | | | **-22,089** |
NOTE: CC tables and ccs (the malloced struct) are still being created, so this only reduces
the CC objects themselves. I will tackle that in a follow-up.
This is not a real-world problem. Marshal.dump never emits this payload.
It can only be hand-crafted. Loading untrusted Marshal data is already
unsafe. This fix only avoids false positives from security scanners.
TYPE_UCLASS ('C') relabels the wrapped object's class with RBASIC_SET_CLASS.
It assumes the object was freshly allocated during load. But with
freeze: true, r_leave() interns a wrapped bare String first, and
interning a bare String returns the shared global fstring. So the relabel
hits that global object instead, corrupting every other reference to the
same interned string.
Load the wrapped object as a partial so r_leave() is deferred, and
finalize it only after the class has been set. The String is then
interned as a non-bare String, which keeps the result a distinct object
instead of the shared fstring.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pathname.new('.').find walked the whole working directory twice, so the run time depended on where the tests were started. With an in-source build that is the entire checkout, which exceeded the 60 second timeout on rubyci's macOS machine. Only the lazy require of find matters here, so walking test/ruby is enough.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An override change made the lockfile count as needing resolution, so `bundle install` fetched the remote index and failed offline when the compact index cache was absent, and `Bundler.setup` re-resolved on boot. The section only needs to be regenerated, which #write_lock already does when the contents differ, and the frozen check still reports the change.
https://github.com/ruby/rubygems/commit/a616f3d1bb
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A locked gem that is not installed and whose metadata rejects the current Ruby is re-resolved during materialization whether or not an override changed, so the man page cannot promise that adding or removing a metadata override never moves a locked version.
https://github.com/ruby/rubygems/commit/c3e0897ff1
Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
timed-out memory leak child
(https://github.com/ruby/test-unit-ruby-core/pull/37)
* Close stdin before terminating a timed-out memory leak child
With a sample range, assert_no_memory_leak keeps the child's stdin open,
and the child's ensure clause reads a "continue" line from it after each
sample. When the timeout fires while the child is running the sampled
code, EnvUtil.terminate sends SIGTERM, and the child's ensure clause
then blocks reading stdin, which was closed only after terminate
returned. SIGTERM cannot end the child, so terminate escalates to
SIGABRT and then waits for the aborted child without a time limit. On
ruby/ruby's YJIT macOS CI jobs the child sometimes did not exit after
the RUBY_ON_BUG core dump, and the test worker hung until the runner
killed it 20 minutes later. In other runs the test passed but left a
crash report for the child.
Close stdin first. The child's read then returns nil, and its ensure
clause breaks out of the sample loop. The break discards the
SignalException raised by SIGTERM, so the child exits with status 0. If
SIGTERM arrives before the child reaches the sample loop, the child
still dies from SIGTERM.
The existing timeout test now also checks that the child exited
normally or by SIGTERM.
https://github.com/ruby/test-unit-ruby-core/commit/ce7a92e724
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
configure created revision.h in $srcdir when it was missing. Since
version.h includes "revision.h", the copy in $srcdir is found before
the one in the build directory.
This used to be harmless because file2lastrev.rb, run with --srcdir,
overwrote that copy in $srcdir on every build. Since 4361d8e48f
(GH-19367), make writes only ./revision.h via lastrev and ifchange, so
the copy in $srcdir was left stale, and every out-of-place build
sharing that $srcdir kept embedding the revision at the time of that
configure run.
draft-release in ruby/actions stages an mswin binary zip next to the draft tarballs, but nothing promotes it to pub/ruby/binaries, so each release has needed a manual publish-binaries run. Dispatch it alongside the other post-release hooks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_for_fd in rbs 4.2.0 calls Dir.for_fd twice with the fd of a Dir it
then closes, and never closes the two Dirs Dir.for_fd returned. Both
still own that fd number. When GC later finalizes them, closedir()
closes whatever file has since been given the same number, and the
owner of that file fails on close with:
Errno::EBADF: Bad file descriptor @ fptr_finalize_flush
In test-bundled-gems this usually hits FileTestSingletonTest, which
runs right after Dir_test.rb and opens files in every test. Whether it
fails depends on GC timing, so it shows up as a flake.
https://github.com/ruby/rbs/pull/3191 fixes the test upstream, but no released rbs has it yet.
Skip the test until the bundled rbs includes that fix.
test_major_sweep_does_not_create_oldmalloc_debt assumed the ring was
already old when the loop started. When Ruby is running with parse.y
this isn't always the case. That only held when GCs ran while
the ring was built. Under parse.y often none did, so the ring was
still young after GC.start.
This means that Minor GCs freed the buffers before they got promoted and
so there isn't any old garbage to sweep.
This commit runs GC.start RVALUE_OLD_AGE times before the loop, so every
buffer in the ring is old and each replacement leaves old garbage for
the next major to sweep, regardless of startup GC history.
In single-ractor mode, we don't want foreign_object_p checks to actually
run (there can be no foreign objspaces). This foreign_object_p check return
false if there was never a (non-main) Ractor created. Previously,
rb_gc_ever_multi_ractor_p() was needlessly complex. This change should
speed up marking and WB implementations in single-ractor mode.
The constant pool starts at source size / 120 entries, 4 at the least,
and doubles at a load of 3/4; each outgrown pair of arrays stays in the
metadata arena. An embedder that interns a fixed set of names into the
pool at every parse (mruby's compiler puts its 58 presyms there, as its
symbol table) grows a short program's pool from 4 to 128 and leaves
about 3 KB behind, as much as the pool itself. With the minimum under
#ifndef it can build with -DPM_CONSTANT_POOL_MIN_CAPACITY=128: measured
on a 32-bit build with 2 KB arena blocks, parsing one REPL line then
holds 6 KB instead of 10 KB. The default stays 4.
https://github.com/ruby/prism/commit/7ed8c71c59
61ebe5123 made defines.h include <sys/types.h> ahead of <inttypes.h>:
with an arm-none-eabi toolchain whose <stdint.h> is GCC's own, newlib's
<inttypes.h> never sees the marker it tests before defining the 64-bit
PRI macros, and PRIu64 goes undefined while PRIu32 does not. 400b21739
("Remove defines") took the header out and the workaround with it, so
a cross build of integer.c and errors_format.c, the two files that
include <inttypes.h> now, fails on such a toolchain (Debian's
gcc-arm-none-eabi 13.2.1 with newlib). Put the include back in those
two files, with the note.
https://github.com/ruby/prism/commit/ff42521e49
The arena's first block is 8 KB, sized for a host. An embedder on a
small heap parses short programs (a REPL line uses about 6 KB in two
arenas) and pays two 8 KB blocks for each; with the constant under
#ifndef it can build with -DPM_ARENA_INITIAL_SIZE=2048 instead.
https://github.com/ruby/prism/commit/f7a062efd4
When formatting error messages, first try to format into a fixed
buffer instead of formatting once to get the length then a second
time to get the actual format.
https://github.com/ruby/prism/commit/557cffe23e
Node ids are assigned sequentially by the parser and compilation emits
instructions in an order correlated with the tree, so the delta between
consecutive instructions are likely to be small. We can store the delta
between consecutive instructions using zigzag encoding.
On Shopify's monolith, this reduces the boot heap memsize by 20MB.
* Time#utc_offset: use America/New_York, as musl parses "EST5EDT" as
a POSIX TZ string instead of loading the zoneinfo file.
* Time#zone: musl accepts "hello-foo" as a POSIX TZ string.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Process.setproctitle and $0 specs used `ps -ocommand= -p`, but busybox
ps does not support -o/-p.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
busybox is a multi-call binary which dispatches on argv[0], so it
cannot run as /bin/sh with an arbitrary argv[0] such as "argv_zero".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`bundle lock --add-checksums --add-platform arm64-darwin` wrote a
lockfile that listed arm64-darwin under PLATFORMS but carried none of
its platform-specific gems. `Definition#add_checksums` resolves the
bundle to fetch the checksums, and it ran before the platform options
were applied, so the resolution never saw the new platform.
This usually went unnoticed. `SpecSet#add_extra_platforms!` tries to
complete every platform the sources know about, and for most bundles
that added the darwin gems anyway. It stops working as soon as one gem
has a precompiled darwin build whose `required_ruby_version` excludes
the current Ruby (bcrypt_pbkdf 1.1.2 on Ruby 3.4): `complete_platform`
then gives up on darwin for every gem, and the lockfile ends up listing
a platform it has no gems for. Running `bundle lock` again does not
repair it, because the platform is already listed.
Apply `--add-checksums` after the platform changes have been recorded.
https://github.com/ruby/rubygems/commit/2f1d325fc2
Resolving through the full-index API lazily downloads individual
gemspecs from /quick/Marshal.4.8/*.gemspec.rz. `Fetcher#fetch_spec` made
that request bare:
Bundler.safe_load_marshal Bundler.rubygems.inflate(downloader.fetch(uri).body)
while `specs_with_retry`, in the same class, wraps its network call in
`Bundler::Retry`. So a single transient failure — a read timeout, say —
aborted the whole resolution, no matter what BUNDLE_RETRY was set to.
That is most painful without a lockfile, where a resolve can fetch a
great many gemspecs one at a time and any one of them can end it.
Wrap the download in the same `Bundler::Retry.new("fetcher", FAIL_ERRORS)`
the index path already uses. FAIL_ERRORS is what keeps this from retrying
things that will not change, such as authentication failures.
Only the download branch is wrapped. The file:// and cached-gemspec
branches do no network I/O, so retrying them would just repeat a local
failure.
Closes https://github.com/ruby/rubygems/pull/9817https://github.com/ruby/rubygems/commit/9578cf9e93
A link check compiles and links conftest.c in one clang invocation, and with -g clang then runs dsymutil, which reads the DWARF of every object pulled in from libruby-static.a. Each check took about 2.2 s instead of 0.17 s on a static build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The check ran after the prompt, so the user was still asked about executables that could not exist. File.directory? also returned false when a parent directory was not searchable, which skipped the executables and removed the gem anyway. Only ENOENT skips now, and other errors still reach the writable check.
https://github.com/ruby/rubygems/commit/ed96042879
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixesruby/rubygems#9959
remove_executables raised Gem::FilePermissionError when the gem's bin
directory did not exist at all, since File.writable? is false for a
missing directory and aborting uninstall entirely was never intended.
Now the missing directory case returns early: there is nothing to
remove.
https://github.com/ruby/rubygems/commit/03d22cfba8
The waiting thread's flock fails with Errno::EINVAL there when another thread of the same JVM already locks the file. Separate processes, which are what the lock guards against, are not affected.
https://github.com/ruby/rubygems/commit/ddc674293b
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gem.open_file_with_lock removed its lock file after releasing the lock.
Since flock locks the file itself rather than its path, a process that
was waiting on the old file could acquire it after it had been removed,
while a newly arriving process created a fresh lock file and locked it
immediately, so both ran at the same time.
Remove the lock file while still holding the lock, and after acquiring
it, check that the locked file is still the one at the lock path,
retrying otherwise.
https://github.com/ruby/rubygems/commit/9a93c235e7
Gems sourced from the same git repository share one cache directory.
Since git sources are preloaded in parallel, two workers could run git
fetch in that directory at the same time, and the fetches collided on
the shallow file and its lock, failing bundle install.
Group the sources by cache path and preload each group in a single job,
so sources sharing a cache run one after another while separate
repositories still run in parallel.
Fixes https://github.com/ruby/rubygems/pull/9946https://github.com/ruby/rubygems/commit/c20901e6a1
busybox lsof ignores the options and prints no header line, so the
FD/TYPE/NODE columns cannot be found and check_fd_leak crashed with
NoMethodError, which killed the test worker. Treat such lsof as
unavailable; leaked fds are still reported.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Takashi Kokubun