realpath_rec reads the names of a symlink target through a raw pointer
into the string returned by rb_readlink, while it calls itself to resolve
those names. RB_GC_GUARD(link_orig) keeps that string alive only for an
absolute target. For a relative target, link_orig is Qnil and nothing
uses `link` after the recursive call, so GC can free the string while the
later names are still being read.
File.realdirpath then returns a wrong path: the unresolved symlink path,
or a path with NUL bytes, which makes Pathname.new raise "pathname
contains null byte". CocoaPods calls Pathname#realdirpath for every
source file, and pnpm installs packages behind relative symlinks, so
`pod install` fails at random in such projects.
File.realpath is not affected on platforms with realpath(3), because only
RB_REALPATH_DIR uses realpath_rec there.
Fixes [Bug #22479]
codespell read "caf\xC3\xA9", the bytes of "café" spelled out in a string literal, as a misspelling of "calf". Any raw UTF-8 bytes serve the test, so use those of a Japanese character instead.
https://github.com/ruby/rubygems/commit/2711d21797
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A flow sequence was split on every comma, so a quoted entry containing one, as in ["x, y", z], came back as the fragments "\"x" and "y\"". Scan the entries so a quoted one is taken whole.
https://github.com/ruby/rubygems/commit/b3de2a6c52
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Psych writes NEL as "\N" and a BOM or other non-printable characters as "\uXXXX" or "\UXXXXXXXX", but those escapes stayed in the loaded string as is. Restore them along with "\_", "\L" and "\P". Metadata is read as binary, so a non-ASCII character unescaped into a scalar that already holds raw UTF-8 bytes is added as bytes instead of raising Encoding::CompatibilityError, which "\x80" and above already did. An escape for a surrogate or a code point beyond U+10FFFF is kept as written, as unknown escapes are.
https://github.com/ruby/rubygems/commit/d8c0371ca1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A block scalar line holding only spaces became an empty line even when it had more spaces than the indentation, which are content in YAML. Gem::YAMLSerializer.dump writes such lines for strings like "x\n \ny\n", so they did not survive a roundtrip.
https://github.com/ruby/rubygems/commit/1255df9644
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The strip indicator only took effect when it was the whole header, so "|2-" written by Psych kept the trailing line break, and even "|-" removed just one of several. Read the chomping indicator next to an indentation indicator and remove every trailing line break, as YAML specifies.
https://github.com/ruby/rubygems/commit/f32f6056d8
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Psych writes a block scalar whose first line starts with spaces as "|2", but the parser ignored the digit and took the indentation from that first line. A description such as " Indented first line\nsecond line\n" lost its leading spaces and the first characters of every later line.
https://github.com/ruby/rubygems/commit/8210dac32c
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A more indented comment line after a plain scalar was folded into the value, so "ssl_ca_cert: /path" followed by an indented comment in ~/.gemrc loaded as "/path # comment". A comment line ends a plain scalar, so treat it like a blank line there. A quoted scalar left open on its first line still runs onto such a line, since "#" inside quotes does not start a comment.
https://github.com/ruby/rubygems/commit/806b1c5698
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gem::YAMLSerializer ended a nested sequence or mapping at the first comment or blank line, so the entries after it were dropped or moved to the root mapping. Commenting out one of the :sources: in ~/.gemrc silently removed every source after it. Skip those lines before looking at the next entry, as parse_node already does.
https://github.com/ruby/rubygems/commit/02f0355460
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The converter can still hold readagain bytes from an earlier rbuf fill,
so fill_cbuf must not put all of them back into the current rbuf. Clamp
the putback to the bytes taken from the current fill; the rest stay in
the converter and are re-read first on the next conversion.
Co-authored-by: Percze Dániel <162882837+Flashmyname@users.noreply.github.com>
Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
A text character can case-fold to more bytes than remain in the
pattern, e.g. U+00DF folds to "ss" against /s/i. Such a character
cannot match, so stop there instead of comparing beyond the pattern.
Reported-by: Paul Ryan (https://hackerone.com/secpaul)
Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
This is not a real-world problem. Marshal.dump never emits this payload.
It can only be hand-crafted. Loading untrusted Marshal data is already
unsafe. This fix only avoids false positives from security scanners.
TYPE_UCLASS ('C') relabels the wrapped object's class with RBASIC_SET_CLASS.
It assumes the object was freshly allocated during load. But with
freeze: true, r_leave() interns a wrapped bare String first, and
interning a bare String returns the shared global fstring. So the relabel
hits that global object instead, corrupting every other reference to the
same interned string.
Load the wrapped object as a partial so r_leave() is deferred, and
finalize it only after the class has been set. The String is then
interned as a non-bare String, which keeps the result a distinct object
instead of the shared fstring.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pathname.new('.').find walked the whole working directory twice, so the run time depended on where the tests were started. With an in-source build that is the entire checkout, which exceeded the 60 second timeout on rubyci's macOS machine. Only the lazy require of find matters here, so walking test/ruby is enough.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_major_sweep_does_not_create_oldmalloc_debt assumed the ring was
already old when the loop started. When Ruby is running with parse.y
this isn't always the case. That only held when GCs ran while
the ring was built. Under parse.y often none did, so the ring was
still young after GC.start.
This means that Minor GCs freed the buffers before they got promoted and
so there isn't any old garbage to sweep.
This commit runs GC.start RVALUE_OLD_AGE times before the loop, so every
buffer in the ring is old and each replacement leaves old garbage for
the next major to sweep, regardless of startup GC history.
When formatting error messages, first try to format into a fixed
buffer instead of formatting once to get the length then a second
time to get the actual format.
https://github.com/ruby/prism/commit/557cffe23e
The check ran after the prompt, so the user was still asked about executables that could not exist. File.directory? also returned false when a parent directory was not searchable, which skipped the executables and removed the gem anyway. Only ENOENT skips now, and other errors still reach the writable check.
https://github.com/ruby/rubygems/commit/ed96042879
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixesruby/rubygems#9959
remove_executables raised Gem::FilePermissionError when the gem's bin
directory did not exist at all, since File.writable? is false for a
missing directory and aborting uninstall entirely was never intended.
Now the missing directory case returns early: there is nothing to
remove.
https://github.com/ruby/rubygems/commit/03d22cfba8
The waiting thread's flock fails with Errno::EINVAL there when another thread of the same JVM already locks the file. Separate processes, which are what the lock guards against, are not affected.
https://github.com/ruby/rubygems/commit/ddc674293b
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gem.open_file_with_lock removed its lock file after releasing the lock.
Since flock locks the file itself rather than its path, a process that
was waiting on the old file could acquire it after it had been removed,
while a newly arriving process created a fresh lock file and locked it
immediately, so both ran at the same time.
Remove the lock file while still holding the lock, and after acquiring
it, check that the locked file is still the one at the lock path,
retrying otherwise.
https://github.com/ruby/rubygems/commit/9a93c235e7
Reading the source in binary mode made the captured path ASCII-8BIT, so a non-ASCII path never matched spec.files and joining it with a non-ASCII directory raised Encoding::CompatibilityError.
https://github.com/ruby/rubygems/commit/8090098280
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gem::SpecificationPolicy#validate_extension_require_relative read each file with File.foreach, which tags lines with Encoding.default_external. Under an unset or ASCII locale (build sandboxes, minimal containers) a valid UTF-8 non-ASCII line raised ArgumentError: invalid byte sequence in US-ASCII, and a non-UTF-8 source did the same under a UTF-8 locale. The error escaped validation and aborted gem build.
The pattern is ASCII-only, so read the file as binary.
Fixes https://github.com/ruby/rubygems/pull/9966https://github.com/ruby/rubygems/commit/e158b757fd
`Gem::StreamUI#ask_for_password` printed the prompt before it turned off
echo, so the terminal could echo back a password written as soon as the
prompt appeared.
Print the prompt inside the `noecho` block.
https://github.com/ruby/rubygems/commit/e648f0dfa1
Two call sites added a negative error code into a jump offset instead
of returning it. It was harmless since compilation fails later anyway.
Reported-by: Owais Lone (thesecguy) <thesecguy45@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
IPAddr.new and IPAddr#zone_id= only accepted \w characters after %, so zone IDs for interfaces such as eth0.100 (VLAN) or br-lan raised InvalidAddressError. RFC 6874 defines ZoneID as unreserved characters (ALPHA, DIGIT, -, ., _, ~), which is also what Resolv::IPv6 accepts (%[-0-9A-Za-z._~]+). Use the same character class in both places.
https://github.com/ruby/ipaddr/commit/8aaee2fa0c
[Bug #22467]
If we perform IO read without the GVL during GC compaction and the string
we are writing to is embedded, then it can get mprotect'd by the read
barrier in the GC, which can cause the IO read operation to fail.
An invalid open_timeout raised before the waiter was assigned, so the cleanup closed fd 0 left by memset and took the process's stdin with it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An interrupted pipe read or write returned right after `CancelIo`, which only requests the cancellation. Wait for the I/O to end and return what it transferred.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The native thread of a terminated thread may have already exited, so
its pthread_t is no longer valid. Passing it to pthread_setname_np()
could crash or rename an unrelated thread.
[Bug #22471]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In the block form, output not yet yielded stays in the buffer between
calls. It was silently dropped when another buffer (or none) was given
for the next call.
data = Zlib::Deflate.deflate("A" * 50000)
half = data.bytesize / 2
inf = Zlib::Inflate.new
out = +""
inf.inflate(data[0, half], buffer: String.new) { |c| out << c }
inf.inflate(data[half..], buffer: String.new) { |c| out << c }
out << inf.finish
out.bytesize
# expected: 50000
# actual: 47806
https://github.com/ruby/zlib/commit/ccb2d0238c
Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
The flag stayed set after a call with buffer: had returned its buffer,
so an internal buffer allocated by a later Inflate#<< was treated as
user-provided and never revealed.
data = Zlib::Deflate.deflate("A" * 50000)
half = data.bytesize / 2
inf = Zlib::Inflate.new
inf.inflate(data[0, half], buffer: String.new)
inf << data[half..]
inf.finish.class
#=> [BUG] Segmentation fault
https://github.com/ruby/zlib/commit/ed296aa113
Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
When a call with buffer: left the stream in progress (it raised
Zlib::NeedDict, or the block form has more input to come), the next run
reused the next_out saved by the previous call. If the buffer had been
modified in between, it pointed into freed or reallocated memory.
d = Zlib::Deflate.new
d.set_dictionary("dict")
data = d.deflate("A" * 12000, Zlib::FINISH)
inf = Zlib::Inflate.new
buf = String.new(capacity: 1 << 14)
begin; inf.inflate(data, buffer: buf); rescue Zlib::NeedDict; end
buf.clear
inf.set_dictionary("dict")
inf.inflate("", buffer: buf)
#=> [BUG] probable buffer overflow: 12000 for 15
Re-derive next_out/avail_out from the buffer at the start of every run,
and reset them when buffer: is dropped.
Reported by richardjiakangsun (https://hackerone.com/richardjiakangsun).
https://github.com/ruby/zlib/commit/e244392b8d
Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
rb_str_subpat_set validated a negative capture index by negating it,
which overflows for INT_MIN and bypasses the range check. The
out-of-bounds read of the match registers then crashed the process.
Rewrite the check without negation.
Reported by: loki (https://hackerone.com/realnits)
Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
Ruby 4 introduced native implementations of Array#find and Array#rfind
(Feature #21678). However, the new implementations changed how they
behave with multiple arguments:
ruby 3.4.8:
```
$ ruby -e "[].find(1, 2)"
-e:1:in 'Enumerable#find': wrong number of arguments (given 2, expected 0..1) (ArgumentError)
from -e:1:in '<main>'
```
ruby 4:
```
$ ruby -e "[].find(1, 2)" # returns an instance of Enumerable and doesn't generate any output
```
This patch restores the original behaviour:
```
$ ruby -e "[].find(1, 2)"
-e:1:in 'Array#find': wrong number of arguments (given 2, expected 0..1) (ArgumentError)
from -e:1:in '<main>'
```
Handle the empty label as a literal string.
OpenSSL::SSL::SSLSocket#post_connection_check can raise NoMethodError
when the server certificate contains a malformed subjectAltName
extension with a dNSName that starts with ".", such as ".example.com".
Reported by 4l1ve on HackerOne.
https://github.com/ruby/openssl/commit/1fb29e7ef5
Instead of the half-baked murmur hash we were using. Also since
it now nicely spreads out the bits, keep all of the nodes in one
hash/allocation, and keep a 4-pointer buffer on the stack for the
majority case where there aren't that many keys.
https://github.com/ruby/prism/commit/cde8d1183c
Consume the leading byte before calling tokadd_mbchar in the regexp Unicode escape fast path. The helper reads the character length from pcur - 1; passing an unconsumed byte corrupted multibyte characters and skipped following bytes.
Add regression coverage for two-, three-, and four-byte UTF-8 characters in invalid Unicode escapes inside extended regexp comments.
test_retained_malloc_triggers_oldmalloc_major_gc relied on GC.start
freeing the garbage array. On MinGW a stale stack slot can keep it
alive, so it died in the first minor GC instead. Those frees brought
oldmalloc_increase back to 0 and no major GC followed.
In this commit we allocate the garbage in a thread that has already
finished. Its stack is no longer scanned, so the full GC frees the
garbage before the retained allocation starts.
Since e631daf17b, gc_sweep_finish calls
gc_malloc_counters_snapshot_free_at_last_gc on both oldcounters and
counters.
That is correct for counters, because gc_reset_malloc_info takes the
baselines at every GC. However for oldcounters, baselines are only taken
at a major GC, so every minor sweep forgot the frees it had just made
but keeps the allocation and oldmalloc_increase grows with allocation
volume.
This means a program churning short-lived malloc'd buffers hit the
oldmalloc limit every few minors.
This commit goes back to having a single oldmalloc_increase counter,
clamped at 0, reset by a major and checked at the end of a minor sweep.
I used the following script to measure the impact of this change.
```ruby
def now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
GC.start
before = GC.stat
start = now
2_000_000.times { "x" * 4096 }
after = GC.stat
elapsed = now - start
majors = after[:major_gc_count] - before[:major_gc_count]
minors = after[:minor_gc_count] - before[:minor_gc_count]
puts [
format("%-60s", RUBY_DESCRIPTION[0, 60]),
"majors=#{majors}",
"minors=#{minors}",
"oldmalloc_inc=#{after[:oldmalloc_increase_bytes]}",
"limit=#{after[:oldmalloc_increase_bytes_limit]}",
format("%.2fs", elapsed),
].join(" ")
```
And observed the following results:
4.0.7:
majors=0 minors=614 oldmalloc_inc=3343152 limit=16777216 0.46s
Previous master:
majors=53 minors=191 oldmalloc_inc=63220807 limit=134217728 0.65s
This commit:
majors=1 minors=243 oldmalloc_inc=20829148 limit=16777216 0.64s
Recently we added some keys to GC.stat in order to measure the GC
performance of Ractor local GC and the associated global GC.
This commit refactors some of those terms: "global" was used in two
places with different meanings which could get confusing.
When it was passed to GC.stat as scope: :global it meant "Give me
aggregate stats for all the GCs running in this process".
Whereas inside the output of GC.stat, global referred to the number of
"global" GCs that are run. That is: stop-the-world GCs that run on
every ractor heap in the process.
Also the global_gc_count that GC.stat reports for a Ractor ignored
global GCs that other Ractors started, even though they swept its heap.
In this commit we fix this confusing behaviour, and make the GC.stat
output more consistent.
We've replaced the scope: parameter to GC.stat with the boolean
process: keyword arg. The default is process: false which means
GC.stat will return stats for the Ractor that executes the code (the
main Ractor if no other Ractors have been created). process: true will
return aggregate stats over all the Ractors in the process (including
dead ones).
Inside GC.stat(process: false), count and global_gc_count now
include every global GC that has swept that Ractor's heap. So Ractor
local gc count == count - global_gc_count.
heap_marked_slots now counts only the marked objects in that Ractor's
own heap. Before, the Ractor that started a global GC counted the
marked objects of every heap, and the other Ractors reported 0.
We also provide a new key global_gc_invocation_count that is the
number of global GCs that have been triggered by the Ractor in question.
We've also moved some keys related to the global page pool out of the
default GC.stat output and into GC.stat(process: true), because they
are process-wide and not Ractor specific.
Data.new turned positional arguments into a keyword Hash and called
initialize with it. When initialize is the default Data#initialize and
every member is given, that call can only store the members and freeze
the instance, so do that directly. A TracePoint on c_call no longer sees
Data#initialize for such calls.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
