SpecGroup#merge compares the sources of candidates, which are nearly always the same object, and every comparison built the credential-less remote strings of both sides. This removes about 300,000 allocations from `bundle update` in the large scenario of tool/bench.
https://github.com/ruby/rubygems/commit/6c17f3b152
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gem metadata is loaded with this parser instead of Psych since it became the default, and it allocated about three times as many objects per gemspec. Most came from stripping lines to measure their indentation, keyword arguments to the node structs and an Array per mapping pair. `bundle install` with a warm cache in the large scenario of tool/bench now allocates about as many objects as RubyGems 4.0.23 did.
https://github.com/ruby/rubygems/commit/b928af7767
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It runs for every platform match during resolution, and Enumerable#partition allocates three arrays plus a memo and an ifunc per call even when no candidate is content addressed. Finding the first content-addressed candidate before sorting the rest leaves the common path without allocations and the other path with fewer than before. This removes about 167,000 allocations from `bundle lock` in the large scenario of tool/bench.
https://github.com/ruby/rubygems/commit/af51b0a476
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A flow sequence was split on every comma, so a quoted entry containing one, as in ["x, y", z], came back as the fragments "\"x" and "y\"". Scan the entries so a quoted one is taken whole.
https://github.com/ruby/rubygems/commit/b3de2a6c52
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Psych writes NEL as "\N" and a BOM or other non-printable characters as "\uXXXX" or "\UXXXXXXXX", but those escapes stayed in the loaded string as is. Restore them along with "\_", "\L" and "\P". Metadata is read as binary, so a non-ASCII character unescaped into a scalar that already holds raw UTF-8 bytes is added as bytes instead of raising Encoding::CompatibilityError, which "\x80" and above already did. An escape for a surrogate or a code point beyond U+10FFFF is kept as written, as unknown escapes are.
https://github.com/ruby/rubygems/commit/d8c0371ca1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A block scalar line holding only spaces became an empty line even when it had more spaces than the indentation, which are content in YAML. Gem::YAMLSerializer.dump writes such lines for strings like "x\n \ny\n", so they did not survive a roundtrip.
https://github.com/ruby/rubygems/commit/1255df9644
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The strip indicator only took effect when it was the whole header, so "|2-" written by Psych kept the trailing line break, and even "|-" removed just one of several. Read the chomping indicator next to an indentation indicator and remove every trailing line break, as YAML specifies.
https://github.com/ruby/rubygems/commit/f32f6056d8
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Psych writes a block scalar whose first line starts with spaces as "|2", but the parser ignored the digit and took the indentation from that first line. A description such as " Indented first line\nsecond line\n" lost its leading spaces and the first characters of every later line.
https://github.com/ruby/rubygems/commit/8210dac32c
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A more indented comment line after a plain scalar was folded into the value, so "ssl_ca_cert: /path" followed by an indented comment in ~/.gemrc loaded as "/path # comment". A comment line ends a plain scalar, so treat it like a blank line there. A quoted scalar left open on its first line still runs onto such a line, since "#" inside quotes does not start a comment.
https://github.com/ruby/rubygems/commit/806b1c5698
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gem::YAMLSerializer ended a nested sequence or mapping at the first comment or blank line, so the entries after it were dropped or moved to the root mapping. Commenting out one of the :sources: in ~/.gemrc silently removed every source after it. Skip those lines before looking at the next entry, as parse_node already does.
https://github.com/ruby/rubygems/commit/02f0355460
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An override change made the lockfile count as needing resolution, so `bundle install` fetched the remote index and failed offline when the compact index cache was absent, and `Bundler.setup` re-resolved on boot. The section only needs to be regenerated, which #write_lock already does when the contents differ, and the frozen check still reports the change.
https://github.com/ruby/rubygems/commit/a616f3d1bb
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A locked gem that is not installed and whose metadata rejects the current Ruby is re-resolved during materialization whether or not an override changed, so the man page cannot promise that adding or removing a metadata override never moves a locked version.
https://github.com/ruby/rubygems/commit/c3e0897ff1
Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
`bundle lock --add-checksums --add-platform arm64-darwin` wrote a
lockfile that listed arm64-darwin under PLATFORMS but carried none of
its platform-specific gems. `Definition#add_checksums` resolves the
bundle to fetch the checksums, and it ran before the platform options
were applied, so the resolution never saw the new platform.
This usually went unnoticed. `SpecSet#add_extra_platforms!` tries to
complete every platform the sources know about, and for most bundles
that added the darwin gems anyway. It stops working as soon as one gem
has a precompiled darwin build whose `required_ruby_version` excludes
the current Ruby (bcrypt_pbkdf 1.1.2 on Ruby 3.4): `complete_platform`
then gives up on darwin for every gem, and the lockfile ends up listing
a platform it has no gems for. Running `bundle lock` again does not
repair it, because the platform is already listed.
Apply `--add-checksums` after the platform changes have been recorded.
https://github.com/ruby/rubygems/commit/2f1d325fc2
Resolving through the full-index API lazily downloads individual
gemspecs from /quick/Marshal.4.8/*.gemspec.rz. `Fetcher#fetch_spec` made
that request bare:
Bundler.safe_load_marshal Bundler.rubygems.inflate(downloader.fetch(uri).body)
while `specs_with_retry`, in the same class, wraps its network call in
`Bundler::Retry`. So a single transient failure — a read timeout, say —
aborted the whole resolution, no matter what BUNDLE_RETRY was set to.
That is most painful without a lockfile, where a resolve can fetch a
great many gemspecs one at a time and any one of them can end it.
Wrap the download in the same `Bundler::Retry.new("fetcher", FAIL_ERRORS)`
the index path already uses. FAIL_ERRORS is what keeps this from retrying
things that will not change, such as authentication failures.
Only the download branch is wrapped. The file:// and cached-gemspec
branches do no network I/O, so retrying them would just repeat a local
failure.
Closes https://github.com/ruby/rubygems/pull/9817https://github.com/ruby/rubygems/commit/9578cf9e93
A link check compiles and links conftest.c in one clang invocation, and with -g clang then runs dsymutil, which reads the DWARF of every object pulled in from libruby-static.a. Each check took about 2.2 s instead of 0.17 s on a static build.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The check ran after the prompt, so the user was still asked about executables that could not exist. File.directory? also returned false when a parent directory was not searchable, which skipped the executables and removed the gem anyway. Only ENOENT skips now, and other errors still reach the writable check.
https://github.com/ruby/rubygems/commit/ed96042879
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixesruby/rubygems#9959
remove_executables raised Gem::FilePermissionError when the gem's bin
directory did not exist at all, since File.writable? is false for a
missing directory and aborting uninstall entirely was never intended.
Now the missing directory case returns early: there is nothing to
remove.
https://github.com/ruby/rubygems/commit/03d22cfba8
Gem.open_file_with_lock removed its lock file after releasing the lock.
Since flock locks the file itself rather than its path, a process that
was waiting on the old file could acquire it after it had been removed,
while a newly arriving process created a fresh lock file and locked it
immediately, so both ran at the same time.
Remove the lock file while still holding the lock, and after acquiring
it, check that the locked file is still the one at the lock path,
retrying otherwise.
https://github.com/ruby/rubygems/commit/9a93c235e7
Gems sourced from the same git repository share one cache directory.
Since git sources are preloaded in parallel, two workers could run git
fetch in that directory at the same time, and the fetches collided on
the shallow file and its lock, failing bundle install.
Group the sources by cache path and preload each group in a single job,
so sources sharing a cache run one after another while separate
repositories still run in parallel.
Fixes https://github.com/ruby/rubygems/pull/9946https://github.com/ruby/rubygems/commit/c20901e6a1
Reading the source in binary mode made the captured path ASCII-8BIT, so a non-ASCII path never matched spec.files and joining it with a non-ASCII directory raised Encoding::CompatibilityError.
https://github.com/ruby/rubygems/commit/8090098280
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Override.from_lock accepted a line such as `"rake", version: "garbage"`, and the bad requirement only failed later as a Gem::Requirement::BadRequirementError outside the parser. Normalizing it while parsing turns it into a LockfileError that names the line.
https://github.com/ruby/rubygems/commit/4f594ea354
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bundler now writes the Gemfile overrides to an OVERRIDES section, one
line per field, sorted, with normalized requirements. The lockfile
parser reads the section without evaluating it. A change to the
overrides now changes the lockfile, and a frozen install fails with a
summary of the added and deleted overrides.
OVERRIDES is a known section, but it is not in
SECTIONS_BY_VERSION_INTRODUCED. Thus Bundler also compares it for
lockfiles that are still bundled with an older Bundler.
Override targets, from: and to: must now be valid gem names.
https://github.com/ruby/rubygems/commit/72bcc33d07
JRuby raises "TypeError: no implicit conversion of Hash into Integer" when File.foreach is called with keyword arguments and no block, as it is when chained with with_index. That broke every gem build in the JRuby Bundler specs. Opening the file with "rb" and iterating each_line reads the same binary lines on CRuby and JRuby.
https://github.com/ruby/rubygems/commit/959b637183
Co-authored-by: Ufuk Kayserilioglu <ufuk.kayserilioglu@shopify.com>
Gem::SpecificationPolicy#validate_extension_require_relative read each file with File.foreach, which tags lines with Encoding.default_external. Under an unset or ASCII locale (build sandboxes, minimal containers) a valid UTF-8 non-ASCII line raised ArgumentError: invalid byte sequence in US-ASCII, and a non-UTF-8 source did the same under a UTF-8 locale. The error escaped validation and aborted gem build.
The pattern is ASCII-only, so read the file as binary.
Fixes https://github.com/ruby/rubygems/pull/9966https://github.com/ruby/rubygems/commit/e158b757fd
`Gem::StreamUI#ask_for_password` printed the prompt before it turned off
echo, so the terminal could echo back a password written as soon as the
prompt appeared.
Print the prompt inside the `noecho` block.
https://github.com/ruby/rubygems/commit/e648f0dfa1
IPAddr.new and IPAddr#zone_id= only accepted \w characters after %, so zone IDs for interfaces such as eth0.100 (VLAN) or br-lan raised InvalidAddressError. RFC 6874 defines ZoneID as unreserved characters (ALPHA, DIGIT, -, ., _, ~), which is also what Resolv::IPv6 accepts (%[-0-9A-Za-z._~]+). Use the same character class in both places.
https://github.com/ruby/ipaddr/commit/8aaee2fa0c
interpreter
(https://github.com/ruby/syntax_suggest/pull/266)
* Prefer Ruby::Prism, the parser built into the interpreter
Ruby 4.1 exposes the prism parser built into the interpreter as
Ruby::Prism, and the prism gem becomes a bundled gem
(https://bugs.ruby-lang.org/issues/22379).
syntax_suggest is loaded when a script fails to parse, so it should not
depend on the prism gem, which may be absent from the Gemfile under
bundler. On Ruby 4.1+ use Ruby::Prism; on older Rubies keep requiring
the prism gem.
Every reference to Prism in syntax_suggest is lexically inside the
SyntaxSuggest module, so defining SyntaxSuggest::Prism is enough and
the call sites are untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Update lib/syntax_suggest/api.rb
* Update CHANGELOG.md
* Apply suggestion from @Earlopain
---------
https://github.com/ruby/syntax_suggest/commit/e11b8842ff
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Earlopain <14981592+Earlopain@users.noreply.github.com>
Returns the binary representation of the address value as a string of
"0" and "1" characters, right adjusted to the address size (32
characters for IPv4, 128 for IPv6).
https://github.com/ruby/ipaddr/commit/63beaa749e
CRuby is now hashing the contents of a file and attaching that to
iseqs in order to properly determine if a file has changed since
it was compiled. This is used in a couple places, but namely it is
used for error_highlight to make sure it isn't highlighting stale
locations.
The hashing algorithm doesn't actually matter since it's just used
to compare across parses, so as long as it is consistent it's okay.
I wanted something fast and standard, so this commit implements the
XXH3_64 algorithm from the spec. Noteably it uses the default
secret and a seed of 0 to make things even simpler. With NEON and
SSE2 implementations, this thing flies quite a bit faster than the
upstream FNV-1a implementation.
I figured this could be useful for other consumers as well, in
particular for tools that generate some kind of metadata around
files (type checkers, linters, etc). Eventually a nice extension
may be to hash a minimal version of the contents (i.e., with
context-insensitive whitespace removed) so that changes to docs or
indentation do not change the hash value.
If you're using the C API, you are only charged for this function
if you actually call it since it is computed lazily. For everyone
else it is automatically called, but truthfully it is so fast that
the difference is completely negligible.
https://github.com/ruby/prism/commit/6b7c5773eb
UnicodeNormalize.normalize and .normalized? read three constants that are
not shareable, so any call from a non-main Ractor raised
Ractor::IsolationError:
* NF_HASH_C / NF_HASH_D, the memoization caches. They cannot be made
shareable because a shareable Hash cannot be written to. Keep them per
Ractor instead: normalization is deterministic, so recomputing an entry
in another Ractor always gives the same string, and the cache is only a
cache. The DoS bound of MAX_HASH_LENGTH now applies per Ractor.
* UNICODE_ENCODINGS, which is splatted in a `when` clause on the path for
UTF-16, UTF-32, GB18030 and UCS-2/4BE. Freeze it.
The other constants are already shareable on master: tables.rb is
frozen_string_literal with frozen tables, and Regexp instances are frozen
since cb03bf43eb.
Keeping the caches keeps main Ractor performance: dropping them instead
costs 2.7x to 11x on real text, and asking the main Ractor for an entry
costs 19.4us per round trip against 3.8us to just compute it. The per
Ractor lookup adds about 60ns per call; normalized? fetches its cache
lazily so a string without a match does not pay it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IPAddr.new("[2001:db8::]/32") is currently accepted and treated the
same as IPAddr.new("2001:db8::/32"). However, square brackets are a
delimiter for embedding an address in a larger syntax such as
host:port (RFC 5952 section 6), and no notation is defined for a
prefix length after the closing bracket.
Print a warning for this form, saying that it will raise
IPAddr::InvalidAddressError in a future version. The result is
unchanged for now. "[2001:db8::]" and "2001:db8::/32" are not
affected.
https://github.com/ruby/ipaddr/commit/a540d51a31
Introduce a perfect hash for keyword checking so we do not have to
do as many comparisons. This results in about a 1% speed up.
Also, introduce a function called by the C extension and the FFI
backend that can check if something is a keyword.
https://github.com/ruby/prism/commit/9d6a9e7d5f
mkmf on mswin moves the drive letter of prefix into DESTDIR in the Makefile, so passing an empty DESTDIR made C extension installs fail on a GNU make built mswin Ruby whenever the current drive was not the one holding prefix. The check now depends on the platform instead of on nmake.
https://github.com/ruby/rubygems/commit/b70ec9230f
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Run recipes with cmd.exe instead of /bin/sh, rewrite the nmake-style `$(VAR:/=\)` substitutions, which GNU make applies only to suffixes, and write the note on failed extensions in the form cmd.exe runs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Windows, parallel jobs creating the same timestamp file, such as the extension builds or `make enc` and `make trans`, fail with a sharing violation while the file exists anyway.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TestResolvDNS#test_query_ipv4_address occasionally fails on Windows CI:
1) Failure:
TestResolvDNS#test_query_ipv4_address [C:/a/ruby/ruby/src/test/resolv/test_dns.rb:236]:
<1> expected but was
<0>.
On Windows, Resolv picks a random source port by itself and binds
0.0.0.0 to it. Windows allows this even when another socket has
already bound 127.0.0.1 to the same port, and then delivers datagrams
sent to 127.0.0.1 to that other socket. When the chosen port collides
like this, the reply from the nameserver on 127.0.0.1 never reaches the
resolver, which waits out every timeout and returns no resources.
This commit sets SO_EXCLUSIVEADDRUSE before binding so that bind fails
on a port another socket already uses, and retries with another port.
Ruby does not define Socket::SO_EXCLUSIVEADDRUSE, so its value,
~SO_REUSEADDR, is used directly.
https://github.com/ruby/resolv/commit/ce593abc84
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Net::HTTP required the whole DNS resolver, which also reads the system DNS configuration at load time, only to match Resolv::IPv4::Regex and Resolv::IPv6::Regex when deciding whether to send SNI. Use the same inlined regexps as the copy vendored in RubyGems, so that the two stay identical.
https://github.com/ruby/net-http/commit/5b42d75812
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The common -v (verbose) was defined after each command's own options and replaced mkmf's `-v ARGS`, so have_var never ran and the argument was passed to create_makefile as the target name. wait_writable's own `v` was dead for the same reason and would now hide the common -v, so drop it.
https://github.com/ruby/un/commit/b52c72da41
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
