Ruby 4 introduced native implementations of Array#find and Array#rfind
(Feature #21678). However, the new implementations changed how they
behave with multiple arguments:
ruby 3.4.8:
```
$ ruby -e "[].find(1, 2)"
-e:1:in 'Enumerable#find': wrong number of arguments (given 2, expected 0..1) (ArgumentError)
from -e:1:in '<main>'
```
ruby 4:
```
$ ruby -e "[].find(1, 2)" # returns an instance of Enumerable and doesn't generate any output
```
This patch restores the original behaviour:
```
$ ruby -e "[].find(1, 2)"
-e:1:in 'Array#find': wrong number of arguments (given 2, expected 0..1) (ArgumentError)
from -e:1:in '<main>'
```
obj_traverse_i called rb_ary_cancel_sharing on every array it visited,
before the traversal knew whether the object graph was shareable.
So Ractor.shareable? modified frozen arrays even when it returned false.
This commit moves rb_ary_cancel_sharing into mark_shareable, which
runs only after the traversal succeeds.
[Bug #22460]
Some example results in the docs no longer match what Ruby prints:
* Array#flatten and Array#flatten!: Set#inspect now shows Set[6, 7]
* Module: Mod.constants also includes Math::DomainError
* Module#const_get and Module#const_set: Floats are shown with full
precision
* Complex#imag: sin(PI/4) is 0.7071067811865475
* String#<<: missing # in #<Encoding:UTF-8>
* Enumerable#inject: typo #=>>
NUM2LEN raised its own RangeError for a value past rb_len_t, so a broken random generator was reported as a conversion failure instead of a number too big.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixnum stays 31 bits on mswin, so bsearch_index gives a Bignum for an index of 2**30 or more, and bsearch returned that index instead of the element.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Array#== walks both element buffers and only calls rb_equal() for a
pair whose VALUEs differ. Array#eql? called rb_ary_elt() twice and
rb_eql() once for every element, even though rb_eql() answers true
for identical VALUEs before doing anything else. For arrays of
Fixnums, Symbols, and nil, which are identical whenever they are
equal, that made eql? several times slower than ==.
recursive_eql() now walks the buffers the way recursive_equal() does.
A pair that is not identical still goes through rb_eql(), which can
run user code that resizes either array, so the length and both
buffers are re-read after every such call. Arrays whose lengths came
apart compare unequal, as they do under ==. Before, the shorter one
read as nil past its end, so the result depended on the remaining
elements of the longer one.
The new tests cover an element whose eql? empties both arrays,
shortens one of them, or grows both past their original capacity
mid-compare.
An int counter overflows once an array has more than INT_MAX elements. On LP64, product with a block then yielded the element at index 1 after the one at INT_MAX, instead of the one at 2**31.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MUL_OVERFLOW_LONG_P falls back to LONG_MIN and LONG_MAX without the overflow builtin, so once rb_len_t widens on mswin it would reject a product of 2**31 or more that the result array can hold.
https://bugs.ruby-lang.org/issues/22400
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rb_int_positive_pow takes the base as long and the exponent as unsigned long, so read the count with NUM2LEN and raise the Integer instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
These still held String and Array lengths and indices in long, or cast them to long. rb_len_t is still long everywhere, so the generated code does not change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rb_random_ulong_limited takes an unsigned long, which stays 32 bits on mswin, so once rb_len_t widens an Array of 2**32 elements or more would never yield its tail. A larger limit goes through rand with an Integer, as rb_random_ulong_limited already does for objects that are not Random.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ARY_MAX_SIZE follows rb_len_t, so the index errors report the value with PRIdLEN. The repeated_permutation exponent stays `long`, because rb_int_positive_pow() takes an `unsigned long` and the value is the user's Integer rather than an array length.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Once rb_len_t widens this changes the `long *` output parameters of rb_range_beg_len(), rb_str_subpos(), rb_arithmetic_sequence_beg_len_step(), ruby_enc_find_basename() and ruby_enc_find_extname(), so extensions passing `long *` to them will need to be updated. The extension guides list these signatures verbatim and follow along.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
When the not_exact case is hit, the initial value could be lost if it is
a float. For example, the following code incorrectly outputs 1.5 but should
output 11.5:
puts [1.5].sum(10.0)
If the initial value is a float but an element of the array is not a float,
then it could be lost and an error won't be raised. For example, the following
code should raise a TypeError but currently does not:
["hello"].sum(10.0)
rb_ary_embeddable_p does not check that the capacity of the array is within
the number of bits used for embedded lengths, so it can cause overflows
on GC that support larger object sizes.
All of our current operations on Array guarantee in-bounds reads using guards. This works +/- fine for `expandarray` and `Array#[]` but at the end of the day, it is pretty common to read out-of-bounds because it's not an error! It just returns `nil`. So in cases where we can't guarantee the array is in-bounds, add a new HIR instruction `ArrayArefChecked`. It pairs with `AdjustBounds`.
This uses `jb` to do the `idx < 0 || idx >= len` check in one instruction.
If the array is modified in the call to to_int in Array#repeated_permutation,
it can trigger an out-of-bounds because the length is cached from before
the call to to_int.
The following script demonstrates teh crash:
ary = (1..1000).to_a
obj = Object.new
obj.define_singleton_method(:to_int) do
ary.replace(Array.new(5) { :x })
2
end
ary.repeated_permutation(obj) { }
The fix in commit 11f57c6 contained a bug where the calculated index should
be `len - i` and not `len - 1`. This script reproduces the crash:
ary = (1..100).to_a
gen = Object.new
gen.define_singleton_method(:rand) { |lim| 8 }
nv = Object.new
nv.define_singleton_method(:to_int) { ary.replace(Array.new(10) { :x }); 10 }
p ary.sample(nv, random: gen)
LONG2FIX does not check the range, so `[1].cycle(2**30).size` returned -1073741824 on mswin. 64-bit platforms hit the same with 2**62.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calling to_int on the count may run Ruby code that modifies the array.
This can cause an out-of-bounds in Array#permutation. For example, the
following code crashes:
ary = (1..1_000).to_a
obj = Object.new
obj.define_singleton_method(:to_int) do
ary.replace(Array.new(5) { :x })
2
end
ary.permutation(obj) { }
When the array is shrunk during rb_arithmetic_sequence_beg_len_step,
ary_subseq_len returns -1. Return an empty array instead of passing the
negative length to ary_make_partial or ary_make_partial_step.
[Bug #22325]
The call to rand (through RAND_UPTO) or to_int (through NUM2LONG) can run
aribtrary Ruby code, which can modify the source array. The size of the
array may shrink, which could cause an out-of-bounds read on the array.
We do check again that all the random numbers are within the new array
size, but the check is incorrect because it checks against len but should
check against len-i (which is the random number requested). The following
script will sometimes crash and other times output corrupted results:
ary = (1..1_000).to_a
obj = Object.new
obj.define_singleton_method(:to_int) do
ary.replace((1..50).to_a)
10
end
gen = Object.new
def gen.rand(lim) = 49
p ary.sample(obj, random: gen)
For example, it can output:
[50, nil, nil, nil, nil, nil, nil, 367219703829, 40, 13294122074]
Array#values_at can execute arbitrary code during execution which may
modify the source array. This can cause a buffer overflow if the new length
of the array has decreased. The following script crashes:
a = (0..100_000).to_a
obj = Object.new
obj.define_singleton_method(:begin) do
a.clear
0
end
obj.define_singleton_method(:end) { 10_000 }
obj.define_singleton_method(:exclude_end?) { false }
puts a.values_at(obj).length
[Bug #22319]
Array#zip triggers an use-after-free it does not account for when the call
to rb_check_array_type modifies the source array. The following script crashes:
a = (1..100_000).to_a
evil = Object.new; $a = a
def evil.to_ary; $a.clear; [1,2,3]; end
a.zip(evil)
[Bug #22318]
Array#flatten triggers an use-after-free it does not account for when the
call to rb_check_array_type modifies the source array. The following script
crashes:
a = (1..100_000).to_a
$a = a
bad = Object.new
def bad.to_ary; $a.clear; [1,2,3]; end
a << bad
a.flatten
This commit changes Array#uniq! to use a temporary Set instead of Hash for
performance and memory savings. We can see in the following benchmark:
a = (0...10_000).to_a + (0...10_000).to_a
1_000.times { a.dup.uniq! }
Results:
Benchmark 1: master
Time (mean ± σ): 242.1 ms ± 3.9 ms [User: 229.6 ms, System: 9.6 ms]
Range (min … max): 235.7 ms … 248.3 ms 12 runs
Benchmark 2: branch
Time (mean ± σ): 197.4 ms ± 6.1 ms [User: 184.2 ms, System: 10.4 ms]
Range (min … max): 185.9 ms … 204.5 ms 14 runs
Summary
branch ran
1.23 ± 0.04 times faster than master
This commit changes Array#uniq to use a temporary Set instead of Hash for
performance and memory savings. We can see in the following benchmark:
a = (0...10_000).to_a + (0...10_000).to_a
1_000.times { a.uniq }
Results:
Benchmark 1: master
Time (mean ± σ): 194.6 ms ± 4.9 ms [User: 183.2 ms, System: 8.7 ms]
Range (min … max): 187.5 ms … 205.1 ms 15 runs
Benchmark 2: branch
Time (mean ± σ): 147.2 ms ± 4.0 ms [User: 134.9 ms, System: 9.5 ms]
Range (min … max): 139.2 ms … 153.4 ms 20 runs
Summary
branch ran
1.32 ± 0.05 times faster than master
This commit changes Array#intersect? to use a temporary Set instead of Hash for
performance and memory savings. We can see in the following benchmark:
a = (0...10_000).to_a
b = (5_000...15_000).to_a
1_000.times { a.intersect?(b) }
Results:
Benchmark 1: master
Time (mean ± σ): 126.8 ms ± 3.8 ms [User: 115.5 ms, System: 8.9 ms]
Range (min … max): 120.9 ms … 134.2 ms 22 runs
Benchmark 2: branch
Time (mean ± σ): 99.3 ms ± 3.5 ms [User: 89.4 ms, System: 7.6 ms]
Range (min … max): 94.1 ms … 109.7 ms 29 runs
Summary
branch ran
1.28 ± 0.06 times faster than master
This commit changes Array#& to use a temporary Set instead of Hash for
performance and memory savings. We can see in the following benchmark:
a = (0...10_000).to_a
b = (5_000...15_000).to_a
1_000.times { a & b }
Results:
Benchmark 1: master
Time (mean ± σ): 196.4 ms ± 9.1 ms [User: 182.9 ms, System: 9.5 ms]
Range (min … max): 186.3 ms … 214.7 ms 13 runs
Benchmark 2: branch
Time (mean ± σ): 163.4 ms ± 3.3 ms [User: 152.7 ms, System: 7.8 ms]
Range (min … max): 157.7 ms … 169.9 ms 17 runs
Summary
branch ran
1.20 ± 0.06 times faster than master
This commit changes Array#difference to use a temporary Set instead of Hash for
performance and memory savings. We can see in the following benchmark:
a = (0...10_000).to_a
b = (5_000...15_000).to_a
1_000.times { a.difference(b) }
Results:
Benchmark 1: master
Time (mean ± σ): 192.2 ms ± 7.1 ms [User: 179.7 ms, System: 9.0 ms]
Range (min … max): 182.4 ms … 208.9 ms 15 runs
Benchmark 2: branch
Time (mean ± σ): 168.1 ms ± 4.7 ms [User: 156.5 ms, System: 8.5 ms]
Range (min … max): 160.2 ms … 174.6 ms 17 runs
Summary
branch ran
1.14 ± 0.05 times faster than master
This commit changes Array#- to use a temporary Set instead of Hash for
performance and memory savings. We can see in the following benchmark:
a = (0...10_000).to_a
b = (5_000...15_000).to_a
1_000.times { a - b }
Results:
Benchmark 1: master
Time (mean ± σ): 200.3 ms ± 3.8 ms [User: 188.0 ms, System: 9.4 ms]
Range (min … max): 194.1 ms … 206.8 ms 14 runs
Benchmark 2: branch
Time (mean ± σ): 170.7 ms ± 6.6 ms [User: 158.7 ms, System: 9.3 ms]
Range (min … max): 160.9 ms … 183.7 ms 17 runs
Summary
branch ran
1.17 ± 0.05 times faster than master
This commit changes Array#union to use a temporary Set instead of Hash for
performance and memory savings. We can see in the following benchmark:
a = (0...10_000).to_a
b = (5_000...15_000).to_a
1_000.times { a.union(b) }
Results:
Benchmark 1: master
Time (mean ± σ): 239.9 ms ± 3.5 ms [User: 224.9 ms, System: 11.7 ms]
Range (min … max): 234.8 ms … 246.0 ms 12 runs
Benchmark 2: branch
Time (mean ± σ): 165.5 ms ± 4.1 ms [User: 150.9 ms, System: 11.5 ms]
Range (min … max): 158.9 ms … 174.0 ms 18 runs
Summary
branch ran
1.45 ± 0.04 times faster than master
This commit changes Array#| to use a temporary Set instead of Hash for
performance and memory savings. We can see in the following benchmark:
a = (0...10_000).to_a
b = (5_000...15_000).to_a
1_000.times { a | b }
Results:
Benchmark 1: master
Time (mean ± σ): 237.5 ms ± 2.0 ms [User: 223.4 ms, System: 10.9 ms]
Range (min … max): 234.6 ms … 240.7 ms 12 runs
Benchmark 2: branch
Time (mean ± σ): 165.2 ms ± 3.8 ms [User: 151.2 ms, System: 10.8 ms]
Range (min … max): 159.8 ms … 174.2 ms 18 runs
Summary
branch ran
1.44 ± 0.04 times faster than master
rb_yield() calls GET_EC() on every element, which on arm64 is an
out-of-line call to rb_current_ec() (RB_THREAD_CURRENT_EC_NOINLINE).
Hoist it out of the loop and use rb_ec_yield(), as ff831eb057 does
for Mutex#synchronize.
benchmark/loop_each.yml: 1.16x on arm64-darwin, 1.05x on aarch64-linux.
This used a hash with all true values, with is simple to convert
to a set to save on memory when flattening array with a large
number of nested elements.
Arrays sharing one buffer have the same data pointer, so rb_ary_splice()
took `a` in `b = a.dup; b.pop; b.concat(a)` for `b` itself and rebased
the source onto `b`'s shorter storage, copying uninitialized memory.
Have the callers tell whether the replacement really is `ary`.
[Bug #22259]
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A common Ruby idiom for variadic methods is to flatten the argument
list so that it can be called with either variadic arguments or an
Array.
e.g. from the sqlite3 gem:
```ruby
def bind_params(*bind_vars)
bind_vars.flatten.each do |var|
# ...
```
However, as soon as `flatten` encounter another array, it has to
protect against recursion, which requires allocating an expensive
identity Hash.
So this pattern has a relatively high cost when called with
a single array argument (e.g. `bind_params [1, 2, 3]`).
We can specialize for that common case without noticeably
impacting other usages of `Array#flatten`:
| |compare-ruby|built-ruby|
|:-----------------------------|-----------:|---------:|
|small_flat_ary.flatten | 7.547M| 7.974M|
| | -| 1.06x|
|small_flat_ary.flatten! | 6.031M| 6.105M|
| | -| 1.01x|
|large_flat_ary.flatten | 465.896k| 485.861k|
| | -| 1.04x|
|large_flat_ary.flatten! | 455.872k| 478.103k|
| | -| 1.05x|
|small_pairs_ary.flatten | 1.327M| 1.407M|
| | -| 1.06x|
|small_pairs_ary.flatten! | 1.165M| 1.153M|
| | 1.01x| -|
|large_pairs_ary.flatten | 96.612k| 96.976k|
| | -| 1.00x|
|large_pairs_ary.flatten! | 94.108k| 97.289k|
| | -| 1.03x|
|mostly_flat_ary.flatten | 399.648k| 417.327k|
| | -| 1.04x|
|mostly_flat_ary.flatten! | 378.315k| 395.946k|
| | -| 1.05x|
|small_nested_ary.flatten | 2.513M| 7.788M|
| | -| 3.10x|
|small_nested_ary.flatten! | 2.024M| 5.914M|
| | -| 2.92x|
|large_nested_ary.flatten | 341.635k| 485.319k|
| | -| 1.42x|
|large_nested_ary.flatten! | 329.837k| 475.014k|
| | -| 1.44x|
|small_nested_ary.flatten(1) | 9.497M| 61.350M|
| | -| 6.46x|
|small_nested_ary.flatten!(1) | 5.136M| 17.668M|
| | -| 3.44x|
|large_nested_ary.flatten(1) | 1.575M| 60.241M|
| | -| 38.26x|
|large_nested_ary.flatten!(1) | 1.369M| 16.835M|
| | -| 12.30x|
BurdetteLamar