Since the fix for Bug #21441, ec->cfp is set to NULL before clearing
the VM stack so that thread_profile_frames() called from a signal
handler returns early instead of walking a stack that is no longer set
on the ec. But these are plain stores to distinct fields of the same
struct, so the compiler is free to reorder them.
With Apple clang 21 and GCC 16 on macOS arm64, and with clang 18 and
GCC 13 on Linux aarch64, both stores of rb_ec_set_vm_stack() were
combined into a single stp emitted before the store to ec->cfp
(vm_stack is at offset 0, vm_stack_size at 0x8 and cfp at 0x10):
stp xzr, xzr, [x0] ; ec->vm_stack = NULL, vm_stack_size = 0
str xzr, [x0, #0x10] ; ec->cfp = NULL
ret
So a signal in between would see a non-NULL ec->cfp with a NULL
ec->vm_stack. Stopping there in lldb when a thread terminates and
calling rb_profile_frames() segfaults in thread_profile_frames(), as
end_cfp is computed from the NULL ec->vm_stack and the loop walks past
the control frames. With this change, ec->cfp is already NULL at that
point and rb_profile_frames() returns 0.
Like in vm_push_frame(), use atomic_signal_fence() to keep the store to
ec->cfp first. With it, all these compilers emit the store to ec->cfp no
later than the stores clearing the stack. The rest of vm.o is unchanged
with Apple clang 21, GCC 16 and GCC 13 on aarch64. Clang 21 and 18:
stp xzr, xzr, [x0, #0x8] ; ec->vm_stack_size = 0, cfp = NULL
str xzr, [x0] ; ec->vm_stack = NULL
ret
GCC 16 and 13:
str xzr, [x0, #0x10] ; ec->cfp = NULL
stp xzr, xzr, [x0] ; ec->vm_stack = NULL, vm_stack_size = 0
ret
With GCC 13 on x86_64, the store to ec->cfp is already emitted first
without the fence, and vm.o is identical with and without it:
endbr64
movq $0x0, 0x10(%rdi) ; ec->cfp = NULL
movq $0x0, (%rdi) ; ec->vm_stack = NULL
movq $0x0, 0x8(%rdi) ; ec->vm_stack_size = 0
ret
[Bug #22477]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rb_vm_empty_cc(), rb_vm_empty_cc_for_super() and rb_vm_uncached_cc()
shouldn't be real function calls. rb_iseq_mark_and_move was calling
these as real functions in a loop.
The assumption is that we're usually calling these methods on boot, and they don't
need CCs. CCs scale with the number of classes that use these methods in the app, so
if you have 10k classes and these classes use 3 of these methods on average, then that
saves 30k CC objects.
Methods on Module that are marked cold are:
public, protected, private, module_function, ruby2_keywords, alias_method, undef_method,
remove_method, attr_reader, attr_writer, attr_accessor, include, prepend.
If these methods ever get redefined, they get new method definitions that are not
marked uncacheable (CCs will get created).
In one of our apps (not the largest), we observed this difference after boot:
| method | before | after | delta |
|-------------------|-------:|------:|------------:|
| `include` | 4518 | 2 | -4516 |
| `alias_method` | 3921 | 0 | -3921 |
| `ruby2_keywords` | 3784 | 0 | -3784 |
| `private` | 3519 | 3 | -3516 |
| `attr_reader` | 2738 | 0 | -2738 |
| `attr_accessor` | 2288 | 0 | -2288 |
| `prepend` | 354 | 1 | -353 |
| `protected` | 303 | 0 | -303 |
| `public` | 302 | 0 | -302 |
| `undef_method` | 178 | 0 | -178 |
| `attr_writer` | 124 | 0 | -124 |
| `module_function` | 42 | 0 | -42 |
| `remove_method` | 24 | 0 | -24 |
| **total** | | | **-22,089** |
NOTE: CC tables and ccs (the malloced struct) are still being created, so this only reduces
the CC objects themselves. I will tackle that in a follow-up.
Under the M:N thread scheduler, a Ruby thread in a non-main Ractor may be
resumed on a different native thread after it blocks. The C compiler
assumes that the native thread never changes within a function, so it may
compute the address of ruby_current_ec once (`__tls_get_addr` in
libruby.so) and reuse it after the switch. The address then refers to the
TLS of the previous native thread, which leads to a SEGV or to using
another thread's ec.
8.times.map do
Ractor.new do
50.times do
",,".split(/(,)/) { sleep 0.001 }
end
:ok
end
end.map(&:value)
#=> [BUG] Segmentation fault at 0x0000000000000030 (with Clang 21)
arm64 and ppc64le already use the out-of-line rb_current_ec()
(RB_THREAD_CURRENT_EC_NOINLINE), which prevents the compiler from caching
the address. Use it on all platforms, and remove the macro.
[Bug #22453]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Any iseq created inside an eval, class_eval or module_eval had its
`location.pathobj` set to an Array of 2 elements where the 2nd element was
`nil`. This is wasteful in terms of memory and also causes longer major GC
times. This was true for the eval forms even when `__FILE__` was given
directly.
Example script:
```
class Foo; end
Foo.class_eval("def hi = 'hi'")
ObjectSpace.each_object(Array) { |obj|
p obj if obj.frozen? && obj.size == 2 && obj[0].to_s.start_with?("(eval at")
}
```
output before:
```
["(eval at ../ruby/test.rb:47)", nil]
["(eval at ../ruby/test.rb:47)", nil]
```
output after: empty
The changes to the 2 structs do not alter the struct sizes (they make use of
padding that was already there).
This also saves ~1000 Arrays at startup, some from the builtin_prelude: arrays
like `["<internal:numeric>", nil]` per iseq created in those files. Around half
of those survive GC and become long-lived Arrays.
This commit adds `jit_calls` and `jit_threshold_reached` to iseq dumps which
dumps the number of calls to the iseq and whether the iseq has been JIT
compiled or not, respectively.
* Make `__FILE__` a frozen core method that return the iseq path.
* Make `load_from_binary` accept `file, path` like compile.
[Feature #17593]
[Feature #22395]
With the default `-O2sy-`, cl.exe compiles the instruction dispatch in vm_exec_core into a binary search. Favoring speed only in vm.c gives it a jump table, while the rest of the DLL and the extensions that inherit OPTFLAGS through RbConfig stay optimized for size.
With reg_pc kept in a local, this makes 20 benchmarks about 20% faster on x64 and grows the DLL by about 1%.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The registered_globals list made every Ractor's GC scan every slot under
a shared lock. This commit splits it into per-Ractor lists.
We also maintain a global registry that references every Ractor that has
ever registered so that unregistering from a Ractor other than the one
that registered can look up the owning list.
When a Ractor is joined, the dead Ractor's slots are moved to the
joining Ractor's heap before the objspace merge happens.
This change tightens up the requirements for values stored in a
registered address.
Previously, Every Ractor's local GC scanned the shared list. So if a
slot held an unshareable object living in a different Ractor's heap,
that object was always kept alive regardless.
After this change, only the registering Ractor's GC scans its list. So if a
slot holds another Ractor's unshareable object, the owning Ractor's
local GC cannot see that reference, so it can collect the object while
the registered slot still points at it.
Storing such an object was already a Ractor containment violation. It
was just hidden by the shared scan.
[Feature #22277]
The box of the immediately previous frame can be the root box, e.g. when
a proc made in the root box calls Marshal.load, and the classes of the
caller's data are invisible there. Walk the caller frames up to the
nearest user box instead, and rename the attribute after what it finds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Builtin methods written in Ruby are defined in the master box, so the
box resolution stops at their frames and cannot see classes defined in
the caller's box. The new attribute makes such methods operate on the
caller's box, as CFUNC frames already do.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
eb38fb670b added it as a thread-safe counterpart to postponed_job, for
MJIT to enqueue work from its own thread [Bug #15316]. MJIT and RJIT are
gone and nothing enqueues anything any more: rb_workqueue_register() has
no caller, no declaration in any header, and hidden visibility, so an
extension cannot reach it either. What is left is rb_postponed_job_flush()
taking workqueue_lock on every call to steal a list that is always empty.
rb_postponed_job_flush() no longer needs the VM: the preregistered job
table it drains is a file-static. Drop the argument.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
env_copy() overwrote the SPECVAL slot of every local env with
VM_BLOCK_HANDLER_NONE. For a TOP/CLASS frame's env that slot stores
the box (VM_ENV_BOX), not a block handler, so a proc defined at a
class/module body and isolated by Ractor.make_shareable lost its box
while keeping its frame type. Any method call inside such a proc then
made rb_current_box() return NULL via current_box_on_cfp(), and the
classext lookup dereferenced the NULL box:
RUBY_BOX=1 ruby -e \
'module M; L = Ractor.make_shareable(->(*a){ Rational(*a) }); end;
p M::L.call(3, 4)'
# => SIGSEGV at 0x0 in rb_vm_search_method_slowpath
Copy the SPECVAL slot through instead when the source env belongs to
a TOP/CLASS frame, so the isolated proc keeps resolving methods in
its defining box.
[Bug #22260]
Using the `container->obj = rb_gc_location(container->obj)` pattern dirties pages unconditionally.
If we're in a forked process, this is bad for CoW. If the object hasn't moved, don't reassign.
create_ractor_alloc_thread() pointed cr->objspace at the child's objspace while
it allocated the child's Thread and root Fiber wrappers, so that they would be
made of objects the child owns. That slot is read without the VM lock by
threads that hold no GVL -- a Ractor's postmortem epilogue, which clears the
TLS EC before freeing the dead thread, thread_sched_reclaim, and any thread
that released the GVL inside rb_nogvl -- to decide which objspace to charge a
free to. One of them could therefore be sent to the child's objspace; a
stillborn child (IsolationError) is then disowned and freed by the orphan merge
job, and the free landed in released memory (ASAN heap-use-after-free at the
malloc accounting in ruby_xfree_sized, seen once in 209 CI runs).
The window allocates exactly two objects, so hand the objspace down to them
instead: rb_newobj_in_objspace() and the two TypedData entry points built on
it carry it to thread_alloc() and fiber_alloc_in(). cr->objspace never moves,
so no other thread can observe anything, and rb_gc_get_objspace() is unchanged.
GC suppression around the wrappers now names the child's objspace too, which is
the one the allocations go to; resolving it through the current Ractor would
suppress the creator's instead and let a cycle collect the half-built child.
The creation cover goes up before the first allocation rather than after the
last one, and the allocation-failure path parks the objspace in the same
VM-lock section, so a global GC can never find it neither covered nor a zombie.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
[Bug #22015]
During a normal method call, the VM creates a control frame, and the Ruby::Box associated with that frame determines the execution context.
However, Symbol#to_proc does not automatically create a frame.
Therefore, we need to manually push a frame associated with the appropriate Box context.
When determining the Box to associate with this new frame, we cannot simply look at the closest Ruby-level control frame.
This is because some built-in methods (like `Enumerable#map`) are implemented in Ruby (e.g., `<internal:enum>`).
These built-in Ruby frames belong to the default Root or Main Box, which would obscure the actual user-defined Box context we need to capture.
To fix this properly, we now traverse the frames and check their inherent Box states.
Since built-in Ruby frames always run in the Root or Main Box, we safely skip them by continuing our traversal until we find a user-defined Box (an Optional Box, indicated by `BOX_OPTIONAL_P`).
An ifunc proc created by Method#to_proc and invoked directly via
Proc#call has no enclosing CFUNC frame. The IFUNC skip loop in
VM_EP_RUBY_LEP returned the caller's ep unconditionally, so a non-local
EVAL ep reached current_box_on_cfp and crashed with
"BUG: Local ep without cme/box". Return the ep early only when the
enclosing frame is a CFUNC, and otherwise continue to the local ep walk.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
[Bug #21977]
When `RUBY_BOX=1` is set, combining `binding` with `Symbol#to_proc` causes a crash (`[BUG] BUG: Local ep without cme/box`).
This occurs because escaping an `IFUNC` frame via `binding` adds the `VM_ENV_FLAG_LOCAL` flag. `VM_EP_RUBY_LEP` previously relied on `VM_EP_LEP`, which blindly stopped at this flag. This caused it to return the `IFUNC` itself instead of its caller, and since `IFUNC` lacks `cme/box` data, the Box resolution crashed.
This commit fixes the issue by traversing `current_cfp` backwards to skip `IFUNC` frames directly, instead of relying on `VM_EP_LEP` and the local flag.
If we have 32 local variables or less (which is most iseqs), then we can
embed lvar_states inside the pointer, which removes the need for an
external memory allocation.
When we run out of JIT memory, we don't compile the iseq, so jit_entry
will be forever NULL. This means that the JIT call counters (jit_entry_calls
and jit_exception_calls) will continue to increase on every call. This
is bad for copy-on-write because it causes the whole rb_iseq_constant_body
to be copied.
We can see an example using the following script (which can only be ran
on Linux) that allocates 200k methods, executes all the methods, then
forks a child process that also executes all the methods and outputs
the change in RSS and PSS:
SYMS = 200_000.times.map { |i| eval("def m#{i} = nil"); :"m#{i}" }
def stats = File.read("/proc/self/smaps_rollup").scan(/^(Rss|Pss):\s+(\d+)/).to_h
SYMS.each { |s| 3.times { send(s) } }
fork do
before = stats
SYMS.each { |s| send(s) }
after = stats
puts "RSS: #{before["Rss"]} -> #{after["Rss"]} kB"
puts "PSS: #{before["Pss"]} -> #{after["Pss"]} kB"
end
Process.wait
When running with YJIT (with `--yjit-call-threshold=1 --yjit-exec-mem-size=1`),
we can see that before this commit, the PSS increases significantly but
stays flat after this commit:
Before:
RSS: 210236 -> 210496 kB
PSS: 104676 -> 162300 kB
After:
RSS: 204308 -> 204692 kB
PSS: 101700 -> 101960 kB
We can see a similar result with ZJIT (with `--zjit-call-threshold=1 --zjit-exec-mem-size=1`):
Before:
RSS: 493564 -> 493732 kB
PSS: 246416 -> 390452 kB
After:
RSS: 493292 -> 493516 kB
PSS: 246286 -> 246428 kB
Windows carried a second, much weaker scheduler: a single Win32 mutex as
the GVL, vm->clock for preemption, its own VM barrier over
vm->ractor.sync.barrier_*, its own Ractor wait/wakeup on a per-Ractor
condvar, no ready queue and no time slice. Keeping two made the rest of
the interpreter branch on which one was compiled in, and left Ractors on
Windows without real scheduling.
Move the scheduler out of thread_pthread.c into thread_sched.c and let
both platforms use it. thread.c now includes thread_sched.c, which
declares the scheduler <-> platform contract, includes THREAD_IMPL_SRC,
and then builds the scheduler on the primitives that file provides.
thread_pthread.c keeps only those primitives: synchronization, thread
local storage, native thread creation and stacks, the communication pipe,
the timer thread backend, the fork lock and the event hooks.
The data structures the scheduler works on move to a new thread_sched.h,
included from both platform headers, so struct rb_thread_sched,
struct rb_native_thread and struct rb_thread_sched_item are now shared.
thread_pthread_mn.c becomes thread_sched_mn.c: it is the M:N scheduler,
not a pthread detail. Most of it -- the coroutine threads, the native
thread stack pool, the timer wheel and the fd -> waiters map -- is
platform independent; only the readiness backend is not, and that section
is now marked as the seam where an epoll/kqueue split (or an IOCP backend,
which is what Windows would need for M:N) would go.
What Windows had to add is small: native_thread_interrupt(),
native_thread_default_max_cpu(), timer_thread_polling(),
timer_thread_wakeup_force(), RB_NATIVE_COND_MONOTONIC_P() and
RB_NATIVE_MUTEX_TRYLOCK_DETECTS_SELF() -- the last because a
CRITICAL_SECTION is recursive, so trylock cannot back a "somebody holds
this" assertion. It sets USE_MN_THREADS to 0 and supplies the stubs
thread_sched_mn.c provides elsewhere, so every thread there is dedicated,
as it already was.
With one scheduler the platform branches go away: all 16 uses of
RUBY_THREAD_PTHREAD_H / RUBY_THREAD_WIN32_H are removed from vm_core.h,
ractor.c, ractor_core.h, ractor_sync.c, vm_sync.c, vm.c and thread.c.
ractor_check_blocking() went with them -- it only ever fed the win32
scheduler's vm->ractor.blocking_cnt -- and so did USE_VM_CLOCK, now that
the common time slice drives preemption on Windows too.
Verified on both platforms. Linux: 11 rounds of make test-all (36236
tests) and btest (2066 tests) with no failures, plus 474 runs of a
scheduler and Ractor stress soak. Windows: no test fails that does not
also fail on the same tree without this change, btest passes, and two
40-minute soaks of 438 and 460 runs are clean. Ractors now run in
parallel there -- 3.6x on four Ractors, where before they could not.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
box_entry_free could run before other T_DATA objects are finalized in
rb_objspace_call_finalizer, and FreeLibrary there unmapped the
rb_data_type_t and dfree functions that later finalization still
dereferences, crashing any boxed process that leaves extension objects
alive at exit. Keep the copied DLLs loaded until ruby_vm_destruct, then
unload and delete them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rather than start from a mutable hash created with `newhash` or
`duphash`, and then iteratively insert into it, it's preferable to
start from a constant hidden hash, so that we can directly allocate
the final hash with the right size.
For instance:
```ruby
{a: 1, B => 2, c: 3}
```
Before:
```
0000 duphash {a: 1} ( 1)[Li]
0002 putspecialobject 1
0004 swap
0005 opt_getconstant_path <ic:0 B>
0007 putobject 2
0009 putobject :c
0011 putobject 3
0013 opt_send_without_block <calldata!mid:core#hash_merge_ptr, argc:5, ARGS_SIMPLE>
```
After:
```
0000 putobject {a: 1} ( 1)[Li]
0002 putspecialobject 1
0004 swap
0005 opt_getconstant_path <ic:0 B>
0007 putobject 2
0009 putobject :c
0011 putobject 3
0013 opt_send_without_block <calldata!mid:core#hash_merge_ptr, argc:5, ARGS_SIMPLE>
```
Since, `duphash` has no idea how large the final hash might be,
it can't right-size the hash it allocates.
Hence it's preferable to use a `putobject` and let `hash_merge_ptr`
compute the final hash right size by assuming all keys are unique.
We know the type of the proc object at allocation time, so we can allocate
exactly the amount of memory needed for that type.
We can also shrink the type from a 4 byte enum since there are only 4 types
of procs. This allows us to put some flags after it, which saves us 8 bytes.
This commit reduces symbol and proc procs by 24 bytes (80 bytes to 56 bytes).
It also reduces iseq and ifunc procs by 8 bytes (80 bytes to 72 bytes).
lvar_states only has 3 states, but an enum uses 4 bytes. This commit shrinks
lvar_states to use only 2 bits (0.25 bytes), which means we save 3.75 bytes
for every local variable in an iseq.
We can see that for this script:
def foo
a = 1
b = 1
c = 1
d = 1
end
iseq = RubyVM::InstructionSequence.of(method(:foo))
puts ObjectSpace.memsize_of(iseq)
It outputs 736 before this commit, and 721 now, which is 15 bytes saved
over the 4 local variables.
A basket is rooted by whoever holds it, and nothing else: the port queue it
waits on, or -- while it is on no queue -- its holder's off_queue_baskets
list. rb_ractor_sync grows that list and the basket a second ccan_list_node,
so it can sit on one or the other.
send alloc the basket, put it on the sender's list, then build the
payload into it: rb_ractor_{move,copy}_courier_build publish the
courier into the basket right after ZALLOC, so even a half-built
one is rooted. A move needs this -- it husks the sources as it
captures them, so what the courier has collected loses its other
root as the walk proceeds.
enqueue off the sender's list, onto the receiver's queue
receive off the queue, onto the receiver's list until the basket is freed
The publish is the handover of ownership too: when the capture raises, the
builder re-raises and leaves the courier where it is, and the basket frees it
exactly once as ractor_basket_new unwinds. (The raise path was pointed out by
the Copilot review on ruby/ruby#18392.)
Before this, a courier was the one payload ractor_basket_mark skipped: it
left the whole job to vm->ractor.move_courier_registry, which only a global
GC walks. Between the send and the next global GC nothing rooted the
payload, and a shareable object reachable from the courier alone was
collected -- a dynamic Symbol loses its fstr, a frozen Array its elements,
and reading it crashes. A shareable String survived only by being an
fstring, rooted elsewhere. That is already broken in master for move; it
becomes reachable from an ordinary send once copy travels by courier. Two
reproducers, both crashing unpatched master and fine on 4.0.2, now pass:
ports = 5.times.map do |i|
port = Ractor::Port.new
r = Ractor.new(port, i) { |p, n| p.send([:"sym_#{n}"], move: true) }
r.join
port
end
5.times { GC.start }
ports.each { |p| puts p.receive[0].inspect }
(the symbol has to be interpolated: a literal one is held by the iseq) The
second sends Ractor.make_shareable([1, "child"]) the same way.
ractor_sync_mark walks the list next to the queues it already walks, so this
needs no new root pass, and it walks both in every collection. "What a
courier holds is shareable, and only a global GC frees a shareable" does not
hold: pinned_roots_mark, which roots a shareable from its page bit, is
skipped once the process is back to a single Ractor
(rb_gc_single_objspace_p), and then an ordinary local GC frees one that
nothing else names. A payload in flight is named by its basket and nothing
else. The list needs no sync lock even so -- only its owner touches it,
unlike the queues a foreign sender writes. The one caller that does skip it
is the single-objspace root pass, where ractor_mark_unshareable_parts has
just walked the same baskets through ractor_sync_mark.
This retires the registry, its lock, its fork re-initialization and
rb_ractor_move_courier_registry_mark. Walking the registry in every
collection instead of the basket would not do: a receiver's local GC, which
materialize's own allocation can trigger, would then walk a courier another
Ractor is inside move_alloc_ref for -- observed as a crash.
move_alloc_node and move_alloc_ref grow by swapping in a fresh array rather
than realloc, for the same reason the nodes are initialized mark-safe: the
courier is a GC root while it is built, and a realloc can leave c->nodes
pointing at a block it has already freed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
With MatchData and an exception's backtrace on the courier, every copy
payload is off-heap: the courier carries the core types and anything else
travels as marshaled bytes. The sender-resident snapshot is gone, and with
it all the machinery that kept one alive across a global GC.
MatchData: rb_match_move_dump takes a release_source flag, since only a move
takes the source's onig region and char_offset apart.
Backtrace: rb_backtrace_blob_dump / _load / _mark copy the frames into an
off-heap blob. A frame only references shareable iseq and method-entry
imemos, so the blob can carry them as they are; it has no compaction update
hook, so the mark pins them (rb_gc_mark, not _movable).
Removed, all of it unreachable once no payload lives in the sender's heap:
rb_ractor_t pin_capture / _cnt / _capa, sending_basket,
gen_fields_capturing, ractor_pin_capture_push
basket p.pinned / p.pinned_cnt, ractor_basket_repin_in_flight,
rb_ractor_repin_in_flight and its queue walks
receiver struct ractor_materialize_frame, ec->materialize_frames and
its mark loop, sync.materializing_copies,
rb_ractor_materializing_p
gc rb_gc_pin_in_flight_message and the whole
rb_gc_impl_pin_in_flight_message hook (default, mmtk, the
modular function table), plus the two verifier relaxations
that existed for a snapshot being materialized
shref bits stay: the write barrier still records them for other reasons.
ractor_copy_native_try and copy_enter stay too, since
Ractor.make_shareable(obj, copy: true) still deep-copies within one objspace.
Message cost, 20k messages per trial, 9 trials, median us per message:
4.0.2 master here
4KB String 3.8 19.4 9.4
20-key Hash 3.3 11.9 7.5
nested Hash/Array/Str 5.3 7.1 5.1
100-elem Integer Array 4.4 5.7 6.9
40B String, Object same across all three
master is well behind 4.0.2 on multi-KB payloads; this recovers about half
of that and puts nested graphs back at 4.0.2's level.
Known issue, already in master and not introduced here: the courier's
registry mark does not traverse a shareable payload, so its children are
collected if a global GC runs while the message is in flight. Unpatched
master crashes on both repros (a dynamic Symbol loses its fstr, a frozen
Array loses its elements) and 4.0.2 does not. It only shows up for move
today; routing copy through the courier makes it reachable from a common
path, so it wants fixing before this ships.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
MMTk marks objects on GC worker threads. Those threads have
no execution context, so GET_EC() inside this VM_CHECK_MODE
helper failed an assertion when imemo_env marking checked
VM_ENV_ESCAPED_P:
vm_core.h:2243: Assertion Failed: rb_current_execution_context:!expect_ec || ec != NULL
Problem: Any C function called after gen_prepare_non_leaf_call() has
the freedom to push values through `cfp->sp`, moving it, and then ask
for a zjit_materialize_frames(). (The C function has to restore
`cfp->sp` before it returns, but it can move it temporarily.) Since
zjit_materialize_frames() used `cfp->sp` as the starting point of
the cursor that writes out the stack, it wrote to the wrong slots.
In practice this happened when the non-leaf C function fails to make
a call and raises ArgumentError.
Solution: Allocate one more native frame slot. Stash `SP`, the VM stack
base pointer in gen_prepare_non_leaf_call(), and add a new stack map
opcode for zjit_materialize_frames() to find it. Materialization during
non-leaf calls now reliably starts from `base_ptr+stack_size` and we
don't look at `cfp->sp`.
* [22224] Ractors, YJIT: Skip re-reporting recorded EP escapes to the JITs
vm_make_env_each reports every EP escape, and both
rb_yjit_invalidate_ep_is_bp and rb_zjit_invalidate_no_ep_escape take
the VM lock with a stop-all-Ractors barrier on every call. For a given
iseq only the first report does anything: afterwards YJIT's
no_ep_escape_iseqs entry is permanently an empty set and ZJIT's
ep_escape_iseqs insert is a no-op, so the barrier is paid for nothing.
Under multiple Ractors this serializes the whole process on every Proc
created from an already-escaped iseq: a workload creating one lambda
per iteration runs 150x slower on 8 Ractors than on 1 with YJIT
enabled.
Record the report in a jit_ep_escape_recorded flag on the iseq body,
set by the active JIT while it holds the VM lock, and skip the calls
in vm_make_env_each once it is set. The flag sits in existing padding
and the body is ZALLOC'd, so it starts false and dies with the iseq.
Setting it in the JIT rather than in the caller keeps mid-process
RubyVM::YJIT.enable correct: escapes reported while no JIT is
initialized are recorded by nobody and must not set the flag. The
unlocked read is benign: a stale false causes one redundant,
idempotent report.
On the micro-benchmark from the bug, --yjit at 8 Ractors goes from
0.04M to 11.07M iterations/s (277x).
* [22224] Deduplicate the EP escape record onto the iseq body
The jit_ep_escape_recorded flag introduced in the previous commit is
now the JITs' only record of an EP escape: YJIT's no_ep_escape_iseqs
keeps only iseqs with live assumptions (the empty-set escape markers
are gone), and ZJIT's ep_escape_iseqs set is removed together with its
update_ep_escape_iseqs compaction pass, since the flag lives in the
iseq body, which the GC does not move. Both JITs read the flag back at
compile time through a new rb_jit_iseq_ep_escape_recorded_p helper.
Co-authored-by: Alan Wu <XrXr@users.noreply.github.com>
The retire GC at notify_exit could not reclaim the terminating
thread's scaffolding -- the Thread and Fiber wrappers, the ThreadGroup,
the Ractor-local stdio, whatever the machine stack still references --
because at that point they were all still rooted. Those ~11 objects
kept ~2 heap pages alive per dead Ractor, merged into the inheritor to
wait for its next major GC, so both a spawn-and-join loop and a
fire-and-forget loop grew the heap by two pages per Ractor with
nothing reclaiming them.
Instead, run the final collection at the tail of thread_start_func_2,
just before the scheduler handoff. The thread still holds the GVL, so
the collection runs lock-free like any local GC (a concurrent global
GC waits for its safepoint), and vm_remove_ractor keeps its light
lock-only section. r->postmortem makes rb_ractor_mark_local_roots
root only what must survive the death -- the join value
(r->sync.legacy) and the rb_gc_register_mark_object pins -- and the
zombie_objspaces entry recorded later measures exactly the pages the
joiner will inherit. While the Ractor is still in vm->ractor.set,
rb_vm_mark and ractor_mark keep the Ractor object itself and its
default port alive as before; everything they mark is foreign to or
pinned by this collection.
Two of the swept wrappers are the dying thread's own: its Thread (the
rest of the collection resolves GET_EC through ec->thread_ptr) and its
root Fiber (whose struct embeds that ec). thread_free and fiber_free
recognize "the struct I am standing on" (ec == the current ec), keep
the struct, and leave an in-band mark (th->self = 0; cont.self = 0,
which already means "no wrapper" on the never-wrapped path). Found as
a 1/30 heap corruption, pinned by ASAN. The collecting thread
captures those two pointers right after the collection, while its GVL
still excludes any global GC, and frees exactly the captured structs
at its very last step -- after vm_remove_ractor unlinks the Ractor,
anything it did not sweep may already be freed by a concurrent global
GC, so the epilogue reads no shared memory at all. Those frees would
resolve their objspace through the TLS ec -- which sits inside the
fiber struct being freed and leads to the retired Ractor -- so
rb_ractor_postmortem_free severs the TLS ec first (the MN epilogue
already had) and they fall back to the main objspace.
While a thread-event hook is registered the postmortem flag stays off
and the collection keeps the ordinary retire roots: callbacks may
retain any Ractor's Thread object (rb_internal_thread_event_hook_t),
and such references are invisible to the reduced roots. The check
takes the hook rwlock, so it happens-after any completed registration;
a hook registered after it gets no event from this thread --
rb_thread_execute_hooks skips a thread whose wrapper is gone
(th->self == 0) -- so it never receives what the collection swept.
On platforms without hook support (win32, none) the check reports
false.
The collection runs after thread_cleanup_func, so the dying thread's
machine stack is already torn down: the root scan skips its machine
context (rb_gc_impl_during_postmortem_p), which holds nothing the
reduced roots do not already cover and faults when scanned on some
platforms (windows-arm).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ractor#value used to be repeatable: the successor could call it again
and get the same object back, because the value stayed reachable from
the dead Ractor's C struct (sync.legacy). Keeping that slot valid
after the Ractor is gone is what forced the successor to root and pin
every Ractor it absorbed, through a VM-wide list guarded by its own
lock, with add, unlink and scan coming from three different threads.
Take the value out of the struct on the first call and raise
Ractor::Error afterwards. The value is then rooted by the caller like
any other return value, so the list, its lock and the pinning go away.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Make the VM-global GC roots per-Ractor so the main Ractor's ordinary
local GC runs lock-free. Add the shareability and objspace-locality
checks to the write barrier, and split the VM-wide tables behind
finer-grained locks.
Co-authored-by: Claude <noreply@anthropic.com>
Since `prc.refined(*ms).refined(*ns)` is behaviorally equivalent to
`prc.refined(*ms, *ns)`, the copy of the block is deferred until
the first call and the memo is shared by these behaviorally
equivalent Procs. A Proc that is never called is no longer copied
at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Method coverage rebuilt its result by walking the heap for method
entries, so a shadowed or removed method that was never called could be
reclaimed by GC and vanish from the result depending on GC timing.
Record each defined method entry in me_set as it is defined, and compute
the result from me_set and cme2counter instead of walking the heap.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A tag established inside a live ZJIT frame can catch a longjmp without unwinding that native frame. Record that fact on rb_vm_tag and materialize only frames above the target, preserving its JITFrame link and keeping JITFrame saves at a single store.
Add Proc#refined(mod, ...), which returns a new Proc that behaves like
the receiver but with the refinements activated by the given modules in
effect inside its body, without affecting the original Proc. This is
an alternative to the previously proposed Proc#using
(https://bugs.ruby-lang.org/issues/16461) that does not modify the
existing block and needs no `using Proc::Refinements` declaration.
Refinements are resolved at run time through the cref, so a refined
proc gets two pieces of its own:
* A cref carrying the additional refinements, built by activating the
modules (rb_using_module_recursive, under the VM lock) on a duplicate
of the block's captured cref. It is stored in a hidden ivar on the
Proc and injected into the block frame on every proc-invocation path
(Proc#call, yield from C methods, invokeblock/opt_call, and
instance_eval/instance_exec/module_eval/class_eval via yield_under).
The captured environment stays shared, so closure variables are still
shared with the original Proc. `using` inside the body is rejected
(CREF_REFINED_PROC), so the cref's refinements never change after
publication.
* A recursive deep copy of the block's instruction sequences
(rb_iseq_dup_with_independent_caches) so the new Proc has its own
inline method caches. This is required for correctness: the VM
caches refinement method resolution per call site assuming a single
lexical cref per iseq, so sharing the iseq would leak the refined
methods into the original Proc. Coverage keeps measuring the copied
iseq.
The copy and the cref are memoized in a hidden identity Hash keyed by
the source iseq (one slot per iseq, matched by the captured cref and
the module arguments, accessed under the VM lock), so repeated calls at
the same site reuse one copy and procs refined with the same modules
share the iseq and the cref. A differing module set or a ruby2_keywords
flag mismatch rebuilds the entry (with a performance warning). The
refinements table of the memoized cref is frozen and marked shareable so
the memo also serves other Ractors.
Procs without an iseq block (Symbol#to_proc, C-function procs), procs
created from methods, and procs that already have refinements are
rejected with ArgumentError.
The YJIT/ZJIT Rust bindings are regenerated for the new rb_proc_t field.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
