A thread that is waiting for the fork lock when another thread forks
exists only in the parent, but on macOS the child's copy of the lock
still counts it. After the child unlocks its copy,
pthread_rwlock_destroy() and pthread_rwlock_init() fail with EBUSY, so
the child aborts with
"[BUG] pthread_rwlock_destroy: Resource busy (EBUSY)". Lookup threads
wait for the lock whenever a fork holds it, so a process that resolves
names in background threads while it forks loses a child now and then.
Reset the child's copy by assigning PTHREAD_RWLOCK_INITIALIZER, as
rb_internal_thread_event_hooks_rw_lock_atfork() does for its lock, and
do not unlock it first. With a waiter counted, macOS's unlock enters the
kernel to wake it, and the state that leaves behind for the lock's
address survives the assignment, so a later waiter in the child can
sleep forever although no thread holds the lock. FreeBSD already skipped
the unlock in the child (dc8c382266), and on glibc the child's unlock
does not release the write lock anyway.
[Bug #22445]
A getaddrinfo thread calls getaddrinfo(3) inside rb_thread_prevent_fork,
but on glibc its thread exit takes the resolver configuration lock again
after the thread has released the fork lock. A fork that lands then
leaves the child with that lock held, and every later name lookup in the
child blocks forever. getnameinfo(3) attaches the same resolver state,
even for a name in /etc/hosts, and its thread ran outside
rb_thread_prevent_fork altogether.
Make the lookup threads joinable and have fork join any that may still
be exiting. As a lookup thread finishes, still inside
rb_thread_prevent_fork, it swaps itself into a single slot with an
atomic exchange and joins the thread it displaced, which has usually
exited by then, and rb_thread_acquire_fork_lock joins whatever is left
in the slot. Once lookups settle, at most one finished thread is left
unjoined. The slot holds pthread_t as a pointer, with NULL for empty,
which assumes pthread_t is scalar and never 0.
getnameinfo threads now run within rb_thread_prevent_fork and register
as they exit, as getaddrinfo threads do, and where getnameinfo runs
without a helper thread it is called within rb_thread_prevent_fork too.
So fork also waits for a reverse lookup in progress, as it already did
for a forward one. raddrinfo_pthread_create now always creates joinable
threads, which drops its pthread_detach fallback for platforms without
pthread_attr_setdetachstate.
The slot is kept across fork, so every fork path that keeps running
Ruby code has to take the fork lock. On the BSDs, Process.daemon forks
with daemon(3) and did not take it, so the daemon would inherit the
registered thread, and its next fork would abort in pthread_join with
ESRCH. Take and release the fork lock around daemon(3) as rb_fork_ruby
does. daemon(3) can fail in the child as well as in the parent, so
compare pids to tell which process returned.
[Bug #22406]
These still held String and Array lengths and indices in long, or cast them to long. rb_len_t is still long everywhere, so the generated code does not change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
None of these has a caller, a declaration in any header, or a place in
libruby's dynamic symbol table, so an extension cannot reach them either.
* rb_vm_opt_newarray_pack: left over from cb9510f539, which moved the
instruction to the _buffer form.
* rb_autoload: a wrapper around rb_autoload_str, deprecated and dropped
from intern.h in Ruby 2.3 [Feature #11664].
* rb_big_fdiv: a wrapper around rb_big_fdiv_double, unused since Bignum
stopped defining #fdiv.
* rb_io_unbuffered: a wrapper around rb_io_synchronized, which is what it
was renamed to before 1.8.0.
* rb_ractor_hooks: an accessor for cr->pub.hooks; only its sibling
rb_ractor_targeted_hooks is used.
* rb_assert_sig: a leftover debugging aid.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`RUBY_MN_THREADS=1` only affects threads created after it is read: the
main thread keeps a native thread of its own and is woken through its
condvar, so handing control to the main thread costs an order of
magnitude more than handing it between any two other threads (6.4us
against 0.56us for a Queue round trip here).
With `RUBY_MN_THREADS=2` the running main thread is turned into an M:N
thread in place (thread_sched_main_to_shared), rather than spawning a
thread for the program body and leaving the main native thread idle:
* the process stack becomes the main thread's coroutine context
(coroutine_initialize_main), the way nt_start's own stack is a shared
thread's context;
* the main native thread joins the shared pool with a stack of its own
for its scheduling loop (nt_loop_co), which starts on the first
transfer into it -- the main thread's first park. From then on any
shared native thread may resume the main thread, and this one serves
any Ractor.
nt_start's shared loop is split out as nt_shared_loop so that both
entries share it. The main native thread never retires, its loop being
on a stack only it could free: ractor_sched_deq takes a can_retire flag
and native_thread_dedicated_dec always lets it rejoin.
The wait in rb_ractor_terminate_all had to give its native thread back
first, or exit hung with the Ractors it waits for having nothing to run
on; that is the commit before this one. The VM barrier waits natively
in the same way and has the same exposure, and is not touched here,
being no more reachable under `=2` than before.
Several places assumed the main thread owns the process's initial native
thread:
* native_thread_init_stack set the process stack's range for any thread
starting on that native thread, over the pool stack an M:N thread
already owns; the case is now keyed on th->sched.context.
* thread_sched_switch passed to_dead for a thread whose status is
THREAD_KILLED, which the main thread already is while
rb_ractor_terminate_all parks it. A thread ends only through
co_start's epilogue transfer.
* thread_sched_atfork left the forker's nt->running_th and nt->retiring
as the parent had them; retire eligibility is decided against the
process's main native thread, refreshed there.
* rb_thread_free_native_thread (RUBY_FREE_AT_EXIT) destroyed the native
thread hosting the main thread even when shared, along with the
altstack registered on the thread running it.
RUBY_FREE_AT_EXIT now leaves more behind: the main native thread and
its context and stack, since a thread is parked in its loop until the
process ends, and the main thread's own coroutine context, since the
free-at-exit path does not reach rb_threadptr_sched_free. Leak-checker
baselines move by that much.
The OS thread name is no longer set from the Ruby thread for M:N
threads. One shared native thread runs many Ruby threads over its life,
so the name described whichever one happened to start on it; under `=2`
that renamed the process itself, the thread group leader's comm being
what ps and pkill show. Thread#name= was already skipped for M:N
threads for the same reason.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A blocking read used to cost four epoll_ctl per wait cycle: ADD before
the wait and DEL after the wake, on both ends of an exchange. The DEL
ran on the timer thread, the one serial resource every io readiness
event already passes through.
With EPOLLONESHOT the delivery itself disarms the fd inside the kernel:
the post-wake DEL becomes no syscall at all, and the next wait re-arms
with one MOD on a registration that persists for the fd's lifetime.
A disarm with no delivery (timeout, interrupt) MODs the event mask to
zero instead of deleting. Registration stays level-triggered, so a
woken reader is under no obligation to drain the fd; the generation tag
already guards against events queued under an old arming, and a stale
registration left by close is repaired by the existing ENOENT fallback
on the next arm.
Two round trips of a 64-byte socketpair exchange drop from 12 to 10
syscalls (epoll_ctl 4.00 -> 2.00) and the timer thread falls from 97%
to 82% of a core. On master 22e4a75d67, sp4 (16 cores), medians:
io_intensive (socketpair round trips, 2 Ractors/pair)
8 pairs 218,839 -> 432,810 rt/s (+98%)
64 pairs 276,556 -> 591,171 rt/s (+114%)
punicorn (one Ractor per connection, wrk from a second box)
keepalive c=64 280,678 -> 309,530 rps (+10%)
4x memcached/req 59,683 -> 67,082 rps (+12%)
SSE probe beside 512 held streams 193,769 -> 205,602 rps (+6%)
Workloads that wait on the timer wheel (sleep), the GVL (cpu), or
connection churn are unchanged; so are Ractor creation, GC and plain
threads (~50 configurations within +-2%).
make btest and full test-all pass with the feature on, as does btest on
a RUBY_DEBUG=1 build, and an adversarial fd close/reuse + cross-thread
close + fork-with-live-io stress. Platforms: epoll only; kqueue keeps
its current path (EV_DISPATCH would be the analogue). EPOLLONESHOT
predates every kernel that can build ruby (Linux 2.6.2), and an epoll
without it now falls back to USE_MN_THREADS=0 at compile time.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Windows carried a second, much weaker scheduler: a single Win32 mutex as
the GVL, vm->clock for preemption, its own VM barrier over
vm->ractor.sync.barrier_*, its own Ractor wait/wakeup on a per-Ractor
condvar, no ready queue and no time slice. Keeping two made the rest of
the interpreter branch on which one was compiled in, and left Ractors on
Windows without real scheduling.
Move the scheduler out of thread_pthread.c into thread_sched.c and let
both platforms use it. thread.c now includes thread_sched.c, which
declares the scheduler <-> platform contract, includes THREAD_IMPL_SRC,
and then builds the scheduler on the primitives that file provides.
thread_pthread.c keeps only those primitives: synchronization, thread
local storage, native thread creation and stacks, the communication pipe,
the timer thread backend, the fork lock and the event hooks.
The data structures the scheduler works on move to a new thread_sched.h,
included from both platform headers, so struct rb_thread_sched,
struct rb_native_thread and struct rb_thread_sched_item are now shared.
thread_pthread_mn.c becomes thread_sched_mn.c: it is the M:N scheduler,
not a pthread detail. Most of it -- the coroutine threads, the native
thread stack pool, the timer wheel and the fd -> waiters map -- is
platform independent; only the readiness backend is not, and that section
is now marked as the seam where an epoll/kqueue split (or an IOCP backend,
which is what Windows would need for M:N) would go.
What Windows had to add is small: native_thread_interrupt(),
native_thread_default_max_cpu(), timer_thread_polling(),
timer_thread_wakeup_force(), RB_NATIVE_COND_MONOTONIC_P() and
RB_NATIVE_MUTEX_TRYLOCK_DETECTS_SELF() -- the last because a
CRITICAL_SECTION is recursive, so trylock cannot back a "somebody holds
this" assertion. It sets USE_MN_THREADS to 0 and supplies the stubs
thread_sched_mn.c provides elsewhere, so every thread there is dedicated,
as it already was.
With one scheduler the platform branches go away: all 16 uses of
RUBY_THREAD_PTHREAD_H / RUBY_THREAD_WIN32_H are removed from vm_core.h,
ractor.c, ractor_core.h, ractor_sync.c, vm_sync.c, vm.c and thread.c.
ractor_check_blocking() went with them -- it only ever fed the win32
scheduler's vm->ractor.blocking_cnt -- and so did USE_VM_CLOCK, now that
the common time slice drives preemption on Windows too.
Verified on both platforms. Linux: 11 rounds of make test-all (36236
tests) and btest (2066 tests) with no failures, plus 474 runs of a
scheduler and Ractor stress soak. Windows: no test fails that does not
also fail on the same tree without this change, btest passes, and two
40-minute soaks of 438 and 460 runs are clean. Ractors now run in
parallel there -- 3.6x on four Ractors, where before they could not.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
coroutine_thread_terminated() leaves the Ractor's living set before handing
the scheduler slot over, then keeps using th for the rest of the function:
thread_sched_to_dead_common() reads th->nt and deregisters th from the
barrier, and the epilogue then stashes th->nt and clears th->sched.context.
Off the set th is unreachable, and once deregistered no barrier waits for
it, so a sweep in another Ractor can free it mid-epilogue. The epilogue
reads freed memory, and co_start() dereferences the NULL it finds in
tctx->nt -- SEGV at 0xc8, the offset of dead_co in struct rb_native_thread.
The removal cannot simply move after the handoff: the GC's root scan walks
r->threads.set without the Ractor lock (relying on the barrier), so the
unlink must stay where no walker can run -- while th is still registered
and before a successor is designated. Nor can the th accesses move before
the removal: deregistration itself writes th's running-list node.
Instead, keep the removal where it is and give the dying thread an explicit
root: r->threads.dying_th, marked by the root scan exactly like a set
member, set just before the removal and cleared inside the scheduler-lock
section after the epilogue's last use of th. Clearing under the lock
serialises the epilogue against its successor -- even a dedicated one woken
by to_dead_common() first runs thread_sched_to_running(), which takes the
same lock -- so successive epilogues cannot overwrite each other's slot,
and the successor cannot publish threads.running_ec before the dying
thread has cleared it. Nothing reads th after the clear: the final unlock
avoids the debug-logging wrapper (which reads th->serial), and whether the
designated successor needs the Ractor enqueued is decided while still
holding the lock, since a dedicated successor may run -- and die, freeing
itself -- the moment the lock is released. The atfork paths reset the slot
so a fork taken mid-epilogue cannot leak a stale root into the child. The
slot is accessed with atomic ops: the root scan may read it while another
Ractor's dying thread writes it.
Also initialise tctx->nt, which native_thread_create_shared() left holding
whatever ruby_xmalloc() returned.
Not addressed, pre-existing on master: rb_postponed_job_trigger_for_ractor()
can copy threads.running_ec and dereference it arbitrarily later (the new
root narrows this but cannot protect an already-copied pointer), and a
freshly created dedicated native thread publishes its EC via
ruby_thread_set_native() before taking the scheduler lock.
Two Ractors looping over Thread.new and GC.start crash master in about
three seconds (19 of 20 runs); an ASAN build reports the heap-use-after-free
directly, at the read of th->nt in coroutine_thread_terminated(), freed by
gc_sweep via rb_thread_free_body. With this change ASAN is clean (0/10 vs
7/10 interleaved), as are the reproducer, s7_nested, both branches of the
epilogue under stress, a Ractor/GC soak, btest and test-all on release and
RUBY_DEBUG=1 builds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every io wait took timer_th.waiting_lock to register and again to wake:
one global lock serialized the fd map, the epoll_ctl calls made under
it, the timer wheel and the wake-pending flags, across all fds.
Split the ownership:
- an fd's waiter list, arming state and the flags of its io waits move
under one of 16 fd shard locks; unrelated fds register and wake in
parallel, and the epoll_ctl calls leave the global lock entirely
- waiting_lock keeps the wheel and fd-less timed waits
- wake_pending becomes a count under its own small lock, so an expiry
hold and an fd event's wake can pin one thread at once
- the untimed waiter list is dropped (its only reader was a VM_ASSERT
pass), so untimed io waits touch no global lock at all
- the fd map's chunk table becomes a fixed array with CAS-installed
chunks, since chunks span shards
An io+timeout wait is registered in both structures; whoever clears its
flags under the fd shard owns the wakeup. The expiry pass pops wheel
nodes under waiting_lock, pins the thread with the pending count, and
claims under the shard by (serial, fd) -- lock order is always
shard -> waiting_lock -> wake_pending_lock.
R ractor pairs (1 thread each) ping-ponging over pipes with blocking
reads, total round-trips/sec, 16-HT machine (Ryzen 9 5900HX),
RUBY_MN_THREADS=1, mean of 2 alternating same-tree runs:
before after
8 pairs 63k 134k
16 pairs 64k 143k
Messaging without fds (port ping-pong pairs) is unchanged.
A thread-heavy shape (256 threads in few ractors doing memcached round
trips) is unchanged: its wakes stay inside each ractor's readyq and the
old lock was not saturated there.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
thread_sched_setup_running_threads runs on every context switch (park,
resume, and both ends of every blocking region) and took the global
ractor.sched.lock to maintain the running_threads and timeslice lists
and running_cnt. With many ractors switching concurrently this lock is
the scaling ceiling.
Replace all three with ntlist, a per-location registration of who runs
where, and no shared counter:
- a thread on a shared nt registers in that nt's running_th, under the
nt's own lock -- uncontended, no cross-core traffic
- a dedicated nt goes on ntlist.running_dnts while its thread runs (the
switch rate of dedicated threads is what it always was; only the lock
shrank), so the fast path writes no shared cache line at all
The consumers walk the registrations: the barrier's interrupt walk also
counts them into a snapshot (barrier_running_cnt) and stamps each nt
with the barrier serial; a stamped nt deregistering during that barrier
decrements the snapshot under sched.lock, so the completion check stays
O(1) however many runners there are.
The timeslice list becomes timeslice.scheds, the scheds whose readyq
holds waiters. The enq that fills an empty readyq (and the add path
taking a turn with waiters already queued) links the sched; the timer
thread interrupts each listed sched's running thread and prunes entries
whose readyq drained. A steadily contended sched stays linked, so the
switch path never touches the list, and idle scheds are not walked at
all. The timer takes each sched's lock by trylock, because the
switchers nest sched.lock -> timeslice.lock; pruning is lazy, so
ractor_free delists the dying ractor's sched.
Barrier pairing needs no fences: the barrier sets barrier_is_waiting
and then walks the registrations under their locks; a switch moves its
registration under the same lock and then reads the flag. A walk that
missed a registration ran before that registration's critical section,
so its flag store is visible to that switcher. Nothing may start
running while the flag stands -- an interrupt flag alone would let a
resuming thread run, and allocate, inside the stop-the-world section --
so the add path takes its registration back and waits for
rb_ractor_sched_barrier_end's broadcast. While a scan holds a
running_th lock the registered thread cannot finish parking, so it
cannot die under the scan. An snt turned dedicated under
rb_thread_lock_native_thread has no creation-time running_thread, so
the running_dnts registration writes nt->running_thread itself.
R ractor pairs ping-ponging via ports (msgs/sec) and R ractors doing
pipe write/read round-trips (rt/sec), 16-HT machine (Ryzen 9 5900HX),
mean of 2 alternating same-tree runs:
before after
port pairs 8 503k 815k +62%
port pairs 32 453k 799k +76%
pipe self 4R 837k 940k +12%
pipe self 16R 820k 898k +10%
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
native_thread_dedicated_inc/dec run on every blocking region boundary and
took ractor.sched.lock each way just to move two counters. Make snt_cnt a
seq-cst atomic: leaving the pool is one FETCH_SUB, and rejoining is a CAS
loop that enforces the max_cpu cap. The lock is taken only when the last
snt goes dedicated, to check grq_cnt and wake the timer thread.
The lost-wakeup gate stays sound as a store-buffer pair: ractor_sched_enq
raises grq_cnt and then reads snt_cnt; dedicated_inc lowers snt_cnt and
then reads grq_cnt under the lock, so at least one side sees the other.
Both increment paths (rejoin and pool widening) check the cap and then
CAS, so max_cpu still holds without the lock.
dnt_cnt only ever fed one RUBY_DEBUG_LOG line, so define and update it
only under USE_RUBY_DEBUG_LOG, without atomicity; nothing may read it
for decisions.
R ractors x 1 thread each doing pipe write/read round-trips on a 16-HT
machine (Ryzen 9 5900HX), round-trips/sec, median of repeated runs:
R master patched
1 422k 430k +2%
2 718k 752k +5%
4 933k 1,036k +11%
8 905k 1,111k +23%
16 735k 1,016k +38%
32 671k 955k +42%
64 690k 952k +38%
128 685k 946k +38%
master peaks at 4 ractors and degrades under lock contention; patched
peaks at 8 and holds up to 128. Single-ractor shapes are unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A dedicated native thread used to sleep in two stages: native_cond_sleep()
parked it on nt->cond.intr inside a blocking region, and on wakeup it went
back to the scheduler to wait for its running turn on nt->cond.readyq. The
thread scheduler's turn wait takes an absolute deadline since d32793fe8e, so
the sleep can happen right there: native_sleep() now parks every dedicated
thread in thread_sched_to_waiting_until_wakeup(), with a deadline when it
has one, and an M:N thread keeps using the timer wheel. The second condvar,
its ubf and native_cond_sleep() go away, and with them the union/struct
dance in struct rb_native_thread for platforms whose condvars remember
their mutex: the one condvar left always pairs with sched->lock_.
Two things keep the handoff as fast as the old path:
The turn wait trusts ETIMEDOUT to say the deadline passed, instead of
reading the clock on every wakeup; on clocksources where clock_gettime is a
real syscall that read was the single biggest cost of a timed wakeup.
ubf_waiting() wakes the target even when the running turn is taken, knowing
it re-parks at once. The old two-stage sleep did the same thing by its
shape, and it is worth doing on purpose: the woken thread's futex wakeup
runs on another core in parallel with the running thread, so by the time
the turn is handed over it is off the handoff path. Measured on a
queue-with-timeout ping-pong, the handoff costs what the two-stage sleep
cost; without the early wake it was three times slower.
The ubf must also leave alone a thread whose deadline already put it back
in the ready queue: waking it a second time would double-enqueue it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The timer thread delivers expiry and fd wakeups in batches: it collects
{thread, serial} pairs under timer_th.waiting_lock, releases the lock (the
scheduler lock a wakeup takes must not nest inside it), and then wakes each
thread. Unlinking an entry is what releases its thread, so from that moment
the thread can be woken by somebody else, exit and be freed while the batch
still holds a bare pointer to it. Waking it then dereferences a freed
thread, and crashes when the Ractor holding it was torn down:
[BUG] Segmentation fault at 0x0000000000000138
timer_thread_check_timeout -> timer_thread_wakeup_thread
-> rb_native_mutex_lock(&TH_SCHED(th)->lock_) # th->ractor is NULL
The serial captured in the batch does not help, and can even match again: a
thread struct reused from the freed one starts counting event serials from
zero, so its first timed wait matches serial 1 held in a stale batch entry,
and the timer wakes a thread whose wheel entry is still armed:
Assertion Failed: thread_sched_wait_running_turn:
th->sched.waiting_reason.flags == thread_sched_waiting_none
Both reproduce on a loop of Ractors that die while one of their threads sits
in a timed receive, in ~15 rounds of 400. They only became reachable when
e1bce29aac cut a dying Ractor's teardown from one second to well under a
millisecond: the batch window used to be dwarfed by the teardown time.
Holding waiting_lock across the wakes would close the window but deadlock:
arming a timer takes the scheduler lock and then waiting_lock, so a wake
taking the scheduler lock under waiting_lock inverts the order. So mark the
threads instead. When the timer thread publishes a batch it sets
in_wake_batch on each thread, under waiting_lock; when it has woken them all
it clears the marks and broadcasts. A dying thread checks its own mark and
waits on the cond until it clears, fencing twice:
Once when it leaves the scheduler for good (thread_sched_to_dead, and the
coroutine epilogue in thread_start_func_2). This is the fence that matters
for a dying Ractor: a stale wakeup reaches the Ractor through TH_SCHED(),
and the rb_ractor_t can be collected as soon as the Ractor is unlinked, so
the wait must happen while it is still alive. Nothing re-arms the thread
after this point, so no later batch can name it.
Once more when the rb_thread_t itself is freed (rb_threadptr_sched_free), as
a backstop for frees that do not come through a thread's own exit.
The wait is bounded by one batch of at most 16 wakeups, and the fence takes
only waiting_lock, which the timer thread never holds while it wakes, so the
two cannot deadlock.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ractor::Port#receive, Ractor.receive and Ractor.select gain a `timeout:`
keyword and return nil when it passes.
The wait itself stays where it is: rb_ractor_sched_wait() keeps parking the
thread in the thread scheduler, so an M:N thread still hands its native
thread back instead of becoming dedicated. How the deadline is taken
depends on which kind of thread waits:
A dedicated native thread parks on its own condvar, so it takes the deadline
there, the way native_cond_sleep() does. Nothing else is involved: the
condvar it waits on is the one a send already signals.
An M:N thread has no condvar of its own, so its deadline is armed on the
timer thread as a timeout-only wheel entry, and the timer thread wakes it
through thread_sched_to_ready_common(), which is exactly how
rb_ractor_sched_wakeup() wakes it for a send. That gives such a waiter two
wakers, so rb_ractor_sched_wakeup() now takes an armed timeout back before
waking, skips a thread that a fired timeout already made runnable, and bumps
the scheduler event serial so a timeout that has not fired cannot wake it a
second time.
`timeout: 0` is a poll: it never blocks, converts nothing and reads no clock,
so it costs what a receive of a waiting message costs. It still delivers the
incoming queue first, so a message another thread just sent is not missed.
A timeout that expires leaves the waiter on the Ractor's waiter list, as a
spurious wakeup does, and ractor_wait_receive() still delivers pending
messages before it reports the timeout, so a message that arrived just before
the deadline is not lost.
Both kinds of wait exist on every pthread platform, including builds without
the timer wheel (USE_MN_THREADS == 0), where every thread is dedicated. On
win32 the wait is a condvar wait, which takes the timeout directly.
coroutine_transfer() restores target->from after the switch, that is,
after this coroutine is resumed -- by then it may have been resumed by
someone else and target may be freed.
With M:N threads it usually is: a Ruby thread parks by transferring to
its shared native thread's context (thread_sched_wait_running_turn), and
that thread, finding the ready queue empty, retires and frees nt and
nt->nt_context. Over 40 retires in the reproducer below, the number of
threads still parked on the retiring context was a median of 19 (max 61,
zero in four cases); each writes to it when it is next resumed.
sizeof(ucontext_t) is 968, so the write lands in a freed 1008-byte
region, and once that is reused it corrupts the new owner:
free(): invalid next size (fast)
ruby_xfree_sized ... st_free_entries -> rb_st_free_table
ractor_free_all_ports -> ractor_notify_exit -> ractor_atexit
thread_start_func_2 -> co_start -> coroutine_trampoline
AddressSanitizer: heap-use-after-free
WRITE of size 8 in coroutine_transfer (Context.h:97), thread T3
968 bytes inside a 1008-byte region freed in nt_start ->
native_thread_destroy_self, allocated in native_thread_alloc
from is read only by the trampoline, when target starts, so the restore
is dead by the time the switch returns. Skip it under
COROUTINE_TARGET_MAY_BE_FREED, which thread_pthread.c defines; cont.c
and other embedders keep the original behaviour. Only ucontext runs
M:N threads today, but the lifetime rule is not specific to them, so
guard every backend that has the restore.
Fibers never hit this: a suspended fiber still references its target, so
it cannot be collected. A native thread's context is not a Ruby object
and is freed by hand.
Reproducer: clang-20 -O1 --enable-shared --with-coroutine=ucontext,
bootstraptest/test_ractor.rb (the case at line 2603 spawns 2000 Ractors)
four times in parallel on four CPUs: 21 of 48 runs died before, 0 of 48
after. Under ASAN, the isolated case eight at a time: 120 of 120
reported before, 0 of 120 after.
CI: https://github.com/ruby/ruby/actions/runs/31942932752/job/95154613098
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
timer_thread_register_waiting kept the timed waiters on a single list
sorted by deadline, inserting with a linear scan. A server whose
connections share one timeout value -- every keep-alive loop -- makes
each insert walk the whole list: with 8000 waiting connections the
scan burns half the CPU (43.71% + 8.30% in thread_sched_wait_events)
and M:N falls behind 1:1.
Bucket timed waiters into a hierarchical timing wheel instead
(Varghese & Lauck, "Hashed and Hierarchical Timing Wheels: Data
Structures for the Efficient Implementation of a Timer Facility",
SOSP '87): 4 levels of 64 slots, 1ms slots at the finest level and
64x coarser per level, ~4.6h of direct range; farther deadlines park
on the coarsest level's farthest slot and re-sort on arrival. A
not-yet-due waiter found while draining a slot is re-bucketed onto a
finer level, so each waiter moves at most once per level. Deadlines
are floored to a slot, so the wheel never fires late, and the
re-bucketing keeps sleep precision.
Insertion becomes O(1) arithmetic. Removal -- the most frequent
operation, since most timed fd waits are cancelled by their fd
becoming ready -- stays O(1): the cached next-expiry is a lower
bound that only insertion has to maintain (waking early is harmless,
so removal does no bookkeeping at all). The timer thread is now
woken only when an insertion shortens that bound, instead of on
every insertion.
Untimed (fd-only) waiters move to their own plain list. The wheel
and its operations are compiled only with USE_MN_THREADS; the slot
width can be widened with -DTIMER_WHEEL_TICK_MS=n for experiments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Since "Ractor: the dying thread collects its own objspace post-mortem"
(e768a87a17), a dying Ractor thread runs rb_ractor_postmortem_free()
after thread_sched_to_dead() (its GVL is gone) and after
rb_ractor_living_threads_remove() -> vm_remove_ractor(), which unlinks
the Ractor, decrements vm->ractor.cnt and signals the terminate waiter.
From that point nothing holds the main Ractor back, but the epilogue
still frees through VM-global state: rb_fiber_free_body() -> cont_free()
takes jit_cont_lock and returns the stack to the shared fiber pool, and
both frees update the main objspace's malloc accounting.
The coroutine epilogue has the same hazard and is covered:
coroutine_thread_terminated() bumps vm->ractor.sched.winding_cnt and
ruby_vm_destruct() spins in rb_thread_sched_wait_winding() until the
reclaim is done. The non-coroutine path -- the ordinary
Ractor.new-spawned dedicated thread -- was not counted, so a main
Ractor that exits while the dying thread is still in the epilogue tears
that state down underneath it.
Count the epilogue in winding_cnt as well. The increment happens before
the Ractor stops being counted, so there is no window where the VM sees
neither the Ractor nor the epilogue. vm->ractor.sched exists only on
the pthread backend, so the count gets the same shape its waiter has:
real work in thread_pthread.c, a no-op in thread_win32.c and
thread_none.c, where rb_thread_sched_wait_winding never waits either.
Reproduced deterministically by delaying the epilogue (50ms) and the
process exit (400ms) on top of e768a87a17: 6/6 runs of
`Ractor.new{ 1 }.value` died with
[BUG] pthread_mutex_lock: Invalid argument (EINVAL)
rb_native_mutex_lock (thread_pthread.c:128)
jit_cont_free (cont.c:1419)
cont_free (cont.c:1185)
rb_ractor_postmortem_free (ractor.c:683)
thread_start_func_2 (thread.c:851)
i.e. the epilogue locking jit_cont_lock after rb_jit_cont_finish()
destroyed it. With this change the same harness is clean 6/6, and
test/ruby/test_ractor.rb, test/objspace/test_ractor.rb, test_thread.rb
and the strscan/digest/io-wait/io-console Ractor tests pass. Without
the injected delay the window is never entered on x86_64 (0 hits in 15
runs of those test files, instrumented), which fits the failures being
confined to the slow emulated runners.
This is the most likely cause of the heap corruption CI has been
reporting since e768a87a17 landed (2026-08-10 18:33 UTC): every failing
case is a child process that runs one Ractor and exits.
Error:
TestIOWaitInRactor#test_ractor [test/io/wait/test_ractor.rb:9]:
pid 68552 exit 0
| free(): corrupted unsorted chunks
1. [2/2] Assertion for "stderr"
| <[]> expected but was
| <["free(): corrupted unsorted chunks"]>.
(also TestStringScannerRactor#test_ractor, TestDigestRactor::*,
TestIOConsoleInRactor#test_ractor -- 16 of the 63 master runs since
2026-08-10, none in the 87 runs before it)
TestObjSpaceRactor#test_copy_finalizer [test/objspace/test_ractor.rb:57]:
==96541==ERROR: AddressSanitizer: heap-use-after-free on address 0x520000000088
WRITE of size 8 at 0x520000000088 thread T5
CI: https://github.com/ruby/ruby/actions/runs/31886577248/job/95016642538
CI: https://github.com/ruby/ruby/actions/runs/31427322788 (ASAN, x86_64)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
thread_io_wait() sent every fd wait with a timeout to the blocking
region (ppoll on a dedicated native thread), even under M:N. A
pattern like IO#wait_readable(0.5) -- WEBrick's keepalive loop, or
any flag-polling wait -- therefore cost a native thread handoff per
call: ~0.7ms to reacquire the running slot after each ppoll under
load, serializing servers to a fraction of the 1:1 throughput.
Register such waits with the timer thread instead, as fd+timeout
combined waiting. That registration path had no caller until now,
and its expiry path timer_thread_deq_wakeup() left the waiter on its
fd's waiter list with the fd still armed; unregister it there the
same way the cancel path does.
A zero timeout keeps using ppoll: it is a plain probe and needs no
parking.
WEBrick c=256: MN 2.3k -> 6.0k rps (hello, now above 1:1's 5.7k),
0.37k -> 1.5k rps (50 memcached accesses per request).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- verify_waiting_list compared w->data.timeout with itself, so the
sorted-order assert was vacuously true and the waiting list's O(n)
insert has been unverified even on debug builds.
- timer_thread_register_waiting repeated the rel && *rel > 0 check that
the branch right above it had just done.
- Remove THREAD_BLOCKING_YIELD: no user, and it calls
thread_sched_to_waiting with a signature that predates the
yield_immediately parameter, so any new user fails to compile.
- native_thread_check_and_create_shared took ractor.sched.lock through
the raw mutex API, bypassing the lock-owner bookkeeping every other
taker goes through (ractor_sched_lock).
- The stack chunk layout comment claimed "0th page is Redzone"; the
header has occupied the head pages since 407dd02c1b.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
None of these is reachable with default settings; each is a mine for a
configuration or platform that steps on it.
- timer_thread_set_timeout: a sleeper's deadline further away than
INT_MAX ms truncated to a negative timeout, i.e. an untimed
epoll_wait, and the sleeper was never woken. Clamp to INT_MAX.
- nt_thread_stack_size: the size sum went through int, so stack sizes
totalling over 2GB went negative (stack_count 0, and nt_alloc_stack
retries mapping 512MB chunks forever). Compute in size_t. Also
page-align the VM stack and machine stack areas: the configured
sizes align only to 4KB, and on 16KB/64KB-page systems the guard
page and the MAP_FIXED machine stack must land on page boundaries.
- nt_alloc_stack: an mprotect/mmap failure had already consumed the
slot (uninitialized_stack_count--); restore it.
- nt_free_stack: prev_free_chunk == NULL cannot double as the "not on
the free list" test -- the list's tail also has it NULL, and
re-pushing the tail self-cycled the list. Track membership in its
own flag.
- ruby_stack_overflowed_p: the __APPLE__ branch runs when th == NULL
yet read th->nt->thread_id; ask pthread_self() instead.
- rb_thread_create_timer_thread: rb_bug on pthread_create failure
instead of silently running without a timer thread.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A retiring shared thread -- ractor_sched_deq giving itself back, or a
thread rejoining a full pool (nt->retiring) -- just returned from
nt_start, leaking its rb_native_thread: the struct, the coroutine
context, the condvars and the sigaltstack. This was the old "how to
free nt and nt->altstack?" TODO, and retiring made it recurring: every
pool shrink leaked one.
Both exits drop the nt from the counts before leaving the loop, so
nothing can reference it; the exiting thread frees it itself, disarming
its sigaltstack registration first so a late signal cannot run on the
freed block.
Verified with SNT_IDLE_RETIRE=0 (retire on the first idle wakeup): 168
retire/free pairs in a bursty ractor workload and a clean btest run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
In the forked child:
- Do not pthread_mutex_destroy timer_th.waiting_lock. glibc returns
EBUSY (which rb_native_mutex_destroy turns into rb_bug_errno) for a
mutex that another ractor's M:N thread held at the fork moment.
Reinitializing it, as the code already does, is the whole story.
- Reinitialize nt_machine_stack_lock, whose holder at the fork moment
does not exist in the child; the next machine-stack alloc or free
would deadlock on it.
- Forget timer_th.event_fd on the kqueue backend instead of leaving the
stale number around. A kqueue is not inherited across fork, so it
must not be closed either: the number may name a reused fd by then.
- Reset dnt_cnt in thread_sched_atfork next to snt_cnt: only the
forking thread's native thread survives. Its consumer is only a
debug log, but the log lied after fork.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ractor_sched_enq only signalled ractor.sched.cond. The signal reaches
a parked shared native thread, and a running one revisits the queue in
ractor_sched_deq (under the same lock) before it can wait, so as long
as any shared native thread exists the entry is served. With none --
each one dedicated to a blocking region or retired -- the signal is
lost, and if the timer thread is also in its untimed sleep
(timeslice_wait_inf), nothing serves the enqueued ractor: its runnable
thread never runs. native_thread_check_and_create_shared, which would
repair the shortage, runs on the timer thread's timeout branch, which
is never reached again.
Wake the timer thread under the same lock its timeout decision is made
under, from the two transitions that can strand an entry: an enqueue
that finds snt_cnt == 0, and the blocking-region entry that dedicates
the last shared native thread away while the queue is non-empty.
Both are off the ordinary path, and timer_thread_wakeup_locked writes
to the communication pipe only when the timer thread sleeps untimed,
so the common case costs nothing. [Bug #21504]
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Holding the pool at RUBY_MAX_CPU is right when that is what the work
needs, but the default is the number of logical CPUs and a server rarely
needs that many. Threads beyond what the work can keep busy do not add
throughput; they drain the global ready queue, so a thread coming back
for more finds it empty and parks, and the next dispatch is a handoff to
a sleeping thread rather than a continuation on the one that just freed
up. A pool sized to the work leaves a backlog instead, and then nothing
has to be woken at all.
Retire on the signal that the queue keeps running dry: a thread woken to
find nothing has had its turn taken by a thread that never waited. The
counter resets on any successful dequeue, so it only fires under
persistent oversupply and stops as soon as the pool is small enough to
keep a backlog. The last shared thread is never given back -- with none
left an enqueue has nobody to signal, and the only code that creates one
runs on the timer thread's timeout branch.
The pool then settles at what the work needs rather than at the cap. At
RUBY_MAX_CPU=16 on 8 cores, one LAN memcached lookup per request:
32 connections 34,593 -> 78,109 req/s (22 -> 8 native threads)
128 connections 70,805 -> 115,424 req/s (32 -> 15)
256 connections 25,764 -> 112,875 req/s (45 -> 16)
A static response gains as much (73,176 -> 174,496 at 32 connections),
and a request that makes 30 such lookups goes from 1,740 to 6,580 req/s
at 256 connections. Requests that do 10ms or more of their own work see
no difference at any connection count.
SNT_IDLE_RETIRE is the number of consecutive fruitless wakeups to
tolerate. 1 through 3 measure the same; 0 retires so eagerly that the
pool cannot grow when it should, costing up to 4x at low connection
counts; a negative value disables the rule.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An idle shared thread that waits SNT_KEEP_SECONDS without being given a
ractor retires itself. It has been defined to 0 since be1bbd5b7d
introduced it in 2023, so it has not been compiled since, and reviving it
as written would not do much: the deadline is recomputed on every turn of
the wait loop, so a thread that keeps being woken to find the queue empty
resets it every time and never times out. It also reads the clock inside
the scheduler lock on every wait.
Its bookkeeping had drifted as well: it decrements running_cnt, which
counts the threads on vm->ractor.sched.running_threads, while a thread
waiting here holds none.
The branch after the wait loop that existed only to assert this case goes
with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
native_thread_check_and_create_shared honours max_cpu, but that is only
half of it. snt_cnt also drops whenever a thread enters a blocking
region and rises again when it leaves, and the rejoining side checks
nothing:
native_thread_dedicated_inc: snt_cnt--; dnt_cnt++;
native_thread_dedicated_dec: snt_cnt++; dnt_cnt--; /* no cap */
So while a thread is away in a read(2) the timer thread reads the dip as
a shortage and hires a replacement, and when the original comes back
nobody is let go. On an HTTP/1.1 Ractor server that transition happens
about once per dispatch, and the pool ratchets past the documented cap:
30 to 52 native threads on 8 cores at RUBY_MAX_CPU=16, where only 16 of
them can run Ruby at once. The surplus does not add throughput. It
drains the global ready queue, so a thread coming back for work finds it
empty and parks, and every dispatch after that is a handoff to a
sleeping thread on another core instead of a continuation on the one
that just freed up -- 2.15 CPU migrations per request against 0.01, 2.2x
the cache misses, IPC 0.51 down to 0.45.
Check the cap where a thread rejoins the shared threads. One that finds
no room left is on its way out: it belongs to neither count and ends
when it next asks for work, which is the first point at which it is not
running anything.
At RUBY_MAX_CPU=16 with 128 connections and up, on requests short enough
for scheduling to be visible:
static response 95,358 -> 204,935 req/s (34 -> 11 threads)
memcached lookup 70,805 -> 115,424 req/s (32 -> 15)
30 such lookups 1,740 -> 6,580 req/s (49 -> 9)
Requests that do 10ms or more of their own work are unaffected in either
direction.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two threads waiting on one fd -- a reader and a writer on the same socket
is the ordinary shape -- made the second one see a lie.
timer_thread_register_waiting() returned false both when the fd was
already ready and when it could not be registered at all (epoll_ctl
giving EEXIST because another thread holds the registration, or EBADF /
EPERM). thread_sched_wait_events() turned "could not register" into
timedout = false, thread_io_wait_events() turned that into "the event
happened", and thread_io_wait() then fabricated fds[0].revents = events.
So IO#wait_readable returned readable for an fd nothing had written to,
and a blocking read behind another waiter spun at 100% CPU
(read -> EAGAIN -> poll -> epoll_ctl -> EEXIST -> retry) until the fd
really became ready.
Distinguish the three outcomes by type rather than packing them into a
bool, at each of the three layers, so that "not registered" can no longer
arrive as readiness. A caller that gets it falls back to its ordinary
blocking path.
Then stop producing that case at all: keep the waiters of an fd in a
table indexed by the fd (POSIX hands out the lowest free descriptor, so
fds stay dense) and arm the backend with the union of what they ask for,
so a second waiter extends the registration instead of colliding with it.
Entries live in fixed chunks because growing the table must not move a
live list head.
Events now carry the fd and the generation of its table entry instead of
a thread pointer. A stale event -- one epoll_wait had already queued
when the fd was disarmed, possibly closed and reopened since -- resolves
to a table lookup under the lock and is dropped on the generation check,
where before it dereferenced a possibly freed rb_thread_t. Managing the
union also means kqueue deletes the filters it registered rather than
only the one that fired, so a registration can no longer be left behind.
Waiters are unlinked and their results recorded while waiting_lock is
held, but woken after releasing it: the scheduler lock is always taken
before waiting_lock, and capturing event_serial under the lock is what
keeps a thread that has since moved on from being woken, as the timeout
path already does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The retire GC at notify_exit could not reclaim the terminating
thread's scaffolding -- the Thread and Fiber wrappers, the ThreadGroup,
the Ractor-local stdio, whatever the machine stack still references --
because at that point they were all still rooted. Those ~11 objects
kept ~2 heap pages alive per dead Ractor, merged into the inheritor to
wait for its next major GC, so both a spawn-and-join loop and a
fire-and-forget loop grew the heap by two pages per Ractor with
nothing reclaiming them.
Instead, run the final collection at the tail of thread_start_func_2,
just before the scheduler handoff. The thread still holds the GVL, so
the collection runs lock-free like any local GC (a concurrent global
GC waits for its safepoint), and vm_remove_ractor keeps its light
lock-only section. r->postmortem makes rb_ractor_mark_local_roots
root only what must survive the death -- the join value
(r->sync.legacy) and the rb_gc_register_mark_object pins -- and the
zombie_objspaces entry recorded later measures exactly the pages the
joiner will inherit. While the Ractor is still in vm->ractor.set,
rb_vm_mark and ractor_mark keep the Ractor object itself and its
default port alive as before; everything they mark is foreign to or
pinned by this collection.
Two of the swept wrappers are the dying thread's own: its Thread (the
rest of the collection resolves GET_EC through ec->thread_ptr) and its
root Fiber (whose struct embeds that ec). thread_free and fiber_free
recognize "the struct I am standing on" (ec == the current ec), keep
the struct, and leave an in-band mark (th->self = 0; cont.self = 0,
which already means "no wrapper" on the never-wrapped path). Found as
a 1/30 heap corruption, pinned by ASAN. The collecting thread
captures those two pointers right after the collection, while its GVL
still excludes any global GC, and frees exactly the captured structs
at its very last step -- after vm_remove_ractor unlinks the Ractor,
anything it did not sweep may already be freed by a concurrent global
GC, so the epilogue reads no shared memory at all. Those frees would
resolve their objspace through the TLS ec -- which sits inside the
fiber struct being freed and leads to the retired Ractor -- so
rb_ractor_postmortem_free severs the TLS ec first (the MN epilogue
already had) and they fall back to the main objspace.
While a thread-event hook is registered the postmortem flag stays off
and the collection keeps the ordinary retire roots: callbacks may
retain any Ractor's Thread object (rb_internal_thread_event_hook_t),
and such references are invisible to the reduced roots. The check
takes the hook rwlock, so it happens-after any completed registration;
a hook registered after it gets no event from this thread --
rb_thread_execute_hooks skips a thread whose wrapper is gone
(th->self == 0) -- so it never receives what the collection swept.
On platforms without hook support (win32, none) the check reports
false.
The collection runs after thread_cleanup_func, so the dying thread's
machine stack is already torn down: the root scan skips its machine
context (rb_gc_impl_during_postmortem_p), which holds nothing the
reduced roots do not already cover and faults when scanned on some
platforms (windows-arm).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Encode the safety argument of the epilogue reorder as VM_ASSERTs: a
non-last dying thread still owns its scheduler slot when it leaves the
living set (its VM-lock work stays a counted running thread), and the
last thread's reverse order is only safe because it has no successor
(running == NULL, so the removal cannot join a barrier). Also check
winding_cnt against underflow at the reclaim decrement.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A fork taken while the parent holds the VM barrier (e.g. rb_gc_before_fork)
leaves the child with barrier_waiting set and a barrier owner that no longer
exists, so the next barrier in the child could never complete. Reset the
barrier fields in thread_sched_atfork like the rest of the scheduler state.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The is_running check added by 3fa1c8708 guarded against a terminating
thread joining the barrier after leaving the running set. Since the
previous commit such a thread leaves the living set before handing over
its scheduler slot, so every joiner is a running-set member again.
Restore the unconditional count and assert the invariant instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`rb_ractor_sched_barrier_join` unconditionally did `barrier_waiting_cnt++`.
A thread can reach it (via `vm_lock_enter`) while its Ractor's
`sched.running` is set but the thread is not yet in the VM running set
(`sched.is_running == false`) -- e.g. resuming from a blocking IO wait.
Counting such a thread pushes `barrier_waiting_cnt` past `running_cnt - 1`
and trips `VM_ASSERT(running_cnt - 1 >= barrier_waiting_cnt)` in
`ractor_sched_barrier_completed_p`; in release builds the unsigned
`(running_cnt - barrier_waiting_cnt) == 1` completion test can underflow.
Increment `barrier_waiting_cnt` only when `cr->threads.sched.is_running`
(O(1), toggled together with running_threads add/del). A not-running
thread is parked at the barrier regardless, so it is safe to wait without
counting.
NOTE (open): this guards the count at the join site. A stricter fix would
prevent a not-running thread from reaching barrier_join at all; that needs
care so it still waits for the barrier rather than proceeding during
compaction. Left for review.
Reproduction is currently coupled with a separate pre-existing compaction
use-after-poison on a blocked-IO thread's STR_TMPLOCK buffer (only the
IO-read blocking path triggers this barrier window), so a standalone
passing test awaits that fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The nt scheduling loop reclaimed a terminated coroutine thread's context using
the return value of coroutine_transfer() (through thread_sched_switch0),
assuming it was the resuming context (the dead coroutine). That holds for the
amd64 asm backend, but the ucontext backend returns the transfer *target*
instead, so the loop never recognized the terminal transfer: the reclaim always
saw a live thread, and the loop always unlocked the sched lock -- which the
dying thread had already released in coroutine_thread_terminated -- and
double-unlocked it. Benign on glibc, but FreeBSD's pthread_mutex_unlock returns
EPERM (crash), and it corrupts the scheduler (hang) elsewhere.
Have the terminating coroutine record its own context in nt->dead_co (per
native thread) right before its final transfer; the loop reads it after switch0
returns, independent of coroutine_transfer()'s backend-dependent return value.
It is set and read on the same native thread with nothing running in between,
so it cannot be overwritten before it is consumed.
coroutine_transfer0() and thread_sched_switch0() no longer need to return the
resuming context, so revert them to void (the ASan-only local remains).
Reproduced on Linux with --with-coroutine=ucontext (100% hang before, clean
after); amd64 and ucontext both pass bootstraptest thread/ractor/fiber.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
thread_sched_atfork empties the global ready queue with
ccan_list_head_init but leaves vm->ractor.sched.grq_cnt at whatever the
parent had, so a child forked while any Ractor was enqueued keeps a stale
non-zero count for an empty list. VM_CHECK_MODE builds then fail
"grq_size(vm, cr) == vm->ractor.sched.grq_cnt" in ractor_sched_deq
(reproducible on master with RUBY_MN_THREADS=1: fork in a loop that also
creates Ractors); on regular builds the counter only feeds assertions and
debug logs, so the corruption stays silent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Redesign the teardown of M:N (coroutine) threads around the natural call
structure nt_start -> co_start -> thread_start_func_2: when the thread body
returns, its epilogue (coroutine_thread_terminated) finishes all rb_thread_t /
rb_ractor_t business while th is still valid, and co_start then makes exactly
one final transfer back to the native thread's own context (nt_start's loop),
where the nt reclaims the dead coroutine's context. The resume itself proves
the final transfer's register save completed, so the reclaim needs no list,
no handshake and no extra locking.
This deletes the zombie-threads machinery (VM-global list, sched.finished
publish, the GC mark/reap pass) and closes its UAF window: publishing
`finished` before the terminal transfer let the GC free the Thread wrapper --
whose coroutine context the transfer still saves into -- mid-teardown.
* struct rb_thread_context bundles a coroutine thread's execution-owned
resources: the coroutine_context (kept first, so th->sched.context points
into the block) plus its machine stack, the stashed final-transfer target
(nt), and a dead flag. co_start's epilogue sets the flag and transfers via
the stashed nt, touching tctx alone -- th may already be collected by then.
The nt loop passes every coroutine it resumes from to
thread_sched_reclaim_from(), which frees dead ones (stack back to the
pool). A live coroutine cannot be resumed elsewhere, die and be freed
while the nt loop still inspects it: a live yield arrives with the sched
lock handed off, and no other nt can schedule that thread again until the
nt loop's own unlock.
* coroutine_thread_terminated (run from thread_start_func_2, where the
upstream non-coroutine teardown also runs, while th is still valid and
listed) does: wake the next thread as upstream's thread_sched_to_dead
does; stash th->nt into tctx and detach th->sched.context; leave the
living set; clear the current ec; and, for a shared-nt non-last thread,
enqueue the Ractor's next runnable thread (not earlier: waking it from
to_dead would let another thread of the same Ractor execute while this
teardown still runs). Everything that reads th happens before the
living-set removal, after which the GC may collect th (and, for a
Ractor's main thread, the Ractor).
* interrupt_lock is now destroyed in thread_free instead of during teardown
(thread_cleanup_func): while th is in its Ractor's living set, anyone --
terminate_all on the Ractor's terminating main thread, Thread#kill /
#raise -- may lock it, and destroying it mid-teardown leaves a window
where a concurrent interrupter locks a destroyed mutex (EINVAL; the
upstream non-coroutine path has the same, smaller, window). At thread_free
time the wrapper is unreachable AND off the living set (listed threads
are marked), so no interrupter can exist. The atfork cleanup reinitializes
the copied lock, which may have been held at the instant of fork.
* The Thread wrapper's dfree only reclaims the context of a thread that
never started; a terminated one cleared th->sched.context in its epilogue.
VM shutdown needs no extra synchronization: past the living-set removal the
epilogue touches only its own context block, the TLS and nt->nt_context,
none of which VM destruct frees (SNT native threads and the machine-stack
pool are intentionally not torn down at exit, as before).
Verified: bootstraptest 2050 PASS; test_thread / test_ractor / test_fiber /
test_gc / test_gc_compact / thread_queue 282 tests 0F0E; Ractor x Thread x
IO teardown churn 150/150 + 60/60 (exercises DNT promotion, terminate_all
racing dying threads, and the double-enqueue fixed in the previous commit);
fork with live Ractors + child GC 20/20; ASAN clean including unreferenced
Thread/Ractor collection (the old zombie window) and mixed stress;
RGENGC_CHECK_MODE=2 btest 2050 + churn; TSan warning profile identical to
master (the new functions appear in no race report); YJIT
move-under-GC.stress; kill/join races; Thread object retention across
Ractor death; exit-while-winding.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A runnable thread whose Ractor was enqueued to the global ready queue can
be served without a dequeue: a thread waking up in wait_running_turn hands
its native thread over by a direct transfer, and the hot-thread path
steals `running` back outright. Both leave the Ractor's grq entry behind,
orphaned. The entry itself is harmless (the dequeuing nt sees a served
Ractor and skips), but the accounting breaks: the next legitimate enqueue
double-lists the one grq_node embedded in the sched, corrupting the queue
(observed on master as pthread_mutex_lock EINVAL / crashes; about 1/20
runs of a Ractor x Thread x IO churn under a saturated shared-nt pool).
Restore the invariant "enqueued <=> runnable and unserved" by cancelling
the entry at both direct-service points (ractor_sched_cancel_enq). The
grq_node is kept self-linked whenever it is not enqueued, so membership is
read off the node itself: under the per-Ractor sched lock a self-linked
read is decisive (enqueuers hold that lock too), and the common direct
switch -- whose transition never enqueued -- pays no lock at all; a linked
read can race only with a dequeue, so recheck under the grq lock before
ccan_list_del_init.
ractor_sched_enq now unconditionally rb_bugs on a still-linked node: the
double-list is queue corruption either way, and a VM_CHECK_MODE-only
assert cannot catch this race in practice (CHECK builds perturb the
timing; the previous VM_CHECK-only scan never fired while plain builds
reproduced the corruption).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ThreadSanitizer cannot follow Ruby's userspace coroutine stack switches, so
its per-thread shadow stack leaks across a switch and eventually faults
inside libtsan. Create a TSan fiber per coroutine and switch to it at each
transfer, mirroring the existing AddressSanitizer annotations. The main
context borrows the OS thread's implicit fiber and must not destroy it.
Only amd64 is annotated here.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The hot thread cannot steal back control when sched->running is an MN
thread, and goes to sleep with sched->runnable_hot_th still pointing to
itself, firing VM_ASSERT(sched->runnable_hot_th != th) at the next
wakeup. Drop the hint before sleeping since the thread is no longer
spinning. On a VM_CHECK_MODE build with RUBY_MN_THREADS=1:
Thread.new { loop {} }
100.times { File.read(IO::NULL) }
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
