The ELFv2 ABI makes f14-f31 and v20-v31 callee-saved (OpenPOWER 64-bit
ELF V2 ABI, 2.2.1 Register Roles), but coroutine_transfer saved only
r14-r31, LR and CR. A coroutine resuming from a transfer therefore
sees whatever the other side left in those registers, and GCC does keep
GPR data there: it vectorizes "store the same pointer twice" as
mtvsrdd vs63 / stxvx, and spills GPRs to VSX under register pressure.
This never bit while nothing on either side of a transfer held a live
value there. ce6f200972 reshaped nt_start, and GCC 13 now keeps a
pointer in v31 for the whole shared-nt loop, the transfer target of
every M:N park. From then on rubyci ppc64le was red on every run:
non-main Ractor threads resumed with a stale VALUE where the VM expected
a block handler (rb_block_given_p() true without a block, "the block
passed to ... may be ignored" warnings, SEGV in invoke_block_from_c_bh
at 0x71/0x75 with the same address on every run). x86_64 has no
callee-saved FP registers and the aarch64 Context.S already saves
d8-d15, which is why only ppc64le failed.
https://rubyci.s3.amazonaws.com/ppc64le/ruby-master/recent.htmlhttps://rubyci.s3.amazonaws.com/ppc64le/ruby-master/log/20260911T003005Z.fail.html.gz
Save f14-f31 with stfd/lfd and v20-v31 with stvx/lvx (VMX only, no VSX
requirement); the frame grows from 160 to 496 bytes and
COROUTINE_REGISTERS from 24 to 66 words. The LR slot stays at index
18, so coroutine_initialize is otherwise unchanged. Fibers use the same
transfer and are covered too. glibc's setjmp/longjmp on ppc64 already
save and restore this set, so the EC tags and
RB_VM_SAVE_MACHINE_CONTEXT were not affected.
Verified on POWER9 (Ubuntu 24.04, gcc 13.3.0, 64K pages, the rubyci
configure) at fef20ce27c: without this patch
TestSetTraceFunc#test_tp_ractor_local_untargeted dies at 0x75 and
TestEnv#test_delete_in_ractor at 0x71 as on rubyci, and
TestTmpdir#test_ractor / TestEnv#test_fetch_in_ractor fail on the same
warnings; with it all of them pass and bootstraptest/test_ractor.rb
passes.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
coroutine_transfer() restores target->from after the switch, that is,
after this coroutine is resumed -- by then it may have been resumed by
someone else and target may be freed.
With M:N threads it usually is: a Ruby thread parks by transferring to
its shared native thread's context (thread_sched_wait_running_turn), and
that thread, finding the ready queue empty, retires and frees nt and
nt->nt_context. Over 40 retires in the reproducer below, the number of
threads still parked on the retiring context was a median of 19 (max 61,
zero in four cases); each writes to it when it is next resumed.
sizeof(ucontext_t) is 968, so the write lands in a freed 1008-byte
region, and once that is reused it corrupts the new owner:
free(): invalid next size (fast)
ruby_xfree_sized ... st_free_entries -> rb_st_free_table
ractor_free_all_ports -> ractor_notify_exit -> ractor_atexit
thread_start_func_2 -> co_start -> coroutine_trampoline
AddressSanitizer: heap-use-after-free
WRITE of size 8 in coroutine_transfer (Context.h:97), thread T3
968 bytes inside a 1008-byte region freed in nt_start ->
native_thread_destroy_self, allocated in native_thread_alloc
from is read only by the trampoline, when target starts, so the restore
is dead by the time the switch returns. Skip it under
COROUTINE_TARGET_MAY_BE_FREED, which thread_pthread.c defines; cont.c
and other embedders keep the original behaviour. Only ucontext runs
M:N threads today, but the lifetime rule is not specific to them, so
guard every backend that has the restore.
Fibers never hit this: a suspended fiber still references its target, so
it cannot be collected. A native thread's context is not a Ruby object
and is freed by hand.
Reproducer: clang-20 -O1 --enable-shared --with-coroutine=ucontext,
bootstraptest/test_ractor.rb (the case at line 2603 spawns 2000 Ractors)
four times in parallel on four CPUs: 21 of 48 runs died before, 0 of 48
after. Under ASAN, the isolated case eight at a time: 120 of 120
reported before, 0 of 120 after.
CI: https://github.com/ruby/ruby/actions/runs/31942932752/job/95154613098
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ThreadSanitizer cannot follow Ruby's userspace coroutine stack switches, so
its per-thread shadow stack leaks across a switch and eventually faults
inside libtsan. Create a TSan fiber per coroutine and switch to it at each
transfer, mirroring the existing AddressSanitizer annotations. The main
context borrows the OS thread's implicit fiber and must not destroy it.
Only amd64 is annotated here.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The hand-written `coroutine_transfer` in `coroutine/ppc64le/Context.S`
lacks the ELFv2 ABI global entry prologue. When the function is reached
via the PLT (lazy resolution path), r2 (TOC pointer) is left pointing
at whatever the resolver had loaded — typically the dynamic linker's
own TOC — instead of libruby's TOC.
The wrong r2 propagates through the fiber switch into `fiber_entry`,
which jumps into `fiber_restore_thread` → `rb_current_ec_set`. The
first PLT thunk inside the new fiber (`__plt___tls_get_addr` for the
`ruby_current_ec` thread-local) computes its GOT entry as
`r2 + offset` and segfaults dereferencing the bogus address.
This manifests as a SIGSEGV on any code path that uses fibers —
including `Enumerator#next`, so even a one-liner crashes:
ruby -e 'a=[1,2,3]; e=a.to_enum; loop { p e.next }'
# [BUG] Segmentation fault at 0x000000081009ac38
Reproduced on FreeBSD 16-CURRENT/powerpc64le with Ruby 4.0.4. The same
asm exists unchanged in 3.3 and 3.4, so they are likely affected as
well. Linux ppc64le may not hit this if its rtld preserves the caller's
r2 across PLT resolution, but per the ELFv2 ABI spec §2.3.1.2 a global
function entered via the PLT is responsible for setting up its own r2
from r12 — relying on the resolver is non-conformant.
The fix adds the standard two-instruction global entry prologue plus a
`.localentry` directive of 8, matching what the compiler emits for
ordinary C functions (e.g. `rb_current_ec_set` carries `<other: 0x60>`,
i.e. STO_PPC64_LOCAL=3). The local entry point now lands at the
original first instruction (`addi 1,1,-160`), so `coroutine_initialize`
in `Context.h` continues to work unchanged — `start + 8` (which skips
fiber_entry's global prologue to reach its local entry) is still
correct because the offset between global and local entry is identical
across all functions in this module.
References:
- Power Architecture 64-Bit ELF V2 ABI Specification, §2.3.1.2
"Function Calling Sequence"
- STO_PPC64_LOCAL encoding: §3.3 "Symbol Values"
When setjmp/longjmp/exceptions are used on Windows it's necessary to store+restore additional information from the TEB.
I didn't find any official documentation about the values to be saved, but found the corresponding boost/context implemenataion:
https://github.com/boostorg/context/commit/abf8e04e23cf05a499594e674d1c90db39117662
This is similar to the special TIB handling on x86/x86_64 on Windows.
Without this fix an exception in a fiber segfaults without any output:
ruby -e "Fiber.new{ raise 'test' }.resume"
These apparently break compilation on old MacOS toolchains, because the
MachO section name is capped to 16 chars (although, on my MacOS, at
least, the section name just gets truncated). Nevertheless, these serve
no purpose on non-ELF platforms (they're part of the LSB Linux ABI).
[Bug #20677]
This wasn't looking at the right macro name for pac-ret support, so if
Ruby was compiled with pac-ret but NOT BTI, then the ELF note would not
be emitted.
We do not implement CET shadow-stack switching in amd64 Context.S. If
you compile Ruby with `-fcf-protection=full` and run it with
`GLIBC_TUNABLES=glibc.cpu.hwcaps=SHSTK` exported, it will crash with a
control flow exception.
Configure the appropriate notes at the end of Context.S
[Bug #18061]
This partially reverts https://github.com/ruby/ruby/pull/10944; now that
we decided to pass CFLAGS to $(CC) when assembling .S files, we don't
need these autoconf macros that capture the state of
__ARM_FEATURE{PAC|BTI}_DEFAULT.
[Bug #20601]
When running on newer Intel processors supporting the feature,
OpenBSD enforces indirect branch tracking. Without this endbr64
instruction, jumps to the coroutine_transfer function result
in SIGILL on OpenBSD/amd64 when using such processors.
The OpenBSD Ruby ports have been using a patch similar to this
for the past two months.
From some research, cet.h has been supported by GCC for about
6 years and LLVM for about 4 years.
This changes the automatic detection of -fstack-protector,
-D_FORTIFY_SOURCE, and -mbranch-protection to write to $hardenflags
instead of $XCFLAGS. The definition of $cflags is changed to
"$hardenflags $orig_cflags $optflags $debugflags $warnflags" to match.
Furthermore, these flags are _prepended_ to $hardenflags, rather than
appended.
The implications of doing this are as follows:
* If a CRuby builder specifies cflags="-mbranch-protection=foobar" at
the ./configure script, and the configure script detects that
-mbranch-protection=pac-ret is accepted, then GCC will be invoked as
"gcc -mbranch-protection=pac-ret -mbranch-protection=foobar". Since
the last flags take precedence, that means that user-supplied values
of these flags in $cflags will take priority.
* Likewise, if a CRuby builder explicitly specifies
"hardenflags=-mbranch-protection=foobar", because we _prepend_ to
$hardenflags in our autoconf script, we will still invoke GCC as
"gcc -mbranch-protection=pac-ret -mbranch-protection=foobar".
* If a CRuby builder specifies CFLAGS="..." at the configure line,
automatic detection of hardening flags is ignored as before.
* C extensions will _also_ be built with hardening flags now as well
(this was not the case by default before because the detected flags
went into $XCFLAGS).
Additionally, as part of this work, I changed how the detection of
PAC/BTI in Context.S works. Rather than appending the autodetected
option to ASFLAGS, we simply compile a set of test programs with the
actual CFLAGS in use to determine what PAC/BTI settings were actually
chosen by the builder. Context.S is made aware of these choices through
some custom macros.
The result of this work is that:
* Ruby will continue to choose some sensible defaults for hardening
options for the C compiler
* Distributors are able to specify CFLAGS that are consistent with their
distribution and override these defaults
* Context.S will react to whatever -mbranch-protection is actually in
use, not what was autodetected
* Extensions get built with hardening flags too.
[Bug #20154]
[Bug #20520]
With `--std=c99` option coroutine/arm64/Context.h errs:
```
In file included from cont.c:26:
coroutine/arm64/Context.h:59:5: error: call to undeclared function 'asm'; ISO C99 and later do not support
implicit function declarations [-Wimplicit-function-declaration]
59 | asm ("hint #8;" : "+r"(r17) : "r"(r16));
| ^
```
Also move the common function header.
In WebAssembly C ABI, the linear stack pointer must be always aligned
to 16 bytes like other archs.
The misaligned stack pointer causes some weird memory corruption since
compiler assumes the aligned stack pointer.
We don't need to save/restore x30 twice, and we can just use `ret`,
which uses x30 as return address register instead of explicit `ret <reg>`
instruction. This also allows us to use `autiasp` instead of `autia1716`
and we can skip setting SP/LR to x16/x17.
Also the size of register save area is shrunk by 16 bytes due to the
removal of extra x30 save/restore.
No encodings are guaranteed in C compilers, and other than UTF-8
encodings may be assumed in some platforms, e.g., CP932 on Windows
Japanese edition, and may result in compilation errors.
* add coroutines for ppc & ppc64
* fix universal coroutine to include ppc & ppc64
* add powerpc*-darwin to configure.ac
* fix thread_pthread for older systems
