Before this change, GC'ing any Ractor object caused you to lose all
enabled tracepoints across all ractors (even main). Now tracepoints are
ractor-local and this doesn't happen. Internal events are still global.
Fixes [Bug #19112]
We're seeing an occasional crash on CI because this ends up inlined all
the way into ractor_wait_receive. On llvm (possibly other compilers) the
thread local address of ec ends up cached (not the value of ec, the
address ec is read from). So if we are migrated to another native
thread, that may be invalid.
Using rb_current_ec_noinline avoids this problems. It would be good to
adjust this code so that ec (or current ractor) is calculated once and
then passed through to both lock and unlock.
`T_DATA` has a flag `RUBY_TYPED_FROZEN_SHAREABLE` which means
if the `T_DATA` object is frozen, it can be sharable.
On the `Ractor.make_sharable(obj)`, rechable objects from the
`T_DATA` object will be apply `Ractor.make_shareable` recursively.
`RUBY_TYPED_FROZEN_SHAREABLE_NO_REC` is similar to the
`RUBY_TYPED_FROZEN_SHAREABLE`, but doesn't apply `Ractor.make_sharable`
recursively for children.
If it refers to unshareable objects, it will simply raise an error.
I'm not sure this pattern is common or not, so it is not in public.
If we find more cases, we can discuss publication.
The "EXIVAR" terminology has been replaced by "gen fields"
AKA "generic fields".
Exivar implies variable, but generic fields include more than
just variables, e.g. `object_id`.
This bug was happening only when the `id2ref` table exists. We need
to replace the generic fields before replacing the object id of the
newly moved object.
Fixes [Bug #21664]
Fixes a segmentation fault when moving nested objects between ractors with GC stress enabled and YJIT.
The issue is a timing problem: `move_enter` allocates new object shells but leaves their contents uninitialized until `move_leave` copies the actual data. If GC runs between these steps (which GC stress makes likely), it tries to follow what appear to be object pointers but are actually uninitialized memory, encountering null or invalid addresses.
The fix zero-initializes the object contents immediately after allocation in `move_enter`, ensuring the GC finds safe null pointers instead of garbage data.
The crash reproduced most consistently with nested hashes and YJIT, likely because nested structures create multiple uninitialized objects simultaneously while YJIT's memory usage increases the probability of GC triggering during moves.
to adopt strict shareable rule.
* (basically) shareable objects only refer shareable objects
* (exception) shareable objects can refere unshareable objects
but should not leak reference to unshareable objects to Ruby world
* `RB_OBJ_SET_SHAREABLE(obj)` makes obj shareable.
All of reachable objects from `obj` should be shareable.
* `RB_OBJ_SET_FROZEN_SHAREABLE(obj)` same as above
but freeze `obj` before making it shareable.
Also `rb_gc_verify_shareable(obj)` is introduced to check
the `obj` does not violate shareable rule (an shareable object
only refers shareable objects) strictly.
The rule has some exceptions (some shareable objects can refer to
unshareable objects, such as a Ractor object (which is a shareable
object) can refer to the Ractor local objects.
To handle such case, `check_shareable` flag is also introduced.
`STRICT_VERIFY_SHAREABLE` macro is also introduced to verify
the strict shareable rule at `SET_SHAREABLE`.
call-seq:
Ractor.sharable_proc(self: nil){} -> sharable proc
It returns shareable Proc object. The Proc object is
shareable and the self in a block will be replaced with
the value passed via `self:` keyword.
In a shareable Proc, the outer variables should
* (1) refer shareable objects
* (2) be not be overwritten
```ruby
a = 42
Ractor.shareable_proc{ p a }
#=> OK
b = 43
Ractor.shareable_proc{ p b; b = 44 }
#=> Ractor::IsolationError because 'b' is reassigned in the block.
c = 44
Ractor.shareable_proc{ p c }
#=> Ractor::IsolationError because 'c' will be reassigned outside of the block.
c = 45
d = 45
d = 46 if cond
Ractor.shareable_proc{ p d }
#=> Ractor::IsolationError because 'd' was reassigned outside of the block.
```
The last `d`'s case can be relaxed in a future version.
The above check will be done in a static analysis at compile time,
so the reflection feature such as `Binding#local_varaible_set`
can not be detected.
```ruby
e = 42
shpr = Ractor.shareable_proc{ p e } #=> OK
binding.local_variable_set(:e, 43)
shpr.call #=> 42 (returns captured timing value)
```
Ractor.sharaeble_lambda is also introduced.
[Feature #21550]
[Feature #21557]
A cross_ractor_require is either a require or an autoload, so we can make
the fields an union. This reduces the size of cross_ractor_require by 8
bytes.
If we malloc when the current Ractor is locked, we can deadlock because
GC requires VM lock and Ractor barrier. If another Ractor is waiting on
this Ractor lock, then it will deadlock because the other Ractor will
never join the barrier.
For example, this script deadlocks:
r = Ractor.new do
loop do
Ractor::Port.new
end
end
100000.times do |i|
r.send(nil)
puts i
end
On debug builds, it fails with this assertion error:
vm_sync.c:75: Assertion Failed: vm_lock_enter:cr->sync.locked_by != rb_ractor_self(cr)
On non-debug builds, we can see that it deadlocks in the debugger:
Main Ractor:
frame #3: 0x000000010021fdc4 miniruby`rb_native_mutex_lock(lock=<unavailable>) at thread_pthread.c:115:14
frame #4: 0x0000000100193eb8 miniruby`ractor_send0 [inlined] ractor_lock(r=<unavailable>, file=<unavailable>, line=1180) at ractor.c:73:5
frame #5: 0x0000000100193eb0 miniruby`ractor_send0 [inlined] ractor_send_basket(ec=<unavailable>, rp=0x0000000131092840, b=0x000000011c63de80, raise_on_error=true) at ractor_sync.c:1180:5
frame #6: 0x0000000100193eac miniruby`ractor_send0(ec=<unavailable>, rp=0x0000000131092840, obj=4, move=<unavailable>, raise_on_error=true) at ractor_sync.c:1211:5
Second Ractor:
frame #2: 0x00000001002208d0 miniruby`rb_ractor_sched_barrier_start [inlined] rb_native_cond_wait(cond=<unavailable>, mutex=<unavailable>) at thread_pthread.c:221:13
frame #3: 0x00000001002208cc miniruby`rb_ractor_sched_barrier_start(vm=0x000000013180d600, cr=0x0000000131093460) at thread_pthread.c:1438:13
frame #4: 0x000000010028a328 miniruby`rb_vm_barrier at vm_sync.c:262:13 [artificial]
frame #5: 0x00000001000dfa6c miniruby`gc_start [inlined] rb_gc_vm_barrier at gc.c:179:5
frame #6: 0x00000001000dfa68 miniruby`gc_start [inlined] gc_enter(objspace=0x000000013180fc00, event=gc_enter_event_start, lock_lev=<unavailable>) at default.c:6636:9
frame #7: 0x00000001000dfa48 miniruby`gc_start(objspace=0x000000013180fc00, reason=<unavailable>) at default.c:6361:5
frame #8: 0x00000001000e3fd8 miniruby`objspace_malloc_increase_body [inlined] garbage_collect(objspace=0x000000013180fc00, reason=512) at default.c:6341:15
frame #9: 0x00000001000e3fa4 miniruby`objspace_malloc_increase_body [inlined] garbage_collect_with_gvl(objspace=0x000000013180fc00, reason=512) at default.c:6741:16
frame #10: 0x00000001000e3f88 miniruby`objspace_malloc_increase_body(objspace=0x000000013180fc00, mem=<unavailable>, new_size=<unavailable>, old_size=<unavailable>, type=<unavailable>) at default.c:8007:13
frame #11: 0x00000001000e3c44 miniruby`rb_gc_impl_malloc [inlined] objspace_malloc_fixup(objspace=0x000000013180fc00, mem=0x000000011c700000, size=12582912) at default.c:8085:5
frame #12: 0x00000001000e3c30 miniruby`rb_gc_impl_malloc(objspace_ptr=0x000000013180fc00, size=12582912) at default.c:8182:12
frame #13: 0x00000001000d4584 miniruby`ruby_xmalloc [inlined] ruby_xmalloc_body(size=<unavailable>) at gc.c:5128:12
frame #14: 0x00000001000d4568 miniruby`ruby_xmalloc(size=<unavailable>) at gc.c:5118:34
frame #15: 0x00000001001eb184 miniruby`rb_st_init_existing_table_with_size(tab=0x000000011c2b4b40, type=<unavailable>, size=<unavailable>) at st.c:559:39
frame #16: 0x00000001001ebc74 miniruby`rebuild_table_if_necessary [inlined] rb_st_init_table_with_size(type=0x00000001004f4a78, size=524287) at st.c:585:5
frame #17: 0x00000001001ebc5c miniruby`rebuild_table_if_necessary [inlined] rebuild_table(tab=0x000000013108e2f0) at st.c:753:19
frame #18: 0x00000001001ebbfc miniruby`rebuild_table_if_necessary(tab=0x000000013108e2f0) at st.c:1125:9
frame #19: 0x00000001001eba08 miniruby`rb_st_insert(tab=0x000000013108e2f0, key=262144, value=4767566624) at st.c:1143:5
frame #20: 0x0000000100194b84 miniruby`ractor_port_initialzie [inlined] ractor_add_port(r=0x0000000131093460, id=262144) at ractor_sync.c:399:9
frame #21: 0x0000000100194b58 miniruby`ractor_port_initialzie [inlined] ractor_port_init(rpv=4750065560, r=0x0000000131093460) at ractor_sync.c:87:5
frame #22: 0x0000000100194b34 miniruby`ractor_port_initialzie(self=4750065560) at ractor_sync.c:103:12
* ZJIT: Implement SingleRactorMode invalidation
* ZJIT: Add macro for compiling jumps
* ZJIT: Fix typo in comment
* YJIT: Fix typo in comment
* ZJIT: Avoid using unexported types in zjit.h
`enum ruby_vminsn_type` is declared in `insns.inc` and is not exported.
Using it in `zjit.h` would cause build errors when the file including it
doesn't include `insns.inc`.
None of the datastructures involved in the require process are
safe to call on a secondary ractor, however when autoloading
encodings, we do so from the current ractor.
So all sorts of corruption can happen when using an autoloaded
encoding for the first time from a secondary ractor.
This object was newly allocated on move_enter, so some GC may happen and
it may have been marked by move_leave, so we need to issue an
rb_gc_writebarrier_remember so that any new references are seen afer the
memcpy from the old object.
Followup: https://github.com/ruby/ruby/pull/13589
This simplify a lot of things, as we no longer need to manually
manage the memory, we can use the Read-Copy-Update pattern and
avoid numerous race conditions.
Co-Authored-By: Étienne Barrié <etienne.barrie@gmail.com>
If an object is shareable and has no capacity left, it isn't
safe to store the object ID in fields as it requires an object
resize which can't be done unless all field reads are synchronized.
In this very specific case we create the object_id in advance,
before the object is made shareable.
[Bug #21090]
The struct was previously allocated on the stack, which could be freed
if the Thread is terminated. Moving this to a T_DATA on the heap should
mean this is no longer an issue.
1000.times { Ractor.new { th = Thread.new { require "rbconfig" }; Thread.pass }.take }
Co-authored-by: Luke Gruber <luke.gruber@shopify.com>
This data is redundant because the shape already contains both the
length and capacity of the object's fields.
So it both waste space and create the possibility of a desync between
the two.
We also do not need to initialize everything to Qundef, this seem
to be a left-over from pre-shape instance variables.
A finalizer registerred in Ractor A can be invoked in B.
```ruby
require "tempfile"
r = Ractor.new{
10_000.times{|i|
Tempfile.new(["file_to_require_from_ractor#{i}", ".rb"])
}
}
sleep 0.1
```
For example, above script makes tempfiles which have finalizers
on Ractor r, but at the end of the process, main Ractor will invoke
finalizers and it violates belonging check. This patch just ignore
the belonging check to avoid CI failure.
Of course it violates Ractor's isolation and wrong workaround.
This issue will be solved with Ractor local GC.
`RUBY_DEBUG=gc_stress ./miniruby -e0` crashes because of this
marking miss.
BTW, to use `RUBY_DEBUG=gc_stress` we need to specify
`--enable-debug-env` configure option. This is why I couldn't repro
on my environments.
see c0c94ab183
John Hawthorn