- ### Problem
When a Ractor isolation error happens due to setting an ivar on a
class, the message doesn't include the name of the ivar and neither
the name of the class. It makes it difficult to track down where the
isolation problem comes from.
I'd like the message to be similar as what we get when a Ractor
access an unshareable ivar.
This is the current behaviour
```ruby
class Foo
def self.foo
@foo ||= "test"
end
end
Ractor.new do
Foo.foo # can not set instance variables of classes/modules by non-main Ractors
end.join
Foo.foo
Ractor.new do
Foo.foo # can not get unshareable values from instance variables of classes/modules from non-main Ractors (@foo from Foo)
end.join
```
### Solution
Modify the error message to include the ivar name and the class/module name.
Co-authored-by: Yaroslav Markin <yaroslav@markin.net>
rb_const_remove deleted the entry from the constant table without the VM
lock and then freed it, while a Ractor that does not own the class looks
the table up under the VM lock but reads the entry after releasing it.
Reading a constant while its owner removes it could crash in the table
lookup or return garbage (an Integer, Float or false that was never
assigned).
Delete under the VM lock, and have the readers that may not own the class
copy the entry while holding it (rb_const_lookup_copy). Class#dup and
the singleton class clone walk the original's table under the lock for
the same reason; the Class#dup check collects the candidates under the
lock and runs rb_ractor_shareable_p after it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A constant entry whose value is Qundef always holds the autoload
registration, so check its type only by VM_ASSERT instead of handling
other cases.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An autoload registration has been kept in a per-module table under the
hidden instance variable __autoload__, in addition to the Qundef entry
in the constant table. With Ruby::Box, the constant table of a class is
copied entry by entry for each box, but the instance variables are
copied shallowly, so all the boxes shared one autoload table.
Once an autoload registered in the master box (e.g., Ruby::Prism) is
resolved in one box, the registration was removed from the shared
table, while the boxes that have their own copy of the class kept their
Qundef entries. There, the constant was neither defined nor
autoloadable, and referring to it raised NameError. Also, autoloads of
the same name registered in two boxes overwrote each other.
Keep the registration in the Qundef constant entry instead (in a union
with the file, which is moved to the registration), so each box has its
own registration with its own constant table, and the two never get out
of sync.
As a dup'ed module now shares the registration (instead of the table),
check that the mutex is locked to see if someone else is loading it, so
that the registration is not considered available after the feature is
loaded.
[Bug #22305]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The constants of an autoload were set in the box that triggered it. For
an autoload registered in the master box (e.g., Ruby::Prism), this
defined the constant only in the first box that accessed it, and the
other boxes got NameError.
Set the constants in the registering box instead. For the master box,
they are set in the prime classext, which is shared by all boxes; a box
that has its own copy of the class (e.g., the one that triggered the
autoload) takes the value from the prime one.
[Bug #22305]
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Argument counts stay `int` or `long`. The int range checks in proc.c and compile.c now depend on SIZEOF_VALUE so that they are kept once rb_len_t widens.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This code should not raise an error:
```
@var = Class.new.freeze.dup # unfrozen copy
@var.instance_variable_set(:@test, 123)
```
It was raising `Ractor::IsolationError` because of a bug in `rb_fields_tbl_copy`.
This patch also fixes the following assertion failure in debug mode:
```
Module.new.freeze.singleton_class.instance_variable_set("@a", 1)
```
Now we get the proper behavior:
```
can't modify frozen Class: #<Class:#<Module:0x00000001094d9900>> (FrozenError)
```
Fixes [Bug #22378]
Under RUBY_BOX=1 an assignment in a user box, the main box included, only
updates the box-local table and never reaches the global entry, so hooks
registered with trace_var never run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
rb_gvar_set and rb_gvar_get key the box table by var->id, so defined? on an aliased name looked for a key that is never written and returned nil.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under RUBY_BOX=1 the box-local gvar table is keyed by the name used for
access, so `alias $b $a` gives the two names separate slots and they
stop being synonyms. Key it by the ID the shared rb_global_variable
was created with instead.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Assignment in a box only writes to box->gvar_tbl and leaves the global
entry with rb_gvar_undef_getter, so defined?($foo) returned nil while
$foo itself returned the assigned value. mkmf's have_devel? stops its
own recursion with `unless defined? $have_devel`, so extconf.rb blew the
stack under RUBY_BOX=1. Reading an undefined gvar no longer caches nil
into the box table, which keeps a key there meaning "defined here".
Previously we had a check that objects which were shareable, but not
frozen on a non-main Ractor would raise an exception if instance
variables were read. This prevented a non-main Ractor reading
non-shareable objects.
This changes the behaviour to forbid all writes to shareable objects
(even when unfrozen). This is done by re-purposing the frozen flag on
the object shape to reflect (FL_FROZEN || FL_SHAREABLE).
With IV writes forbidden on all shareable objects whether or not they
are frozen (with the exception of classes and modules), we can also
remove the check against reading instance variables, since no unsafe
instance variables can be set.
[Bug #22291]
None of these has a caller, a declaration in any header, or a place in
libruby's dynamic symbol table, so an extension cannot reach them either.
* rb_vm_opt_newarray_pack: left over from cb9510f539, which moved the
instruction to the _buffer form.
* rb_autoload: a wrapper around rb_autoload_str, deprecated and dropped
from intern.h in Ruby 2.3 [Feature #11664].
* rb_big_fdiv: a wrapper around rb_big_fdiv_double, unused since Bignum
stopped defining #fdiv.
* rb_io_unbuffered: a wrapper around rb_io_synchronized, which is what it
was renamed to before 1.8.0.
* rb_ractor_hooks: an accessor for cr->pub.hooks; only its sibling
rb_ractor_targeted_hooks is used.
* rb_assert_sig: a leftover debugging aid.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Requested by matz on [Feature #22226]: a non-owner freezing somebody else's
class is a hole, because it lets a third party make a class permanently
unmodifiable. Naming is the same kind of change to the class itself.
Module#freeze checks only when it would actually change something, so an
already frozen class stays a no-op for every Ractor and idempotent .freeze
calls keep working.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A class variable is shared across the inheritance chain and the class it is
stored in can migrate (cvar_overtaken), so the deciding Ractor is the owner of
the class the variable is actually stored in, not of the receiver it was looked
up through. That is a single well-defined owner at any moment, it is the same
single writer the rest of that class's fields already have, and it keeps the
cross-boundary writes out: main cannot write into a Ractor's class, and a
Ractor cannot write into main's.
Two holes turned up on the way:
- Module#remove_class_variable had no ownership check at all, so any Ractor
could delete a class variable out of a foreign class.
- The class variable inline caches keyed the fast path on rb_ractor_main_p(),
and the fast path writes straight into ic->entry->class_value without going
through rb_cvar_set. Main running a method of a foreign class therefore
skipped the ownership check entirely. The condition is now the owner of
class_value, matching what the instance variable fast paths already do -- and
a non-main owner gets the fast path it could not reach before.
cvar_overtaken() no longer deletes the duplicated entry from a class owned by
another Ractor: that deletion is reachable from read paths (rb_cvar_find), so
doing it there would be a cross-Ractor write, and it is only clean-up.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Only the Ractor which created a class/module may modify it: define, remove or
undefine methods, alias, include/prepend, refine, define or remove constants,
register an autoload, and set the class's own instance variables. Reading is
unchanged. This replaces the assorted "main Ractor only" rules with one
creator-based rule, and in exchange a Ractor gets full use of the classes it
creates itself -- unshareable constant and instance variable values included,
which even the main Ractor could not do for its own classes before.
Three paths reach a class's tables without going through rb_class_modify_check,
so they get the check of their own: Module#refine (which writes its refinement
tables into the receiver as well as into the refined class), Class#initialize
(which writes the superclass of a not-yet-initialized class, and had no frozen
check either), and Module#ruby2_keywords.
The constant inline cache has to move with the rule. Its hit condition was
"shareable value, or we are the main Ractor", which was sound only because a
non-main Ractor could not have an unshareable constant in the first place. Now
it can, so an owner priming the cache would let any other Ractor read that value
straight out of it, never reaching the owner check in vm_get_ev_const. The
entry records the Ractor which filled it instead -- the one that passed the
check -- which also gives a non-main owner a fast path it never had.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Using the `container->obj = rb_gc_location(container->obj)` pattern dirties pages unconditionally.
If we're in a forked process, this is bad for CoW. If the object hasn't moved, don't reassign.
Calling these hook methods shouldn't create CCs because these are not hot methods.
These CCs were only looked up in the gccct table (global call cache lookup table)
for rb_funcall and friends.
For one of our apps after bootup we reduced CCs significantly:
| | A control | B (your commit) | B − A |
|-----------------------|------------|-----------------|-----------------------|
| callcaches (after GC) | 705,957 | 651,747 | −54,210 (−7.7%) |
| cc tables | 46,361 | 43,051 | −3,310 |
| total cc memory | 86,441,345 | 79,823,142 | −6,618,203 (−6.3 MB) |
With RUBY_BOX=1, firing an autoload invoked Ruby::Box#require on the
registered box, bypassing any Kernel#require decoration (RubyGems,
Zeitwerk, ...) prepended in that box. Zeitwerk's implicit namespaces
rely on the decorated require, so a plain Rails app failed with
LoadError on the first request. Call require on the box's top self in
a frame running in that box instead, which dispatches the decorated
method and still loads the feature into the box that registered the
autoload.
Adding a generic-fields entry takes the VM-wide generic_fields_lock and then
calls st_insert, which mallocs when the table grows. With this Ractor's GC
disabled the allocation cannot fall back to a collection, so it can raise
NoMemoryError -- and the raise skipped both gf_unlock() and the matching
rb_gc_local_enable(). A leaked generic_fields_lock hangs every later access
to generic fields, including the marking of any Ractor's local GC, so an
otherwise recoverable NoMemoryError turned into a deadlock. Unwind through
a tag instead.
While here, drop three things that no longer match the code: a basket
payload comment describing a generic-ivar map field that was removed, the
unused st_table placeholder left in ractor_basket_new, and two comments in
variable.c still describing the per-Ractor generic-fields tables that the
single global table replaced.
This commit also carries the mechanical translation of the Japanese comments
in these two files, which was in progress in the same working tree.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Keep the generic-ivar host-to-fields_obj mapping in one global table,
serializing mutators with a dedicated lock. A local GC marks only the
values living in its own objspace; the global GC marks the values of
live keys in the weak pass that runs after marking. This removes the
table-ownership transitions that used to happen on shareable promotion,
on send, and on absorb.
THe downside is that we now always reserve one 8B reference for the
IMEMO/fields, even for structs that never have any ivars.
But it is an acceptable tradeoff given that ivars on structs aren't
rare.
Co-Authored-By: John Hawthorn <john@hawthorn.email>
Numerous codepath have the same `switch/case` with very little
differing logic compared to `rb_obj_fields`, might as well
use it everywhere and share more code.
Followup: https://github.com/ruby/ruby/pull/17631
Now that complex `RObject` always have an IMEMO/fields, we no
longer need to have an external `st_table` to replicate the
memory layout of `RObject`.
Instead of spilling into a raw buffer, `RObject` now spills
into an `IMEMO/fields` object like other types.
From an instance variable layout standpoint, an extended `RObject`
is now identical to a `RTypedData`, as in they both store the
reference to their `IMEMO/fields` at the same offset (`VALUE * 2`).
One positive consequence of this is that the only case where a `T_OBJECT`
needs sweeping is if a finalizer was registered.
YJIT now side exit when it need to write a new ivar into a `RObject`
that is out of space. This is unlikely to cause a performance regression
as this codepath isn't supposed to happen after warmup given `max_iv_count`
is recorded on classes, so future objects should be large enough.
Hence it's best not to waste executable memory for such codepath.
ZJIT lost support for writting into extended `RObject`.
Most of the code to support it is there it's just missing an
implementation of `RBASIC_SET_SHAPE_ID`, which recently changed
to strip some bits out of the shape (see comments).
I will leave it to the ZJIT team to implement it (sorry).
There are a few future cleanups planned that I keep for followups:
- We can now get rid of the `ROBJECT_HEAP` flag, it's redundant with
the shape layout bits.
- `imemo_fields` can now embed the `st_table` when complex.
Co-Authored-By: John Hawthorn <john@hawthorn.email>
Co-authored-by: Randy Stauner <randy@r4s6.net>
Like the capacity part, the layout part of an object shape
almost never changes. The few exceptions are:
- On allocation.
- On being compacted by GC.
- When RObject oberflows.
As such it simplifies a lot of code if `RBASIC_SET_SHAPE_ID` strips
the layout bits, as we often copy the shape from IMEMO/fields to
the owner object and vice-versa.
Also change `RBASIC_SET_SHAPE_ID_WITH_CAPACITY` into
`RBASIC_SET_FULL_SHAPE_ID` so it can be used for assigning both
capacity and layout.
Set RUBY_TYPED_THREAD_SAFE_FREE on TypedData types whose dfree function
is trivially safe, and only frees its own memory
Types not yet marked as THREAD_SAFE_FREE:
- id2ref_tbl_type: sets the process-global id2ref_tbl to NULL.
- mutex_data_type: unlinks itself from another thread's keeping_mutexes list.
- autoload_data_type: deletes nodes from a shared intrusive list.
- autoload_const_type: deletes itself from a shared list.
- rb_cont_data_type: mutates the shared fiber pool and the global first_jit_cont list.
- rb_fiber_data_type: delegates to cont_free.
- FiberPoolDataType: mutates shared fiber pool free-lists.
- ractor_data_type: reconfigures global VM event-hook flags and call caches.
- exported_object_registry: frees a global table under the VM lock.
- rb_box_data_type (Box::Entry): unlinks classext from other live class and module objects.
- box_ext_cleanup_type: dereferences another String object and calls unlink.
- monitor_data_type: uses the default free, so the flag has no effect.
Co-authored-by: Luke Gruber <luke.gruber@shopify.com>
* Reserve 2 bits for expressing object layout
We would like to make instance variable reads in the JIT compiler faster
(as well as simplify the JIT implementation). Currently, in order to
read an instance variable, we have to:
1. Test for heap object
2. Load object to a 64 bit register
3. Mask the object header
4. Bit test against the masked header
5. JNE
6. Load field
We would like to:
1. Test for heap object
2. Load object shape to a 32 bit register
3. Bit test against the shape
4. JNE
5. Load field
The way we fetch instance variables is not consistent across objects.
In order to realize our goal, we need to encode object layout inside the
shape. If we encode object layout inside the shape, then the shape
itself will guarantee that the access pattern generated by the JIT
compiler is correct.
We should encode the following load patterns into the shape tag bits.
This way we can share shapes on transitions, but be able to
differentiate the access patterns for the JIT compiler. In other words,
two objects can have an `@a -> @b -> @c` transition and share the same
shape, but the tag bits can differentiate the access pattern so that the
JIT compiler can be confident that the machine code is correct.
Here are the patterns:
1. Embedded/Extended T_OBJECT Instance Variables
Objects with direct references to instance variables or via malloc
buffer
2. Objects with fields_objects fields
These are Data and TypedData objects. They have an associated axillary
imemo/fields object that stores the instance variables. The access
pattern is `object[2] + 2`. The fields object is the 3rd field, and the
instance variables start at +2 inside the fields object. The fields
object itself is a Ruby object, so it contains the usual header bits +
class headers.
3. Non Boxable Classes / Modules
This is similar to Objects with fields_objects, but the fields object is
stored at a different offset. We’re differentiating this from boxable
classes and modules because those are harder to support.
4. Other
"Other" pattern is for objects that are rare, or have
difficult-to-implement access patterns. This includes:
* Boxable classes and modules
* Structs (for now)
* Objects that use the geniv table
Proposed shape bit layout:
```
Current shape_id_t is 32 bits:
31 28 27 26 25 24 23 22 19 18 0
+-----------+--+--+--+--+--+------------+----------------------------+
| unused |L1|L0|OI|FR|CX| heap index | shape tree offset |
+-----------+--+--+--+--+--+------------+----------------------------+
| | | | | | |
| | | | | | +-- bits 0-18: SHAPE_ID_OFFSET_MASK
| | | | | +--------------- bits 19-22: SHAPE_ID_HEAP_INDEX_MASK
| | | | +------------------ bit 23: SHAPE_ID_FL_COMPLEX
| | | +--------------------- bit 24: SHAPE_ID_FL_FROZEN
| | +------------------------ bit 25: SHAPE_ID_FL_HAS_OBJECT_ID
+--+--------------------------- bits 26-27: SHAPE_ID_LAYOUT_MASK
```
The important part about these layout patterns is that they do not
reflect the _type_ of object, only how the object is laid out in memory.
For example, we currently treat structs as "other", but we can refactor
them to have the same layout as "Objects with fields_objects", and when
we do that they should get a different bit in the shape header.
This commit only reserves the two bits, it doesn't use them in the JIT
compiler yet.
Co-Authored-By: John Hawthorn <john@hawthorn.email>
Co-Authored-By: Max Bernstein <tekknolagi@gmail.com>
* Update gc.c
Co-authored-by: Nobuyoshi Nakada <nobu.nakada@gmail.com>
* Update shape.h
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
* fix function name
* Update shape.c
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
* fix function name
* Revert "Update shape.c"
This reverts commit 900711defc6c541a93f3393a350819ae88cf87f1.
* add comment
---------
Co-authored-by: John Hawthorn <john@hawthorn.email>
Co-authored-by: Max Bernstein <tekknolagi@gmail.com>
Co-authored-by: Nobuyoshi Nakada <nobu.nakada@gmail.com>
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
This allows us to treat the two types of T_DATA the same way when
reading instance variables.
Non-TypedData T_DATA are deprecated, so making them faster isn't
particularly important, but it is helpful to avoid needing to test for
TypedData and adding another special case.
Now that we have 1024B slots, we can store up to 126 fields inline.
Objects larger than this are rare if not non-existent, hence we can
get rid of the `malloc` path for imemo/fields and simply transition
to `TOO_COMPLEX`.
This additionally allows to shrink `attr_index_t` from 16 to 8B.
Note: the ZJIT "ivar on extended" tests are renamed as "complex" because
"extended" AKA malloc allocated imemo/fields no longer exists.
They're now complex fields, AKA st tables.
rb_class_allocate_instance: start as complex when over max_fields
If `RCLASS_MAX_IV_COUNT` is over `max_fields`, allocating a large
slot to end up transitioning to `TOO_COMPLEX` is wasteful.
We might as well start as complex directly.
