rb_class_duplicate_classext builds the whole copy first and the caller only
then puts it in the class's table, so a GC in the middle collects the tables
it has made and marking the class afterwards walks into a freed object. Put
the classext in the table first instead. Marking it while it is still half
built is fine, since ZALLOC leaves every field zero.
$ RUBY_BOX=1 ruby --disable-gems -e 'GC.stress = true; module Enumerable; def m; end; end'
-e:1: [BUG] try to mark T_NONE object (obj: out-of-heap:0x..., parent: ... T_MODULE/Module Enumerable)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ensure_origin read the m_tbl before the first write created the box-local classext, so the origin ICLASS captured the prime m_tbl while the subclass ICLASSes referred the box-local copy made by the CoW. rb_prepend_module then missed the shared m_tbl check and inserted the prepended module after the target module instead of before it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Prepending a module to a module that already has includers backfills an
origin iclass into each includer's ancestor chain using rb_prepend_module.
The backfilled iclass is never added to a subclasses list, so
rb_clear_method_cache can't reach it when one of the module's methods is
later redefined.
Calling super through a call site that has cached the backfilled iclass
calls the old entry, even though it should be redefined.
```ruby
module M; def foo; :m; end; end
class D; include M; end
M.prepend(Module.new { def foo; super; end })
D.new.foo # prime the super cache
M.send(:define_method, :foo) { :hooked }
D.new.foo # => got :m, expected :hooked
```
This commit makes sure that the backfilled iclass is registered in the
subclasses list.
Since a2531ba293, class_associate_super()
maintains subclass lists only for direct T_CLASS -> T_CLASS links. When
a class that includes or prepends a module is cloned, rb_mod_init_copy()
connects the clone (or its origin) to a T_ICLASS, so the clone was never
added to its real superclass's subclass list and Class#subclasses did
not return it.
Register the clone with its superclass explicitly after the chain is
built. rb_class_superclass() can be used because the superclass table
is already up to date at this point and the superclass always exists
(the root class cannot be cloned). Clones of classes without modules
are registered by class_associate_super() as before.
[Backport #22190]
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Using obj_id in classext_cow_classes is slow and also is a problem when
freeing classes because the lookup requires the EC, which is not available
if it is running on a GC thread. This will cause it to crash on MMTk.
We don't need to use the obj_id as the key in the classext_cow_classes
st_table because the class itself is pinned and won't move so we can use
it as the key instead.
This change separates the master box from the root box, as the single
master copy of boxes.
Before this change, the root box is the source of copies, and also
it runs builtin classes' code. Builtin code makes changes by
requiring files, thus the source of copies is mutable, be different
from the past one. This causes different internal states of boxes,
depending on when it is created.
After this change, all boxes are created from the master box, and
the master box never runs code, so it realizes the immutable source
of copies.
This also makes is possible to separate RubyGems (and other default
gems) from each boxes. All boxes will have their own copies of RubyGems.
RubyGems has its internal state, but it'll be separated between boxes
after this change.
class.c: rb_class_duplicate_classext also dup content of cvc_tbl
[Bug #21952]
Shallow copying the table result in the same memory being shared
between multiple box, causing double free when one of the box
is garbage collected.
---
class.c: Make cvc_tbl a managed object
[Bug #21952]
Solves the double-free or use after-free concern with boxes.
Now entries can safely be used for copy-on-write.
Also is likely necessary to make it save to read cvar from
secondary ractors, as allowed since: ab32c0e690
Currently we maintain the subclasses list for two separate purposes (we essentially have to different relationships we're putting into the same list):
1. On a T_MODULE, we track the T_ICLASSes created to include it into
other classes. Used for method invalidation and propagating includes
on the module that happen after it's been used
2. On a T_CLASS/T_ICLASS, we track the T_CLASS/T_ICLASS which are the
immediate children of the class. We use this for method invalidation,
some cvar things, and to iterate through subclasses.
Purpose 1 does not have any issues with box, the T_ICLASS always belongs
to one specific module and that's immutable. This list can be box-global
(always use the prime classext or hoist it out) and only needs to be
pruned during free. If we care about behaviour under a particular box
(ie. the propagating includes), we should look up the current box being
modified on the ICLASS itself.
Purpose 2 is more complicated. It currently tracks the immediate
children, the T_CLASS or T_ICLASS whose super points back. Because super
is per-box and is mutable (include/prepend insert ICLASSes into the
chain) we need to update the list on include/prepend, entries must be
per-box, and we can have multiple entries per-box. *I propose we
simplify this by no longer tracking the immediate subclass*, but instead
tracking the T_CLASS -> ... -> T_CLASS relationship, ie. the inverse of
rb_class_superclass. That relationship is the same across all boxes and
immutable after Class creation.
As a special case the ICLASS for refinements are also added to the
purpose 2 list (on T_CLASS). As those ICLASS do not chain to an eventual
leaf T_CLASS.
When we need to find the classes which have included a module, we can
use the module subclasses list to find the ICLASS and then use
RCLASS_INCLUDER. If we needed to iterate all T_ICLASS, we could then
walk up the CLASS_SUPER chain, but I didn't find anywhere we needed to
do that.
Without this change, classes (including iclass) are allocated
as un-boxable classes after initializing user boxes (after starting
script evaluation). Under this situation, iclasses are created as
un-boxabled class when core modules are included by a class in the
root box, then it causes problems because it's in the root box but
it can't have multiple classexts.
This change makes it possible to allocate boxable classes even after
initializing user boxes. Classes create in the root box will be
boxable, and those can have 2 or more classexts.
Since singleton classes are created lazily, we need to make sure that
we lock around their creation. Unfortunately, that means we need to
lock around every shareable object's call to `singleton_class`,
including classes and modules.
Safe multi-ractor subclass list mutation
We need to lock around mutation and accesses of a class's subclasses
list. Unfortunately we also need to do this when creating singleton
classes, as the singleton class does need to go into `super`'s
subclasses list for CC invalidation purposes.
Found by wbcheck
It seems like here the classext was associated with the class, but it
already had Ruby objects attached.
rb_gc_writebarrier_remember works around that issue, but I suspect if we
enabled autocompaction the values copied into the classext before it was
attached could be broken.
This reverts commit 2998c8d6b9.
We need to find a better way to fix this bug. Even with this refcount
change, errors were still being seen in CI. For now we need to remove
this failing test.
We don't decrement the super and module subclasses count for iclasses that
are having their classext replaced. This causes the reference count to be
incorrect and leak memory.
The following script demonstrates the memory leak:
module Foo
refine(Object) do
define_method(:<=) {}
end
end
class Bar
include Comparable
end
With RUBY_FREE_AT_EXIT and ASAN, we can see many memory leaks, including:
Direct leak of 16 byte(s) in 1 object(s) allocated from:
#0 0x599f715adca2 in calloc (miniruby+0x64ca2)
#1 0x599f716bd779 in calloc1 gc/default/default.c:1495:12
#2 0x599f716d1370 in rb_gc_impl_calloc gc/default/default.c:8216:5
#3 0x599f716b8ab1 in ruby_xcalloc_body gc.c:5221:12
#4 0x599f716b269c in ruby_xcalloc gc.c:5215:34
#5 0x599f715eab23 in class_alloc0 class.c:790:22
#6 0x599f715e4bec in class_alloc class.c:836:12
#7 0x599f715e60c9 in module_new class.c:1693:17
#8 0x599f715e60a2 in rb_module_new class.c:1701:12
#9 0x599f715e6303 in rb_define_module class.c:1733:14
#10 0x599f715ebc5f in Init_Comparable compar.c:315:22
#11 0x599f716e35f5 in rb_call_inits inits.c:32:5
#12 0x599f7169cbfd in ruby_setup eval.c:88:9
#13 0x599f7169cdac in ruby_init eval.c:100:17
#14 0x599f715b0fa9 in rb_main main.c:41:5
#15 0x599f715b0f59 in main main.c:62:12
#16 0x739b2f02a1c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
#17 0x739b2f02a28a in __libc_start_main csu/../csu/libc-start.c:360:3
#18 0x599f7157c424 in _start (miniruby+0x33424)
Classes/modules defined in a namespace are defined under ::Object
as usual (as without namespaces), and it'll be set into the const_tbl
of ::Object.
In namespaces, namespace objects' const_tbl is equal to the one of ::Object.
So constants of ::Object are just equal to constants of the namespace.
That means, top level classes/modules in a namespace can be referred
as namespace::KlassName without calling rb_define_class_under_id().
Calling the usual rb_iclass_classext_free() causes SEGV because
duplicating a newer classext of iclass had set the reference from superclass
to the newer classext, but calling rb_iclass_classext_free() deletes it.
to adopt strict shareable rule.
* (basically) shareable objects only refer shareable objects
* (exception) shareable objects can refere unshareable objects
but should not leak reference to unshareable objects to Ruby world
The st_insert in RCLASS_SET_NAMESPACE_CLASSEXT may overwrite an existing
rb_classext_t, causing it to leak memory. This commit changes it to use
st_update to free the existing one before overwriting it.
We duplicate the method table in rb_class_duplicate_classext, so we should
set RCLASSEXT_ICLASS_IS_ORIGIN and unset RCLASSEXT_ICLASS_ORIGIN_SHARED_MTBL
to signal that the iclass owns the method table and it should be freed.
remove_class_from_subclasses calls st_insert, which mallocs. Malloc is not
allowed in GC. This commit replaces the st_insert with an st_update since
we know that ns_id exists in the st_table.
The following script reproduces the crash:
require "tempfile"
Tempfile.create do |file|
ns = Namespace.new
ns.require(file)
end
* ZJIT: Add NoSingletonClass patch point
This patch point makes sure that when the object has a singleton class,
the JIT code is invalidated. As of now, this is only needed for C call
optimization.
In YJIT, the singleton class guard only applies to Array, Hash, and String.
But in ZJIT, we may optimize C calls from gems (e.g. `sqlite3`). So the
patch point needs to be applied to a broader range of classes.
* ZJIT: Only generate NoSingletonClass guard when the type can have singleton class
* ZJIT: Update or forget NoSingletonClass patch point when needed
to fix inconsistent and wrong current namespace detections.
This includes:
* Moving load_path and related things from rb_vm_t to rb_namespace_t to simplify
accessing those values via namespace (instead of accessing either vm or ns)
* Initializing root_namespace earlier and consolidate builtin_namespace into root_namespace
* Adding VM_FRAME_FLAG_NS_REQUIRE for checkpoints to detect a namespace to load/require files
* Removing implicit refinements in the root namespace which was used to determine
the namespace to be loaded (replaced by VM_FRAME_FLAG_NS_REQUIRE)
* Removing namespaces from rb_proc_t because its namespace can be identified by lexical context
* Starting to use ep[VM_ENV_DATA_INDEX_SPECVAL] to store the current namespace when
the frame type is MAGIC_TOP or MAGIC_CLASS (block handlers don't exist in this case)
Previously we had an assertion that the method table was only set on
young objects, and a comment stating that was how it needed to be used.
I think that confused the complexity of the write barriers that may be
needed here.
* Setting an empty M_TBL never needs a write barrier
* T_CLASS and T_MODULE should always fire a write barrier to newly added
methods
* T_ICLASS only needs a write barrier to methods when
RCLASSEXT_ICLASS_IS_ORIGIN(x) && !RCLASSEXT_ICLASS_ORIGIN_SHARED_MTBL(x)
We shouldn't assume that the object being young is sufficient, because
we also need write barriers for incremental marking and it's unreliable.
When creating a new origin in ensure_origin, we need to fire a write
barrier after RCLASS_WRITE_ORIGIN. rb_class_set_super allocates, so GC
could happen there, either incrementally marking or promoting the newly
allocated class, and only after RCLASS_WRITE_ORIGIN will origin mark
object in the M_TBL.
The `FL_FREEZE` flag is redundant with `SHAPE_ID_FL_FROZEN`, so
ideally it should be eliminated in favor of the later.
Doing so would eliminate the risk of desync between the two, but
also solve the problem of the frozen status being global in namespace
context (See Bug #21330).
Even when namespaces are enabled, only a few core classes created
during init will eventually be namespaced.
For these it's OK to allocate a 320B slot to hold the extra namespace
stuff.
But for any class created post init, we know we'll never need the
namespace and we can fit in a 160B slot.
This behave almost exactly as a T_OBJECT, the layout is entirely
compatible.
This aims to solve two problems.
First, it solves the problem of namspaced classes having
a single `shape_id`. Now each namespaced classext
has an object that can hold the namespace specific
shape.
Second, it open the door to later make class instance variable
writes atomics, hence be able to read class variables
without locking the VM.
In the future, in multi-ractor mode, we can do the write
on a copy of the `fields_obj` and then atomically swap it.
Considerations:
- Right now the `RClass` shape_id is always synchronized,
but with namespace we should likely mark classes that have
multiple namespace with a specific shape flag.
