A getaddrinfo thread calls getaddrinfo(3) inside rb_thread_prevent_fork,
but on glibc its thread exit takes the resolver configuration lock again
after the thread has released the fork lock. A fork that lands then
leaves the child with that lock held, and every later name lookup in the
child blocks forever. getnameinfo(3) attaches the same resolver state,
even for a name in /etc/hosts, and its thread ran outside
rb_thread_prevent_fork altogether.
Make the lookup threads joinable and have fork join any that may still
be exiting. As a lookup thread finishes, still inside
rb_thread_prevent_fork, it swaps itself into a single slot with an
atomic exchange and joins the thread it displaced, which has usually
exited by then, and rb_thread_acquire_fork_lock joins whatever is left
in the slot. Once lookups settle, at most one finished thread is left
unjoined. The slot holds pthread_t as a pointer, with NULL for empty,
which assumes pthread_t is scalar and never 0.
getnameinfo threads now run within rb_thread_prevent_fork and register
as they exit, as getaddrinfo threads do, and where getnameinfo runs
without a helper thread it is called within rb_thread_prevent_fork too.
So fork also waits for a reverse lookup in progress, as it already did
for a forward one. raddrinfo_pthread_create now always creates joinable
threads, which drops its pthread_detach fallback for platforms without
pthread_attr_setdetachstate.
The slot is kept across fork, so every fork path that keeps running
Ruby code has to take the fork lock. On the BSDs, Process.daemon forks
with daemon(3) and did not take it, so the daemon would inherit the
registered thread, and its next fork would abort in pthread_join with
ESRCH. Take and release the fork lock around daemon(3) as rb_fork_ruby
does. daemon(3) can fail in the child as well as in the parent, so
compare pids to tell which process returned.
[Bug #22406]
(cherry picked from commit 44f7c6b00d)
box_entry_free could run before other T_DATA objects are finalized in
rb_objspace_call_finalizer, and FreeLibrary there unmapped the
rb_data_type_t and dfree functions that later finalization still
dereferences, crashing any boxed process that leaves extension objects
alive at exit. Keep the copied DLLs loaded until ruby_vm_destruct, then
unload and delete them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
rb_class_duplicate_classext builds the whole copy first and the caller only
then puts it in the class's table, so a GC in the middle collects the tables
it has made and marking the class afterwards walks into a freed object. Put
the classext in the table first instead. Marking it while it is still half
built is fine, since ZALLOC leaves every field zero.
$ RUBY_BOX=1 ruby --disable-gems -e 'GC.stress = true; module Enumerable; def m; end; end'
-e:1: [BUG] try to mark T_NONE object (obj: out-of-heap:0x..., parent: ... T_MODULE/Module Enumerable)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`rb_load_entrypoint` is used only by `rb_box_load`; it checks that the
argument has two-elements (implicitly assuming it is an array),
extracts its contents, and then simply discards the argument.
FL_TEST -> FL_TEST_RAW
FL_SET -> FL_SET_RAW
I was seeing bad performance on the ruby-bench getivar-module benchmark
when using yjit. This was with other changes in a branch, but changes
totally unrelated to accessing ivars on a class or using boxes. The
change to RCLASS_PRIME_CLASSEXT_WRITABLE_P was the one that made the
difference, but I changed a couple of other FL_SET uses as well. This
change reduced the benchmark runs from 60ms to 40ms.
This change separates the master box from the root box, as the single
master copy of boxes.
Before this change, the root box is the source of copies, and also
it runs builtin classes' code. Builtin code makes changes by
requiring files, thus the source of copies is mutable, be different
from the past one. This causes different internal states of boxes,
depending on when it is created.
After this change, all boxes are created from the master box, and
the master box never runs code, so it realizes the immutable source
of copies.
This also makes is possible to separate RubyGems (and other default
gems) from each boxes. All boxes will have their own copies of RubyGems.
RubyGems has its internal state, but it'll be separated between boxes
after this change.
Previously, rb_ivar_foreach would walk up the shape tree, but yield
instance variables to the callback as it went. If the object shape was
modified during this callback, particularly with removing an instance
variable, it could result in reading free'd memory.
This commit solves this by adding a version which buffering all instance
variable names and values before calling the callback, giving a snapshot
of the object at the time rb_ivar_foreach_buffered is called.
The buffer is made with ALLOCV_N, so the performance difference should
be minimal, and I don't think this method is particuarly heavily used.
I also added a missing variable assignment fixed by 96cd1bc765.
[Bug #21996] (Backport)
class.c: rb_class_duplicate_classext also dup content of cvc_tbl
[Bug #21952]
Shallow copying the table result in the same memory being shared
between multiple box, causing double free when one of the box
is garbage collected.
---
class.c: Make cvc_tbl a managed object
[Bug #21952]
Solves the double-free or use after-free concern with boxes.
Now entries can safely be used for copy-on-write.
Also is likely necessary to make it save to read cvar from
secondary ractors, as allowed since: ab32c0e690
[PATCH] Fix Box regexp match vars after non-match
[PATCH] Use box_ready for $&, $`, $\', $+
These variables have rb_gvar_readonly_setter, so box_ready is sufficient.
Only $~ needs box_dynamic due to its custom match_setter.
Currently we maintain the subclasses list for two separate purposes (we essentially have to different relationships we're putting into the same list):
1. On a T_MODULE, we track the T_ICLASSes created to include it into
other classes. Used for method invalidation and propagating includes
on the module that happen after it's been used
2. On a T_CLASS/T_ICLASS, we track the T_CLASS/T_ICLASS which are the
immediate children of the class. We use this for method invalidation,
some cvar things, and to iterate through subclasses.
Purpose 1 does not have any issues with box, the T_ICLASS always belongs
to one specific module and that's immutable. This list can be box-global
(always use the prime classext or hoist it out) and only needs to be
pruned during free. If we care about behaviour under a particular box
(ie. the propagating includes), we should look up the current box being
modified on the ICLASS itself.
Purpose 2 is more complicated. It currently tracks the immediate
children, the T_CLASS or T_ICLASS whose super points back. Because super
is per-box and is mutable (include/prepend insert ICLASSes into the
chain) we need to update the list on include/prepend, entries must be
per-box, and we can have multiple entries per-box. *I propose we
simplify this by no longer tracking the immediate subclass*, but instead
tracking the T_CLASS -> ... -> T_CLASS relationship, ie. the inverse of
rb_class_superclass. That relationship is the same across all boxes and
immutable after Class creation.
As a special case the ICLASS for refinements are also added to the
purpose 2 list (on T_CLASS). As those ICLASS do not chain to an eventual
leaf T_CLASS.
When we need to find the classes which have included a module, we can
use the module subclasses list to find the ICLASS and then use
RCLASS_INCLUDER. If we needed to iterate all T_ICLASS, we could then
walk up the CLASS_SUPER chain, but I didn't find anywhere we needed to
do that.
Mutexes spend a significant amount of time in `rb_fiber_serial`
because it can't be inlined (except with LTO).
The fiber struct is opaque the so function can't be defined as inlineable.
Ideally the while fiber struct would not be opaque to the rest of
Ruby core, but it's tricky to do.
Instead we can store the fiber serial in the execution context
itself, and make its access cheaper:
```
$ hyperfine './miniruby-baseline --yjit /tmp/mut.rb' './miniruby-inline-serial --yjit /tmp/mut.rb'
Benchmark 1: ./miniruby-baseline --yjit /tmp/mut.rb
Time (mean ± σ): 4.011 s ± 0.084 s [User: 3.977 s, System: 0.011 s]
Range (min … max): 3.950 s … 4.245 s 10 runs
Benchmark 2: ./miniruby-inline-serial --yjit /tmp/mut.rb
Time (mean ± σ): 3.495 s ± 0.150 s [User: 3.448 s, System: 0.009 s]
Range (min … max): 3.340 s … 3.869 s 10 runs
Summary
./miniruby-inline-serial --yjit /tmp/mut.rb ran
1.15 ± 0.05 times faster than ./miniruby-baseline --yjit /tmp/mut.rb
```
```ruby
i = 10_000_000
mut = Mutex.new
while i > 0
i -= 1
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
end
```
The changes are to `io.c` and `thread.c`.
I changed the API of 2 exported thread functions from `internal/thread.h` that
didn't look like they had any use in C extensions:
* rb_thread_wait_for_single_fd
* rb_thread_io_wait
I didn't change the following exported internal function because it's
used in C extensions:
* rb_thread_fd_select
I added a comment to note that this function, although internal, is used
in C extensions.
This rewrites the class allocator search to be faster. Instead of using
RCLASS_SUPER, which is now even slower due to Box, we can scan the
superclasses list to find a class where the allocator is defined.
This also disallows allocating from an ICLASS. Previously I believe that
was only done for FrozenCore, and that was changed in
e596cf6e93.
If we don't have uint128, then rb_int128_to_numeric emits a strict
aliasing warning:
numeric.c:3641:39: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
3641 | return rb_uint128_to_numeric(*(rb_uint128_t*)&n);
| ^~~~~~~~~~~~~~~~~
The #ifdef is currently not taken because include/ruby/backward.h is
not included at this point. The attribute is unnecessary in an internal
header, so remove it.
[Bug #21710]
- struct.c: `struct_alloc`
It is possible for a `NEWOBJ` tracepoint call back to write fields
into a newly allocated object before `struct_alloc` had the time
to set the `RSTRUCT_GEN_FIELDS` flags and such.
Hence we can't blindly initialize the `fields_obj` reference to `0`
we first need to check no fields were added yet.
- object.c: `rb_class_allocate_instance`
Similarly, if a `NEWOBJ` tracepoint tries to set fields on the object,
the `shape_id` must already be set, as it's required on T_OBJECT to
know where to write fields.
`NEWOBJ_OF` had to be refactored to accept a `shape_id`.
Previously this held a pointer to the Fiber itself, which requires
marking it (which was only implemented recently, prior to that it was
buggy). Using a monotonically increasing integer instead allows us to
avoid having a free function and keeps everything simpler.
My main motivations in making this change are that the root fiber lazily
allocates self, which makes the writebarrier implementation challenging
to do correctly, and wanting to avoid sending Mutexes to the remembered
set when locked by a short-lived Fiber.
This reverts commit 2998c8d6b9.
We need to find a better way to fix this bug. Even with this refcount
change, errors were still being seen in CI. For now we need to remove
this failing test.
Calling the usual rb_iclass_classext_free() causes SEGV because
duplicating a newer classext of iclass had set the reference from superclass
to the newer classext, but calling rb_iclass_classext_free() deletes it.
to adopt strict shareable rule.
* (basically) shareable objects only refer shareable objects
* (exception) shareable objects can refere unshareable objects
but should not leak reference to unshareable objects to Ruby world
* `RB_OBJ_SET_SHAREABLE(obj)` makes obj shareable.
All of reachable objects from `obj` should be shareable.
* `RB_OBJ_SET_FROZEN_SHAREABLE(obj)` same as above
but freeze `obj` before making it shareable.
Also `rb_gc_verify_shareable(obj)` is introduced to check
the `obj` does not violate shareable rule (an shareable object
only refers shareable objects) strictly.
The rule has some exceptions (some shareable objects can refer to
unshareable objects, such as a Ractor object (which is a shareable
object) can refer to the Ractor local objects.
To handle such case, `check_shareable` flag is also introduced.
`STRICT_VERIFY_SHAREABLE` macro is also introduced to verify
the strict shareable rule at `SET_SHAREABLE`.
The st_insert in RCLASS_SET_NAMESPACE_CLASSEXT may overwrite an existing
rb_classext_t, causing it to leak memory. This commit changes it to use
st_update to free the existing one before overwriting it.
Douglas Eichelberger