These still held String and Array lengths and indices in long, or cast them to long. rb_len_t is still long everywhere, so the generated code does not change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ring buffer of Queue and the limit of SizedQueue are element counts that are filled from and compared with Array lengths, so they would truncate on mswin once rb_len_t widens.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
monitor_enter0() writes the owner and count after do_mutex_lock(),
which may block with the GVL released. The monitor was stored only in
a stack struct field that nothing read, which the compiler may drop, so
nothing was guaranteed to keep the embedded data in place.
Same pattern as https://bugs.ruby-lang.org/issues/22397
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Using the `container->obj = rb_gc_location(container->obj)` pattern dirties pages unconditionally.
If we're in a forked process, this is bad for CoW. If the object hasn't moved, don't reassign.
It was [previously][1] moved from C to Ruby, but the doc on the Ruby
method was copied from `#lock` while the correct doc was left in C.
[1]: 07b2356a6a
Set RUBY_TYPED_THREAD_SAFE_FREE on TypedData types whose dfree function
is trivially safe, and only frees its own memory
Types not yet marked as THREAD_SAFE_FREE:
- id2ref_tbl_type: sets the process-global id2ref_tbl to NULL.
- mutex_data_type: unlinks itself from another thread's keeping_mutexes list.
- autoload_data_type: deletes nodes from a shared intrusive list.
- autoload_const_type: deletes itself from a shared list.
- rb_cont_data_type: mutates the shared fiber pool and the global first_jit_cont list.
- rb_fiber_data_type: delegates to cont_free.
- FiberPoolDataType: mutates shared fiber pool free-lists.
- ractor_data_type: reconfigures global VM event-hook flags and call caches.
- exported_object_registry: frees a global table under the VM lock.
- rb_box_data_type (Box::Entry): unlinks classext from other live class and module objects.
- box_ext_cleanup_type: dereferences another String object and calls unlink.
- monitor_data_type: uses the default free, so the flag has no effect.
Co-authored-by: Luke Gruber <luke.gruber@shopify.com>
While freeing a locked mutex, if there is a waiter for the
mutex it could call `rb_fiber_scheduler_unblock` which calls
into Ruby code.
The function `rb_mutex_unlock_th` was a footgun because it was used
during normal Ruby code and also during GC. The fix is to not call this
function during GC and simply remove the mutex from th->keeping_mutexes.
This reverts commit 994257ab06.
I saw some failures in CI that are probably related to the change.
Example:
```
1) Failure:
TestMonitor#test_timedwait [/Users/runner/work/ruby/ruby/src/test/monitor/test_monitor.rb:282]:
```
This starvation problem has not been an issue in real apps afaik, so for now it's best to revert
it and think of a better solution.
Continually locking a mutex m can lead to starvation if all other threads are on the waitq of m.
See https://bugs.ruby-lang.org/issues/21840 for more details.
Solution:
When a thread `T1` wakes up `T2` during mutex unlock but `T1` or any other thread successfully acquires it
before `T2`, then we record the `running_time` of the thread during mutex acquisition. Then during unlock, if
that thread's running_time is less than the saved running time, we set it back to the saved time.
Fixes [Bug #21840]
Thread::Queue spends a significant amount of time in array functions,
checking for invariants we know aren't a problem, and whether the backing
array need to reordered.
By using a ring buffer we can remove a lot of overhead (~23% faster).
```
$ hyperfine './miniruby --yjit /tmp/q.rb' './miniruby-qrb --yjit /tmp/q.rb'
Benchmark 1: ./miniruby --yjit /tmp/q.rb
Time (mean ± σ): 1.050 s ± 0.191 s [User: 0.988 s, System: 0.004 s]
Range (min … max): 0.984 s … 1.595 s 10 runs
Benchmark 2: ./miniruby-qrb --yjit /tmp/q.rb
Time (mean ± σ): 844.2 ms ± 3.1 ms [User: 840.4 ms, System: 2.8 ms]
Range (min … max): 838.6 ms … 848.9 ms 10 runs
Summary
./miniruby-qrb --yjit /tmp/q.rb ran
1.24 ± 0.23 times faster than ./miniruby --yjit /tmp/q.rb
```
```
q = Queue.new([1, 2, 3, 4, 5, 6, 7, 8])
i = 2_000_000
while i > 0
i -= 1
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
q.push(q.pop)
end
```
Mutexes spend a significant amount of time in `rb_fiber_serial`
because it can't be inlined (except with LTO).
The fiber struct is opaque the so function can't be defined as inlineable.
Ideally the while fiber struct would not be opaque to the rest of
Ruby core, but it's tricky to do.
Instead we can store the fiber serial in the execution context
itself, and make its access cheaper:
```
$ hyperfine './miniruby-baseline --yjit /tmp/mut.rb' './miniruby-inline-serial --yjit /tmp/mut.rb'
Benchmark 1: ./miniruby-baseline --yjit /tmp/mut.rb
Time (mean ± σ): 4.011 s ± 0.084 s [User: 3.977 s, System: 0.011 s]
Range (min … max): 3.950 s … 4.245 s 10 runs
Benchmark 2: ./miniruby-inline-serial --yjit /tmp/mut.rb
Time (mean ± σ): 3.495 s ± 0.150 s [User: 3.448 s, System: 0.009 s]
Range (min … max): 3.340 s … 3.869 s 10 runs
Summary
./miniruby-inline-serial --yjit /tmp/mut.rb ran
1.15 ± 0.05 times faster than ./miniruby-baseline --yjit /tmp/mut.rb
```
```ruby
i = 10_000_000
mut = Mutex.new
while i > 0
i -= 1
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
mut.synchronize { }
end
```
Previously this held a pointer to the Fiber itself, which requires
marking it (which was only implemented recently, prior to that it was
buggy). Using a monotonically increasing integer instead allows us to
avoid having a free function and keeps everything simpler.
My main motivations in making this change are that the root fiber lazily
allocates self, which makes the writebarrier implementation challenging
to do correctly, and wanting to avoid sending Mutexes to the remembered
set when locked by a short-lived Fiber.
Mutexes were being improperly unlocked on thread cleanup. This bug was
introduced in 050a895439.
We must keep a reference from the mutex to the thread, because if the fiber
is collected before the mutex is, then we cannot unlink it from the thread in
`mutex_free`. If it's not unlinked from the the thread when it's freed, it
causes bugs in `rb_thread_unlock_all_locking_mutexes`.
We now mark the fiber when a mutex is locked, and the thread is marked
as well. However, a fiber can still be freed in the same GC cycle as the
mutex, so the reference to the thread is still needed.
The reason we need to mark the fiber is that `mutex_owned_p()` has an ABA
issue where if the fiber is collected while it's locked, a new fiber could be
allocated at the same memory address and we could get false positives.
Fixes [Bug #21342]
Co-authored-by: John Hawthorn <john@hawthorn.email>
Documented the necessity of calling `wait` in a loop. We modified the
example to demonstrate the idiomatic use, and added a third thread `a2`
to demonstrate another reason that necessitates the loop.
Mentioned spurious wake-up in the doc.
assert does not print the bug report, only the file and line number of
the assertion that failed. RUBY_ASSERT prints the full bug report, which
makes it much easier to debug.
This patch introduce M:N thread scheduler for Ractor system.
In general, M:N thread scheduler employs N native threads (OS threads)
to manage M user-level threads (Ruby threads in this case).
On the Ruby interpreter, 1 native thread is provided for 1 Ractor
and all Ruby threads are managed by the native thread.
From Ruby 1.9, the interpreter uses 1:1 thread scheduler which means
1 Ruby thread has 1 native thread. M:N scheduler change this strategy.
Because of compatibility issue (and stableness issue of the implementation)
main Ractor doesn't use M:N scheduler on default. On the other words,
threads on the main Ractor will be managed with 1:1 thread scheduler.
There are additional settings by environment variables:
`RUBY_MN_THREADS=1` enables M:N thread scheduler on the main ractor.
Note that non-main ractors use the M:N scheduler without this
configuration. With this configuration, single ractor applications
run threads on M:1 thread scheduler (green threads, user-level threads).
`RUBY_MAX_CPU=n` specifies maximum number of native threads for
M:N scheduler (default: 8).
This patch will be reverted soon if non-easy issues are found.
[Bug #19842]
* Revert "Extract `do_mutex_lock_check_interrupts` to try and fix `ppc64le`. (#8393)"
This reverts commit 5184b40dd4.
* .travis.yml: Try default gcc 9.4.0 instead of gcc-10 in ppc64le and s390x.
Use gcc 9.4.0 instead of gcc-10 to avoid the current failures by a possible GCC
10 compiler bug in the Travis ppc64le and s390x cases. And it also aligns with
RubyCI Ubuntu ppc64le and s390x where the default gcc is used.
---------
Co-authored-by: Jun Aruga <jaruga@ruby-lang.org>
We found some tests were hanging in `do_mutex_lock`, specifically the
fiber scheduler autoload test. After much investigation, it may be a code
generation bug. Because we didn't change the code, but only extracted it
into a separate function, and it appears to fix the problem.
It's possible (but very rare) to have a race condition between setting
`mutex->fiber = NULL` and `thread_mutex_remove(th, mutex)` which results
in the following bug:
```
[BUG] invalid keeping_mutexes: Attempt to unlock a mutex which is not locked
```
Fixes <https://bugs.ruby-lang.org/issues/19480>.
Fixes the following compilation warnings:
thread_sync.c:908:48: warning: taking address of packed member of `struct rb_queue` may result in an unaligned pointer value [-Waddress-of-packed-member]
thread_sync.c:1181:48: warning: taking address of packed member of `struct rb_queue` may result in an unaligned pointer value [-Waddress-of-packed-member]
[Bug #19105]
If no fiber scheduler is registered and the fiber that
owns the lock and the one that try to acquire it
both belong to the same thread, we're in a deadlock case.
Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
