Fix race between timer thread dequeuing waiting thread and thread
skipping sleeping due to being dequeued. We now use `th->event_serial` which
is protected by `thread_sched_lock`. When a thread is put on timer thread's waiting
list, the event serial is saved on the item. The timer thread checks
that the saved serial is the same as current thread's serial before
calling `thread_sched_to_ready`.
The following script (taken from a test in `test_thread.rb` used to crash on
scheduler debug assertions. It would likely crash in non-debug mode as well.
```ruby
def assert_nil(val)
if val != nil
raise "Expected #{val} to be nil"
end
end
def assert_equal(expected, actual)
if expected != actual
raise "Expected #{expected} to be #{actual}"
end
end
def test_join2
ok = false
t1 = Thread.new { ok = true; sleep }
Thread.pass until ok
Thread.pass until t1.stop?
t2 = Thread.new do
Thread.pass while ok
t1.join(0.01)
end
t3 = Thread.new do
ok = false
t1.join
end
assert_nil(t2.value)
t1.wakeup
assert_equal(t1, t3.value)
ensure
t1&.kill&.join
t2&.kill&.join
t3&.kill&.join
end
rs = 30.times.map do
Ractor.new do
test_join2
end
end
rs.each(&:join)
```
Add the macro `RB_THREAD_CURRENT_EC_NOINLINE` to manage the condition to use
no-inline version rb_current_ec for a better maintainability.
Note that the `vm_core.h` includes the `THREAD_IMPL_H` by the
`#include THREAD_IMPL_H`. The `THREAD_IMPL_H` can be `thread_none.h`,
`thread_pthread.h` or `thread_win32.h` according to the
`tool/m4/ruby_thread.m4` creating the `THREAD_IMPL_H`.
The change in this commit only defining the `RB_THREAD_CURRENT_EC_NOINLINE` in
the `thread_pthread.h` is okay in this situation. Because in the
`thread_none.h` case, the thread feature is not used at all, including
Thread-Local Storage (TLS), and in the `thread_win32.h` case, the
`RB_THREAD_LOCAL_SPECIFIER` is not defined. In the `thread_pthread.h` case, the
`RB_THREAD_LOCAL_SPECIFIER` is defined in the `configure.ac`. In the
`thread_none.h` case, the `RB_THREAD_LOCAL_SPECIFIER` is defined in the
`thread_none.h`.
This commit fixes the failures in bootstraptest/test_ractor.rb reported on
the issue ticket <https://bugs.ruby-lang.org/issues/21534>.
TLS (Thread-Local Storage) may not be accessed across .so on ppc64le too.
I am not sure about that. The comment "// TLS can not be accessed across
.so on ..." in this commit comes from the following commit.
https://github.com/ruby/ruby/commit/319afed20fba8f9b44611d16e4930260f7b56b86#diff-408391c43b2372cfe1fefb3e1c2531df0184ed711f46d229b08964ec9e8fa8c7R118
> // on Darwin, TLS can not be accessed across .so`
This failures only happened when configuring with cppflags=-DRUBY_DEBUG and -O3
on ppc64le.
The reproducing steps were below.
```
$ ./autogen.sh
$ ./configure -C --disable-install-doc cppflags=-DRUBY_DEBUG
$ make -j4
$ make btest BTESTS=bootstraptest/test_ractor.rb
...
FAIL 2/147 tests failed
make: *** [uncommon.mk:913: yes-btest] Error 1
```
The steps with a reproducing script based on the `bootstraptest/test_ractor.rb`
were below.
```
$ cat test_ractor_1.rb
counts = []
counts << Ractor.count
p counts.inspect
ractors = (1..2).map { Ractor.new { Ractor.receive } }
counts << Ractor.count
p counts.inspect
ractors[0].send('End 0').join
sleep 0.1 until ractors[0].inspect =~ /terminated/
counts << Ractor.count
p counts.inspect
ractors[1].send('End 1').join
sleep 0.1 until ractors[1].inspect =~ /terminated/
counts << Ractor.count
p counts.inspect
$ make run TESTRUN_SCRIPT=test_ractor_1.rb
...
vm_core.h:2017: Assertion Failed: rb_current_execution_context:ec == rb_current_ec_noinline()
...
```
The assertion failure happened at the following line.
https://github.com/ruby/ruby/blob/f3206cc79bec2fd852e81ec56de59f0a67ab32b7/vm_core.h#L2017
This fix is similar with the following commit for the arm64.
https://github.com/ruby/ruby/commit/f7059af50a31a4d27a04ace0beadb60616f3f971
Fixes [Bug #21534]
* Added `Ractor::Port`
* `Ractor::Port#receive` (support multi-threads)
* `Rcator::Port#close`
* `Ractor::Port#closed?`
* Added some methods
* `Ractor#join`
* `Ractor#value`
* `Ractor#monitor`
* `Ractor#unmonitor`
* Removed some methods
* `Ractor#take`
* `Ractor.yield`
* Change the spec
* `Racotr.select`
You can wait for multiple sequences of messages with `Ractor::Port`.
```ruby
ports = 3.times.map{ Ractor::Port.new }
ports.map.with_index do |port, ri|
Ractor.new port,ri do |port, ri|
3.times{|i| port << "r#{ri}-#{i}"}
end
end
p ports.each{|port| pp 3.times.map{port.receive}}
```
In this example, we use 3 ports, and 3 Ractors send messages to them respectively.
We can receive a series of messages from each port.
You can use `Ractor#value` to get the last value of a Ractor's block:
```ruby
result = Ractor.new do
heavy_task()
end.value
```
You can wait for the termination of a Ractor with `Ractor#join` like this:
```ruby
Ractor.new do
some_task()
end.join
```
`#value` and `#join` are similar to `Thread#value` and `Thread#join`.
To implement `#join`, `Ractor#monitor` (and `Ractor#unmonitor`) is introduced.
This commit changes `Ractor.select()` method.
It now only accepts ports or Ractors, and returns when a port receives a message or a Ractor terminates.
We removes `Ractor.yield` and `Ractor#take` because:
* `Ractor::Port` supports most of similar use cases in a simpler manner.
* Removing them significantly simplifies the code.
We also change the internal thread scheduler code (thread_pthread.c):
* During barrier synchronization, we keep the `ractor_sched` lock to avoid deadlocks.
This lock is released by `rb_ractor_sched_barrier_end()`
which is called at the end of operations that require the barrier.
* fix potential deadlock issues by checking interrupts just before setting UBF.
https://bugs.ruby-lang.org/issues/21262
Many libraries should be loaded on the main ractor because of
setting constants with unshareable objects and so on.
This patch allows to call `requore` on non-main Ractors by
asking the main ractor to call `require` on it. The calling ractor
waits for the result of `require` from the main ractor.
If the `require` call failed with some reasons, an exception
objects will be deliverred from the main ractor to the calling ractor
if it is copy-able.
Same on `require_relative` and `require` by `autoload`.
Now `Ractor.new{pp obj}` works well (the first call of `pp` requires
`pp` library implicitly).
[Feature #20627]
This patch introduce M:N thread scheduler for Ractor system.
In general, M:N thread scheduler employs N native threads (OS threads)
to manage M user-level threads (Ruby threads in this case).
On the Ruby interpreter, 1 native thread is provided for 1 Ractor
and all Ruby threads are managed by the native thread.
From Ruby 1.9, the interpreter uses 1:1 thread scheduler which means
1 Ruby thread has 1 native thread. M:N scheduler change this strategy.
Because of compatibility issue (and stableness issue of the implementation)
main Ractor doesn't use M:N scheduler on default. On the other words,
threads on the main Ractor will be managed with 1:1 thread scheduler.
There are additional settings by environment variables:
`RUBY_MN_THREADS=1` enables M:N thread scheduler on the main ractor.
Note that non-main ractors use the M:N scheduler without this
configuration. With this configuration, single ractor applications
run threads on M:1 thread scheduler (green threads, user-level threads).
`RUBY_MAX_CPU=n` specifies maximum number of native threads for
M:N scheduler (default: 8).
This patch will be reverted soon if non-easy issues are found.
[Bug #19842]
Checking by `__STDC_VERSION__` is unreliable because old gcc 4.8
supports `-std=gnu11` option but does not implement `_Thread_local`.
Check the implementation directly instead.
Show native thread's serial on `RUBY_DEBUG_LOG`.
`nt->serial` is also stored into `ruby_nt_serial` if the compiler
supports `RB_THREAD_LOCAL_SPECIFIER`.
`NON_SCALAR_THREAD_ID` shows `pthread_t` is non-scalar (non-pointer)
and only s390x is known platform. However, the supporting code is
very complex and it is only used for deubg print information.
So this patch removes the support of `NON_SCALAR_THREAD_ID`
and make the code simple.
`rb_thread_t` contained `native_thread_data_t` to represent
thread implementation dependent data. This patch separates
them and rename it `rb_native_thread` and point it from
`rb_thraed_t`.
Now, 1 Ruby thread (`rb_thread_t`) has 1 native thread (`rb_native_thread`).
Now GVL is not process *Global* so this patch try to use
another words.
* `rb_global_vm_lock_t` -> `struct rb_thread_sched`
* `gvl->owner` -> `sched->running`
* `gvl->waitq` -> `sched->readyq`
* `rb_gvl_init` -> `rb_thread_sched_init`
* `gvl_destroy` -> `rb_thread_sched_destroy`
* `gvl_acquire` -> `thread_sched_to_running` # waiting -> ready -> running
* `gvl_release` -> `thread_sched_to_waiting` # running -> waiting
* `gvl_yield` -> `thread_sched_yield`
* `GVL_UNLOCK_BEGIN` -> `THREAD_BLOCKING_BEGIN`
* `GVL_UNLOCK_END` -> `THREAD_BLOCKING_END`
* removed
* `rb_ractor_gvl`
* `rb_vm_gvl_destroy` (not used)
There are GVL functions such as `rb_thread_call_without_gvl()` yet
but I don't have good name to replace them. Maybe GVL stands for
"Greate Valuable Lock" or something like that.
To make some kind of Ractor related extensions, some functions
should be exposed.
* include/ruby/thread_native.h
* rb_native_mutex_*
* rb_native_cond_*
* include/ruby/ractor.h
* RB_OBJ_SHAREABLE_P(obj)
* rb_ractor_shareable_p(obj)
* rb_ractor_std*()
* rb_cRactor
and rm ractor_pub.h
and rename srcdir/ractor.h to srcdir/ractor_core.h
(to avoid conflict with include/ruby/ractor.h)
To access TLS, it is faster to use language TLS specifier instead
of using pthread_get/setspecific functions.
Original proposal is: Use native thread locals. #3665
This commit introduces Ractor mechanism to run Ruby program in
parallel. See doc/ractor.md for more details about Ractor.
See ticket [Feature #17100] to see the implementation details
and discussions.
[Feature #17100]
This commit does not complete the implementation. You can find
many bugs on using Ractor. Also the specification will be changed
so that this feature is experimental. You will see a warning when
you make the first Ractor with `Ractor.new`.
I hope this feature can help programmers from thread-safety issues.
According to MSVC manual (*1), cl.exe can skip including a header file
when that:
- contains #pragma once, or
- starts with #ifndef, or
- starts with #if ! defined.
GCC has a similar trick (*2), but it acts more stricter (e. g. there
must be _no tokens_ outside of #ifndef...#endif).
Sun C lacked #pragma once for a looong time. Oracle Developer Studio
12.5 finally implemented it, but we cannot assume such recent version.
This changeset modifies header files so that each of them include
strictly one #ifndef...#endif. I believe this is the most portable way
to trigger compiler optimizations. [Bug #16770]
*1: https://docs.microsoft.com/en-us/cpp/preprocessor/once
*2: https://gcc.gnu.org/onlinedocs/cppinternals/Guard-Macros.html
`acquired' was an old boolean variable, but nowadays it is a
rb_thread_t pointer; "gvl.owner" seems like a more appropriate
name. And document the contended path including waitq, timer,
and timer_err.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64581 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This matches the behavior of old timer thread more closely
and seems to fix [Bug #14999] when limited to a single CPU.
I cannot reproduce the error on a multi-core system unless
I use schedtool to force affinity to a single CPU:
schedtool -a 0x01 -e make test-spec \
MSPECOPT='-R1000 spec/ruby/library/conditionvariable/wait_spec.rb'
While it may be good enough to pass the spec, I don't have
huge degree of confidence in the interrupt handling robustness
under extremely heavy load (these may be ancient bugs, though).
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64467 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Do not waste extra memory for each thread, but make
thread_pthread.c easier-to-follow as a result.
[ruby-core:88475] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64375 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This reverts commit 194a6a2c68 (r64203).
Race conditions which caused the original reversion will be fixed
in the subsequent commit.
[ruby-core:88360] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64352 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Maybe some platforms have strange condition variable implementations
which have a "memory" of which mutexes they're associated with.
In any case, it makes documentation easier even on GNU/Linux and
FreeBSD.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64124 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
To reduce resource use and reduce CI failure; remove
timer-thread. Single-threaded Ruby processes (including forked
children) will never see extra thread overhead. This prevents
glibc and jemalloc from going into multi-threaded mode and
initializing locks or causing fragmentation via arena explosion.
The GVL is implements its own wait-queue as a ccan/list to
permit controlling wakeup order. Timeslice under contention is
handled by a designated timer thread (similar to choosing a
"patrol_thread" for current deadlock checking).
There is only one self-pipe, now, as wakeups for timeslice are
done independently using condition variables. This reduces FD
pressure slightly.
Signal handling is handled directly by a Ruby Thread (instead
of timer-thread) by exposing signal self-pipe to callers of
rb_thread_fd_select, native_sleep, rb_wait_for_single_fd, etc...
Acquiring, using, and releasing the self-pipe is exposed via 4
new internal functions:
1) rb_sigwait_fd_get - exclusively acquire timer_thread_pipe.normal[0]
2) rb_sigwait_fd_sleep - sleep and wait for signal (and no other FDs)
3) rb_sigwait_fd_put - release acquired result from rb_sigwait_fd_get
4) rb_sigwait_fd_migrate - migrate signal handling to another thread
after calling rb_sigwait_fd_put.
rb_sigwait_fd_migrate is necessary for waitpid callers because
only one thread can wait on self-pipe at a time, otherwise a
deadlock will occur if threads fight over the self-pipe.
TRAP_INTERRUPT_MASK is now set for the main thread directly in
signal handler via rb_thread_wakeup_timer_thread.
Originally, I wanted to use POSIX timers
(timer_create/timer_settime) for this. Unfortunately, this
proved unfeasible as Mutex#sleep resumes on spurious wakeups and
test/thread/test_cv.rb::test_condvar_timed_wait failed. Using
pthread_sigmask to mask out SIGVTALRM fixed that test, but
test/fiddle/test_function.rb::test_nogvl_poll proved there'd be
some unavoidable (and frequent) incompatibilities from that
approach.
Finally, this allows us to drop thread_destruct_lock and
interrupt current ec directly.
We don't need to rely on vm->thread_destruct_lock or a coherent
vm->running_thread on any platform. Separate timer-thread for
time slice and signal handling is relegated to thread_win32.c,
now.
[ruby-core:88088] [Misc #14937]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64107 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
There's no reason to use CLOCK_REALTIME for any condvars in Ruby.
Indeed, we initialized all condvars with RB_CONDATTR_CLOCK_MONOTONIC
anyway; so simplify our code and reduce ifdefs.
[ruby-core:85639] [Misc #14497]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63238 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This will make it easier for us to try alternative mutex/condvar
implementations while still using pthreads for thread management.
[Feature #10134]
* thread_pthread.h: define RB_NATIVETHREAD_LOCK_INIT and
RB_NATIVETHREAD_COND_INIT macros
* thread_pthread.c (native_mutex_lock, native_mutex_unlock,
native_mutex_trylock, native_mutex_initialize,
native_mutex_destroy, native_cond_wait):
use rb_nativethread_lock_t instead of pthread_mutex_t
* thread_pthread.c (native_mutex_debug): make argument type-agnostic
to avoid later cast.
* thread_pthread.c (register_cached_thread_and_wait):
replace PTHREAD_COND_INITIALIZER with RB_NATIVETHREAD_COND_INIT,
use native_mutex_{lock,unlock}
* thread_pthread.c (use_cached_thread):
use native_mutex_{lock,unlock}
* thread_pthread.c (native_sleep):
use rb_nativethread_lock_t to match th->interrupt_lock,
use native_mutex_{lock,unlock}
* thread_pthread.c (timer_thread_lock): use rb_nativethread_lock_t type
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47185 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* thread_pthread.h (struct rb_global_vm_lock_struct):
do not expose pthread type for lock
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46761 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* thread_pthread.h: remove unneeded semaphore.h include
We currently do not use POSIX semaphores.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46760 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Move native thread related lines from vm_core.h.
And declare several functions "rb_nativethread_lock_*",
manipulate locking.
* common.mk: add thread_native.h.
* thread.c: add functions "rb_nativethread_lock_*".
* thraed.c, thread_[pthread,win32].[ch]: rename rb_thread_lock_t
to rb_nativethread_lock_t to make it clear that this lock is for
native thraeds, not for ruby threads.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42133 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
Jean Boussier