A thread that is waiting for the fork lock when another thread forks
exists only in the parent, but on macOS the child's copy of the lock
still counts it. After the child unlocks its copy,
pthread_rwlock_destroy() and pthread_rwlock_init() fail with EBUSY, so
the child aborts with
"[BUG] pthread_rwlock_destroy: Resource busy (EBUSY)". Lookup threads
wait for the lock whenever a fork holds it, so a process that resolves
names in background threads while it forks loses a child now and then.
Reset the child's copy by assigning PTHREAD_RWLOCK_INITIALIZER, as
rb_internal_thread_event_hooks_rw_lock_atfork() does for its lock, and
do not unlock it first. With a waiter counted, macOS's unlock enters the
kernel to wake it, and the state that leaves behind for the lock's
address survives the assignment, so a later waiter in the child can
sleep forever although no thread holds the lock. FreeBSD already skipped
the unlock in the child (dc8c382266), and on glibc the child's unlock
does not release the write lock anyway.
[Bug #22445]
A getaddrinfo thread calls getaddrinfo(3) inside rb_thread_prevent_fork,
but on glibc its thread exit takes the resolver configuration lock again
after the thread has released the fork lock. A fork that lands then
leaves the child with that lock held, and every later name lookup in the
child blocks forever. getnameinfo(3) attaches the same resolver state,
even for a name in /etc/hosts, and its thread ran outside
rb_thread_prevent_fork altogether.
Make the lookup threads joinable and have fork join any that may still
be exiting. As a lookup thread finishes, still inside
rb_thread_prevent_fork, it swaps itself into a single slot with an
atomic exchange and joins the thread it displaced, which has usually
exited by then, and rb_thread_acquire_fork_lock joins whatever is left
in the slot. Once lookups settle, at most one finished thread is left
unjoined. The slot holds pthread_t as a pointer, with NULL for empty,
which assumes pthread_t is scalar and never 0.
getnameinfo threads now run within rb_thread_prevent_fork and register
as they exit, as getaddrinfo threads do, and where getnameinfo runs
without a helper thread it is called within rb_thread_prevent_fork too.
So fork also waits for a reverse lookup in progress, as it already did
for a forward one. raddrinfo_pthread_create now always creates joinable
threads, which drops its pthread_detach fallback for platforms without
pthread_attr_setdetachstate.
The slot is kept across fork, so every fork path that keeps running
Ruby code has to take the fork lock. On the BSDs, Process.daemon forks
with daemon(3) and did not take it, so the daemon would inherit the
registered thread, and its next fork would abort in pthread_join with
ESRCH. Take and release the fork lock around daemon(3) as rb_fork_ruby
does. daemon(3) can fail in the child as well as in the parent, so
compare pids to tell which process returned.
[Bug #22406]
(cherry picked from commit 44f7c6b00d)
`extra_fd` was leaked if `fd_set_cloexec` fails -- I can't think of any
chance of that happening here, but just in case.
Coverity Scan found this issue.
So that it doesn't get included in the generated binaries for builds
that don't support loading shared GC modules
Co-Authored-By: Peter Zhu <peter@peterzhu.ca>
Do not release GVL around get{pwuid,pwnam,grgid,grnam} calls,
as doing so is not thread-safe. Another C extension could have
a concurrent call, and derefencing the returned pointer from
these calls could result in a segfault.
Have rb_home_dir_of call rb_getpwdirnam_for_login if available,
so it can use getpwnam_r and release GVL in a thread-safe manner.
This is related to GVL releasing work in [Bug #20587].
[Feature #20590]
For better of for worse, fork(2) remain the primary provider of
parallelism in Ruby programs. Even though it's frowned uppon in
many circles, and a lot of literature will simply state that only
async-signal safe APIs are safe to use after `fork()`, in practice
most APIs work well as long as you are careful about not forking
while another thread is holding a pthread mutex.
One of the APIs that is known cause fork safety issues is `getaddrinfo`.
If you fork while another thread is inside `getaddrinfo`, a mutex
may be left locked in the child, with no way to unlock it.
I think we could reduce the impact of these problem by preventing
in for the most notorious and common cases, by locking around
`fork(2)` and known unsafe APIs with a read-write lock.
- Extract functions to check not-found conditions
- Set the length to the result of `rb_getlogin`
- Reentrant versions return an error numeber but not `errno`
- Check maybe-undefined macros with `defined`
The macro SafeStringValue() became just StringValue() in c5c05460ac,
and it is deprecated nowadays.
This patch replaces remaining macro usage. Some occurrences are left in
ext/stringio and ext/win32ole, they should be fixed upstream.
The macro itself is not deleted, because it may be used in extensions.
* Allows macOS users to use M:N threads (and technically FreeBSD, though it has not been verified on FreeBSD)
* Include sys/event.h header check for macros, and include sys/event.h when present
* Rename epoll_fd to more generic kq_fd (Kernel event Queue) for use by both epoll and kqueue
* MAP_STACK is not available on macOS so conditionall apply it to mmap flags
* Set fd to close on exec
* Log debug messages specific to kqueue and epoll on creation
* close_invalidate raises an error for the kqueue fd on child process fork. It's unclear rn if that's a bug, or if it's kqueue specific behavior
Use kq with rb_thread_wait_for_single_fd
* Only platforms with `USE_POLL` (linux) had changes applied to take advantage of kernel event queues. It needed to be applied to the `select` so that kqueue could be properly applied
* Clean up kqueue specific code and make sure only flags that were actually set are removed (or an error is raised)
* Also handle kevent specific errnos, since most don't apply from epoll to kqueue
* Use the more platform standard close-on-exec approach of `fcntl` and `FD_CLOEXEC`. The io-event gem uses `ioctl`, but fcntl seems to be the recommended choice. It is also what Go, Bun, and Libuv use
* We're making changes in this file anyways - may as well fix a couple spelling mistakes while here
Make sure FD_CLOEXEC carries over in dup
* Otherwise the kqueue descriptor should have FD_CLOEXEC, but doesn't and fails in assert_close_on_exec
This patch introduce M:N thread scheduler for Ractor system.
In general, M:N thread scheduler employs N native threads (OS threads)
to manage M user-level threads (Ruby threads in this case).
On the Ruby interpreter, 1 native thread is provided for 1 Ractor
and all Ruby threads are managed by the native thread.
From Ruby 1.9, the interpreter uses 1:1 thread scheduler which means
1 Ruby thread has 1 native thread. M:N scheduler change this strategy.
Because of compatibility issue (and stableness issue of the implementation)
main Ractor doesn't use M:N scheduler on default. On the other words,
threads on the main Ractor will be managed with 1:1 thread scheduler.
There are additional settings by environment variables:
`RUBY_MN_THREADS=1` enables M:N thread scheduler on the main ractor.
Note that non-main ractors use the M:N scheduler without this
configuration. With this configuration, single ractor applications
run threads on M:1 thread scheduler (green threads, user-level threads).
`RUBY_MAX_CPU=n` specifies maximum number of native threads for
M:N scheduler (default: 8).
This patch will be reverted soon if non-easy issues are found.
[Bug #19842]
`Process::Status#&` and `Process::Status#>>` are provided only for
the backward compatibility with older Ruby than 1.8 where `$?` was
a `Fixnum`, and the knowledge about internals of system dependent
macros is necessary to use them. Modern programs and libraries
should not need these methods.
Douglas Eichelberger