Recently we added some keys to GC.stat in order to measure the GC
performance of Ractor local GC and the associated global GC.
This commit refactors some of those terms: "global" was used in two
places with different meanings which could get confusing.
When it was passed to GC.stat as scope: :global it meant "Give me
aggregate stats for all the GCs running in this process".
Whereas inside the output of GC.stat, global referred to the number of
"global" GCs that are run. That is: stop-the-world GCs that run on
every ractor heap in the process.
Also the global_gc_count that GC.stat reports for a Ractor ignored
global GCs that other Ractors started, even though they swept its heap.
In this commit we fix this confusing behaviour, and make the GC.stat
output more consistent.
We've replaced the scope: parameter to GC.stat with the boolean
process: keyword arg. The default is process: false which means
GC.stat will return stats for the Ractor that executes the code (the
main Ractor if no other Ractors have been created). process: true will
return aggregate stats over all the Ractors in the process (including
dead ones).
Inside GC.stat(process: false), count and global_gc_count now
include every global GC that has swept that Ractor's heap. So Ractor
local gc count == count - global_gc_count.
heap_marked_slots now counts only the marked objects in that Ractor's
own heap. Before, the Ractor that started a global GC counted the
marked objects of every heap, and the other Ractors reported 0.
We also provide a new key global_gc_invocation_count that is the
number of global GCs that have been triggered by the Ractor in question.
We've also moved some keys related to the global page pool out of the
default GC.stat output and into GC.stat(process: true), because they
are process-wide and not Ractor specific.
GC.stat(:global_gc_count) now reports the stop-the-world global cycles
the current Ractor started, instead of mirroring the process-wide count,
so the values can be summed across Ractors. GC.stat(:global_gc_count,
scope: :global) gives the process total, including Ractors that have
since exited.
A global cycle no longer counts as a major GC on the Ractor that started
it, so count == minor_gc_count + major_gc_count + global_gc_count.
Replace the RVALUE_SLOT_SIZE-multiplier based pool sizes with explicit
power-of-two (and near-power-of-two) slot sizes. On 64-bit this gives
12 heaps (32, 40, 64, 80, 96, 128, 160, 256, 512, 640, 768, 1024)
instead of 5, providing finer granularity and less internal
fragmentation. On 32-bit the layout is 5 heaps (32, 64, 128, 256, 512).
gc_config_set returned rb_gc_impl_config_get, but gc_config_get also added
the implementation key to the return value. This caused the return value
of GC.config to differ depending on whether the optional hash argument is
provided or not.
This feature provides a new method `GC.config` that configures internal
GC configuration variables provided by an individual GC implementation.
Implemented in this PR is the option `full_mark`: a boolean value that
will determine whether the Ruby GC is allowed to run a major collection
while the process is running.
It has the following semantics
This feature configures Ruby's GC to only run minor GC's. It's designed
to give users relying on Out of Band GC complete control over when a
major GC is run. Configuring `full_mark: false` does two main things:
* Never runs a Major GC. When the heap runs out of space during a minor
and when a major would traditionally be run, instead we allocate more
heap pages, and mark objspace as needing a major GC.
* Don't increment object ages. We don't promote objects during GC, this
will cause every object to be scanned on every minor. This is an
intentional trade-off between minor GC's doing more work every time,
and potentially promoting objects that will then never be GC'd.
The intention behind not aging objects is that users of this feature
should use a preforking web server, or some other method of pre-warming
the oldgen (like Nakayoshi fork)before disabling Majors. That way most
objects that are going to be old will have already been promoted.
This will interleave major and minor GC collections in exactly the same
what that the Ruby GC runs in versions previously to this. This is the
default behaviour.
* This new method has the following extra semantics:
- `GC.config` with no arguments returns a hash of the keys of the
currently configured GC
- `GC.config` with a key pair (eg. `GC.config(full_mark: true)` sets
the matching config key to the corresponding value and returns the
entire known config hash, including the new values. If the key does
not exist, `nil` is returned
* When a minor GC is run, Ruby sets an internal status flag to determine
whether the next GC will be a major or a minor. When `full_mark:
false` this flag is ignored and every GC will be a minor.
This status flag can be accessed at
`GC.latest_gc_info(:needs_major_by)`. Any value other than `nil` means
that the next collection would have been a major.
Thus it's possible to use this feature to check at a predetermined
time, whether a major GC is necessary and run one if it is. eg. After
a request has finished processing.
```ruby
if GC.latest_gc_info(:needs_major_by)
GC.start(full_mark: true)
end
```
[Feature #20443]
This commit splits gc.c into two files:
- gc.c now only contains code not specific to Ruby GC. This includes
code to mark objects (which the GC implementation may choose not to
use) and wrappers for internal APIs that the implementation may need
to use (e.g. locking the VM).
- gc_impl.c now contains the implementation of Ruby's GC. This includes
marking, sweeping, compaction, and statistics. Most importantly,
gc_impl.c only uses public APIs in Ruby and a limited set of functions
exposed in gc.c. This allows us to build gc_impl.c independently of
Ruby and plug Ruby's GC into itself.
The documentation for GC.start was incorrect. It said that the
`immediate_sweep` keyword argument would defer sweeping when set to
`true`, but it should have said that it would defer sweeping when set to
`false`.
The commit will also fix the typo "immedate_sweep" to "immediate_sweep".
This commit adds key force_incremental_marking_finish_count to
GC.stat_heap. This statistic returns the number of times the size pool
has forced incremental marking to finish due to running out of slots.
There is a `time` key in GC.stat that gives us the total time spent in
GC. However, we don't know what proportion of the time is spent between
marking and sweeping. This makes it difficult to tune the GC as we're
not sure where to focus our efforts on.
This PR adds keys `marking_time` and `sweeping_time` to GC.stat for the
time spent marking and sweeping, in milliseconds.
[Feature #19437]
Matt Valentine-House