rb_vm_empty_cc(), rb_vm_empty_cc_for_super() and rb_vm_uncached_cc()
shouldn't be real function calls. rb_iseq_mark_and_move was calling
these as real functions in a loop.
The assumption is that we're usually calling these methods on boot, and they don't
need CCs. CCs scale with the number of classes that use these methods in the app, so
if you have 10k classes and these classes use 3 of these methods on average, then that
saves 30k CC objects.
Methods on Module that are marked cold are:
public, protected, private, module_function, ruby2_keywords, alias_method, undef_method,
remove_method, attr_reader, attr_writer, attr_accessor, include, prepend.
If these methods ever get redefined, they get new method definitions that are not
marked uncacheable (CCs will get created).
In one of our apps (not the largest), we observed this difference after boot:
| method | before | after | delta |
|-------------------|-------:|------:|------------:|
| `include` | 4518 | 2 | -4516 |
| `alias_method` | 3921 | 0 | -3921 |
| `ruby2_keywords` | 3784 | 0 | -3784 |
| `private` | 3519 | 3 | -3516 |
| `attr_reader` | 2738 | 0 | -2738 |
| `attr_accessor` | 2288 | 0 | -2288 |
| `prepend` | 354 | 1 | -353 |
| `protected` | 303 | 0 | -303 |
| `public` | 302 | 0 | -302 |
| `undef_method` | 178 | 0 | -178 |
| `attr_writer` | 124 | 0 | -124 |
| `module_function` | 42 | 0 | -42 |
| `remove_method` | 24 | 0 | -24 |
| **total** | | | **-22,089** |
NOTE: CC tables and ccs (the malloced struct) are still being created, so this only reduces
the CC objects themselves. I will tackle that in a follow-up.
[Bug #22436]
The check_funcall_exec function incorrectly restored the pending block
handler which causes it to leak.
For example, the following code crashes:
o = Object.new
def o.method_missing(*) = nil # Array#zip calls o.each via method_missing
out = Object.new
def out.write(*) = nil
def out.puts(*) = yield # receives a block it was never given
$stdout = out
[1].zip(o)
puts 1 # Kernel#puts -> $stdout.puts gets a stale ifunc block
iseq.h does not reference any of the pm_* types or functions declared in
prism_compile.h, so include the latter from the files that actually use
it instead. Only 6 of the 19 files including iseq.h need the prism
declarations, so this keeps prism/prism.h out of the include closure of
ast.c, box.c, builtin.c, cont.c, eval.c, jit.c, proc.c, thread.c,
vm_dump.c, vm_trace.c, yjit.c, zjit.c and internal/jit.h.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Argument counts stay `int` or `long`. The int range checks in proc.c and compile.c now depend on SIZEOF_VALUE so that they are kept once rb_len_t widens.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
We know the type of the proc object at allocation time, so we can allocate
exactly the amount of memory needed for that type.
We can also shrink the type from a 4 byte enum since there are only 4 types
of procs. This allows us to put some flags after it, which saves us 8 bytes.
This commit reduces symbol and proc procs by 24 bytes (80 bytes to 56 bytes).
It also reduces iseq and ifunc procs by 8 bytes (80 bytes to 72 bytes).
Since `prc.refined(*ms).refined(*ns)` is behaviorally equivalent to
`prc.refined(*ms, *ns)`, the copy of the block is deferred until
the first call and the memo is shared by these behaviorally
equivalent Procs. A Proc that is never called is no longer copied
at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Audit of every EC_PUSH_TAG user, following the LLVM 19 longjmp
rematerialization bug worked around in rb_fiber_start: locals and
arguments that are read on the longjmp path but not modified inside
the tag region are made volatile. C99 7.13.2.1p3 guarantees such
unmodified locals, but LLVM 17 and 19 have been observed to
reconstruct their values incorrectly after a longjmp (cfp in
rb_iterate0, fiber in rb_fiber_start). volatile on a parameter may
not be effective (67d0f4821f,
https://bugs.ruby-lang.org/issues/19145), so arguments are copied
into volatile locals instead.
* vm_eval.c (rb_iterate0): cfp is compared against the escaping
frame after a longjmp; clang 17 at -O1 with -fPIC was observed to
clobber it during the development of the ifunc invalidation for
https://bugs.ruby-lang.org/issues/22019. ec is read next to it to
rewind the frame and to rethrow.
* eval_error.c (rb_ec_error_print_detailed0): opt and highlight are
read on the path reached by a longjmp from rb_get_backtrace while
the other locals of the function are already volatile.
* scheduler.c (rb_fiber_scheduler_unblock, fiber_interrupt): ec and
saved_interrupt_mask are read after a longjmp to restore
ec->interrupt_mask; cfp next to them was already volatile.
* eval.c (rb_vrescue2): r_proc and data2 are used when the raise is
handled. args is left as is; a va_list cannot portably be
volatile-qualified and is memory-backed in practice.
* eval.c (rb_protect): pstate is written through after the tag is
popped.
* eval.c (rb_ec_ensure): ec, e_proc and data2 run the ensure proc
after a longjmp; a clobbered e_proc would miscall it.
* vm_eval.c (vm_catch_protect): stateptr is written through after
the pop. tag has its address taken and stays in memory.
* vm_trace.c (exec_hooks_protected): ec and list are passed to
exec_hooks_postcheck after the pop.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Clang 17 at -O1 with -fPIC can incorrectly rematerialize cfp after longjmp when a nearby stack layout changes, even though cfp is not modified in the tag region. This makes rb_iterate0 miss the matching escape frame and propagate TAG_BREAK as an unexpected break.
Copy the ec argument into a volatile local and make the saved cfp volatile so both values are reloaded from their pre-setjmp storage. This keeps the direct ZJIT fields added by #17953.
This is the rb_iterate0 portion of #17931.
Add Proc#refined(mod, ...), which returns a new Proc that behaves like
the receiver but with the refinements activated by the given modules in
effect inside its body, without affecting the original Proc. This is
an alternative to the previously proposed Proc#using
(https://bugs.ruby-lang.org/issues/16461) that does not modify the
existing block and needs no `using Proc::Refinements` declaration.
Refinements are resolved at run time through the cref, so a refined
proc gets two pieces of its own:
* A cref carrying the additional refinements, built by activating the
modules (rb_using_module_recursive, under the VM lock) on a duplicate
of the block's captured cref. It is stored in a hidden ivar on the
Proc and injected into the block frame on every proc-invocation path
(Proc#call, yield from C methods, invokeblock/opt_call, and
instance_eval/instance_exec/module_eval/class_eval via yield_under).
The captured environment stays shared, so closure variables are still
shared with the original Proc. `using` inside the body is rejected
(CREF_REFINED_PROC), so the cref's refinements never change after
publication.
* A recursive deep copy of the block's instruction sequences
(rb_iseq_dup_with_independent_caches) so the new Proc has its own
inline method caches. This is required for correctness: the VM
caches refinement method resolution per call site assuming a single
lexical cref per iseq, so sharing the iseq would leak the refined
methods into the original Proc. Coverage keeps measuring the copied
iseq.
The copy and the cref are memoized in a hidden identity Hash keyed by
the source iseq (one slot per iseq, matched by the captured cref and
the module arguments, accessed under the VM lock), so repeated calls at
the same site reuse one copy and procs refined with the same modules
share the iseq and the cref. A differing module set or a ruby2_keywords
flag mismatch rebuilds the entry (with a performance warning). The
refinements table of the memoized cref is frozen and marked shareable so
the memo also serves other Ractors.
Procs without an iseq block (Symbol#to_proc, C-function procs), procs
created from methods, and procs that already have refinements are
rejected with ArgumentError.
The YJIT/ZJIT Rust bindings are regenerated for the new rb_proc_t field.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Instead of spilling into a raw buffer, `RObject` now spills
into an `IMEMO/fields` object like other types.
From an instance variable layout standpoint, an extended `RObject`
is now identical to a `RTypedData`, as in they both store the
reference to their `IMEMO/fields` at the same offset (`VALUE * 2`).
One positive consequence of this is that the only case where a `T_OBJECT`
needs sweeping is if a finalizer was registered.
YJIT now side exit when it need to write a new ivar into a `RObject`
that is out of space. This is unlikely to cause a performance regression
as this codepath isn't supposed to happen after warmup given `max_iv_count`
is recorded on classes, so future objects should be large enough.
Hence it's best not to waste executable memory for such codepath.
ZJIT lost support for writting into extended `RObject`.
Most of the code to support it is there it's just missing an
implementation of `RBASIC_SET_SHAPE_ID`, which recently changed
to strip some bits out of the shape (see comments).
I will leave it to the ZJIT team to implement it (sorry).
There are a few future cleanups planned that I keep for followups:
- We can now get rid of the `ROBJECT_HEAP` flag, it's redundant with
the shape layout bits.
- `imemo_fields` can now embed the `st_table` when complex.
Co-Authored-By: John Hawthorn <john@hawthorn.email>
Co-authored-by: Randy Stauner <randy@r4s6.net>
Passing an array of const pointers within a struct as an argument
seems to be too complex for cl.exe:
```
vm_eval.c(501): warning C4090: 'function': different 'const' qualifiers
```
Rename for consistency with ISEQ_BODY naming convention. Remove the
separate rb_cfp_has_iseq/rb_cfp_has_pc wrappers — CFP_ISEQ and CFP_PC
return NULL for missing values, which works directly in if conditions.
Per tenderlove's review, every cfp->iseq read should go through an
accessor that accounts for JITFrame. Rename the field to _iseq so
direct access causes a compile error, and rename the accessors to
drop the zjit prefix (rb_cfp_iseq, rb_cfp_has_iseq, rb_cfp_pc,
rb_cfp_has_pc).
Prism changed structure quite a bit. Most of the previously-public
structs are now opaque. This requires quite a bit of changes
internally.
It also triggered some unrelated changes, which were necessary
because Prism's main header used to pull in standard headers. So
box.c and jit.c are now no longer transitively getting all the
headers they needed.
Profiling of `ruby --disable-all -e 1` shows between 5 and 10%
of the time is spent in `_platform_memset`, called from `rb_gccct_clear_table()`,
itself called from `clear_method_cache_by_id_in_class()`.
Which makes sense, during boot we define numerous methods, and after
each we have to invalidate the `gccct`, which is done by zeroing
1023 pointers, so roughtly 8kiB of memory.
By keeping track of whether the table was used since the last clear,
we can save a lot of useless memory writes.
This may also be beneficial when loading user defined code, as it's unlikely
the `gccct` would be dirtied between two `def`.
NB: profiling such a short lived process gives a lot of variance, but
with a very high sampling rate and multiple attempts, `clear_method_cache_by_id_in_class`
is always in the top 2 hotspots.
For now the provided size is just for GC statistics, but in the future
we may want to forward it to C23's `free_sized` and passing an incorrect
size to it is undefined behavior.
One of the biggest remaining contention point is `RClass.cc_table`.
The logical solution would be to turn it into a managed object, so
we can use an RCU strategy, given it's read heavy.
However, that's not currently possible because the table can't
be freed before the owning class, given the class free function
MUST go over all the CC entries to invalidate them.
However if the `CC->klass` reference is weak marked, then the
GC will take care of setting the reference to `Qundef`.
When the inner code cannot represent the name of the locals in the
outer code, do not bother putting them into the constant pool as
they will not be referenced.
Fixes [Bug #20992]
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
We need to GC guard the iseq because the code above it malloc memory which
could trigger a GC. Since we only use ISEQ_BODY, the compiler may optimize
out the iseq local variable so we need to GC guard it.
Fixes: http://ci.rvm.jp/results/trunk_asan@ruby-sp1/5484752
If we have local variables outside of the eval, the local variables names
are IDs. We convert these IDs to char * using rb_id2name. However, these
char * are actually Ruby strings, which may be embedded. This means that
it is not safe to rb_id2name and call any potential GC entrypoints because
if a GC compaction runs, the embedded string may move and the pointer will
change.
For example, if you compile with `-DRGENGC_CHECK_MODE=1`, then the following
script will crash:
GC.auto_compact = :empty
GC.stress = true
o = Object.new
eval("def o.m(k: 0) k end")
The crash message is:
test.rb:6: [BUG] Local with constant_id 1 does not exist
ruby 3.4.0dev (2024-12-17T18:34:57Z prism-local-compac.. 434346726c) +PRISM [arm64-darwin24]
-- C level backtrace information -------------------------------------------
miniruby(rb_print_backtrace+0x24) [0x10312fec4] vm_dump.c:823
miniruby(rb_print_backtrace) (null):0
miniruby(rb_vm_bugreport+0x2d4) [0x1031301b8] vm_dump.c:1155
miniruby(rb_bug_without_die_internal+0xa8) [0x102dd6a94] error.c:1097
miniruby(rb_bug+0x28) [0x102dd6b00] error.c:1115
miniruby(pm_lookup_local_index+0xec) [0x102d61e4c] prism_compile.c:1237
miniruby(pm_compile_node+0x45d0) [0x102d252f4] prism_compile.c:9334
miniruby(pm_compile_node+0x1864) [0x102d22588] prism_compile.c:8650
miniruby(pm_compile_node+0x65ec) [0x102d27310] prism_compile.c:9897
miniruby(pm_compile_scope_node+0x3008) [0x102d77bcc] prism_compile.c:6584
miniruby(pm_compile_node+0x5ec4) [0x102d26be8] prism_compile.c:9768
miniruby(pm_iseq_compile_node+0xac) [0x102d20bf0] prism_compile.c:10069
miniruby(pm_iseq_new_with_opt_try+0x2c) [0x102e7d088] iseq.c:1029
miniruby(rb_protect+0x108) [0x102dea9bc] eval.c:1033
miniruby(pm_iseq_new_with_opt+0x264) [0x102e7c444] iseq.c:1082
miniruby(pm_iseq_new_eval+0xec) [0x102e7c8e0] iseq.c:961
miniruby(pm_eval_make_iseq+0x594) [0x1031209cc] vm_eval.c:1770
miniruby(eval_make_iseq+0x54) [0x103120068] vm_eval.c:1799
Luke Gruber