Add a new `Primitive.attr :always_inline`. Force inline ISEQs with this
attribute. This is helpful for e.g. `BasicObject#!=`, which is otherwise
megamorphic, and very small.
Other JITs also have a "always inline small methods" policy but I didn't
want to wade into that yet.
The bindings now declare rb_len_t as i64 on every platform, so the 32-bit branch of rb_len_mem can no longer be reached, and load_mem had no other caller.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
long is 32 bits on mswin while pointers are 64 bits, so String, Array and MatchData could not reach 2GiB there. config.h now sets SIZEOF_RB_LEN_T to 8 on mswin64, and value.h then defines rb_len_t as int64_t. mingw still uses long.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When gen_block_series() fails on a block after the first one, for
example because code memory is full, it frees the blocks it already
compiled with free_block(blockref, false). That skips
remove_from_graph(). A batch block can have a branch to a block
outside the batch, such as a JITReturn branch to an existing
continuation. That block keeps the freed branch in its incoming list.
Invalidating it later, as rb_yjit_invalidate_ep_is_bp() does, reads
the freed branch and crashes.
Free the batch blocks with graph_intact=true. They were compiled in
this call and every block they link to still exists, so it is safe to
unlink them.
The length fields of String and Array are rb_len_t, which is not necessarily long. The Fixnum range checks stay based on long. The bindings still declare rb_len_t as c_long, so the build now fails if rb_len_t gets wider without them, instead of YJIT reading only part of each length.
https://bugs.ruby-lang.org/issues/22400
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`make yjit-bindgen` and `make zjit-bindgen` pick the typedef up from the C API, so the Rust bindings are regenerated here too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nothing in it was specific to lengths, and building the C long memory operand inside saves each caller from spelling out its width.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixnum operands fit in 32 bits where long is 32-bit, so the 64-bit overflow check never fires there and the range check alone decides. Code generated where long is 64-bit is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
They emit instructions, which a call like asm.cmp_fixnum_fits_long(val) makes clearer than a free function taking asm.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bindgen now finds rb_jit_get_page_size and rb_jit_reserve_addr_space in internal/jit.h, which moves them in both generated files.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The disasm options and RubyVM::YJIT.disasm need a dev build, which win32/configure.bat cannot make. The warning for builds without YJIT also mentions --enable-yjit, since installing rustc alone does not enable it on mswin.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/tmp/perf-{pid}.map resolves to a usually missing directory on the current drive, so the first compilation panicked, and Linux perf could not read the map there anyway. --yjit-perf=fp is unaffected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
UCRT's isatty returns 0x40 for a character device, so the comparison with 1 never held and --yjit-dump-disasm was never colored. Checking for non-zero instead would also color output sent to NUL, which is a character device too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RtlUnwindEx(), which longjmp() and C++ exceptions use, needs unwind info for every frame, so raising through JIT frames skipped __finally blocks and C++ destructors. Exception dispatch also could not walk through JIT frames to reach the unhandled exception filter that the compaction read barrier relies on. Describe the entry frame, which now always has a frame pointer on Windows, for the whole region.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reserve the region with VirtualAlloc and commit pages when they become writable, in place of mmap and mprotect. Freed pages stay committed with their contents discarded, since rb_jit_mark_executable() covers the whole mapped region and VirtualProtect() fails on decommitted or reserved-only pages, and YJIT frees only the mapped part of the last code page for the same reason.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Generated code called C functions and Rust callbacks with the SysV convention. On Windows, pass the first four arguments in RCX, RDX, R8 and R9 and the rest on the stack above the 32-byte shadow space, and keep RSI and RDI out of the temp registers since they are callee-saved there, which limits --yjit-temp-regs to 3.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stats and counters can exceed the Fixnum range, which is 31 bits on Windows, and failed the assertion in VALUE::fixnum_from_usize.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Fixnum holds a C long, which is 32-bit on Windows, so an arithmetic result or a length that did not overflow 64 bits could still be out of the Fixnum range and make an unnormalized Fixnum. Side-exit there as each instruction does on a 64-bit overflow, with opt_mult chaining to send, and treat String#length as a possible Bignum, whose allocation needs the PC and SP saved.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The bindings are generated on LP64, where ID and st_data_t are unsigned long and long is 64-bit. On Windows long is 32-bit and both are unsigned long long, so pin them to u64, cast long arguments with the platform width, and widen the RString and RArray length fields before mixing them with 64-bit operands.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
std::os::unix::io::RawFd is not available on Windows. Leaking the opened File works everywhere and needs no fd round trip on each write.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
LoadSExt exists to widen a narrower operand, but load_sext typed its output with the operand width, so the result could not be mixed with 64-bit operands and the arm64 emitter passed a 32-bit destination to sxtw. The register allocator still sizes the out register by the operand, so both emitters widen it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
find_safe_move compared register sources with a destination including their width and ignored memory bases, so a C argument could be written into a register before another argument read it through a 32-bit view or loaded through it as its base. Compare reg_no for both, as ZJIT's reorder_reg_moves does for memory bases.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The assert panicked across the extern "C" boundary and aborted the process. Report the value like the other invalid ones, together with the limit, which the generic invalid option error does not show.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fallback in find_unused_reg searched all MAX_MAPPED_REGS slots, so fewer registers, as with --yjit-temp-regs=3, could get an index beyond them and panic with an out of bounds access in TEMP_REGS.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
We are not going to optimized switch-based dispatch with the hash table, at
least not yet, but we should let the interpreter profile the execution paths
before ZJIT runs them.
This should inject a lot more type information for the compiler. For example,
on rubyboy, we go from ~1080ms/iter to ~820ms/iter.
Later, we can make the dispatch faster too.
[Feature #22236]
Allows to give a time budget to YJIT. When YJIT goes over the budget
it pauses compilation until the budget is raised.
It is best effort, YJIT may go over, or have no choice but to compile
in some cases, but the goal is to be able to limit the CPU time overhead
of YJIT during warmup, so that it's spread over a longer period of
time and doesn't impact service latency as much.
The box of the immediately previous frame can be the root box, e.g. when
a proc made in the root box calls Marshal.load, and the classes of the
caller's data are invisible there. Walk the caller frames up to the
nearest user box instead, and rename the attribute after what it finds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Builtin methods written in Ruby are defined in the master box, so the
box resolution stops at their frames and cannot see classes defined in
the caller's box. The new attribute makes such methods operate on the
caller's box, as CFUNC frames already do.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Only the Ractor which created a class/module may modify it: define, remove or
undefine methods, alias, include/prepend, refine, define or remove constants,
register an autoload, and set the class's own instance variables. Reading is
unchanged. This replaces the assorted "main Ractor only" rules with one
creator-based rule, and in exchange a Ractor gets full use of the classes it
creates itself -- unshareable constant and instance variable values included,
which even the main Ractor could not do for its own classes before.
Three paths reach a class's tables without going through rb_class_modify_check,
so they get the check of their own: Module#refine (which writes its refinement
tables into the receiver as well as into the refined class), Class#initialize
(which writes the superclass of a not-yet-initialized class, and had no frozen
check either), and Module#ruby2_keywords.
The constant inline cache has to move with the rule. Its hit condition was
"shareable value, or we are the main Ractor", which was sound only because a
non-main Ractor could not have an unshareable constant in the first place. Now
it can, so an owner priming the cache would let any other Ractor read that value
straight out of it, never reaching the owner check in vm_get_ev_const. The
entry records the Ractor which filled it instead -- the one that passed the
check -- which also gives a non-main owner a fast path it never had.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ISEQ_TRANSLATED is defined through a chain of macros that ends in an
enum value (IMEMO_FL_USER3 -> FL_USER8 -> RUBY_FL_USER8), which bindgen
cannot evaluate, so YJIT and ZJIT had been defining the constant
manually in cruby.rs. ZJIT doesn't even use the constant, so drop it
there, and re-expose it for YJIT through enum yjit_bindgen_constants
in yjit.c so that the Rust side can no longer go out of sync with
iseq.h.
https://github.com/ruby/ruby/pull/18597#discussion_r3914719597
We are out of imemo types because IMEMO_MASK uses 4 bits and we have 16
imemo types. This makes it not possible to experiment with new imemo types,
so we can increase it by one bit to have 16 more imemo types.
Use the profiled splat array length to expand caller splats into fixed
positional arguments for SendDirect. Guard the runtime length and fall
back to the original Send when it changes or ruby2_keywords semantics
apply.
Validate direct sends using the expanded argument count, then apply the
existing keyword setup and rest parameter packing. Move the
ruby2_keywords splat predicate into shared JIT code so both YJIT and
ZJIT can use it.
This commit adds support for using AR tables for identity hashes (hashes
created by method Hash#compare_by_identity). Before this change, identity
hashes always used ST tables. However, this used a lot more memory because
ST tables are quite large. This commit adds a new flag RHASH_COMPARE_BY_IDENTITY
for AR tables that are compare-by-identity.
For example, for an empty identity hash, the size goes from 256 bytes to
160 bytes:
require "objspace"
h = {}.compare_by_identity
puts ObjectSpace.memsize_of(h)
