iseq.h does not reference any of the pm_* types or functions declared in
prism_compile.h, so include the latter from the files that actually use
it instead. Only 6 of the 19 files including iseq.h need the prism
declarations, so this keeps prism/prism.h out of the include closure of
ast.c, box.c, builtin.c, cont.c, eval.c, jit.c, proc.c, thread.c,
vm_dump.c, vm_trace.c, yjit.c, zjit.c and internal/jit.h.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
These still held String and Array lengths and indices in long, or cast them to long. rb_len_t is still long everywhere, so the generated code does not change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Argument counts stay `int` or `long`. The int range checks in proc.c and compile.c now depend on SIZEOF_VALUE so that they are kept once rb_len_t widens.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Line numbers and ioctl/fcntl/syscall arguments stay `long`, including the request size setup_narg() asks narg_len() for, which is named apart from the string length it is compared against.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Make `__FILE__` a frozen core method that return the iseq path.
* Make `load_from_binary` accept `file, path` like compile.
[Feature #17593]
[Feature #22395]
The JIT optimizer sees an `invokebuiltin` and would like to assume that calling it does not write back to the frame. A `Primitive` should never write to the frame by taking a pointer to a local, so this PR removes that functionality.
Partial revert of https://github.com/ruby/ruby/commit/bb8924240d329091eaa38f46791036ac13ccd3ed.
----
* Don't allow writing to locals in cexpr
This makes the few remaining cases of `cexpr`-that-looks-at-locals a
read-only view.
* Remove uses of LOCAL_PTR from ractor.rb
Use two new builtin functions instead.
* Remove LOCAL_PTR from mk_builtin_loader
* Don't bother making the intermediate pointer
* Revert bb8924240d
* Add Primitive.local_self!
[Bug #22334]
If we return a with a keyword splat, then each element does not necessarily
push two values onto the stack, so we cannot compile this as a simple sequence
of key-value pairs combined with newhash. The following script reproduces
the issue and crashes:
def r
return **{foo: 1}
end
puts r()
Rather than start from a mutable hash created with `newhash` or
`duphash`, and then iteratively insert into it, it's preferable to
start from a constant hidden hash, so that we can directly allocate
the final hash with the right size.
For instance:
```ruby
{a: 1, B => 2, c: 3}
```
Before:
```
0000 duphash {a: 1} ( 1)[Li]
0002 putspecialobject 1
0004 swap
0005 opt_getconstant_path <ic:0 B>
0007 putobject 2
0009 putobject :c
0011 putobject 3
0013 opt_send_without_block <calldata!mid:core#hash_merge_ptr, argc:5, ARGS_SIMPLE>
```
After:
```
0000 putobject {a: 1} ( 1)[Li]
0002 putspecialobject 1
0004 swap
0005 opt_getconstant_path <ic:0 B>
0007 putobject 2
0009 putobject :c
0011 putobject 3
0013 opt_send_without_block <calldata!mid:core#hash_merge_ptr, argc:5, ARGS_SIMPLE>
```
Since, `duphash` has no idea how large the final hash might be,
it can't right-size the hash it allocates.
Hence it's preferable to use a `putobject` and let `hash_merge_ptr`
compute the final hash right size by assuming all keys are unique.
The branch coverage structure was keyed by the address of the AST node,
which is needed to let multiple compilations of the same node (e.g., an
ensure clause) share one entry [Bug #16967]. However, ASTs are freed
after compilation, so when code is eval'ed against the same path, a
reused node address could merge unrelated branches into one entry,
nondeterministically.
Key the structure by [source_hash, node_id, first_lineno] instead.
This is deterministic: different code eval'ed at the same path always
gets separate entries, and re-evaluating the very same code at the same
path and line accumulates the counters, like line coverage does.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
After each prism parse, compute a source hash of the parsed source and
keep it in the scope node. The data section after an __END__ marker is
not part of the code, so the hash covers the source only up to the end
of the __END__ line, which matches the range that parse.y hashes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
If number of keyword arguments are not taken into account when comparing CI
shapes, invalid bindings happen:
```
class Foo
def bar(a: nil, b: nil)
p [:body, a: a, b: b]
end
end
foo = Foo.new
foo.bar(a: 1, b: 2) #=> [:body, {a: 1, b: 2}] # prime CC
foo.bar(99, a: 0) #=> [:body, {a: 99, b: nil}] # BAD!
```
Also, set max keywords to 2**16-1 so we don't grow the cc_entries_entry struct.
Fixes [Bug #22099]
pm_setup_args mallocs one kw_arg buffer with references = 0. The inline_new branch then feeds that same pointer
to three callinfo constructions:
- 3789 — new_callinfo(... method_id="new" ..., kw_arg, 0) -> the opt_new ci
- 3795 — PUSH_SEND_R(... "initialize", ..., flags | VM_CALL_FCALL, kw_arg)
- 3800 — PUSH_SEND_R(... method_id="new", ..., flags, kw_arg) (fallback)
Tracing the refcount through rb_vm_ci_lookup()
1. It increments kwarg->references and allocates a fresh new_ci before the dedup st_update
2. The dedup (vm_ci_hash_cmp) compares kwarg contents, not the pointer. So if an earlier line already interned a new ci with the
identical keyword set, st_update returns that pre-existing ci (which holds a different buffer) and discards our new_ci.
3. Our kw_arg is now orphaned: references == 1, but the only holder is the discarded new_ci, which is a normal collectable imemo
4. An allocation like PUSH_INSN2 opt_new at or new_callinfo() can trigger a GC. References back to 0, kw_arg buffer freed.
5. new_callinfo() using the freed buffer, does argc += kw_arg->keyword_len (use-after-free)
The fix: Keep the buffer alive across the allocations in inline_new.
Fixes [Bug #22104]
RHash isn't a good fit for storing `cdhash` as this force to allow
arbitrary hash types into RHash, which doesn't work with AR tables.
It also cause the cdhash to be larger than needed.
As well as `putchilledstring` as `dupchilledstring`.
This is more consistent with similar `duparray` and `duphash`
instructions and better reflect it's behavior.
The value is always needed. Now prism emits the same instructions as parse.y for the added test case.
`defined?`/`flip-flop` caused `argument stack underflow`, `and`/`or` segfaulted during runtime.
When we allocate a RHash using `rb_hash_new_capa()`, if `capa` is
larger than `8` it's directly allocated as an `st_stable` in a `80B`
slot.
However if the requested size if lesser or equal to 8, we allocate
it as an `ar_table` in a `160B` slot.
Since most hashes are allocated as mutable, we have to be able to
accomodate as much as 8 AR_TABLE entries regardless.
However there are case where we know the Hash won't ever be resized,
that notably the case of all the "literal" hashes allocated by the
compiler.
These are immediately frozen and hidden upon being constructed, hence
we can know for sure they won't ever be resized. This allows us to
allocate the smaller ones in smaller slots.
```
size: 0, slot_size: 32
size: 1, slot_size: 48
size: 2, slot_size: 64
size: 3, slot_size: 80
size: 4, slot_size: 96
size: 5, slot_size: 112
size: 6, slot_size: 128
size: 7, slot_size: 144
size: 8, slot_size: 160
```
```ruby
require "objspace"
p ObjectSpace.memsize_of({}.freeze) # => 40
p ObjectSpace.memsize_of({a: 1}.freeze) # => 80
p ObjectSpace.memsize_of({a: 1, b: 2}.freeze) # => 80
p ObjectSpace.memsize_of({a: 1, b: 2, c: 3}.freeze) # => 80
p ObjectSpace.memsize_of({a: 1, b: 2, c: 3, d: 4}.freeze) # => 160
p ObjectSpace.memsize_of({a: 1, b: 2, c: 3, d: 4, e: 5, }.freeze) # => 160
p ObjectSpace.memsize_of({a: 1, b: 2, c: 3, d: 4, e: 5, f: 6}.freeze) # => 160
p ObjectSpace.memsize_of({a: 1, b: 2, c: 3, d: 4, e: 5, f: 6, g: 7}.freeze) # => 160
p ObjectSpace.memsize_of({a: 1, b: 2, c: 3, d: 4, e: 5, f: 6, g: 7, h: 8}.freeze) # => 160
```
The inline constant cache previously used `RCLASS_SINGLETON_P` to detect
"unstable" CREFs that need ic_cref stored and checked on every IC hit.
This caused the `class << self` pattern to create inline caches which
requires extra checks and can't optimize as well by the JITs.
We can avoid this by replacing the `RCLASS_SINGLETON_P` check with a
`VM_DEFINECLASS_FLAG_DYNAMIC_CREF` flag added to the defineclass
instruction at compile time to indicate a dynamic class scope and
specifically avoid setting it for the `class << self` pattern.
We can apply the same logic to fix dynamic CREF on `module (expr)::Foo`.
We can say that any class definition which uses only constant references
is stable, or at least as stable as the cref it was declared inside.
[Bug #20948]
pm_parse_process initializes the index_lookup_table but nothing seems to
use it after it has been allocated. However, pm_compile_scope_node will
overwrite the index_lookup_table and cause it to leak memory. This can
be seen during bootup with the following memory leaks reported by ASAN:
#0 0x60dba31b7af3 in malloc
#1 0x60dba32e0718 in rb_gc_impl_malloc gc/default/default.c:8287:5
#2 0x60dba32c7aa7 in ruby_xmalloc_body gc.c:5373:12
#3 0x60dba32c4a54 in ruby_xmalloc gc.c:5355:34
#4 0x60dba3260314 in pm_index_lookup_table_init_heap prism_compile.h:89:29
#5 0x60dba3209388 in pm_parse_process prism_compile.c:11366:5
Prism changed structure quite a bit. Most of the previously-public
structs are now opaque. This requires quite a bit of changes
internally.
It also triggered some unrelated changes, which were necessary
because Prism's main header used to pull in standard headers. So
box.c and jit.c are now no longer transitively getting all the
headers they needed.
For now the provided size is just for GC statistics, but in the future
we may want to forward it to C23's `free_sized` and passing an incorrect
size to it is undefined behavior.
In the C API, we want to use slices instead of locations in the
AST. In this case a "slice" is effectively the same thing as the
location, expect it is represented using a 32-bit offset and a
32-bit length. This will cut down on half of the space of all of
the locations in the AST.
Note that from the Ruby/Java/JavaScript side, this is effectively
an invisible change. This only impacts the C/Rust side.
Kevin Newton