One of the biggest remaining contention point is `RClass.cc_table`.
The logical solution would be to turn it into a managed object, so
we can use an RCU strategy, given it's read heavy.
However, that's not currently possible because the table can't
be freed before the owning class, given the class free function
MUST go over all the CC entries to invalidate them.
However if the `CC->klass` reference is weak marked, then the
GC will take care of setting the reference to `Qundef`.
When the inner code cannot represent the name of the locals in the
outer code, do not bother putting them into the constant pool as
they will not be referenced.
Fixes [Bug #20992]
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
We need to GC guard the iseq because the code above it malloc memory which
could trigger a GC. Since we only use ISEQ_BODY, the compiler may optimize
out the iseq local variable so we need to GC guard it.
Fixes: http://ci.rvm.jp/results/trunk_asan@ruby-sp1/5484752
If we have local variables outside of the eval, the local variables names
are IDs. We convert these IDs to char * using rb_id2name. However, these
char * are actually Ruby strings, which may be embedded. This means that
it is not safe to rb_id2name and call any potential GC entrypoints because
if a GC compaction runs, the embedded string may move and the pointer will
change.
For example, if you compile with `-DRGENGC_CHECK_MODE=1`, then the following
script will crash:
GC.auto_compact = :empty
GC.stress = true
o = Object.new
eval("def o.m(k: 0) k end")
The crash message is:
test.rb:6: [BUG] Local with constant_id 1 does not exist
ruby 3.4.0dev (2024-12-17T18:34:57Z prism-local-compac.. 434346726c) +PRISM [arm64-darwin24]
-- C level backtrace information -------------------------------------------
miniruby(rb_print_backtrace+0x24) [0x10312fec4] vm_dump.c:823
miniruby(rb_print_backtrace) (null):0
miniruby(rb_vm_bugreport+0x2d4) [0x1031301b8] vm_dump.c:1155
miniruby(rb_bug_without_die_internal+0xa8) [0x102dd6a94] error.c:1097
miniruby(rb_bug+0x28) [0x102dd6b00] error.c:1115
miniruby(pm_lookup_local_index+0xec) [0x102d61e4c] prism_compile.c:1237
miniruby(pm_compile_node+0x45d0) [0x102d252f4] prism_compile.c:9334
miniruby(pm_compile_node+0x1864) [0x102d22588] prism_compile.c:8650
miniruby(pm_compile_node+0x65ec) [0x102d27310] prism_compile.c:9897
miniruby(pm_compile_scope_node+0x3008) [0x102d77bcc] prism_compile.c:6584
miniruby(pm_compile_node+0x5ec4) [0x102d26be8] prism_compile.c:9768
miniruby(pm_iseq_compile_node+0xac) [0x102d20bf0] prism_compile.c:10069
miniruby(pm_iseq_new_with_opt_try+0x2c) [0x102e7d088] iseq.c:1029
miniruby(rb_protect+0x108) [0x102dea9bc] eval.c:1033
miniruby(pm_iseq_new_with_opt+0x264) [0x102e7c444] iseq.c:1082
miniruby(pm_iseq_new_eval+0xec) [0x102e7c8e0] iseq.c:961
miniruby(pm_eval_make_iseq+0x594) [0x1031209cc] vm_eval.c:1770
miniruby(eval_make_iseq+0x54) [0x103120068] vm_eval.c:1799
If there's a syntax error during iseq compilation then prism would leak
memory because it would not free the pm_parse_result_t.
This commit changes pm_iseq_new_with_opt to have a rb_protect to catch
when an error is raised, and return NULL and set error_state to a value
that can be raised by calling rb_jump_tag after memory has been freed.
For example:
10.times do
10_000.times do
eval("/[/=~s")
rescue SyntaxError
end
puts `ps -o rss= -p #{$$}`
end
Before:
39280
68736
99232
128864
158896
188208
217344
246304
275376
304592
After:
12192
13200
14256
14848
16000
16000
16000
16064
17232
17952
* Remove 1 allocation in Enumerable#each_with_index
Previously, each call to Enumerable#each_with_index allocates 2
objects, one for the counting index, the other an imemo_ifunc passed
to `self.each` as a block.
Use `struct vm_ifunc::data` to hold the counting index directly to
remove 1 allocation.
* [DOC] Brief summary for usages of `struct vm_ifunc`
This function accepts flags:
RB_NO_KEYWORDS, RB_PASS_KEYWORDS, RB_PASS_CALLED_KEYWORDS:
Works as the same as rb_block_call_kw.
RB_BLOCK_NO_USE_PACKED_ARGS:
The given block ("bl_proc") does not use "yielded_arg" of rb_block_call_func_t.
Instead, the block accesses the yielded arguments via "argc" and "argv".
This flag allows the called method to yield arguments without allocating an Array.
This patch removes the `VALUE flags` member from the `rb_ast_t` structure making `rb_ast_t` no longer an IMEMO object.
## Background
We are trying to make the Ruby parser generated from parse.y a universal parser that can be used by other implementations such as mruby.
To achieve this, it is necessary to exclude VALUE and IMEMO from parse.y, AST, and NODE.
## Summary (file by file)
- `rubyparser.h`
- Remove the `VALUE flags` member from `rb_ast_t`
- `ruby_parser.c` and `internal/ruby_parser.h`
- Use TypedData_Make_Struct VALUE which wraps `rb_ast_t` `in ast_alloc()` so that GC can manage it
- You can retrieve `rb_ast_t` from the VALUE by `rb_ruby_ast_data_get()`
- Change the return type of `rb_parser_compile_XXXX()` functions from `rb_ast_t *` to `VALUE`
- rb_ruby_ast_new() which internally `calls ast_alloc()` is to create VALUE vast outside ruby_parser.c
- `iseq.c` and `vm_core.h`
- Amend the first parameter of `rb_iseq_new_XXXX()` functions from `rb_ast_body_t *` to `VALUE`
- This keeps the VALUE of AST on the machine stack to prevent being removed by GC
- `ast.c`
- Almost all change is replacement `rb_ast_t *ast` with `VALUE vast` (sorry for the big diff)
- Fix `node_memsize()`
- Now it includes `rb_ast_local_table_link`, `tokens` and script_lines
- `compile.c`, `load.c`, `node.c`, `parse.y`, `proc.c`, `ruby.c`, `template/prelude.c.tmpl`, `vm.c` and `vm_eval.c`
- Follow-up due to the above changes
- `imemo.{c|h}`
- If an object with `imemo_ast` appears, considers it a bug
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
This patch is part of universal parser work.
## Summary
- Decouple VALUE from members below:
- `(struct parser_params *)->debug_lines`
- `(rb_ast_t *)->body.script_lines`
- Instead, they are now `rb_parser_ary_t *`
- They can also be a `(VALUE)FIXNUM` as before to hold line count
- `ISEQ_BODY(iseq)->variable.script_lines` remains VALUE
- In order to do this,
- Add `VALUE script_lines` param to `rb_iseq_new_with_opt()`
- Introduce `rb_parser_build_script_lines_from()` to convert `rb_parser_ary_t *` into `VALUE`
## Other details
- Extend `rb_parser_ary_t *`. It previously could only store `rb_parser_ast_token *`, now can store script_lines, too
- Change tactics of building the top-level `SCRIPT_LINES__` in `yycompile0()`
- Before: While parsing, each line of the script is added to `SCRIPT_LINES__[path]`
- After: After `yyparse(p)`, `SCRIPT_LINES__[path]` will be built from `p->debug_lines`
- Remove the second parameter of `rb_parser_set_script_lines()` to make it simple
- Introduce `script_lines_free()` to be called from `rb_ast_free()` because the GC no longer takes care of the script_lines
- Introduce `rb_parser_string_deep_copy()` in parse.y to maintain script_lines when `rb_ruby_parser_free()` called
- With regard to this, please see *Future tasks* below
## Future tasks
- Decouple IMEMO from `rb_ast_t *`
- This lifts the five-members-restriction of Ruby object,
- So we will be able to move the ownership of the `lex.string_buffer` from parser to AST
- Then we remove `rb_parser_string_deep_copy()` to make the whole thing simple
[Feature #20205]
As a path toward enabling frozen string literals by default in the future,
this commit introduce "chilled strings". From a user perspective chilled
strings pretend to be frozen, but on the first attempt to mutate them,
they lose their frozen status and emit a warning rather than to raise a
`FrozenError`.
Implementation wise, `rb_compile_option_struct.frozen_string_literal` is
no longer a boolean but a tri-state of `enabled/disabled/unset`.
When code is compiled with frozen string literals neither explictly enabled
or disabled, string literals are compiled with a new `putchilledstring`
instruction. This instruction is identical to `putstring` except it marks
the String with the `STR_CHILLED (FL_USER3)` and `FL_FREEZE` flags.
Chilled strings have the `FL_FREEZE` flag as to minimize the need to check
for chilled strings across the codebase, and to improve compatibility with
C extensions.
Notes:
- `String#freeze`: clears the chilled flag.
- `String#-@`: acts as if the string was mutable.
- `String#+@`: acts as if the string was mutable.
- `String#clone`: copies the chilled flag.
Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
In preparation for https://bugs.ruby-lang.org/issues/20205.
The `frozen_string_literal` compilation option will no longer
be a boolean but a tri-state: `on/off/default`.
This `st_table` is used to both mark and pin classes
defined from the C API. But `vm->mark_object_ary` already
does both much more efficiently.
Currently a Ruby process starts with 252 rooted classes,
which uses `7224B` in an `st_table` or `2016B` in an `RArray`.
So a baseline of 5kB saved, but since `mark_object_ary` is
preallocated with `1024` slots but only use `405` of them,
it's a net `7kB` save.
`vm->mark_object_ary` is also being refactored.
Prior to this changes, `mark_object_ary` was a regular `RArray`, but
since this allows for references to be moved, it was marked a second
time from `rb_vm_mark()` to pin these objects.
This has the detrimental effect of marking these references on every
minors even though it's a mostly append only list.
But using a custom TypedData we can save from having to mark
all the references on minor GC runs.
Addtionally, immediate values are now ignored and not appended
to `vm->mark_object_ary` as it's just wasted space.
- Don't use `build_options_scopes` We can inline the code here instead
and avoid allocating all the extra arrays.
- Create `pm_scope_node_t` objects with the correct local table, for the
scope node returned from the parser.
Co-Authored-By: Kevin Newton <kddnewton@gmail.com>
Jean Boussier