These still held String and Array lengths and indices in long, or cast them to long. rb_len_t is still long everywhere, so the generated code does not change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pattern of allocating an empty hash with `rb_hash_new()` and
then appending to it is relatively wasteful, because every time
we go over the hash capacity, we'll have to `realloc`, rehash etc.
It may even get worse if https://github.com/ruby/ruby/pull/18345
lands, as right now an empty hash starts with a capacity of 8,
but it may very well start with a capacity of 2.
Hence in cases where the final hash size is easy to predict, we
should use `rb_hash_new_capa()`.
enc_list_update makes the frozen encoding list shareable by setting
RUBY_FL_SHAREABLE directly. Use the public RB_OBJ_SET_SHAREABLE() helper
instead, which is the intended way to mark an object shareable
(rb_obj_set_shareable). No behavioural change on the shared interpreter;
it keeps the encoding-list sharing off the raw-flag path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Set RUBY_TYPED_THREAD_SAFE_FREE on TypedData types whose dfree function
is trivially safe, and only frees its own memory
Types not yet marked as THREAD_SAFE_FREE:
- id2ref_tbl_type: sets the process-global id2ref_tbl to NULL.
- mutex_data_type: unlinks itself from another thread's keeping_mutexes list.
- autoload_data_type: deletes nodes from a shared intrusive list.
- autoload_const_type: deletes itself from a shared list.
- rb_cont_data_type: mutates the shared fiber pool and the global first_jit_cont list.
- rb_fiber_data_type: delegates to cont_free.
- FiberPoolDataType: mutates shared fiber pool free-lists.
- ractor_data_type: reconfigures global VM event-hook flags and call caches.
- exported_object_registry: frees a global table under the VM lock.
- rb_box_data_type (Box::Entry): unlinks classext from other live class and module objects.
- box_ext_cleanup_type: dereferences another String object and calls unlink.
- monitor_data_type: uses the default free, so the flag has no effect.
Co-authored-by: Luke Gruber <luke.gruber@shopify.com>
When debugging the fstring table, I found "UTF-8" to be the most common
interned strings in many benchmarks.
We have a fixed, limited number of these strings, so we might as well permanently
cache their fstrings.
to adopt strict shareable rule.
* (basically) shareable objects only refer shareable objects
* (exception) shareable objects can refere unshareable objects
but should not leak reference to unshareable objects to Ruby world
Previously we were looping over the enc_table, but when I added an
assertion the only thing that loop was doing is the equivalent of
ENC_TO_ENCINDEX(base). However we don't even need the index of base.
Instead we should be able to just use the badirectly.
Previously we would sometimes call enc_register_at several times in
order to update the encoding values after the base encoding may have
been loaded.
This updates enc_register_at to only be used via enc_register, when an
actual new encoding at a new index is being registered.
Other callers (which in all cases found the index by the name matching)
now call enc_load_from_base which is only responsibly for loading the
encoding from the base values.
Make sure VM lock is not held when calling `load_transcoder_entry`, as
that causes deadlock inside ractors. `String#encode` now works inside
ractors, among others.
Atomic load the rb_encoding_list
Without this, wbcheck would sometimes hit a missing write barrier.
Co-authored-by: John Hawthorn <john.hawthorn@shopify.com>
Hold VM lock when iterating over global_enc_table.names
This st_table can be inserted into at runtime when autoloading
encodings.
minor optimization when calling Encoding.list
Using `encoding->max_enc_len` as a way to check if the encoding
has been loaded isn't atomic, because it's not atomically set
last.
Intead we can use a dedicated atomic value inside the encoding table.
None of the datastructures involved in the require process are
safe to call on a secondary ractor, however when autoloading
encodings, we do so from the current ractor.
So all sorts of corruption can happen when using an autoloaded
encoding for the first time from a secondary ractor.
This fixes segfaults and errors of the type "Encoding not found" when
using encoding-related methods and internal encoding c functions across
ractors.
Example of a possible segfault in release mode or assertion error in debug mode:
```ruby
rs = []
100.times do
rs << Ractor.new do
"abc".force_encoding(Encoding.list.shuffle.first)
end
end
while rs.any?
r, obj = Ractor.select(*rs)
rs.delete(r)
end
```
While profiling msgpack-ruby I noticed a very substantial amout of time
spent in `rb_enc_associate_index`, called by `rb_utf8_str_new`.
On that benchmark, `rb_utf8_str_new` is 33% of the total runtime,
in big part because it cause GC to trigger often, but even then
`5.3%` of the total runtime is spent in `rb_enc_associate_index`
called by `rb_utf8_str_new`.
After closer inspection, it appears that it's performing a lot of
safety check we can assert we don't need, and other extra useless
operations, because strings are first created and filled as ASCII-8BIT
and then later reassociated to the desired encoding.
By directly allocating the string with the right encoding, it allow
to skip a lot of duplicated and useless operations.
After this change, the time spent in `rb_utf8_str_new` is down
to `28.4%` of total runtime, and most of that is GC.
[Bug #20598]
Just like [Bug #20595], Encoding#name_list and Encoding#aliases can have
their strings corrupted when Encoding.default_internal is set to nil.
Co-authored-by: Matthew Valentine-House <matt@eightbitraptor.com>
[Bug #20595]
enc_set_default_encoding will free the C string if the encoding is nil,
but the C string can be used by the encoding name string. This will cause
the encoding name string to be corrupted.
Consider the following code:
Encoding.default_internal = Encoding::ASCII_8BIT
names = Encoding.default_internal.names
p names
Encoding.default_internal = nil
p names
It outputs:
["ASCII-8BIT", "BINARY", "internal"]
["ASCII-8BIT", "BINARY", "\x00\x00\x00\x00\x00\x00\x00\x00"]
Co-authored-by: Matthew Valentine-House <matt@eightbitraptor.com>
[Feature #18576]
Since outright renaming `ASCII-8BIT` is deemed to backward incompatible,
the next best thing would be to only change its `#inspect`, particularly
in exception messages.
This `st_table` is used to both mark and pin classes
defined from the C API. But `vm->mark_object_ary` already
does both much more efficiently.
Currently a Ruby process starts with 252 rooted classes,
which uses `7224B` in an `st_table` or `2016B` in an `RArray`.
So a baseline of 5kB saved, but since `mark_object_ary` is
preallocated with `1024` slots but only use `405` of them,
it's a net `7kB` save.
`vm->mark_object_ary` is also being refactored.
Prior to this changes, `mark_object_ary` was a regular `RArray`, but
since this allows for references to be moved, it was marked a second
time from `rb_vm_mark()` to pin these objects.
This has the detrimental effect of marking these references on every
minors even though it's a mostly append only list.
But using a custom TypedData we can save from having to mark
all the references on minor GC runs.
Addtionally, immediate values are now ignored and not appended
to `vm->mark_object_ary` as it's just wasted space.
There is a memory leak in Encoding.default_external= and
Encoding.default_internal= because the duplicated name is not freed
when overwriting.
10.times do
1_000_000.times do
Encoding.default_internal = nil
end
puts `ps -o rss= -p #{$$}`
end
Before:
25664
41504
57360
73232
89168
105056
120944
136816
152720
168576
After:
9648
9648
9648
9680
9680
9680
9680
9680
9680
9680
when the RUBY_FREE_ON_SHUTDOWN environment variable is set, manually free memory at shutdown.
Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>
Co-authored-by: Peter Zhu <peter@peterzhu.ca>
