long is 32 bits on mswin while pointers are 64 bits, so String, Array and MatchData could not reach 2GiB there. config.h now sets SIZEOF_RB_LEN_T to 8 on mswin64, and value.h then defines rb_len_t as int64_t. mingw still uses long.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit renames rb_str_tmp_frozen_no_embed_acquire to rb_str_no_gvl_safe_acquire
and makes it part of the public C API.
There is currently no way for C extensions that use no GVL operations to
safely read from a string because it has no way of guaranteeing that the
string is not embedded. Embedded strings are not safe for no GVL operations
because GC compaction can run, which will mprotect the memory region and
will cause any read to SEGV.
See https://github.com/ruby/ruby/pull/9817 for when rb_str_tmp_frozen_no_embed_acquire
was introduced.
The buffer is copied over based on the new string's termlen
so we can just set the new encoding raw instead of recalculating
based on the old encoding's termlen.
capa does not include the terminator, so when a buffer moves
between strings with different terminator lengths the capacity has to
be converted or we will try to free the buffer with the wrong size:
[BUG] buffer 0x00005030039ff768 freed with old_size=18, but was allocated with size=17
[BUG] buffer 0x00005631adc48f48 freed with old_size=8268, but was allocated with size=8269
Three related fixes:
* `rb_str_tmp_frozen_release`: convert tmp's capacity to orig's
terminator length when handing the buffer back. This is the inverse
of a calculation added to `rb_str_tmp_frozen_no_embed_acquire` in
91619f0230.
* `heap_str_make_shared`: since bdf3032e35
the root created by `rb_str_tmp_frozen_acquire` does not copy the
encoding of the string it takes the buffer from, so the two can have
different terminator lengths and the root's capacity has to be
adjusted as well, otherwise the root frees fewer bytes than were
allocated once its only child stops sharing the buffer. termlen is
passed in because the caller assigns the encoding of the root after
this function returns.
* `rb_str_tmp_frozen_no_embed_acquire`: unset `STR_BORROWED` on the
root. `STR_SET_SHARED` sets `STR_BORROWED` on roots with
`klass == 0` to mark the ones that may have other children, but this
root only ever has orig as its child, so unset it like
`heap_str_make_shared` does. Any string that starts sharing the
root later will set the flag again. Without this
`rb_str_tmp_frozen_release` never gives the buffer back, so strings
written by `io_fwrite` stay shared and are copied on their next
modification.
The release and make_shared adjustments go together
as they were previously canceling each other out.
45a2c95d0f mistakenly rewrote
FL_TEST_RAW(orig, STR_SHARED) && !FL_TEST_RAW(orig, STR_TMPLOCK|RUBY_FL_FREEZE)
as
FL_TEST_RAW(orig, STR_SHARED | STR_TMPLOCK) == STR_TMPLOCK
requiring orig to not be shared, the opposite of the original
condition. orig always shares the buffer with tmp here, so the buffer
was never given back and the string stayed shared until its next
modification copied it. The code then read aux.shared from strings
that are not shared, where the union holds aux.capa.
This should be backported to 4.0
We should cap the max length for embedding duplicated strings because it
could be costly in performance and memory usage for very large embedded
strings. Instead, we can use a view into the parent string and perform
copy-on-write.
This makes `RBobject` `4B` larger on 32 bit systems
but simplifies the implementation a lot.
[Feature #21353]
Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
[Bug #20585]
This was changed in 36a06efdd9 because
`String.new(1024)` would end up allocating `1025` bytes, but the problem
with this change is that the caller may be trying to right size a String.
So instead, we should just better document the behavior of `capacity:`.
[Feature #20205]
Now that chilled strings no longer appear as frozen, there is no
need to offer an API to check for chilled strings.
We however need to change `rb_check_frozen_internal` to no
longer be a macro, as it needs to check for chilled strings.
Some extensions (like stringio) may need to differentiate between
chilled strings and frozen strings.
They can now use rb_str_chilled_p but must check for its presence since
the function will be removed when chilled strings are removed.
[Bug #20389]
[Feature #20205]
Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
The documentation for `rb_enc_interned_str_cstr` notes that `enc` can be
a null pointer, but this currently causes a segmentation fault when
trying to autoload the encoding. This commit fixes the issue by checking
for NULL before calling `rb_enc_autoload`.
* Unify length field for embedded and heap strings
The length field is of the same type and position in RString for both
embedded and heap allocated strings, so we can unify it.
* Remove RSTRING_EMBED_LEN
Remove !USE_RVARGC code
[Feature #19579]
The Variable Width Allocation feature was turned on by default in Ruby
3.2. Since then, we haven't received bug reports or backports to the
non-Variable Width Allocation code paths, so we assume that nobody is
using it. We also don't plan on maintaining the non-Variable Width
Allocation code, so we are going to remove it.
This commit adds a `capacity` field to shapes, and adds shape
transitions whenever an object's capacity changes. Objects which are
allocated out of a bigger size pool will also make a transition from the
root shape to the shape with the correct capacity for their size pool
when they are allocated.
This commit will allow us to remove numiv from objects completely, and
will also mean we can guarantee that if two objects share shapes, their
IVs are in the same positions (an embedded and extended object cannot
share shapes). This will enable us to implement ivar sets in YJIT using
object shapes.
Co-Authored-By: Aaron Patterson <tenderlove@ruby-lang.org>
Add a new macro BASE_SLOT_SIZE that determines the slot size.
For Variable Width Allocation (compiled with USE_RVARGC=1), all slot
sizes are powers-of-2 multiples of BASE_SLOT_SIZE.
For USE_RVARGC=0, BASE_SLOT_SIZE is set to sizeof(RVALUE).
Fixes [Feature #13381]
When passed a `fake_str`, `register_fstring` would create new strings
with `str_new_static`. That's not what was expected, and answer
almost no use cases.
This modifies the following String methods to return String instances
instead of subclass instances:
* String#*
* String#capitalize
* String#center
* String#chomp
* String#chop
* String#delete
* String#delete_prefix
* String#delete_suffix
* String#downcase
* String#dump
* String#each/#each_line
* String#gsub
* String#ljust
* String#lstrip
* String#partition
* String#reverse
* String#rjust
* String#rpartition
* String#rstrip
* String#scrub
* String#slice!
* String#slice/#[]
* String#split
* String#squeeze
* String#strip
* String#sub
* String#succ/#next
* String#swapcase
* String#tr
* String#tr_s
* String#upcase
This also fixes a bug in String#swapcase where it would return the
receiver instead of a copy of the receiver if the receiver was the
empty string.
Some string methods were left to return subclass instances:
* String#+@
* String#-@
Both of these methods will return the receiver (subclass instance)
in some cases, so it is best to keep the returned class consistent.
Fixes [#10845]
This removes the related tests, and puts the related specs behind
version guards. This affects all code in lib, including some
libraries that may want to support older versions of Ruby.
The buffer deduplication codepath in rb_fstring can be used to free the buffer
of shared string roots, which leads to use-after-free.
Introudce a new flag to tag strings that at one point have been a shared root.
Check for it in rb_fstring to avoid freeing buffers that are shared by
multiple strings. This change is based on nobu's idea in [ruby-core:94838].
The included test case test for the sequence of calls to internal functions
that lead to this bug. See attached ticket for Ruby level repros.
[Bug #16151]
When a string is #frozen, it's capacity is resized to fit (if it is much
larger), since we know it will no longer be mutated.
> puts ObjectSpace.dump(String.new("a"*30, capacity: 1000))
{"type":"STRING", "class":"0x7feaf00b7bf0", "bytesize":30, "capacity":1000, "value":"...
> puts ObjectSpace.dump(String.new("a"*30, capacity: 1000).freeze)
{"type":"STRING", "class":"0x7feaf00b7bf0", "frozen":true, "bytesize":30, "value":"...
(ObjectSpace.dump doesn't show capacity if capacity is equal to bytesize)
Previously, if we dedup into an fstring, using String#-@, capacity would
not be reduced.
> puts ObjectSpace.dump(-String.new("a"*30, capacity: 1000))
{"type":"STRING", "class":"0x7feaf00b7bf0", "frozen":true, "fstring":true, "bytesize":30, "capacity":1000, "value":"...
This commit makes rb_fstring call rb_str_resize, the same as
rb_str_freeze does.
Closes: https://github.com/ruby/ruby/pull/2256
This is a follow up for 3f9562015e.
Before this commit, it was possible to create a shared string which
shares with another shared string by passing a frozen shared string
to `str_duplicate`.
Such string looks like:
```
-------- -----------------
| root | ------ owns -----> | root's buffer |
-------- -----------------
^ ^ ^
----------- | |
| shared1 | ------ references ----- |
----------- |
^ |
----------- |
| shared2 | ------ references ---------
-----------
```
This is bad news because `rb_fstring(shared2)` can make `shared1`
independent, which severs the reference from `shared1` to `root`:
```c
/* from fstr_update_callback() */
str = str_new_frozen(rb_cString, shared2); /* can return shared1 */
if (STR_SHARED_P(str)) { /* shared1 is also a shared string */
str_make_independent(str); /* no frozen check */
}
```
If `shared1` was the only reference to `root`, then `root` can be
reclaimed by the GC, leaving `shared2` in a corrupted state:
```
----------- --------------------
| shared1 | -------- owns --------> | shared1's buffer |
----------- --------------------
^
|
----------- -------------------------
| shared2 | ------ references ----> | root's buffer (freed) |
----------- -------------------------
```
Here is a reproduction script for the situation this commit fixes.
```ruby
a = ('a' * 24).strip.freeze.strip
-a
p a
4.times { GC.start }
p a
```
- string.c (str_duplicate): always share with the root string when
the original is a shared string.
- test_rb_str_dup.rb: specifically test `rb_str_dup` to make
sure it does not try to share with a shared string.
[Bug #15792]
Closes: https://github.com/ruby/ruby/pull/2159
FrozenError will be used instead of RuntimeError for exceptions
raised when there is an attempt to modify a frozen object. The
reason for this change is to differentiate exceptions related
to frozen objects from generic exceptions such as those generated
by Kernel#raise without an exception class.
From: Jeremy Evans <code@jeremyevans.net>
Signed-off-by: Urabe Shyouhei <shyouhei@ruby-lang.org>
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61131 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
