[PATCH] Reject overflowed repeat/null-check IDs in huge regexps (#16801)
Add explicit errors for excessive range-repeat and null-check IDs;
map new errors in regerror; add regression tests for oversized generated
patterns.
Concatenated large repeats reach a program past INT_MAX from a few hundred
kilobytes of source, and the offsets emitted into it wrap. Cap the size
while the program is emitted, which does not depend on the length pass that
overflows along with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
[PATCH] [Bug #21842] Let `rb_interned_str` return US-ASCII if possible
[PATCH] [DOC] Update docs for rb_interned_str and related functions (#15897)
Related to [Bug #21842].
* rb_interned_str: document what decides whether the returned string is
in US-ASCII or BINARY encoding.
* rb_interned_str_cstr: include the same description as rb_interned_str
for the encoding. This one was still missing the update for US-ASCII
and erroneously said the returned string was alwasy in BINARY encoding
* rb_str_to_interned_str: document how the encoding of the result is
defined.
Co-authored-by: Herwin <herwinw@users.noreply.github.com>
[PATCH] Rename to `struct rbimpl_size_overflow_tag`
This struct is used for addition not only for multiplication, so
remove the word `mul`, and make the member names more descriptive.
Suppress gcc 15 unterminated-string-initialization warnings
Separate `__has_attribute` from `defined(__has_attribute)`
Fix Visual C warnings:
```
regenc.h(121): warning C4067: unexpected tokens following preprocessor directive - expected a newline
```
Cast up `int` instruction code to `VALUE`
Fix Visual C warnings:
```
iseq.c(3793): warning C4312: 'type cast': conversion from 'int' to 'void *' of greater size
iseq.c(3794): warning C4312: 'type cast': conversion from 'int' to 'void *' of greater size
```
Do not let files depend on a phony target
Detect `clock_gettime` and `clock_getres` for winpthreads
As reported in <https://bugs.ruby-lang.org/issues/21340>, older autoconf
have an AC_HEADER_STDBOOL that's incompatible with C23. Autoconf 2.72
fixed the macro, but also mentions that it's obsolescent since all
current compilers have this header.
Since we require C99 [1] and VS 2015 [2], should be able take
that suggestion and include stdbool.h without a check. This worked
fine on rubyci.org.
[Backport #21340]
[1]: https://bugs.ruby-lang.org/issues/15347
[2]: https://bugs.ruby-lang.org/issues/19982
Fix redefinition of `clock_gettime` and `clock_getres`
winpthreads-git 12.0.0.r720 provides `clock_gettime` and
`clock_getres` as inline functions.
digest.so needs ruby/digest.h which is installed by build-ext
Copy to path with the base name
The macro MAYBE_UNUSED, prepared by ./configure, may not be defined in
some environments such as Oracle Developer Studio 12.5 on Solaris 10.
This fixes [Bug #20963]
Explaining this by reference to rb_id2str() obscures a few important
details because IDs and symbols don't map to each other perfectly (you
can have a dynamic symbol without an ID!) Also, it used to take 2
redirections to get to concrete information, and I think being more
direct is friendlier.
* Use FL_USER0 for ELTS_SHARED
This makes space in RString for two bits for chilled strings.
* Mark strings returned by `Symbol#to_s` as chilled
[Feature #20350]
`STR_CHILLED` now spans on two user flags. If one bit is set it
marks a chilled string literal, if it's the other it marks a
`Symbol#to_s` chilled string.
Since it's not possible, and doesn't make much sense to include
debug info when `--debug-frozen-string-literal` is set, we can't
include allocation source, but we can safely include the symbol
name in the warning message, making it much easier to find the source
of the issue.
Co-Authored-By: Étienne Barrié <etienne.barrie@gmail.com>
---------
Co-authored-by: Étienne Barrié <etienne.barrie@gmail.com>
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
If `long` and `int` are the same size, `unsigned int` max would exceed
`signed long` range. It is guaranteed by `RB_POSFIXABLE` that `v` can
be casted to `long` safely here.
to show unused block warning strictly.
```ruby
class C
def f = nil
end
class D
def f = yield
end
[C.new, D.new].each{|obj| obj.f{}}
```
In this case, `D#f` accepts a block. However `C#f` doesn't
accept a block. There are some cases passing a block with
`obj.f{}` where `obj` is `C` or `D`. To avoid warnings on
such cases, "unused block warning" will be warned only if
there is not same name which accepts a block.
On the above example, `C.new.f{}` doesn't show any warnings
because there is a same name `D#f` which accepts a block.
We call this default behavior as "relax mode".
`strict_unused_block` new warning category changes from
"relax mode" to "strict mode", we don't check same name
methods and `C.new.f{}` will be warned.
[Feature #15554]
gcc 14 for aarch64 with `-O3` may emit a false positive warning for a
pointer access of `RB_BUILTIN_TYPE` called from `RB_TYPE_P`. `Qfalse`
shouldn't get there because of `RB_SPECIAL_CONST_P`, but the optimizer
seems to ignore this condition in some cases (`ASSUME` just before the
access doesn't seem to have any effect either). Only by reversing the
order in `RB_SPECIAL_CONST_P` to compare with 0 first does the warning
seem to go away.
```
../../.././include/ruby/internal/special_consts.h:349:36: error: conversion to ‘VALUE’ {aka ‘long unsigned int’} from ‘int’ may change the sign of the result [-Werror=sign-conversion]
349 | return RB_SPECIAL_CONST_P(obj) * RUBY_Qtrue;
| ~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
```
[Bug #20650]
The capture group allocates memory that is leaked when it times out.
For example:
re = Regexp.new("^#{"(a*)" * 10_000}x$", timeout: 0.000001)
str = "a" * 1000000 + "x"
10.times do
100.times do
re =~ str
rescue Regexp::TimeoutError
end
puts `ps -o rss= -p #{$$}`
end
Before:
34688
56416
78288
100368
120784
140704
161904
183568
204320
224800
After:
16288
16288
16880
16896
16912
16928
16944
17184
17184
17200
Since 730e3b2ce0
("Stop exposing `rb_str_chilled_p`"), we noticed a speed loss on a few
benchmarks that are string operations heavy. This is partially due to
routines no longer having the options to inline rb_check_frozen_inline()
in non-LTO builds. Make it an inlining candidate again to recover speed.
Testing this patch on my machine, the fannkuchredux benchmark gets a
1.15 speed-up with YJIT and 1.03 without YJIT.
This feature is no longer possible under current design; now that our GC
is pluggable, we cannot assume what was achieved by this compiler flag
is always possble by the dynamically-loaded GC implementation.
This function accepts flags:
RB_NO_KEYWORDS, RB_PASS_KEYWORDS, RB_PASS_CALLED_KEYWORDS:
Works as the same as rb_block_call_kw.
RB_BLOCK_NO_USE_PACKED_ARGS:
The given block ("bl_proc") does not use "yielded_arg" of rb_block_call_func_t.
Instead, the block accesses the yielded arguments via "argc" and "argv".
This flag allows the called method to yield arguments without allocating an Array.
Previously, RBIMPL_ASSERT_TYPE() used Check_Type() only in RUBY_DEBUG
builds. It raised TypeError, but only in debug builds. For people testing
type mismatch using debug builds looking for a Ruby exception, this can
be misleading -- the code could be missing a type check in non-debug builds
if it is relying on for example RSTRING_LEN() to raise.
Also, Check_Type() can obscure the true cause of error in debug mode.
When type check fails because the object is corrupt, instead of crashing
with a clear type assertion message, it can crash while trying to
construct an exception object to raise. You can see this for example in
<https://github.com/ruby/ruby/actions/runs/9489999591/job/26152506434?pr=10985>,
where RB_ENCODING_GET() is used on a corrupt object, but the crash
happens later and says "Assertion Failed:
../src/vm_method.c:1477:callable_method_entry_or_negative".
RBIMPL_ASSERT_TYPE() should assert right away.
RBIMPL_ASSERT_OR_ASSUME() asserts when RUBY_DEBUG and assumes in release
builds, as desired.
This should help investigate flaky CI failures that show up as TypeError
from `Kernel#require`, e.g.
"'Kernel#require': wrong argument type false (expected String) (TypeError)".
Same CI failure examples:
- https://github.com/ruby/ruby/actions/runs/9034787861/job/24828147431
- https://github.com/ruby/ruby/actions/runs/9418303667/job/25945492440
- https://github.com/ruby/ruby/actions/runs/9505650952/job/26201031314
The failure occurs with and without use of YJIT.
[Feature #20205]
Now that chilled strings no longer appear as frozen, there is no
need to offer an API to check for chilled strings.
We however need to change `rb_check_frozen_internal` to no
longer be a macro, as it needs to check for chilled strings.
[Feature #20507]
This was missed from the initial commit.
```
../../.././include/ruby/internal/value_type.h:446:27: error: implicit conversion changes signedness: 'enum ruby_value_type' to 'int' [-Werror,-Wsign-conversion]
rb_unexpected_type(v, t);
~~~~~~~~~~~~~~~~~~ ^
```
nagachika