`nth_char` and the four `SliceableSequenceOp` methods each walked the buffer
to reach a character index, so `s[i]` and `s[a:b]` on a non-ASCII string were
O(i) and O(b), and a loop over either was quadratic. They now resolve through
`char_index_to_byte`, which is what the index table was added for: a plain
slice becomes a byte reslice, and a stepped slice one lookup per collected
character.
An index within four code points of an end is still walked to, and so is a
slice that reaches within four of both. PyPy draws the same line with
`MAX_UNROLL_NEXT_CODEPOINT_POS`, in a guard that also asks the JIT whether the
index is a constant; there is no JIT here, but the reason to skip the build
survives it -- `s[0]` on a long string should not pay for a table.
The stepped slices took their character count from `(range.len() / step) + 1`,
which overshoots whenever the last step lands short: `"aéc"[::3]` reported a
length of 2 for a one-character string, and `reversed()` on it read past the
end of the buffer and panicked. The count is now the index iterator's own
length, so it cannot drift from the characters actually collected.
Assisted-by: Claude
* common: index a WTF-8 buffer's code points for random access
`Wtf8`'s iterators are sequential, so resolving a code point index through
them is O(n) and code that indexes the same string repeatedly walks it once
per index. `Wtf8Index` is a side table -- one 24-byte group per 64 code
points, 0.375 bytes per code point -- that answers the same question in
constant time; the layout is PyPy's `UTF8_INDEX_STORAGE`.
`StrData` builds one on the first call to the new `char_index_to_byte`, in a
slot published by compare-exchange, and drops it with the string. ASCII
strings answer from the index itself and never build a table. A clone gets an
empty slot, since it indexes its own copy of the buffer.
Assisted-by: Claude
* _sre: drive a non-ASCII str subject through the string's character index
The `&Wtf8` drive answers `count` and `create_cursor` by decoding from the
start of the subject, so a scan that restarts at successive positions walks
the subject once per position, and `slice` walks it again per extracted
group. `Utf8Str` holds the `PyStr` and asks it instead: `count` is the cached
character length, and `create_cursor` and `slice` resolve their positions
through `char_index_to_byte`. Stepping is the `&Wtf8` drive's, unchanged.
The table lives on the string, so a `Match` that outlives the scan shares it
-- `group` has no cursor of its own to move relative to.
`SreStr for &Wtf8` has no callers left; the `StrDrive` impl stays, since
`Utf8Str` steps through it. The three subject helpers now share one downcast.
Assisted-by: Claude
`do_stepped_slice` and `do_stepped_slice_reverse` took the character count of
a non-ASCII result as `(range.len() / step) + 1`, which overshoots by one
whenever the span is an exact multiple of the step: `"aéc"[::3]` is one
character and reported two, `"가나다라"[::2]` two and reported three. ASCII
subjects were unaffected, since that arm collects into an `AsciiString` whose
length comes from the data.
The count is stored as the string's character length, so the result then
claimed a character its buffer does not hold, and `reversed()` on it indexed
past the end and panicked:
s = "".join(["a", "é", "c"])
list(reversed(s[::3])) # index out of bounds: the len is 1 but the index is 1
Use `div_ceil`, which is the number of elements the underlying range yields.
A differential over 44331 subscript and slice shapes -- every combination of
start, stop and step over strings straddling the ASCII split, the surrogate
range and the astral plane -- now matches CPython 3.14 exactly, where it
differed on 3100 lines before.
Assisted-by: Claude
* sre_engine: document that StrDrive positions are character indices
StrDrive carried no documentation, so nothing recorded that a cursor's
position is a character index rather than a byte offset. Every implementation
satisfies it -- skip(n) advances position by exactly n in all three -- and the
engine depends on it: _count bounds a repeat with position + max_count and
reports the repeat length as a difference of positions, ASSERT compares
position against a lookbehind width, and search_info recovers a match start as
position - (len - 1). A drive over a variable-width encoding that stored byte
offsets would leave those type-correct and wrong, and would index a lookbehind
out of bounds.
Write the invariant down on StringCursor and on each trait method.
Also walk the group's own cursor in GROUPREF instead of counting to the
group's width, so the loop bound is the thing being stepped. Generated
machine code is unchanged.
Assisted-by: Claude
* _sre: drive an all-ASCII str subject over its bytes
with_sre_str handed every str subject to the &Wtf8 drive, which answers
count() by counting every code point and create_cursor(n) by stepping over the
first n. Both run once per Request, so a scan that restarts at successive
positions walked the subject again on every call: finditer over an ASCII
subject of n tokens was quadratic in n.
PyStr already records whether it is ASCII -- StrKind is decided when the
string is built -- and for ASCII a character index is a byte index, so the
&[u8] drive's cursor arithmetic already applies: count() is the byte length
and create_cursor() is a pointer offset. Add AsciiStr, which delegates every
StrDrive method to that impl and differs only in slice(), which reslices the
span and returns str rather than bytes.
Matching is unaffected: StrDrive carries no unicode semantics, because the
engine keys every unicode decision on the compiled pattern's opcode rather
than on the subject type.
Also bind the subject once in with_sre_str, so callers passing a temporary
(`&x.clone()`) build it once rather than per arm.
finditer over an ASCII subject, n tokens, this machine:
n before after
5000 41.60ms 1.37ms
10000 217.30ms 2.68ms
20000 1206.28ms 5.17ms
40000 5016.65ms 10.31ms
Per-doubling x5.22/x5.55/x4.16 becomes x1.95/x1.93/x1.99. Collecting
m.group(0) for every match goes 5061.02ms -> 17.80ms at n=40000.
test.test_re is unchanged at 166 tests, OK (skipped=14, expected failures=6),
and a 2731-line differential over the is_ascii() boundary -- findall,
finditer spans and groups, sub, split, match, search, fullmatch across
subjects that are empty, ASCII, non-ASCII, or mixed -- is byte-identical to
CPython 3.14.6. Introducing an off-by-one in AsciiStr::slice moves 1224 of
those lines, so the comparison reaches the new code.
Assisted-by: Claude
* _imp: read get_frozen_object's data as a whole marshal value
The data argument went through deserialize_code(), which reads a code body
without the type byte in front of it, so nothing marshal.dumps() produces was
accepted. Read it with marshal.loads() and require a code object back.
Assisted-by: Claude
* _imp: bind find_frozen's arguments with FromArgs
The arity check was done by hand on a FuncArgs. Take the arguments through a
FromArgs struct instead, which makes withdata keyword-only, and fill in the
data it asks for: the frozen encoding is not marshal, so the code is
re-serialized into what get_frozen_object() reads back.
Assisted-by: Claude
* itertools: give tee's shared buffer a Python type
The buffer was a PyRc<PyItertoolsTeeData>, which is not a Python object, so the
collector could not walk into it and any cycle running through a tee was
uncollectable. Make it the _tee_dataobject type with a traverse, held by
PyRef, and split the rest along the same lines: tee() is a function, _tee is
the iterator type it builds, and _tee takes a single iterable rather than
returning a tuple from __new__. _tee is weak-referenceable, tee() rejects a
negative n with ValueError and reserves its tuple fallibly.
test_itertools.test_tee passes now; its expectedFailure marker is removed.
Assisted-by: Claude
* contextvars: hold the shared context state under locks
The variable map was a RefCell, the enter flag, the context index, the token's
used flag and the variable hash were Cells, and each carried an unsafe impl
Sync. A Context or ContextVar shared between threads overlapped their borrows
and panicked. The map and the per-variable cache are now PyMutex, the flags and
the index are atomics, entering a context is a compare_exchange, and the three
unsafe impl Sync are gone.
The cache also stopped being read through AtomicCell::as_ptr, which raced a
concurrent store on a value holding a PyObjectRef.
Values displaced from the map or the cache are dropped after the lock is
released: __del__ can come straight back into the same context, and the locks
are not reentrant.
Assisted-by: Claude
* generator: read the frame state under the running claim
send(), send_none(), throw() and close() read `closed` and `frame.lasti()`
before `running` was compare_exchanged, so the frame they went on to resume
could be one another thread had already advanced. A resume that decided from
`lasti() == 0` that the generator had not started pushes no value onto the
value stack, and the code after the yield pops one, which underflows the stack.
The compare_exchange now hands back a guard, taken before those reads and
released after maybe_close(), so the generator is retired while it is still
claimed.
Assisted-by: Claude
* itertools: claim a tee's position and its buffer as one step
`_tee::next` read `index` and moved it on afterwards, and
`_tee_dataobject::get_item` released `running` before the value it fetched
from the source was cached.
Two callers on one `_tee` then read the same index, hand out the same value
twice and advance past a value that was never cached, which leaves `index`
past `values.len()` and indexes the buffer out of bounds. Two callers at the
same index on separate tees each fetch a value from the source, and one of the
two is dropped without reaching a caller.
Both claims now cover the read and the update.
Assisted-by: Claude
* _asyncio: require InvalidStateError to be a type
new_invalid_state_error() called whatever `asyncio.exceptions.InvalidStateError`
names and unwrapped the downcast of the result, so a future asked for a result
it does not have panicked once that attribute was rebound to something that is
not an exception:
asyncio.InvalidStateError = lambda *args: 42
_asyncio.Future(loop=object()).result()
The type is looked up the way get_cancelled_error_type() looks its own up, and
raised with new_exception_msg; a lookup that does not produce an exception type
falls back to RuntimeError, as the other arms already did.
Assisted-by: Claude
`w_ref` marks a code or slice entry incomplete until `w_complete`, because
the reader rebuilds both from their fields and a `TYPE_REF` issued while
those fields are still on the wire names an object that does not exist
yet. `WriterRefTable` carries that marker and `write_object_depth`
raises `cannot marshal recursion <type> objects` instead of emitting the
reference.
`test_reference_loop_code`, `test_unmarshallable` and
`test_reference_loop_slice` lose their RustPython markers; `test_marshal`
is 75 run, 16 skipped.
Assisted-by: Claude
* Keep abnormal marshal loop test aligned with CPython
Assisted-by: OpenAI Codex:gpt-5.5
* marshal: retain runtime code constants while loading
Decode code fields through the runtime MarshalBag so co_consts values that do not fit the compiler constant enum remain available to VM code wrappers while the compiler table receives shape placeholders.
Assisted-by: OpenAI Codex:GPT-5.4
* marshal: write and read co_consts through the runtime bag
serialize_code gains a serialize_code_with variant that writes each
co_consts entry through a caller-supplied writer; serialize_code keeps
the BorrowedConstant writer as its default. The VM writer passes its own
write_object_depth, so a code constant that Literal holds but
BorrowedConstant cannot describe reaches the stream instead of panicking
in borrow_obj_constant, and a constant shared with the enclosing object
takes an entry in the writer's reference table.
PyMarshalBag implements constant_ref_from_value, bytes_from_value,
str_from_value and tuple_elements_from_value, which
deserialize_code_value_inner requires to decode code fields through the
runtime bag.
Assisted-by: Claude
* mmap: guard move() against dest/src past the mapping size
An offset larger than the mapping made `size - dest` underflow, producing
an out-of-range slice index panic. Check the high side first, as write() does.
Assisted-by: Claude
* exceptions: keep ImportError name/path/name_from out of an empty instance dict
ImportError.__reduce__ unwrapped get_arg(0), which is None when the exception
was constructed with no positional argument, aborting on pickle.dumps(ImportError()).
Fall back to the full args tuple there, and expose name/path/name_from as class
attributes defaulting to None so a bare ImportError() reduces to (cls, ()).
Assisted-by: Claude
* asyncio: stop unwrapping the _current_tasks downcast in current_task()
_asyncio._current_tasks is a reassignable module attribute; the three sibling
functions already degrade gracefully when it is not a dict, current_task() did not.
Assisted-by: Claude
* asyncio: raise TypeError when FutureIter.throw()'s exception class returns a non-exception
exc_type.call() goes through a Python-controlled __new__, so its result can be
any object; the downcast was unwrapped. Report it as a TypeError instead.
Assisted-by: Claude
* sequence: use a fallible reservation for sequence repetition
The guard only rejects a repeat whose per-element size crosses MAX_MEMORY_SIZE,
so (1,) * (10**12) still reached Vec::with_capacity and aborted in the allocator.
Reserve fallibly and surface a MemoryError.
Assisted-by: Claude
* collections: reject an oversized deque repetition with MemoryError
deque * sys.maxsize reached the allocator and aborted with a capacity overflow.
Apply the MAX_MEMORY_SIZE guard the other sequences already carry.
Assisted-by: Claude
* itertools: raise OverflowError for an out-of-range r argument
combinations/combinations_with_replacement/permutations narrowed r with
to_usize().unwrap(), so r=2**64 panicked instead of raising.
Assisted-by: Claude
* math: stream the generic sumprod path instead of collecting both iterables
The big-int path collected each argument into a Vec before multiplying, so an
unbounded iterable exhausted memory. Advance both iterators in lockstep and
accumulate, reporting a length mismatch when only one is exhausted.
Assisted-by: Claude
* _imp: raise TypeError for a second positional argument to find_frozen
withdata is keyword-only and unimplemented; passing it positionally hit an
unimplemented!() and aborted.
Assisted-by: Claude
* _typing: check _idfunc arity before indexing args
_typing._idfunc() with no argument indexed args[0] out of bounds.
Assisted-by: Claude
* exceptions: require a sequence for the ExceptionGroup excs argument
The argument was collected before being validated, so an unbounded iterable
such as itertools.count() exhausted memory. Reject a non-sequence up front.
Assisted-by: Claude
* mmap: treat an inverted find/rfind range as empty
find(b"x", 5, 2) built a slice whose start exceeded its end and panicked.
Assisted-by: Claude
* collections: give deque and defaultdict a GC traverse
Neither type opted into traversal, so a reference cycle through a deque or a
defaultdict was never collected.
Assisted-by: Claude
* itertools: opt the iterator types into GC traverse
cycle and its siblings hold Python references but declared no traverse, so a
cycle built through one of them leaked.
Assisted-by: Claude
* _ctypes: mask an out-of-range int instead of panicking
c_char_p(2**64) and pointer item assignment called .expect() on the narrowing
conversion. Wrap the value to the target width, which is what the C
implementation stores.
Assisted-by: Claude
* hashlib: import _hashlib when a hash module is loaded
The hash object type is a static type owned by _hashlib; calling _md5.md5()
without _hashlib imported hit an uninitialized static type and panicked.
Assisted-by: Claude
* _csv: fall back to the built-in dialect defaults when none is registered
The dialect table is empty until csv.py registers 'excel', so _csv.reader([])
unwrapped a missing entry and panicked.
Assisted-by: Claude
* builtins: reject surrogates in compile()/eval() source instead of panicking
expect_str() panics on a str containing surrogates; convert with try_as_utf8
so eval(chr(0xd800)) raises.
Assisted-by: Claude
* _suggestions: require a list for _generate_suggestions candidates
The argument was collected before validation, so an unbounded iterable
exhausted memory.
Assisted-by: Claude
* lzma: size the filter chain before consuming it
filters= was collected into a Vec before the length check, so an unbounded
iterable exhausted memory. Take the length through the sequence protocol first.
Assisted-by: Claude
* classmethod: opt into GC traverse
staticmethod already declares traverse; classmethod did not, so a cycle through
the wrapped callable leaked.
Assisted-by: Claude
* posix: reject unbounded iterables in posix_spawn and setgroups
argv, setsigdef, setsigmask and setgroups bound an ArgIterable and collected it
before validating, so an infinite generator exhausted memory. Require a
list/tuple for argv, validate signals while streaming, and take setgroups
through the sequence protocol.
Assisted-by: Claude
* _ctypes: size Array slice assignment and _argtypes_ before collecting
Both eagerly collected their argument, so an unbounded iterable exhausted
memory before the length check ran.
Assisted-by: Claude
* sys: propagate the breakpointhook warning failure
warn() was unwrapped, so an unimportable $PYTHONBREAKPOINT under
-W error panicked instead of raising.
Assisted-by: Claude
* structseq: require the sequence argument when constructing a struct sequence
A no-argument construction produced an empty backing tuple, and reading any
named field then indexed out of bounds.
Assisted-by: Claude
* Guard the hash slot dispatch against unbounded recursion
`PyObject::hash` invoked the type's hash slot directly, so an element-wise
`__hash__` following a deeply nested object graph recursed one native frame
per level and overflowed the stack. Wrap the dispatch in `with_recursion`,
matching the repr and rich-compare dispatches in the same file, so
`hash(x)` on a nested tuple/GenericAlias/slice raises `RecursionError`.
Assisted-by: Claude
* genericalias: guard the __parameters__ walk against unbounded recursion
`make_parameters_from_slice` recursed into every list/tuple argument with
nothing counting the frames, so `list[L]` for a self-referential or deeply
nested `L` overflowed the native stack at subscript time. Wrap the descent
in `with_recursion`, which makes the walk fallible; `PyGenericAlias::new`,
`from_args` and `make_parameters` now return `PyResult` and every caller
propagates.
Assisted-by: Claude
* Fix the type confusion in PyAtomicRef's Debug impl
`PyAtomicRef<T>` stores a pointer to a `Py<T>`, as `Deref`, `load_raw`,
`swap` and `Drop` all read it, but `Debug` cast it to a bare `T` and
formatted the object header as payload bytes. For `PyFunction`, whose
`code: PyAtomicRef<PyCode>` has a pointer-chasing `Debug`, that
dereferenced header words and segfaulted.
Cast to `PyObject` instead, which is what `Drop` already does and which
also covers the `PyAtomicRef<PyObject>` and `PyAtomicRef<Option<T>>`
instantiations that have no `Py<T>`.
`_asyncio._enter_task` reached this through `{:?}` in its "Cannot enter
into task" message; format the two tasks with their Python repr, which is
what the message is meant to show.
Assisted-by: Claude
* _sre: disallow instantiating Match
`Match` inherited `object.__new__`, so `M.__new__(M)` produced an instance
whose `regs`/`string`/`pattern` were never filled in by a match run; the
mapping subscript path read them and dereferenced garbage. The type has no
public constructor, so mark it `DISALLOW_INSTANTIATION`, which makes
`M.__new__(M)` raise `TypeError: cannot create 're.Match' instances`.
Assisted-by: Claude
* utils: return the empty repr instead of asserting a non-empty collection
`collection_repr` took the first element with an `.expect()` justified by the
caller's preceding non-empty check. Another thread clearing the collection
between that check and the iteration made the iterator yield nothing and
panicked the worker. Fall back to the caller-supplied empty form, which is
the text those callers already produce for an empty collection.
Assisted-by: Claude
* itertools: advance cycle's index atomically
`cycle.__next__` did `fetch_add(1)` and then reset the index to 0 in a
separate store, so two threads replaying the saved items could both read an
index past the end of `saved` and panic on the slice access. Do the advance
and the wrap in one `fetch_update`.
Assisted-by: Claude
* _ctypes: reject a float argument to a foreign call without argtypes
`conv_param`, the conversion used when `argtypes` is not set, converted its
argument with `try_int`, which goes through `__int__` and so accepts a
float. `libc.strlen(1.5)` therefore passed 1 where the callee expects a
`char *` and the callee dereferenced it. Match `ConvParam`, which does a
`PyLong_Check` and converts nothing: take the branch only for an `int` (or
a subclass, so `True` still converts), and let a float fall through to
"Don't know how to convert parameter".
The branch below it converted a float to a C double, but `try_int` claimed
every float before it could run, so it was dead; `CArgValue::Double`
existed only for that branch and both go. Typed doubles are unaffected —
they travel as `CArgValue::Typed` with code 'd'.
Assisted-by: Claude
* Report the iterator itself from PyIter's traverse
`Traverse for PyIter<O>` delegated to the inherent `PyObject::traverse` of
the object it wraps, so it reported that iterator's referents instead of
the iterator. The iterator's own reference to those referents was then
never subtracted during the collector's reference-subtraction pass, the
referents kept a non-zero gc_refs, and every object reachable from them
was classified as a root.
Any cycle running through a type with a `PyIter` field therefore survived
collection: `map`, `filter`, `zip`, `enumerate`, `reversed` and the
`itertools` iterators all leaked, while the same cycle through a `list`,
`tuple` or `list_iterator` collected. Report the wrapped object, as the
`PyObjectRef`, `PyRef<T>` and `PyStackRef` impls do.
`itertools.tee` still leaks: its shared buffer is a `PyRc<PyItertoolsTeeData>`
rather than a Python object, so the collector cannot see through it.
Assisted-by: Claude
* Remove the obsolete expectedFailure on test_code_module.test_unicode_error
Compiling a source string containing a lone surrogate now raises
UnicodeEncodeError, so the test passes.
Assisted-by: Claude
* itertools: reserve the combination indices fallibly
`combinations` and `combinations_with_replacement` built their index vector
with an infallible allocation, so an `r` that passes the ssize_t check but
does not fit in memory aborted the process instead of raising MemoryError.
Assisted-by: Claude
* Apply the struct sequence constructor's dict argument
`structseq(sequence, dict)` discarded its second argument, so the hidden
fields past `n_sequence_fields` — `tm_zone`, `st_atime` and friends — were
always None when constructed directly or restored from a `(sequence, dict)`
pickle, and a non-dict second argument was accepted silently.
Take the dict, require it to be a dict, and fill the hidden slots the
sequence did not cover from it. A key that names a field the sequence
already supplied, or no field at all, is now a
"got duplicate or unexpected field name(s)" TypeError instead of being
dropped. Both arguments are bindable by name, as `sequence` and `dict`.
`os.stat_result` and `os.statvfs_result` did not accept a second argument
at all; they and `time.struct_time` now share the parsing.
Assisted-by: Claude
* _imp: report the argument count in find_frozen's arity error
Assisted-by: Claude
* Add regression tests for the reproduced crashers
One case per catalog entry, each asserting the behavior the fix produces:
recursion guards, the memory-unsafety sites, the overflow and unbounded
allocation guards, the eager-collection rejections, the unwrap sites, and
the cycles the collector now breaks. Every expected value was checked
against CPython 3.14.
Assisted-by: Claude
* Tolerate a changed-size RuntimeError in the set repr stress test
The mutator and reader threads race on purpose; a "changed size during
iteration" RuntimeError is a valid outcome of that race and should not fail
the test. The panic it guards against is not.
Assisted-by: Claude
* Move the crash regression tests into per-module snippets
crash_regressions.py collected every reproduced crasher in one file. Split it
into the snippet for the module each case exercises, and add stdlib_gc.py,
stdlib_asyncio.py, stdlib_lzma.py, stdlib_threading_set_repr.py and
stdlib_threading_itertools_cycle.py for the cases with no existing home.
The suite runs every snippet under the host CPython too, so the checks only
RustPython raises are guarded by sys.implementation.name: the hash and
__parameters__ recursion depth, and the deque repeat overflow. The float
ctypes argument accepts either TypeError or ctypes.ArgumentError.
Assisted-by: Claude
* Reject extra positional arguments to float()
PyFloat::slot_new's exact-float fast path tested args.args.first(), which
succeeds for any number of positional arguments, so float(1.5, True)
returned the first one and never reached args.bind(vm). CPython's clinic
generated float_new runs _PyArg_CheckPositional("float", nargs, 0, 1)
before it fetches the first argument.
Add args.args.len() == 1 to the condition, as PyInt::slot_new,
PyStr::slot_new and PyComplex::slot_new already do; the extra argument
then falls through to args.bind(vm) and raises TypeError.
Assisted-by: Claude Code:claude-opus-5
* Bind float() arguments before the fast path
The exact-float fast path repeated float()'s argument count inline, which
duplicates what FromArgs::arity already knows. Bind first and match on the
resulting OptionalArg instead, so the positional and keyword rules stay in
one place and the bound value is reused by py_new.
Assisted-by: Claude Code:claude-opus-5
PropertyArgs declared a fifth positional-or-keyword field, name, so the
derived arity was 0..=5 and property(None, None, None, None, None) built
a property object with the fifth argument stored in the __name__ slot.
CPython's property.__init__ is Argument Clinic generated with maxpos = 4
and the keyword list {fget, fset, fdel, doc}, so it rejects both that call
and property(name='x').
Drop the field. Nothing passed it: clone_property_with always supplied
None and copies the name separately, and the name slot is still filled by
__set_name__ and the __name__ setter.
Assisted-by: Claude Code:claude-opus-5
int.__round__ returned the receiver unchanged on the no-rounding path, so
round() on an int subclass handed back an instance of that subclass:
round(True) was True, not 1. CPython routes both no-op paths (ndigits is
None, and ndigits >= 0) through long_long(), which reuses the reference
only when the value is an exact int and otherwise copies it into a fresh
one.
Return zelf.__int__(vm).into_pyref(), which goes through into_exact_or()
and is the same normalization the neighbouring __trunc__, __floor__ and
__ceil__ already perform.
Assisted-by: Claude Code:claude-opus-5
Store the `dropwhile` predicate as a Python object and call it while
advancing the iterator. This defers callable validation until the predicate
is first needed, matching CPython for empty input while preserving exception
propagation during iteration.
Remove the now-passing `test_dropwhile` expected-failure marker.
Assisted-by: Codex:gpt-5.6-sol
* Reuse stored hashes in dict.fromkeys()
Closes#8490.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Make fromkeys_known_hashes a PyDict associated fn
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
RustPython's tail call machinery pre-allocates and reuses a vector.
The code drains the vector into a new vector which is stored elsewhere.
Clippy warns that this pattern causes a spurious location. Clippy is
usually right that this pattern is suspect, but in this case the initial
vector is reused so we want to keep the initial location.
* Fix marshal recursive reference loading
Create reference-tracked containers before reading their children so recursive list, dict, set, and tuple graphs can be unmarshaled. Preserve interned string markers through the runtime bag and add an initialization-only tuple construction path.
Assisted-by: Codex:gpt-5
* Preserve marshal container insertion errors
Keep Python exceptions raised while constructing unmarshaled sets, frozensets, and dictionaries instead of collapsing them into ValueError. This makes abnormal recursive hash-container streams report TypeError like CPython and removes the remaining test_marshal expected failure.
Assisted-by: Codex:gpt-5
* Enable full abnormal marshal reference loop test
* Test direct marshal tuple reference loop
Assisted-by: Codex:gpt-5
* Reuse stored hashes when building a set from a set/frozenset/dict
set and frozenset recomputed __hash__ for every element even when the
source object already stored a hash per entry. CPython's
set_update_internal branches on PyAnySet_Check / PyDict_CheckExact and
feeds set_add_entry the hash read from the source table; RustPython
always iterated generically.
Split the hash computation out of the Dict entry points so callers can
supply a hash they already hold, add keys_with_hashes() to hand out
(key, hash) pairs, and take the fast path in the set constructors and
in the set operations whose argument is a set/frozenset/exact dict.
ArgIterable::as_object() exposes the pre-__iter__ object so the set
operations can dispatch on the source type without changing any of
their signatures.
Closes#8489. dict.fromkeys() is the dict-target counterpart and is
tracked in #8490, so test_do_not_rehash_dict_keys keeps its
expectedFailure marker until that lands too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Rename the hash-carrying variants to *_known_hash
`_with_hash` said nothing about which direction the hash travels, and the
same suffix was already used both ways in this file: keys_with_hashes()
hands hashes out, while insert_with_hash() takes one in. The pre-existing
insert_with_hint()/get_with_hint() pair has the same problem.
Follow CPython's "KnownHash" variants (_PyDict_SetItem_KnownHash,
_PyDict_Contains_KnownHash, _PyDict_DelItem_KnownHash) instead, so the
suffix marks the argument direction and matches the name a reader
familiar with CPython already expects. keys_with_hashes() keeps `with`
because it really does return the hashes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Narrow visibility of the new helpers and trim their docs
from_object() and ArgIterable::as_object() had no caller outside their own
file and crate respectively, so drop them to private and pub(crate). The
dict_inner helpers stay pub(crate) because builtins::set calls them.
Also shorten the doc comments to match the density of the surrounding
code; only insert_known_hash keeps a real note, since a wrong hash there
corrupts the table silently.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* map: raise TypeError when called with no iterables
Assisted-by: Claude Code:claude-fable-5
* add test for map with no iterables
Assisted-by: Claude Code:claude-fable-5
* removed expectedFailures
* removed test for map constructed with no iterables - covered by existing in test_itertools
* Add new_payload_exception helper for built-in payload exceptions
* Use new_payload_exception for SystemExit raise sites
* Use new_payload_exception for BlockingIoError raise sites
* Use new_payload_exception for OSError errno dispatch in slot_new
* Run rustfmt
* Run clippy
* Add type guard to new_payload_exception and extract new_system_exit
* Extract new_system_exit and fix stale expect messages
The four SystemExit raise sites repeated the same construction, so route them through a new_system_exit helper next to new_stop_iteration. The expect messages still described invoke_exception's downcast failure, which no longer applies after moving to new_payload_exception.
* Update crates/vm/src/vm/vm_new.rs
---------
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
Rust supports C string literals which automatically create a CStr with a
trailing NUL. `windows-sys` provides an analogous macro for wide
strings. Both of these avoid allocations which is nice for constants.
code.replace() called as_interned_str().unwrap() on its string
arguments, which panics when the caller passes a string that has not
been interned. modulefinder's replace_paths_in_code() builds a fresh
co_filename, so any use of ModuleFinder(replace_paths=...) aborted the
interpreter.
Intern the incoming strings instead, and raise TypeError rather than
panicking when a non-string appears in co_names/co_varnames/
co_cellvars/co_freevars.
Unskips test_modulefinder.test_replace_paths.
Co-authored-by: Pablo Garcia <pablito@MacBook-Neo-de-Pablo.local>
`search_info_literal`'s len>1 arm reset the tail-match cursor by advancing one
character past the matched prefix. That position equals the INFO block's
prefix_skip boundary only when the prefix ends where the skip does; for a
pattern like `ab(cd)`, `_get_literal_prefix` reports prefix_len=4 with
prefix_skip=2, so the tail resumed two characters too far.
`state.cursor` already holds `req.start + skip`, which is what
`sre_lib.h` SRE(search) computes as `ptr - (prefix_len - prefix_skip - 1)`,
and which the len==1 arm above already uses. Take it unconditionally.
Searching "xabcdcd" for `ab(cd)` returned span (1, 7) instead of (1, 5);
over a 600-case corpus stamped with CPython's own answers, 94 cases disagreed
before this change and none after. The wrong resume also dropped matches
outright, not only widened them.
Assisted-by: Claude
According to POSIX, `time_t` should be 64 bits. `musl` changed its
`time_t` to an `i64` over five years ago. `glibc` provides compatibility
features that declare `time_t` as either `i32` or `i64`. Rustix uses the
raw Linux syscall for stat which returns an `i64`. For our purposes, an
`i64` makes sense because it's modern and avoids the year 2038 problem.
It also reduces our dependency on what `libc` defines.
Sources:
* https://www.man7.org/linux/man-pages/man3/time_t.3type.html
* https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/sys_types.h.html
* Allow automated commits through pre-commit hooks
* Fix agent C-API test instructions
Exclude rustpython-capi from root workspace test commands and run its tests from the crate directory so its Cargo configuration applies. Document the project AI policy and required commit trailer.
Assisted-by: Codex:gpt-5
CPython's _PyFrame_GetLocals only syncs fastlocals/cells into locals()
for function scope; module/class scope just returns the namespace dict
as-is. sync_visible_locals_to_mapping did this for every scope, so the
implicit __conditional_annotations__ cell (used for PEP 649/749
deferred annotations) broke two ways: at module scope its cell is
always empty, so syncing overwrote the dict's real value with None,
causing NameError on the next annotated statement; at class scope its
cell is the only real value, so syncing leaked it into locals()/dir(),
unlike CPython.
Skip cell/free slots outside function scope to match CPython, and add
a regression snippet.
Closes#8379
Assisted-by: Claude Code:claude-sonnet-5
dependabot[bot]