74 Commits

Include renames

Author	SHA1	Message	Date
Jeong, YunWon	819b4b86fd	Fix malformed Unicode error constructors (#8497) ... * Fix malformed Unicode error constructors Replace message-only Unicode decode and encode errors with fully initialized exceptions, preserve source objects and failure ranges, and remove the obsolete constructors. Assisted-by: Codex:gpt-5 * Fix Windows Unicode CI regressions Assisted-by: Codex:gpt-5	2026-08-17 00:33:56 +09:00
Jeong, YunWon	11de045ef8	Per-interpreter runtime state and a process-wide GC stop-the-world (#8517) ... * vm: add per-interpreter runtime state and interpreter registry - vm/runtime.rs: process-global interpreter registry (monotonic ids, weak entries), InterpreterWhence/InterpreterInfo, process main id recording via main_interpreter_id(), a threading-gated owner map (store_owned_interpreter/ take_owned_interpreter/is_owned_interpreter/owned_interpreter_count), and the SUPPORTS_ISOLATED_INTERPRETERS constant. - PyGlobalState gains interpreter_id/whence/is_main and is_main_interpreter(); PyConfig/Settings derive Clone so a subinterpreter can clone parent config. - Interpreter: id()/whence()/is_main()/is_process_main(), create_subinterpreter() and create_owned_subinterpreter(); unregister on Drop. - thread.rs: per-interpreter thread slots (INTERP_THREAD_SLOTS), slot swap when switching interpreters on one OS thread, cleanup keyed by interpreter id. - Install signal handlers and init the main-thread ident only on the main interpreter; _thread._is_main_interpreter reflects the current interpreter. - sys.implementation.supports_isolated_interpreters reads the constant. - Guard the registry static for non-threading builds where rc::Weak is !Send. Assisted-by: Claude Code:claude-opus-4-8 * gc: stop every interpreter while collecting The generation lists are process-global, so a collection reads and frees objects owned by every interpreter. CollectStopTheWorld stopped only the collecting interpreter, leaving other interpreters' threads free to mutate the same object graph during the reference-subtraction, reachability and snapshot phases. Stop all live interpreters instead, in runtime id order, and restart them in reverse. The global `collecting` mutex serializes collectors process-wide, so no second collector takes these exclusions in another order; fork acquires a single interpreter's exclusion, so the orders cannot cycle. - runtime: add live_interpreter_states(), ordered by interpreter id. - StopTheWorldState methods take &PyGlobalState instead of &VirtualMachine, so an interpreter's world can be stopped without a VM for it; update the call sites in frame, _thread, posix, faulthandler and capi. - Document in gc_state() that the collector is process-wide: gc.disable(), thresholds, gc.garbage and gc.get_objects() observe process-wide state, and a per-interpreter collector additionally needs untrack_object (called from default_dealloc with no VM in scope) routed to the owning interpreter. Tests: stop_the_world_parks_threads_of_another_interpreter asserts a thread entered in one interpreter parks another interpreter's threads (it fails without this change), plus a collect-while-another-interpreter-churns test. Assisted-by: Claude Code:claude-opus-4-8 * vm: scope the interpreter registry per thread without threading The registry held `rc::Weak` in a process-global `OnceLock` and covered the resulting `!Send`/`!Sync` with an `unsafe impl` justifying it as "non-threading builds are single-threaded". That is not this codebase's model: `static_cell!` is thread-local without the `threading` feature precisely so each OS thread can own its own `Context::genesis()` and `GcState`, so two threads could reach the same `Rc` counts through the registry. Use `static_cell!` for the registry as well, matching `gc_state()`, and drop the `unsafe impl`. Ids are consequently unique per registry rather than per process in non-threading builds, which is documented on `alloc_interpreter_id`. Move the recorded main interpreter id into the registry so it follows the same scoping instead of living in a separate global `OnceLock`. Also make `CollectStopTheWorld::new` accumulate into a live guard: it built a bare `Vec` and only moved it into the restarting `Drop` type after stopping every interpreter, so an unwind partway through the loop left the already stopped interpreters parked forever with their exclusion held. Assisted-by: Claude Code:claude-opus-4-8 * vm: keep the interpreter registry usable across bootstrap, drop and fork Three gaps where the registry did not describe the interpreters that actually exist, each of which hides an interpreter from the collector's stop-the-world. Register before `initialize()`. Registration ran as the last step of `initialize_vm`, so the whole bootstrap — which executes Python bytecode and allocates GC-tracked objects — was invisible to `live_interpreter_states()`. It still cannot run any earlier than this: the init hooks take `PyRc::get_mut` on the state, which fails as soon as the registry holds a weak reference to it. Stop unregistering in `Interpreter::drop`. The handle does not decide the interpreter's lifetime — every `ThreadedVirtualMachine` from `new_thread()` holds its own `PyRc<PyGlobalState>` — so an interpreter with running workers disappeared from the registry while its threads kept mutating the object graph. The entries are weak, so lifetime is already correct without the removal; dead entries are now reaped when registering instead. Interpreters are consequently released rather than unregistered at a fixed point, so the two tests asserting disappearance now wait for it: a collection in progress legitimately holds a reference to every live interpreter. Repair other interpreters after fork. `py_os_after_fork_child` only fixed the forking interpreter, leaving every other one with slots for threads that did not survive (still ATTACHED if they were running bytecode) plus locks and stop-the-world flags held by them. Since a collection stops all interpreters, the child's first collection would wait for threads that no longer exist. Reset their locks, stop-the-world state and thread tables, drop this thread's cached slots for them, and reinit the registry's own locks first, since enumerating interpreters now takes them. Tests: test_gc, test_threading and test_fork1 pass, as do the vm tests in both the threading and default configurations. Assisted-by: Claude Code:claude-opus-4-8 * vm: attach and detach thread slots when switching interpreters `enter_vm` decided whether to attach from `was_outermost` (an empty VM_STACK), which held while a thread could only ever be in one interpreter. With a slot per (thread, interpreter) pair, entering interpreter B from a thread already inside interpreter A's section switched CURRENT_THREAD_SLOT to B's slot but attached nothing: the thread then ran B's bytecode with B's slot DETACHED while A's slot stayed ATTACHED. A collector stopping B force-parks the DETACHED slot and concludes B is stopped, and then walks the object graph this thread is still mutating. Pair the attach/detach with the slot switch instead (≈ `_PyThreadState_Swap`): `begin_interpreter_section` detaches the enclosing interpreter's slot, makes the target slot current and attaches it, and `end_interpreter_section` undoes that and re-attaches the enclosing interpreter. Both live in `set_current_vm`, which every path making a VM current already goes through, so `enter_vm` and `VmBootstrapGuard` no longer track outermost-ness themselves. `nested_enter_of_subinterpreter_is_stoppable` covers this: it runs a subinterpreter nested inside the parent's section and asserts the sub's threads park when the sub's world is stopped. It fails with the previous attach-at-outermost-only behavior. Assisted-by: Claude Code:claude-opus-4-8 * vm: list only the current interpreter's subclasses Interpreters share the context, so `class Foo(int)` in one of them pushes onto the same `int.subclasses` every other one reads, and `int.__subclasses__()` returned types no other interpreter can reach. Record the creating interpreter on `HeapTypeExt` and filter `__subclasses__` by it, the way `lookup_tp_subclasses` reads `tp_subclasses` out of per-interpreter state for static builtin types. Types built before any interpreter exists — the ones the shared context creates, including the exception hierarchy — carry no id and stay visible to every interpreter, which is what `_PyStaticType_InitBuiltin` produces by registering the builtin subclass links once per interpreter. The other walks over `subclasses` (version-tag invalidation, abc flag propagation, mro updates, slot propagation) are left as they are: each starts from a type being mutated, so from a heap type, whose subclasses all live in the interpreter that created it. `subinterpreter_subclasses_are_scoped_to_their_interpreter` covers this and fails without the filter. Assisted-by: Claude Code:claude-opus-5 * gc: give each interpreter its own collector state The generation lists stay process-wide, because an object is untracked from `default_dealloc`, where no interpreter is in scope to route to. What changes is that a collection no longer acts on every interpreter's objects, and the gc module no longer reports one interpreter's state to another. `track_object` stamps the running interpreter into a new `gc_owner` word on the object header, and a collection takes as candidates only the objects carrying its own tag plus the ones carrying none. `gc_owner` fits in the padding the header's alignment already forces, so objects do not grow; an assertion on the header size keeps it that way. Objects allocated with no interpreter running — everything the shared context builds — carry no owner and stay candidates for every interpreter, which is where they were before. Objects that outlive the interpreter that tracked them are adopted the same way by the next full collection, rather than being left to a collector that will never come. `enabled`, the thresholds, the debug flags, the statistics, `gc.garbage` and `gc.callbacks` move onto `PyGlobalState` — the last two off the shared `Context` — so `gc.disable()`, `gc.set_threshold()`, `gc.get_stats()`, `gc.get_objects()` and `gc.garbage` describe the interpreter that asks. The occupancy counts behind `gc.get_count()` and `gc.get_freeze_count()` stay process-wide: they measure how full the shared lists are. Their decrements are saturating now, since a collection zeroes the generations it emptied while another interpreter's objects are still sitting in them. Stop-the-world still stops every interpreter. Unowned objects are candidates and any interpreter can incref one, so the refcounts a collection reads are only stable while all of them are parked. This also fixes a deadlock it exposed: `CollectStopTheWorld` dropped its references to the stopped interpreters while the collection still held the generation read locks, so releasing the last reference to one — which frees its objects, and so untracks them — waited for a write lock behind that read lock. The references are now held until the guard itself drops. `collections_only_reach_the_collecting_interpreter` and `get_objects_only_reports_the_calling_interpreter` cover this and both fail without the owner check. Assisted-by: Claude Code:claude-opus-5 * _queue, _thread: detach before taking locks held across waits `_queue.Semaphore` holds its mutex across the `allow_threads` condvar wait, and `join_internal` holds a thread handle's completion mutex the same way. Stop-the-world can stop a thread while it holds either one. The remaining acquisitions ran attached, so a thread blocking on such a mutex had no safepoint left to reach: the stop never completed, and the holder was never resumed to release it. Route those acquisitions through helpers that detach first. The fork-child reinit paths keep their direct locks. Assisted-by: Claude * _io, _winapi: detach on the remaining stopped-holdable lock takes `TextIOWrapper.__repr__` took `data` directly while every other method takes it through `lock_opt`, which detaches. `Overlapped` holds `inner` across the `allow_threads` in `GetOverlappedResult`, and all four of its takes were direct. A thread stopped by stop-the-world can be holding either mutex, so taking one while attached left the blocked thread with no safepoint to reach. Assisted-by: Claude * gc: size the interpreter owner tag to the header padding `gc_owner` was a u32. With 4-byte pointers its alignment pushed it out of the padding that follows the gc bits and generation, growing every object by a word and tripping the `SIZEOF_PYOBJECT_HEAD` assertion on 32-bit targets. Introduce `GcOwner = u16`, which the `repr(C)` layout places at offset 10 on 32-bit and 18 on 64-bit, leaving the header at 6 words on both. Tags now run out after 65535 interpreters; `alloc_owner` already falls back to `GC_NO_OWNER`, so an interpreter past that collects as it did before tagging. Also widen three test deadlines that measure liveness, not speed. Assisted-by: Claude * vm: reuse cleared frame blocks and shorten interpreter hot paths - `datastack` remembers the most recently popped frame block. `push_frame` reports an exact LIFO reuse, and `setup_datastack_frame` then skips zero-filling localsplus. - Small-int loads push the context's cached int as a borrowed stack reference instead of taking a new one. - Calls to jitted functions go straight to `execute_call_vectorcall`. - Binary-op specialization reads ints through the new `PyInt::try_to_i64_fast` instead of the generic primitive conversion, and `try_to_bool` moves its non-bool path into a `#[cold]` helper. - Dict caches read an entry through a keys-version stamp (`get_index_if_keys_version`) rather than an entry-index hint. - Frame publishing caches a pointer to `ThreadSlot::top_iframe` in a thread-local `Cell` instead of borrowing `CURRENT_THREAD_SLOT`. * gc, vm: address review notes on owner tags and interpreter docs Look up retired owner tags with a sorted binary search instead of a linear scan, which every scanned object paid for once per dropped interpreter. Drop the claim that clearing a tag frees it for reuse; `alloc_owner` only ever hands out new tags. Also correct the tag space it mentions, which is 16-bit since the tag was sized to the header padding. Document `is_main` as "top-level interpreter" rather than "the process main": every top-level interpreter sets it, and only the first registered one becomes the main `main_interpreter_id` reports. Assert membership rather than an exact owned-interpreter count delta; the owned table is process-global and other tests store into it in parallel. Assisted-by: Claude * vm: gate interpreter registration on stop-the-world admission A collection snapshots the registry, stops the interpreters it found, and then reads tracked objects with their threads parked. An interpreter that registered after the snapshot was taken was absent from it, so nothing stopped it and its bootstrap ran Python — allocating and mutating the shared generation lists — underneath that scan. Registration and the stop now share a process-global gate: the collection holds it from before the snapshot until the restart, and registration takes it around the registry insert. The insert runs detached, since a thread that waited for the gate, or re-attached while holding it, would leave the stop it waits for no safepoint to complete at. Assisted-by: Claude * gc: keep the tracking counters off the barrier path Every tracked allocation and every free went through sequentially consistent counter updates, and each free through a `fetch_update` CAS loop. The counters drive only the gen0 threshold and `gc.get_count()`, and the generation locks — not the counters — order the list changes they describe, so they are relaxed now and the decrement is a load plus a conditional `fetch_sub`. `is_enabled` and `threshold`, both read once per allocation, are relaxed for the same reason: an allocation racing `gc.disable()` may use either value. Drop `alloc_count`, which nothing has ever read. Assisted-by: Claude * dict: settle lookups and iteration steps under one read guard A lookup probed under a read guard, dropped it with the matched entry in hand, and then took the lock again to re-find the entry. `lookup_extract` reads the entry while the probe still holds the guard when key identity settles the match, which is the case that cannot run Python. Dict and set iterators took one lock to compare the size and another to read the entry, then cloned both the key and the value even though a keys or values view keeps only one of them. `next_entry_checked` does the size check and the read under one guard, and clones through a per-view projection. A store that missed its inline-cache hint re-probed the dict afterwards only to recover the entry index the store had just computed; `unchecked_push` reports that index instead. Assisted-by: Claude * vm: build a call's argument vector once The specialized CALL handlers collected the positional arguments into one vector and then copied them into a second one to put `self` in front, so every specialized builtin, method-descriptor, class and non-Python call paid two allocations, a copy and two frees. The arguments are already laid out on the value stack in vectorcall order, so `take_call_args` fills one vector by index — the shape `execute_call_vectorcall` already used. `vectorcall_native_function` and the keyword path of `vectorcall_function` then cloned that vector again to build `FuncArgs`; both now move it in through `from_vectorcall_owned`, as the other vectorcall slots do. Assisted-by: Claude * vm: reach an instance dict without cloning it The specialized attribute instructions cloned the instance dict — a rwlock round-trip plus a refcount round-trip — for two things that only look at it: `LoadAttrMethodLazyDict` asking whether the dict exists, and the keys-version stamp check that is the whole of `shadowing_instance_attr`'s fast path. Both now read it borrowed, through `has_instance_dict` / `with_instance_dict`. `generic_getattr_opt` probed the dict with the name's `&Wtf8`, which hashes the name on every lookup and can never match a key by pointer. Passing the `Py<PyStr>` uses the string's cached hash and the interned-key identity check, and drops an allocation when a stored key is not an exact `str`. Assisted-by: Claude * vm: shorten the per-instruction safepoint and the call preamble The dispatch loop asks once per instruction whether stop-the-world wants this thread, and that read went through `CURRENT_THREAD_SLOT` — a `RefCell` borrow, so two stores to thread-local memory around an `Option` test. Cache the `stop_requested` pointer in a plain `Cell` at the same three places the frame pointers are cached, and the safepoint becomes one relaxed load. `lasti` is advanced from the index the loop just read, rather than reloaded to increment it, and `Resume` reads `quickened` before swapping it, so a call to an already-quickened code object costs a load instead of an atomic read-modify-write. An exact-args vectorcall to a Python function built a heap `FrameObject` where the equivalent `invoke` path uses a data stack frame; it now does the same when tracing is off. Assisted-by: Claude * vm: give KwArgs a zero-sized hasher `KwArgs::default()` is built for every call, keyword-less ones included, and it seeded a `RandomState` each time — a thread-local read and 16 bytes in every `FuncArgs`. Keyword names come from the program text, so the map now uses `BuildHasherDefault`, whose `Default` is a zero-init. Assisted-by: Claude * vm: promote borrowed stack refs before a yield `yield <small int>` tripped the "borrowed refs on stack at yield point" assertion: `LoadSmallInt` pushes a borrowed ref, and a yield saves the stack with the frame, which is exactly what that assertion forbids. Promote the stack first, so a suspended frame owns everything it holds. Assisted-by: Claude * vm: run the exact-args vectorcall on a heap frame again `vectorcall_function` took the data stack path for an exact-args call whenever tracing was off. A data stack frame that materializes — one traceback entry is enough — copies its localsplus into the `FrameObject`, so every value it holds gains a reference for as long as the frame runs, and a tail call holds the callee's function object until the caller's next call or return. Both are reachable without this hunk, through the `CallPyExactArgs` specialization, but routing the vectorcall entry point through the data stack made them apply from a call site's first execution: test_enumerate, test_memoryio, test_sys, test_tempfile and test_traceback all read a refcount or a finalization that moved. Assisted-by: Claude * vm: release a tail call's function at the callee's return The trampoline moved the callee's function object into the caller's `SuspendedFrame` and dropped it only once the caller had run again, so a function entered through a tail call kept one reference past its return: `sys.getrefcount` on it read one high for the caller's next stretch of bytecode, and anything the function reached stayed alive that much longer. The callee's frame is released before the `ReturnValue` or `Unwind` action is formed, and a materialized frame object holds its own references, so the owner is dropped where the suspended caller is popped instead. Assisted-by: Claude * vm: stop copying a running frame's locals when it materializes `materialize_slow` cloned the source frame's fast locals into the new `FrameObject`, so every value in a data stack frame gained a reference lasting as long as that frame object: one traceback entry through a frame kept all of its locals alive, and `sys.getrefcount` read one high for each of them. Nothing read that copy while the frame ran. `find_live_source_iframe` already routes reads to the live frame, and `exit_iframe` fills the slots from it at return, so the copy now starts empty as `materialize_slow_chain`'s does, and `snapshot_to_heap` goes with its last caller. Two readers had to be brought to the live frame the same way. `has_active_hidden_locals` read the frame object's own slots, which let `locals()` in a class body write an inlined comprehension's hidden variable into the class namespace; it now shares `live_fastlocals` with the other two. Another thread's frame has no live frame to reach from here, so the six cross-thread sites fill the copy under stop-the-world through `materialize_with_locals`. Assisted-by: Claude * vm: key data stack frame reuse on the exact frame size push_frame and pop_frame compared 16-byte-aligned sizes, so two frames whose sizes differ by less than the alignment matched as a reuse. The caller reads that as "every slot was cleared by the previous frame", but the larger frame's tail slots were never touched by it. Assisted-by: Claude * vm: store the frame object payload address in the cross-thread slot set_current_frame and reinit_frame_slot_after_fork wrote the address of the Py<FrameObject>, while the cross-thread readers hand that pointer to Py::from_payload_ptr, which subtracts the object header from it. Both writers now store the payload address. Assisted-by: Claude * vm: copy a foreign thread's frames instead of linking them sys._current_frames(), _thread._current_exceptions() and the cross-thread f_back path handed back frame objects still linked to the running frames. The owning thread wrote its locals into those same buffers at exit_iframe while the reader was cloning out of them. materialize_detached and materialize_detached_chain copy the values under stop-the-world and leave nothing for the owner to write into; materialize_with_locals is gone. FrameColdData gains attached_tid, the thread still running the frame a frame object was materialized from, which exit_iframe clears once it has written the values in. check_locals_access now rejects a read from any other thread while it is set; the FrameOwner test could not, because materialize_slow forces FrameObject ownership. Also removes with_iframe, which had no callers. Assisted-by: Claude * vm: resolve the two rustdoc links this branch added MAIN_INTERPRETER_ID is not in scope in interpreter.rs, and new_thread only exists in a threading build, so neither link resolved. Assisted-by: Claude	2026-08-16 21:21:31 +09:00
Jeong, YunWon	d5d19e0611	Fix 33 reproduced fuzzing and static-review defects (#8514) ... * mmap: guard move() against dest/src past the mapping size An offset larger than the mapping made `size - dest` underflow, producing an out-of-range slice index panic. Check the high side first, as write() does. Assisted-by: Claude * exceptions: keep ImportError name/path/name_from out of an empty instance dict ImportError.__reduce__ unwrapped get_arg(0), which is None when the exception was constructed with no positional argument, aborting on pickle.dumps(ImportError()). Fall back to the full args tuple there, and expose name/path/name_from as class attributes defaulting to None so a bare ImportError() reduces to (cls, ()). Assisted-by: Claude * asyncio: stop unwrapping the _current_tasks downcast in current_task() _asyncio._current_tasks is a reassignable module attribute; the three sibling functions already degrade gracefully when it is not a dict, current_task() did not. Assisted-by: Claude * asyncio: raise TypeError when FutureIter.throw()'s exception class returns a non-exception exc_type.call() goes through a Python-controlled __new__, so its result can be any object; the downcast was unwrapped. Report it as a TypeError instead. Assisted-by: Claude * sequence: use a fallible reservation for sequence repetition The guard only rejects a repeat whose per-element size crosses MAX_MEMORY_SIZE, so (1,) * (10**12) still reached Vec::with_capacity and aborted in the allocator. Reserve fallibly and surface a MemoryError. Assisted-by: Claude * collections: reject an oversized deque repetition with MemoryError deque * sys.maxsize reached the allocator and aborted with a capacity overflow. Apply the MAX_MEMORY_SIZE guard the other sequences already carry. Assisted-by: Claude * itertools: raise OverflowError for an out-of-range r argument combinations/combinations_with_replacement/permutations narrowed r with to_usize().unwrap(), so r=2**64 panicked instead of raising. Assisted-by: Claude * math: stream the generic sumprod path instead of collecting both iterables The big-int path collected each argument into a Vec before multiplying, so an unbounded iterable exhausted memory. Advance both iterators in lockstep and accumulate, reporting a length mismatch when only one is exhausted. Assisted-by: Claude * _imp: raise TypeError for a second positional argument to find_frozen withdata is keyword-only and unimplemented; passing it positionally hit an unimplemented!() and aborted. Assisted-by: Claude * _typing: check _idfunc arity before indexing args _typing._idfunc() with no argument indexed args[0] out of bounds. Assisted-by: Claude * exceptions: require a sequence for the ExceptionGroup excs argument The argument was collected before being validated, so an unbounded iterable such as itertools.count() exhausted memory. Reject a non-sequence up front. Assisted-by: Claude * mmap: treat an inverted find/rfind range as empty find(b"x", 5, 2) built a slice whose start exceeded its end and panicked. Assisted-by: Claude * collections: give deque and defaultdict a GC traverse Neither type opted into traversal, so a reference cycle through a deque or a defaultdict was never collected. Assisted-by: Claude * itertools: opt the iterator types into GC traverse cycle and its siblings hold Python references but declared no traverse, so a cycle built through one of them leaked. Assisted-by: Claude * _ctypes: mask an out-of-range int instead of panicking c_char_p(2**64) and pointer item assignment called .expect() on the narrowing conversion. Wrap the value to the target width, which is what the C implementation stores. Assisted-by: Claude * hashlib: import _hashlib when a hash module is loaded The hash object type is a static type owned by _hashlib; calling _md5.md5() without _hashlib imported hit an uninitialized static type and panicked. Assisted-by: Claude * _csv: fall back to the built-in dialect defaults when none is registered The dialect table is empty until csv.py registers 'excel', so _csv.reader([]) unwrapped a missing entry and panicked. Assisted-by: Claude * builtins: reject surrogates in compile()/eval() source instead of panicking expect_str() panics on a str containing surrogates; convert with try_as_utf8 so eval(chr(0xd800)) raises. Assisted-by: Claude * _suggestions: require a list for _generate_suggestions candidates The argument was collected before validation, so an unbounded iterable exhausted memory. Assisted-by: Claude * lzma: size the filter chain before consuming it filters= was collected into a Vec before the length check, so an unbounded iterable exhausted memory. Take the length through the sequence protocol first. Assisted-by: Claude * classmethod: opt into GC traverse staticmethod already declares traverse; classmethod did not, so a cycle through the wrapped callable leaked. Assisted-by: Claude * posix: reject unbounded iterables in posix_spawn and setgroups argv, setsigdef, setsigmask and setgroups bound an ArgIterable and collected it before validating, so an infinite generator exhausted memory. Require a list/tuple for argv, validate signals while streaming, and take setgroups through the sequence protocol. Assisted-by: Claude * _ctypes: size Array slice assignment and _argtypes_ before collecting Both eagerly collected their argument, so an unbounded iterable exhausted memory before the length check ran. Assisted-by: Claude * sys: propagate the breakpointhook warning failure warn() was unwrapped, so an unimportable $PYTHONBREAKPOINT under -W error panicked instead of raising. Assisted-by: Claude * structseq: require the sequence argument when constructing a struct sequence A no-argument construction produced an empty backing tuple, and reading any named field then indexed out of bounds. Assisted-by: Claude * Guard the hash slot dispatch against unbounded recursion `PyObject::hash` invoked the type's hash slot directly, so an element-wise `__hash__` following a deeply nested object graph recursed one native frame per level and overflowed the stack. Wrap the dispatch in `with_recursion`, matching the repr and rich-compare dispatches in the same file, so `hash(x)` on a nested tuple/GenericAlias/slice raises `RecursionError`. Assisted-by: Claude * genericalias: guard the __parameters__ walk against unbounded recursion `make_parameters_from_slice` recursed into every list/tuple argument with nothing counting the frames, so `list[L]` for a self-referential or deeply nested `L` overflowed the native stack at subscript time. Wrap the descent in `with_recursion`, which makes the walk fallible; `PyGenericAlias::new`, `from_args` and `make_parameters` now return `PyResult` and every caller propagates. Assisted-by: Claude * Fix the type confusion in PyAtomicRef's Debug impl `PyAtomicRef<T>` stores a pointer to a `Py<T>`, as `Deref`, `load_raw`, `swap` and `Drop` all read it, but `Debug` cast it to a bare `T` and formatted the object header as payload bytes. For `PyFunction`, whose `code: PyAtomicRef<PyCode>` has a pointer-chasing `Debug`, that dereferenced header words and segfaulted. Cast to `PyObject` instead, which is what `Drop` already does and which also covers the `PyAtomicRef<PyObject>` and `PyAtomicRef<Option<T>>` instantiations that have no `Py<T>`. `_asyncio._enter_task` reached this through `{:?}` in its "Cannot enter into task" message; format the two tasks with their Python repr, which is what the message is meant to show. Assisted-by: Claude * _sre: disallow instantiating Match `Match` inherited `object.__new__`, so `M.__new__(M)` produced an instance whose `regs`/`string`/`pattern` were never filled in by a match run; the mapping subscript path read them and dereferenced garbage. The type has no public constructor, so mark it `DISALLOW_INSTANTIATION`, which makes `M.__new__(M)` raise `TypeError: cannot create 're.Match' instances`. Assisted-by: Claude * utils: return the empty repr instead of asserting a non-empty collection `collection_repr` took the first element with an `.expect()` justified by the caller's preceding non-empty check. Another thread clearing the collection between that check and the iteration made the iterator yield nothing and panicked the worker. Fall back to the caller-supplied empty form, which is the text those callers already produce for an empty collection. Assisted-by: Claude * itertools: advance cycle's index atomically `cycle.__next__` did `fetch_add(1)` and then reset the index to 0 in a separate store, so two threads replaying the saved items could both read an index past the end of `saved` and panic on the slice access. Do the advance and the wrap in one `fetch_update`. Assisted-by: Claude * _ctypes: reject a float argument to a foreign call without argtypes `conv_param`, the conversion used when `argtypes` is not set, converted its argument with `try_int`, which goes through `__int__` and so accepts a float. `libc.strlen(1.5)` therefore passed 1 where the callee expects a `char *` and the callee dereferenced it. Match `ConvParam`, which does a `PyLong_Check` and converts nothing: take the branch only for an `int` (or a subclass, so `True` still converts), and let a float fall through to "Don't know how to convert parameter". The branch below it converted a float to a C double, but `try_int` claimed every float before it could run, so it was dead; `CArgValue::Double` existed only for that branch and both go. Typed doubles are unaffected — they travel as `CArgValue::Typed` with code 'd'. Assisted-by: Claude * Report the iterator itself from PyIter's traverse `Traverse for PyIter<O>` delegated to the inherent `PyObject::traverse` of the object it wraps, so it reported that iterator's referents instead of the iterator. The iterator's own reference to those referents was then never subtracted during the collector's reference-subtraction pass, the referents kept a non-zero gc_refs, and every object reachable from them was classified as a root. Any cycle running through a type with a `PyIter` field therefore survived collection: `map`, `filter`, `zip`, `enumerate`, `reversed` and the `itertools` iterators all leaked, while the same cycle through a `list`, `tuple` or `list_iterator` collected. Report the wrapped object, as the `PyObjectRef`, `PyRef<T>` and `PyStackRef` impls do. `itertools.tee` still leaks: its shared buffer is a `PyRc<PyItertoolsTeeData>` rather than a Python object, so the collector cannot see through it. Assisted-by: Claude * Remove the obsolete expectedFailure on test_code_module.test_unicode_error Compiling a source string containing a lone surrogate now raises UnicodeEncodeError, so the test passes. Assisted-by: Claude * itertools: reserve the combination indices fallibly `combinations` and `combinations_with_replacement` built their index vector with an infallible allocation, so an `r` that passes the ssize_t check but does not fit in memory aborted the process instead of raising MemoryError. Assisted-by: Claude * Apply the struct sequence constructor's dict argument `structseq(sequence, dict)` discarded its second argument, so the hidden fields past `n_sequence_fields` — `tm_zone`, `st_atime` and friends — were always None when constructed directly or restored from a `(sequence, dict)` pickle, and a non-dict second argument was accepted silently. Take the dict, require it to be a dict, and fill the hidden slots the sequence did not cover from it. A key that names a field the sequence already supplied, or no field at all, is now a "got duplicate or unexpected field name(s)" TypeError instead of being dropped. Both arguments are bindable by name, as `sequence` and `dict`. `os.stat_result` and `os.statvfs_result` did not accept a second argument at all; they and `time.struct_time` now share the parsing. Assisted-by: Claude * _imp: report the argument count in find_frozen's arity error Assisted-by: Claude * Add regression tests for the reproduced crashers One case per catalog entry, each asserting the behavior the fix produces: recursion guards, the memory-unsafety sites, the overflow and unbounded allocation guards, the eager-collection rejections, the unwrap sites, and the cycles the collector now breaks. Every expected value was checked against CPython 3.14. Assisted-by: Claude * Tolerate a changed-size RuntimeError in the set repr stress test The mutator and reader threads race on purpose; a "changed size during iteration" RuntimeError is a valid outcome of that race and should not fail the test. The panic it guards against is not. Assisted-by: Claude * Move the crash regression tests into per-module snippets crash_regressions.py collected every reproduced crasher in one file. Split it into the snippet for the module each case exercises, and add stdlib_gc.py, stdlib_asyncio.py, stdlib_lzma.py, stdlib_threading_set_repr.py and stdlib_threading_itertools_cycle.py for the cases with no existing home. The suite runs every snippet under the host CPython too, so the checks only RustPython raises are guarded by sys.implementation.name: the hash and __parameters__ recursion depth, and the deque repeat overflow. The float ctypes argument accepts either TypeError or ctypes.ArgumentError. Assisted-by: Claude	2026-08-14 00:57:10 +09:00
Jeong, YunWon	cdf006cfa1	Split InterpreterFrame from FrameObject for stack-allocated execution (#8354) ... * Add LightFrame for unobserved Python-to-Python calls Allocate a lightweight LightFrame header on the DataStack instead of a full Frame PyObject for specialized exact-args call paths. The frame is materialized lazily only when observed (sys._getframe, traceback, tracing). Key changes: - LightFrame struct with borrowed pointers and no refcount overhead - FrameSource enum (Heavy/Light) replacing the object field on ExecutingFrame - CURRENT_LIGHT_FRAME TLS for the light frame chain - Interleaved heavy/light frame walking in frame_at_offset_vm and current_thread_frame_vm - invoke_light_slots on PyFunction for the fast call path Call overhead reduced from ~92ns to ~58ns (~37% reduction). fib(28) improved from ~152ms to ~111ms (~27% reduction). Assisted-by: Claude * Streamline with_frame: inline recursion check, skip tracing dispatch - Inline check_recursive_call instead of wrapping in with_recursion closure - Amortize C stack overflow check to every 64th call depth - Skip dispatch_traced_frame when use_tracing is false (hot path) - Add recursion_depth_increment/decrement helpers Assisted-by: Claude * Address review findings for LightFrame correctness - Fix materialize_light_frame previous pointer: use payload pointer (&**frame) instead of Py<Frame> pointer, matching the chain convention - Fix predecessor ref management: use retained_back instead of mem::forget to keep materialized predecessors alive - Fix materialized ref reclaim: use FrameRef::from_raw to drop the leaked keeper ref without double-incrementing - Use PyAtomic<u32> for LightFrame.lasti instead of AtomicU32, removing the unsound cast on non-threading targets - Add amortized C stack overflow check to light frame path - Add scopeguard to light frame execution for panic safety - Remove unnecessary Send/Sync impls from LightFrame - Make check_c_stack_overflow pub(crate) for light frame path access Assisted-by: Claude * Fix CI failures: cspell, stale materialized state, f_back chain gaps - Fix cspell: 'amortised' → 'amortized' - Sync lasti, prev_line, and fastlocals when re-observing a materialized light frame, fixing stale f_locals/f_lineno in test_inspect and others - In with_frame, materialize any active light frame as the heavy frame's predecessor and store it in retained_back, so f_back and inspect.stack see the correct interleaved order (H_new → L_mat → H_old) - Guard invoke_light_slots on NEWLOCALS | OPTIMIZED code flags Assisted-by: Claude * Unify frame chain: replace dual-chain with single FrameChainPtr Replace the dual-chain frame management (CURRENT_FRAME AtomicPtr<Frame> + CURRENT_LIGHT_FRAME Cell<*const LightFrame>) with a single unified chain using FrameChainPtr, a tagged pointer where bit 0 distinguishes heavy (*const Frame) from light (*const LightFrame) entries. Changes: - Add FrameChainPtr type with tagged pointer encoding - Change LightFrame: replace previous_light + saved_current_frame with single previous: FrameChainPtr field - Change InterpreterFrame.previous from AtomicPtr<Frame> to PyAtomic<usize> - Change CURRENT_FRAME TLS from AtomicPtr<Frame> to AtomicUsize - Remove CURRENT_LIGHT_FRAME TLS entirely - Simplify all chain walkers (frame_at_offset, frame_at_offset_vm, find_owned_chain_frame, for_each_current_frame, current_thread_frame) - Remove light-frame materialization block from with_frame - Update f_back to dispatch on FrameChainPtr tags - Update faulthandler and gc_state to walk unified chain - Adaptive C stack check: every call in debug, every 16th in release Assisted-by: Claude * Fix light frame materialization: current_frame, f_back, and GC tracking - current_frame() now uses current_thread_frame_vm() to materialize light frames, fixing super()/compile()/exec() when called from light frame context - materialize_light_frame stores retained_back for heavy predecessors too, so f_back resolves after the caller returns - Escaped materialized frames (traceback, sys._getframe) are GC-tracked at cleanup so reference cycles are collectible - Restore pre-existing @expectedFailure markers for test_frame proxy tests Assisted-by: Claude * Limit light frame materialization to super() only current_frame() returns heavy-only again to avoid unnecessary materialization that extends object lifetimes. super() init uses current_thread_frame_vm() directly since it must inspect the light frame's code and freevars to find __class__. Assisted-by: Claude * Fix remaining CI failures: C stack check, GC tracking threshold, expectedFailure - Check C stack overflow every call in light frame path (not every 16th) to prevent stack overflow on CI workers with smaller stacks - Fix GC tracking threshold: escaped materialized frame detection uses strong_count > 1 (was > 2, missed frames referenced only by traceback in the result) - current_frame() materializes light frames so warnings, compile(), and other callers see the correct frame - Mark test_futures2 PyFutureTests.test_task_exc_handler_correct_context as expectedFailure (pre-existing on main: PyTask __del__ timing) Assisted-by: Claude * Avoid materialization in hot paths: add light-frame-aware accessors current_frame() stays heavy-only to avoid costly materialization on every call. Instead, add current_globals(), current_code(), and current_builtins() that read directly from light frame raw pointers without creating frame PyObjects. Callers migrated: - PyFunction:🆕 current_builtins() for fallback builtins lookup - compile()/eval(): current_code() for future feature flags - type.__new__: current_globals() for __module__ detection - typevar/typing: current_globals() for caller module name - _io: current_code() for source path check - super(): current_thread_frame_vm() for __class__ cell access - warnings.warn: current_thread_frame_vm() for stack level walk - import: current_globals() for module resolution Assisted-by: Claude * Fix current_locals() to include light frames current_locals() must materialize light frames so locals() builtin returns the correct scope. This fixes test_zipimport doctest and other tests that rely on locals() in light frame context. Assisted-by: Claude * Fix clippy, profile tracing, resume_gen_frame, and faulthandler issues - Fix clippy: collapsible_if in materialize_light_frame, map_unwrap_or in import - Fix profile breakage: use current_thread_frame_vm in trace_event_inner to materialize light frames, preventing frame identity mismatch when profiling is enabled inside a light frame (e.g. profile.Profile.runctx) - Disable light frames inside trace/profile callbacks (tracing_is_suppressed) - Fix resume_gen_frame: skip light frames in previous chain to avoid dangling pointers after generator suspend - Fix faulthandler: emit '<no Python frame>' when chain has only light frames Assisted-by: Claude * Fix materialized frame refcount: transfer localsplus ownership on exit Remove @expectedFailure from test_futures2: the underlying issue was that materialized light frames cloned localsplus values (refcount +1 each), keeping objects alive until the next GC cycle collection. Fix: after run_light_frame returns, transfer ownership of localsplus values from the light frame to the materialized frame, replacing the clones. This eliminates the extra refcount, allowing objects to be GC'd immediately when their last reference is dropped. Also stabilize materialized frame's localsplus on the heap in sync_materialized_on_exit before GC tracking — the data-stack backing will be popped imminently. Assisted-by: Claude * Fix rustfmt, increase stack margin, address review comments - Fix rustfmt formatting issues (lint CI failure) - Increase STACK_MARGIN_BYTES from 2048 to 3072 words to prevent C stack overflow in deep recursion tests on macOS CI (test_functools SIGSEGV) - Skip dispatch_traced_frame when use_tracing is false (review comment) - Fix post-fork top_frame initialization: walk past light frames to find nearest heavy frame (review comment) Assisted-by: Claude * Remove redundant test override in test_futures2 The override only called super() without any marker, making it a no-op that prek flags as a redundant test patch. Assisted-by: Claude * Fix test_pdb and test_functools CI failures - Revert with_frame dispatch_traced_frame skip: tracing can be enabled mid-execution (pdb.set_trace), so Return events must always be checked. This fixes 3 test_pdb doctest failures across all platforms. - Check C stack overflow on every with_frame call instead of every 64th: the previous sampling approach missed overflows between checks when native stack frames are large (invoke_light_slots + lru_cache recursion). - Add #[inline(never)] to invoke_light_slots to prevent the optimizer from merging its stack frame into callers. - Add early C stack check before DataStack allocation in invoke_light_slots. Assisted-by: Claude * Rename Frame to FrameObject, FrameRef to FrameObjectRef Prepare for _PyInterpreterFrame/PyFrameObject separation by renaming the PyObject-backed frame type to FrameObject. Python-visible name ("frame") is unchanged. Assisted-by: Claude * Remove Deref<Target = InterpreterFrame> from FrameObject Replace implicit field access through Deref with explicit iframe()/iframe_ref()/iframe_mut() calls. This decouples FrameObject from InterpreterFrame field layout, enabling independent modification of InterpreterFrame in future commits. Assisted-by: Claude * Move InterpreterFrame identity fields to raw pointers InterpreterFrame's code/globals/builtins/func_obj fields are now borrowed raw pointers instead of owned PyRef/PyObjectRef. Ownership of these references is anchored in FrameObject's new owned_code, owned_globals, owned_builtins, owned_func_obj fields. This aligns InterpreterFrame's layout with LightFrame's existing raw-pointer pattern, enabling future DataStack-based allocation without PyObject heap allocation. - Add code()/globals()/builtins()/func_obj() accessor methods - Add FrameObject::new_ref() for combined create+allocate+patch - Update ExecutingFrame to use &Py<PyCode> / &Py<PyDict> / &PyObject - Update monitoring.rs functions to take &Py<PyCode> instead of &PyRef<PyCode> - Add Send+Sync impls for InterpreterFrame (raw pointers are !Send by default) Assisted-by: Claude * Remove LightFrame, FrameChainPtr, and dual-frame-type infrastructure Delete LightFrame struct, FrameChainPtr tagged pointer, FrameSource enum, and all supporting functions: materialize_light_frame, sync_light_to_materialized, transfer_localsplus_to_materialized, sync_materialized_on_exit, run_light_frame, invoke_light_slots body, LocalsPlus::from_datastack_raw. All call paths now use a single frame type (FrameObject wrapping InterpreterFrame). The frame chain is a simple *const FrameObject linked list through InterpreterFrame.previous. invoke_light_slots delegates to invoke_exact_args_slots. ExecutingFrame.frame_source replaced with ExecutingFrame.frame. Frame chain walking simplified throughout (no more light/heavy dispatch). Net: -724 lines of dual-frame-type complexity. Assisted-by: Claude * Clean up invoke_light_slots and _vm function variants - Replace invoke_light_slots calls with invoke_exact_args_slots - Delete the invoke_light_slots delegate method - Replace current_thread_frame_vm/frame_at_offset_vm with their non-_vm equivalents (no more light frame materialization needed) - Remove stale light-frame comment in callable.rs Assisted-by: Claude * Fix CI lint issues - Move github context expressions from run blocks to env blocks (zizmor/template-injection) - Add workflow-level permissions: {} to lib-deps-check.yaml - Suppress zizmor excessive-permissions for lib-deps-check.yaml (pull_request_target is required for PR comments) - Apply rustfmt formatting Assisted-by: Claude * Stack-allocate InterpreterFrame for non-generator function calls - Extract InterpreterFrame::new() from FrameObject::new() - Change frame chain from *const FrameObject to *const InterpreterFrame - Add InterpreterFrame.materialized field for lazy FrameObject creation - Add vm.with_iframe()/run_frame_fast() for stack-allocated frame execution - Move ExecutingFrame.frame to iframe: *const InterpreterFrame - Implement lazy materialize() for on-demand FrameObject creation - Update invoke_with_locals/invoke_prepared_exact_args to skip heap allocation - Update frame chain walking in faulthandler, gc_state, builtins/frame - super() reads InterpreterFrame directly without materialization Assisted-by: Claude * Remove unnecessary refcount and atomic ops from fast call path - Use raw pointer for func_obj instead of cloning PyObjectRef (saves 2 atomic RMW) - Skip owner AcqRel swap for stack-allocated frames (always Thread-owned) Assisted-by: Claude * Change trace field to Option<PyObjectRef>, remove vm from InterpreterFrame::new - trace: PyMutex<PyObjectRef> -> PyMutex<Option<PyObjectRef>> - Eliminates vm.ctx.none() refcount inc/dec per frame init - Remove vm parameter from InterpreterFrame::new() (no longer needed) - f_trace getter returns None when trace is unset - f_trace setter stores None instead of the None singleton Assisted-by: Claude * Store current frame pointer on VirtualMachine for fast lookup - Add current_frame_ptr Cell<usize> field to VirtualMachine - Hot path (with_iframe) reads/writes vm.current_frame_ptr instead of TLS - TLS CURRENT_FRAME still maintained for signal-safe traceback walking - Use set_current_frame_nosave() to skip cross-thread top_frame update - super(), frame_at_offset, current_thread_frame_materialize use vm Cell Assisted-by: Claude * Apply rustfmt Assisted-by: Claude * Fix clippy use_self warnings, add inline hints for hot path Assisted-by: Claude * Remove code refcount clone from fast call path Defer code.to_owned() to the generator/coroutine path only. On the fast path, use &Py<PyCode> directly — the code object is alive via the PyFunction on the caller's stack. Assisted-by: Claude * Remove VM current_frame_ptr Cell (fix SIGSEGV in threading builds) The Cell<usize> on VirtualMachine could desynchronize from the TLS CURRENT_FRAME in nested VM scenarios (enter_vm called with different VMs on the same thread). Revert to TLS-only frame chain management. Keep set_current_frame_nosave() for the stack-frame fast path: it skips cross-thread top_frame publication but still does the TLS swap. Fix clippy not_unsafe_ptr_arg_deref in set_current_frame. Fix unnecessary Result wrapper in make_generator_or_coro. Assisted-by: Claude * Fix CI failures: GC frame assertion, faulthandler Radium import, dead code warning - Fix SIGSEGV in threading builds: GC debug assertion walked frame chain via top_frame, which is not updated by set_current_frame_nosave used by stack-allocated iframes. Walk CURRENT_FRAME TLS chain instead. - Add InterpreterFrame::get_lasti() accessor, remove direct Radium import from faulthandler.rs (fixes unused import warning). - Add cfg_attr allow(dead_code) for from_payload_ptr without threading (fixes WASM build warning). Assisted-by: Claude * Fix materialized frame: copy localsplus, clear stale previous pointers - materialize_slow: copy fastlocals snapshot to heap instead of empty localsplus (fixes index-out-of-bounds panic in locals()/f_locals) - materialize_slow: set previous to null (stack iframes become dangling) - with_frame/resume_gen_frame: clear iframe.previous on pop (prevents dangling pointers to freed stack iframes) - f_back: walk TLS CURRENT_FRAME chain instead of top_frame for cross-thread lookup (top_frame is stale for stack-allocated iframes) - check_locals_access: match materialized frames by comparing the Py<FrameObject> address stored in the stack iframe's materialized field Assisted-by: Claude * Fix Windows clippy: gate unix-only imports and variables FrameObject, Py imports and top_iframe variable are only used in unix+threading cfg blocks. Gate them properly for non-unix builds. Assisted-by: Claude * Fall back to FrameObject when tracing; fix cross-thread frame access - invoke_with_locals: fall back to heap-allocated FrameObject path when use_tracing is active, so trace/profile callbacks fire correctly. This fixes test_trace, test_bdb, test_monitoring regressions. - set_current_frame: publish top_iframe alongside top_frame in ThreadSlot so cross-thread readers (sys._current_frames) can materialize stack-allocated frames from other threads under stop-the-world. - get_all_current_frames: use TLS CURRENT_FRAME for current thread, fall back to top_iframe materialization for other threads. - Use set_current_frame (not nosave) in with_iframe for proper cross-thread visibility. Assisted-by: Claude * Apply rustfmt Assisted-by: Claude * Fix Windows: gate unix-only top_iframe_ptr variable Assisted-by: Claude * Fix frame chain: f_back, retained_back, owner, faulthandler - Set materialized frame owner to FrameObject (not Thread) so frame.clear() works on traceback frames - Fix f_back for materialized frames: walk TLS chain to find source iframe and materialize its previous frame - Fix faulthandler dump_traceback to use dump_live_frames (reads live lasti from stack iframes instead of stale materialized copies) - Add retained_back propagation in with_frame cleanup: when a frame escapes, materialize caller and mark it escaped for chain propagation - Fix release_datastack_frame: don't overwrite retained_back if already set by with_frame cleanup Assisted-by: Claude * Apply rustfmt Assisted-by: Claude * Refine with_frame retained_back: use frame_obj instead of materialize Avoid materializing caller frames in with_frame cleanup to prevent creating localsplus snapshots that keep extra refcounts. Only use existing FrameObjects (frame_obj) for retained_back. Also keep previous pointer alive in release_datastack_frame since the caller is still executing at that point. Assisted-by: Claude * Fix f_lineno: use Cell<u32> for prev_line, update on every instruction prev_line was only updated when tracing was active, causing f_lineno to return stale values when observed mid-CALL (e.g. from warnings.warn or sys._getframe). Changed prev_line from u32 to Cell<u32> for interior mutability so it can be read safely while ExecutingFrame holds a shared reference, and update it on every instruction so f_lineno always returns the correct line. Assisted-by: Claude * Fix f_lineno for materialized frames: read live prev_line from TLS chain Materialized FrameObjects have a snapshot of prev_line from materialize time. When f_lineno is called on a materialized frame that is still executing (its source iframe is on the TLS chain), walk the TLS chain to find the source iframe and read its live prev_line. This fixes lineno tracking in warnings, gettext, and other modules that read frame line numbers during function calls. Also fixed the pointer comparison: use from_payload_ptr to convert FrameObject payload address to Py<FrameObject> address for matching against the materialized field. Assisted-by: Claude * Sync live frame state for materialized frames - Add find_live_source_iframe() to walk TLS chain and find the live source InterpreterFrame for a materialized FrameObject - f_lasti: read live lasti from source iframe when available - f_lineno: use read_volatile for live prev_line from source iframe - sync_visible_locals_to_mapping: read live localsplus from source iframe - framelocalsproxy_getval: read live localsplus from source iframe - with_iframe cleanup: sync localsplus, prev_line, lasti to materialized FrameObject when the frame returns - sync_fastlocals_from: new LocalsPlus method to update fastlocals Fixes test_inspect (stale f_locals, stale lineno in positions), test_listcomps (stale iteration variable in locals()). Assisted-by: Claude * Fix retained_back: use materialize_chain and strong_count check - Add materialize_chain() that creates a lightweight FrameObject with empty localsplus for f_back chain building only. This avoids cloning local variables which would create extra refcounts (fixes test_memoryview). - with_frame cleanup: use strong_count > 1 instead of escaped flag to determine if retained_back is needed, ensuring chain propagation. - with_iframe cleanup: use materialize_chain for retained_back to avoid extra refcounts from localsplus snapshots. Fixes test_memoryview (refcount from retained_back chain) and test_traceback.test_extract_stack (f_back chain depth). Assisted-by: Claude * Fix cross-thread f_back chain for sys._current_frames/exceptions - get_all_current_frames: materialize entire frame chain and link retained_back during stop-the-world for other threads - f_back: use stop-the-world to safely materialize cross-thread frame chains on unix when prev iframe is on another thread - f_back: check frame_obj() before STW for fast path Fixes test_sys.test_current_frames and test_sys.test_current_exceptions. Assisted-by: Claude * Fix monitoring LINE events and faulthandler thread dump - Skip all instrumented opcodes in bytecode loop prev_line update to avoid defeating InstrumentedLine de-duplication - Add prev_line update in execute_instrumented for non-RESUME/non-LINE instrumented opcodes - update_events_mask: walk TLS iframe chain to re-instrument all frames including stack-allocated ones (fixes missing LINE events for inline frames already past RESUME) - faulthandler: use top_iframe fallback for stack-allocated frames in dump_all_threads and watchdog_thread - faulthandler: use dump_live_frames for current thread dump Fixes test_monitoring LINE/CALL tests, test_faulthandler thread dumps. Assisted-by: Claude * Fix clippy: remove duplicate Radium import, use previous() helper Assisted-by: Claude * Fix materialized frame issues: GC tracking, live locals, f_trace propagation - Track materialized FrameObjects in GC so cycle collection works (fixes __del__ not called at exit for exception traceback cycles) - materialize_chain returns owned PyRef without temporary_refs to avoid defeating GC cycle detection from non-GC-tracked storage - framelocalsproxy_setval writes to live source iframe when available (fixes f_locals proxy writes not affecting executing frame) - f_lineno uses lasti-based line for non-executing frames instead of prev_line (fixes PEP 626 wrong line after exception unwind) - f_trace/f_trace_lines/f_trace_opcodes setters propagate to live source iframe (fixes pdb set_trace not working through materialized frames) - frame.clear() checks find_live_source_iframe to reject clearing a frame that is backed by a live stack-allocated iframe - Fix clippy warnings and unused imports in faulthandler - Remove unsafe top_iframe access from faulthandler watchdog thread Assisted-by: Claude * Fix faulthandler crashes, GC tracking timing, and test_generators - Use top_iframe instead of top_frame for frame chain walking in faulthandler (dump_all_threads and watchdog), avoiding stale or GC-cleared FrameObject pointer dereference - Defer GC tracking of materialized FrameObjects to with_iframe cleanup (after set_current_frame restores old chain), preventing premature collection while temporary_refs still holds the only ref - Add FrameObject::try_iframe() for safe access to potentially GC-cleared frames - Remove expectedFailure for test_exhausted_generator_frame_cycle (now passes with GC tracking fix) - Add 'noalias' to cspell dictionary Assisted-by: Claude * Fix Windows frame dump and remove dead code - Publish top_iframe on all platforms (not just unix) so Windows faulthandler and sys._current_frames can see stack-allocated frames - Use top_iframe instead of frames Mutex in Windows faulthandler dump and watchdog paths - Use top_iframe in non-unix get_all_current_frames for sys._current_frames - Remove unused dump_traceback_thread_chain (replaced by top_iframe walk) - Mark test_pdb_await_support as expected failure Assisted-by: Claude * Fix ThreadSlot init on all platforms, skip test_pdb_await_support - Add top_iframe to ThreadSlot initializer in init_thread_slot_if_needed (fixes wasm32, Windows compile errors) - Skip test_pdb_await_support on RustPython (async pdb exception callback receives None exc argument) Assisted-by: Claude * Remove dead faulthandler code (dump_frame_from_ref, dump_traceback_thread_frames, try_iframe) These functions are no longer called after switching all frame chain walking to use top_iframe directly. Assisted-by: Claude * Fix Windows sys._current_frames: materialize full frame chain The non-unix get_all_current_frames path was only materializing the top iframe without linking retained_back, so f_back chain walking could not find deeper frames like f123() in test_sys tests. Replicate the full chain materialization from the unix path. Assisted-by: Claude * Use STW-based cross-thread f_back on all platforms Replace the non-unix frames-mutex fallback with the same stop-the-world iframe chain materialization used on unix. Fixes test_current_exceptions on Windows where stack-allocated iframes were not found in the FrameObject-only frames list. Assisted-by: Claude * Fix set_f_lineno to write to live iframe, use STW for cross-thread access - set_f_lineno: write lasti/pending_stack_pops to the live source iframe instead of the materialized copy so debugger jumps take effect on stack-allocated frames - f_back cross-thread: enter STW before dereferencing the prev pointer to prevent use-after-free if the owning thread returns - get_all_current_frames (non-unix): add STW protection when walking cross-thread iframe chains, matching the unix path Assisted-by: Claude * Guard current_location() against lasti==0, always capture retained_back - current_location(): return first_line_number instead of panicking when lasti is 0 (before first instruction executes) - with_frame retained_back: materialize the caller iframe when needed so f_back always resolves for escaped FrameObjects Assisted-by: Claude * Fix format, revert eager retained_back materialization Remove double blank line in frame.rs (lint failure). Revert with_frame retained_back to only use already-materialized caller FrameObjects — eagerly materializing the caller added refcounts on local variables, preventing timely deallocation and causing test_io, test_memoryview, and test_futures failures. Assisted-by: Claude * Remove rustpython-unicode-isolation-issue.md from gitignore Assisted-by: Claude * Remove test_pdb_await_support skip: async pdb now works correctly Assisted-by: Claude * Fix STOP_ITERATION monitoring: wrap value in StopIteration instance fire_stop_iteration was passing the raw iterator return value to callbacks, but the STOP_ITERATION event callback signature expects a StopIteration exception instance. Wrap non-StopIteration values in a new StopIteration(value), matching PyMonitoring_FireStopIterationEvent. This fixes test_pdb_await_support where bdb's exception_callback received None instead of a StopIteration instance. Assisted-by: Claude	2026-08-01 00:59:41 +09:00
박효종	6a7d0fd144	Allow lone-surrogate keyword keys in f(**d) (#8409) ... * Allow lone-surrogate keyword keys in f(**d) `f(**{'\udc81': 2})` raised `TypeError: keywords must be strings` even though the key is a valid `str`, because `KwArgs` stored keys as Rust `String` (strict UTF-8) and `collect_ex_args` narrowed each key to `PyUtf8Str` (valid UTF-8 required). CPython only checks that the key is a `str` (`Py_TPFLAGS_UNICODE_SUBCLASS`), never encoding validity. Change `KwArgs<T>`'s key type from `String` to `Wtf8Buf` (the WTF-8 storage `PyStr` already uses) and relax the keyword-key downcast from `PyUtf8Str` to `PyStr` in `collect_ex_args`, `from_vectorcall`/ `from_vectorcall_owned` (which previously panicked on surrogate keys), the `functools.partial` keyword merge, and the C-API `dict_to_kwargs`. `Wtf8Buf` borrows only as `Wtf8`, so inherent `get`/`contains_key`/ `swap_remove`/`shift_remove(&str)` on `KwArgs` restore the `&str` lookup interface `String: Borrow<str>` used to provide (via the zero-cost `Wtf8::new` cast), and a generic `FromIterator<(K: Into<Wtf8Buf>, T)>` keeps construction sites unchanged. WTF-8 awareness stays localized to `function/argument.rs`. Fixes #8228 Assisted-by: Claude Code:claude-opus-4-8 * Avoid cloning kwargs keys in _ast, borrow via as_ref instead `new_str` only needs a `&Wtf8`, so pass `key.as_ref()` rather than cloning the key into an owned `Wtf8Buf`. The key is reused afterwards (error message, `intern_str`), so a borrow is the right fit. Assisted-by: Claude Code:claude-opus-4-8	2026-07-30 22:13:22 +09:00
Bas Schoenmaeckers	4d80a3320f	Fix some pyo3 testcase against the c-api (#8349)	2026-07-25 20:07:44 +09:00
Bas Schoenmaeckers	04a203c955	Fix deadlock in c-api when a thread enters a stop_the_world (#8355)	2026-07-24 02:43:31 +09:00
Bas Schoenmaeckers	455d22b353	Add more unicode functions to the c-api (#8272) ... * Add more unicode functions Review Add unicode decode functions Add more functions * Remove * Add helper	2026-07-22 20:28:51 +09:00
강동안	12057b3a03	Take a borrow in invoke_exception instead of an owned type (#8327)	2026-07-20 17:06:44 +09:00
Jeong, YunWon	c7d265e8dd	ctypes: fix Windows pip truststore (#8314) ... * ctypes: preserve result when errcheck returns args * ctypes: cover Windows pip truststore * capi: implement PyEval_SaveThread * capi: merge saved thread state into PyThreadState	2026-07-20 15:21:42 +09:00
김은빈	d26972e3bc	weakref: reuse proxy objects (#8307) ... * weakref: reuse callback-less proxy objects like CPython Make PyWeakProxy a #[repr(transparent)] view sharing PyWeak's payload (manual PyPayload with shared PAYLOAD_TYPE_ID + class check), so a proxy is the weakref-list node itself and add()'s reuse applies to it directly. Port CPython's insert_weakref/get_basic_refs slot discipline: genericref at head, generic proxy right after it, with callback/class re-verificationbefore reuse. Constructor moves to a slot_new override since reuse returns an existing object. Drops the hidden __weakproxy marker subclass and the proxy branch in PyWeakref_GetRef. Assisted-by: Claude Code:claude-opus-4-8 * weakref: always validate class in basic-proxy slot lookup * weakref: rustfmt find_generic_proxy_ptr * weakref: unify reuse check shape	2026-07-18 23:47:03 +09:00
Bas Schoenmaeckers	c5efd71f2a	Add codecs support to c-api (#8267) ... * Add codecs support to c-apo * Reduce code duplication	2026-07-17 00:19:31 +09:00
Bas Schoenmaeckers	96fde50f75	Add AsyncIterator support to c-api (#8285)	2026-07-15 23:41:36 +09:00
Bas Schoenmaeckers	ac28fff085	Add descriptor support to c-api (#8283) ... * Add descriptor support to c-api * Review * Extract getset builder * Extract memberdef builder	2026-07-15 23:41:18 +09:00
Bas Schoenmaeckers	57b630a6c7	Add str compare functions to c-api (#8268)	2026-07-15 23:24:26 +09:00
Bas Schoenmaeckers	7f81b2491c	Add more dict functions to c-api (#8269) ... * Add more dict functions to c-api * Review	2026-07-15 23:19:41 +09:00
Bas Schoenmaeckers	b6ade8bde6	Add more eval functions to c-api (#8244)	2026-07-09 14:14:56 +09:00
Bas Schoenmaeckers	5ca7d17f60	Add memoryview support to c-api (#8240)	2026-07-09 14:12:10 +09:00
Bas Schoenmaeckers	61d91d6fb4	Add garbage collection c-api (#8239)	2026-07-09 14:08:23 +09:00
Bas Schoenmaeckers	3a4b4a8c65	Add more build-info functions to c-api (#8216) ... * Add more build-info functions to c-api * Use CStr for COMPILER/COPYRIGHT/PLATFORM	2026-07-09 14:07:28 +09:00
Jeong, YunWon	210ccc889c	Thread-safe type operations: type lock, QSBR type-cache reclamation, GC stop-the-world, and interpreter optimizations (#7416) ... * type lock * Drop old PyObjectRef outside type lock to prevent deadlock Dropping values inside with_type_lock can trigger weakref callbacks, which may access attributes (LOAD_ATTR specialization) and re-acquire the non-reentrant type mutex, causing deadlock. Return old values from lock closures so they drop after lock release. * Align type lock behavior with CPython * Add PUBLISHED flag bit to RefCount state word Assisted-by: Claude * Add QSBR module for deferred memory reclamation Assisted-by: Claude * Defer memory free of cache-published objects via QSBR Assisted-by: Claude * Wire QSBR checkpoints into thread lifecycle and eval breaker Assisted-by: Claude * Publish type-cache values to QSBR and bypass cache without a VM Mark cached type-method values as QSBR-published before storing their pointer in TYPE_CACHE, so racing readers never try-incref freed memory. Debug-assert that the current thread is ATTACHED whenever a lock-free cache read happens. Delete find_name_in_mro_without_vm and its unsound direct SeqLock read outside any VM/thread registration; find_name_in_mro now falls back to the locked, uncached MRO walk when no VM is current. This also removes assign_version_tag()'s last caller, so its unlocked fallback is deleted along with it (version_for_specialization keeps the locked path). VirtualMachine::initialize() runs Python bytecode (e.g. importing codecs/encodings) before any enter_vm scope exists, which left the bootstrap thread not ATTACHED for cache reads during startup. Add thread::VmBootstrapGuard, an RAII counterpart to enter_vm usable across statements that need &mut VirtualMachine, and wrap initialize() with it. Assisted-by: Claude * Use try-incref reads and QSBR-backed swaps in specialization cache Assisted-by: Claude * Reset QSBR thread registry after fork in the child Add Qsbr::reset_after_fork, which clears the registered thread slots before draining the retire queue. Call it in py_os_after_fork_child, right after type_cache_after_fork() and before _thread::after_fork_child() re-registers the surviving thread. Dead parent threads' Arc<QsbrSlot> handles live on in the child's copied memory with no destructor ever running, so without this the registry keeps them 'online' forever and future grace periods never complete. Assisted-by: Claude * Add threaded stress test for type cache mutation races Assisted-by: Claude * Restrict qsbr internals to pub(crate) and fix clippy lints Downgrade pub items in the private qsbr::threading module (QsbrSlot, Qsbr, QSBR, and its methods) to pub(crate), since they were never reachable outside the crate. Downgrade ThreadSlot::qsbr accordingly to avoid a private-interfaces warning. Import Arc/Weak from alloc instead of std to match clippy::std_instead_of_alloc. Rewrite the process() comment: goals are not strictly ordered by push order since advance() and the queue push aren't atomic together, so concurrent free_delayed calls can interleave; the drained prefix stays sound because each item individually passed poll, with reordering only affecting reclamation latency. Assisted-by: Claude * Gate per-instruction QSBR check behind a global pending flag eval_breaker_tripped() and check_signals() called thread::qsbr_break_requested() on every bytecode instruction, which does a thread-local lookup, RefCell borrow, and atomic load even when no QSBR retirement is pending. This caused a measured 10-18% slowdown on tight interpreter loops. Add Qsbr::pending, a global AtomicBool set under the queue lock while free_delayed() holds a retired allocation, and cleared under the same lock by process()/drain_all() once the queue empties. The hot path now checks Qsbr::break_pending() (a single relaxed static load) before touching thread-local state, and only pays the TLS cost while a retirement is actually in flight. Assisted-by: Claude * Merge signal and QSBR eval-breaker flags into one atomic word Replace the separate ANY_TRIGGERED AtomicBool and QSBR pending AtomicBool checks in the per-instruction eval-breaker path with a single AtomicU8 (EVAL_BREAKER) holding a bit per source. The global QSBR instance mirrors its pending state into the QSBR bit under the same queue-lock critical sections that already toggle pending, while local QSBR instances used by unit tests are left untouched. eval_breaker_tripped() and check_signals() now read/act on the merged word (eval_breaker_pending(), qsbr_bit_set()) instead of issuing two separate atomic loads per instruction. set_triggered() switches from store to fetch_or so a signal handler never clobbers the QSBR bit. Removed the now-unreferenced is_triggered() helper; break_pending() is kept for unit tests only and marked allow(dead_code) outside test builds. Assisted-by: Claude * Skip freelist reuse for published objects in default_dealloc Freelist-eligible payloads (tuple, int, list, dict, float, ...) were routed to T::freelist_push before ever reaching PyInner::dealloc, so a published object (e.g. a tuple cached as a type attribute) bypassed the is_published -> free_delayed QSBR hook entirely. That let PyRef::new_ref reuse the slot and rewrite the refcount word with a non-atomic core::ptr::write racing a reader's atomic try-incref, and let FreeList::drop call alloc::dealloc directly while a reader could still be mid-read. Gate the freelist branch in default_dealloc on RefCount::is_published(), checked before any teardown. A published object now always falls through to PyInner::dealloc, whose existing hook defers the memory free via QSBR. Document the non-unix (e.g. Windows) reclamation-latency limitation in the QSBR design doc's Lifecycle integration section (previously only noted in a code comment in vm/thread.rs). Extend the type-cache stress test so the mutator also churns a freelist-eligible published tuple class attribute (C.shape), read by readers each inner loop and deleted on the same cadence as C.m, to exercise the fixed bypass. Assisted-by: Claude * Use itoa for PyInt::to_str_radix_10 i64 fast path Format small integers with itoa::Buffer instead of i64::to_string() in PyInt::to_str_radix_10(), which backs both str(int)/repr(int) for exact int and PyObject::str()'s exact-int fast path. Adds the itoa crate as a workspace dependency. Assisted-by: Claude * Make object attribute dunders wrapper descriptors Remove the __getattribute__, __setattr__ and __delattr__ #[pymethod]s from PyBaseObject so add_operators installs PyWrapper slot wrappers for them from the #[pyslot] functions instead of method descriptors. lookup_slot_in_mro now classifies these entries as NativeSlot, so heap types without Python-level overrides keep the native getattro/setattro slot functions, which lets LOAD_ATTR/STORE_ATTR specialization apply to instances of Python-defined classes. Since __setattr__ and __delattr__ share the setattro slot, resolve both names together in update_one_slot: overriding only one of them no longer lets the other name's native resolution overwrite the dispatching wrapper. Assisted-by: Claude * Resolve TpSetattro pair on attribute deletion, not just addition update_one_slot's TpSetattro branch only ran the combined __setattr__/__delattr__ resolution (added in the previous commit) when ADD was true. On deletion (e.g. del C.__setattr__), it fell back to inherit_from_mro, which copies the base class's slot and discards the class's own remaining Python-level override of the other name. lookup_slot_in_mro reads the current attribute dicts, so the same resolution is correct for both addition and deletion; the ADD-only guard is removed and the logic now always runs. Assisted-by: Claude * Generate Opcode::cache_entries/deopt as table lookups generate_rs_opcode_metadata.py now emits a 256-entry const array for each function instead of a chained match over opcode names. Opcode::deopt() indexes an [Option<Opcode>; 256] table built from the existing specialization-to-base mapping, and Opcode::cache_entries() indexes a [u8; 256] table that bakes in the same deopt/to_base composition deoptimize() previously performed before its own match. Regenerated opcode_metadata.rs from the updated script; PseudoOpcode's bodies are unchanged since it has no cache/deopt entries to table. Added an exhaustive test in instruction.rs that checks every valid u8 opcode against a frozen copy of the old chained-match bodies. Assisted-by: Claude * Make Opcode/Instruction numeric conversions O(1) Opcode::as_numeric, Opcode::as_instruction and Instruction::as_opcode were each a per-variant match over ~230 arms in the define_opcodes! macro. Opcode now carries the same explicit $op_id discriminants and #[repr($typ)] as Instruction, which makes the conversions sound as: - Opcode::as_numeric: a plain `self as $typ` identity cast. - Opcode::as_instruction / Instruction::as_opcode: a mem::transmute, relying on both enums sharing one-$typ-wide layout (Instruction's payload fields are all the zero-sized Arg<T> marker, checked by a new per-instantiation size_of assertion). try_from_numeric is left as a match: $op_id values have gaps (specialized/instrumented opcode ranges), so a range check can't replace it. Adds #[inline] to the rewritten conversions plus deoptimize, and to the small generated wrapper/table-lookup functions from the previous opcode-metadata change (as_u8/as_u16, cache_entries, deopt, to_base) via generate_rs_opcode_metadata.py, then regenerates opcode_metadata.rs. Extends the instruction.rs equivalence-test pattern with two exhaustive tests (u8 and u16 instantiations) checking the new identity-cast/transmute conversions against the untouched try_from_numeric/TryFrom paths. Assisted-by: Claude * Pop CallIsinstance arguments directly instead of collecting into Vecs The specialized handler built two heap-allocated Vecs per call (pop_multiple().collect() plus a with_capacity(2) buffer) before invoking is_instance. Pop the class and instance straight off the stack for the effective two-argument case, removing the per-call allocations. Assisted-by: Claude * Inherit tp_new instead of always installing new_wrapper update_one_slot's TpNew branch stored new_wrapper for every ADD, so all heap types had slots.new == new_wrapper and the CallAllocAndEnterInit specialization never fired. Now __new__ is resolved through the MRO dicts: an own or Python-level definition installs new_wrapper, while a builtin __new__ entry (or none) inherits slots.new from the solid base. The now-reachable CallAllocAndEnterInit path gains the missing guards: the specializer requires co_argcount == oparg + 1 (via can_specialize_call) and rejects generator-like __init__, and the handler re-checks the argcount against the class-level init cache. Without the argcount check, __init__ defaults broke re._parser (UnboundLocalError in SubPattern.__init__). Remove the now-unused is_simple_for_call_specialization. Assisted-by: Claude * Skip locals dict allocation for the init-cleanup shim frame Mark the shim code object NEWLOCALS and create the shim frame with no locals mapping so FrameLocals::lazy() is used. The shim only executes ExitInitCheck/ReturnValue and never touches locals, so this removes one dict allocation per specialized instantiation. Assisted-by: Claude * Guard CALL_ALLOC_AND_ENTER_INIT with cached function version The init specialization cache stored only the type version, so swapping __init__.__code__ (which zeroes func_version) did not deopt the warmed call site and stale code assumptions (argcount shape, no varargs/kwonly, not a generator) were applied to the new code object. Store get_version_for_current_state() in the specialization cache when caching __init__ and re-check func_version() in the handler before using the cached function, deopting to the generic call path on mismatch, the same way the getitem specialization cache does. Assisted-by: Claude * Apply rustfmt and ruff-format fixes to files from earlier commits Assisted-by: Claude * Add frame object freelist Frame.iframe becomes FrameUnsafeCell<Option<InterpreterFrame>> so a dead frame can be left as an empty husk. Traverse::clear extracts all owned child references (including localsplus values via LocalsPlus::clear_into) and empties the cell. Dead frame allocations are cached in a thread-local FreeList<Frame> (up to 200 entries) and reused by PyRef::new_ref. clear_generator returns early when the frame was already cleared by cycle collection; generator drops can run after tp_clear of their frame. gc: skip tp_clear for objects saved to gc.garbage by DEBUG_SAVEALL, since they remain reachable from Python. Assisted-by: Claude * Drop frame children directly in tp_clear instead of extracting Pushing the frame's ~10 child references into the clear buffer grew a Vec on every frame deallocation. Take the interpreter frame out of the cell and drop it in place; LocalsPlus::clear_into is removed. Assisted-by: Claude * Untrack objects before tp_clear and guard cross-thread f_locals The cycle collector called tp_clear on unreachable objects while they were still linked into the GC generation lists, so another thread could obtain a strong reference to an already-cleared object (e.g. a frame husk with iframe == None) via gc.get_objects() and access the cleared payload. Untrack the dead set before the clear phase, mirroring the untrack-then-clear ordering of the refcount dealloc path. Objects that gained an external reference before untracking are found by comparing each object's strong count against the references coming from within the dead set; such late-resurrected objects and everything reachable from them are re-tracked and skipped, letting a later collection retry once the external references are released. Also reject f_locals access for a frame currently executing on another thread. Reading fastlocals of such a frame races with the executing thread overwriting the slots (torn reads of dropped values). Access from the executing thread itself (locals(), trace callbacks) is still allowed via the current-frame chain check. Assisted-by: Claude * Skip localsplus heap copy for uniquely referenced dying frames release_datastack_frame now checks the frame's strong count after untracking it from the GC generation lists. When the caller holds the only reference, localsplus values are dropped in place and the data stack storage is released without the heap copy; escaped frames (traceback, sys._getframe, trace callbacks) keep the copy path. The three inline materialize+pop call sites in function.rs are routed through release_datastack_frame. Assisted-by: Claude * Stage exact-args call arguments in stack slot buffers CallPyExactArgs, CallBoundMethodExactArgs and CallAllocAndEnterInit no longer collect popped arguments into a per-call Vec (plus a second Vec for self-prepending). Arguments are popped into a fixed-size Option<PyObjectRef> slot buffer (CallArgBuffer, 8 inline slots, heap fallback for larger arities) and moved into fastlocals via the new invoke_exact_args_slots / a take() iterator. - prepare_exact_args_frame now accepts an ExactSizeIterator of args - specialization_run_init_cleanup_shim takes the slot buffer and fills slot 0 with new_obj, dropping one redundant clone - LoadAttrGetattributeOverridden, LoadAttrProperty and BinaryOpSubscrGetitem pass inline slot arrays instead of vec![] Assisted-by: Claude * Reduce allocations in generic call paths - FuncArgs::prepend_arg: use reserve instead of reserve_exact so exact-capacity vectors do not realloc on every prepend. - IntoFuncArgs::into_method_args: build the final args vec once with capacity len + 1 instead of prepending into a full vector. - PyType::call: skip cloning FuncArgs when no init slot can run after slot_new (no init slot, not `type`, and slot_new is not new_wrapper). Assisted-by: Claude * Skip redundant exc_info restore on frame exit with_frame_impl saves the current exc_info slot value and restores it when the frame exits. When the slot still holds the same value, the restore rewrites the slot and recomputes the thread-exception mirror for no effect. Add restore_exception, which compares the slot against the saved value by pointer identity and skips the store when they match. The saved value is a strong reference held for the whole frame scope, so the pointed-to object cannot be freed and its address reused while the frame runs, making the pointer comparison free of ABA. Assisted-by: Claude * Apply rustfmt to gc_state.rs Assisted-by: Claude * Avoid empty FuncArgs clone in PyType::call Cloning args for the init call clones the kwargs map even when args is empty, which shows up in no-argument instantiation profiles. Prepare the init-call args before slot_new: a default FuncArgs when args is empty (indistinguishable from a clone of empty args), the existing clone otherwise. The empty check also keeps the init-slot load of the clone-elision path off the no-argument call path. Assisted-by: Claude * Push freelist husks only after tp_clear and child drops default_dealloc pushed the object onto the thread-local freelist before running clear_fn. Payloads that drop children inside clear (Frame) can run __del__ there, and a reentrant allocation could pop the husk and overwrite its payload while clear_fn still held a &mut borrow of it. Tuples had the same window through the extracted-edges drop, which ran after the push. Reorder default_dealloc to clear, drop the extracted children, and only then attempt the freelist push, so the husk becomes reusable only when no borrows into the payload remain. The tuple freelist bucketed husks by element count, which required reading the payload during push; after this reordering the elements are already cleared. PyInner<PyTuple> is a fixed-size allocation (the elements box is dropped and replaced on reuse), so replace the per-size buckets with the shared single-list FreeList. Remove the now-unused pyinner_layout helper. Assisted-by: Claude * Resolve __set__ and __delete__ together for the descr_set slot The descr_set slot serves both __set__ and __delete__, but the TpDescrSet accessor resolved only the single modified name and fell back to MRO inheritance on delete. Deleting one of the pair could disable the surviving operation: after `del D.__set__`, an instance delete stopped calling the remaining __delete__. Resolve both names together like the TpSetattro accessor does: any Python-level definition selects the dispatching wrapper, matching native functions are stored directly, a single native function with the other name absent is stored directly, and the slot is inherited from the MRO only when neither name resolves. Assisted-by: Claude * Update type base on __bases__ assignment set_bases replaced bases and mro but never updated the stored base, so __base__ kept reporting the old value after reassignment and slot resolution (tp_new inheritance, set_new/set_alloc during init_slots) read the stale solid base. Change the base field to PyAtomicRef<Option<PyType>> so it can be swapped under the type lock while remaining lock-free for readers, and recompute it in set_bases with the same best_base validation type creation uses (BASETYPE flag, instance layout conflict among bases). The swapped-out base is parked in the frame's temporary refs; other references released inside the critical section are dropped after the lock is released. Also in set_bases: - remove this class from the old bases' subclass lists (pruning dead entries), so repeated assignment no longer accumulates duplicates - roll back bases, base, and all updated mros when the recursive mro update fails, instead of leaving the type half-reparented - fix the misformatted empty-tuple error message Remove the expectedFailure marker from test_unsubclassable_types, which now passes. Assisted-by: Claude * Fix rollback order, dead weakrefs, and lock-held drops in set_bases - Restore recorded mros in reverse on rollback so a class visited multiple times through diamond inheritance ends with its original mro instead of an intermediate one. - Skip dead weakrefs in the subclass list during the recursive mro update instead of panicking on upgrade. - Retire the replaced mros on the success path instead of dropping them while the type lock is held. Assisted-by: Claude * Reify number sub-slot wrapper once in update_one_slot The update_sub_slot! macro expanded the Python-method wrapper closure at both the own and inherited store sites. Each expansion is a distinct fn item, so a base and a non-overriding subclass stored wrappers with distinct addresses in unmerged debug builds. binary_op1 compares slot fn addresses to detect whether a subclass overrides the operator, so the inherited slot was misread as an override and C() // E() dispatched to __rfloordiv__ instead of __floordiv__. Bind the wrapper store in a single closure and call it from both branches so the fn item is reified once and the slot value stays identical across a base and its subclasses. Document the conservative-on-mismatch nature of the new_wrapper address guard in call_wrapped. Assisted-by: Claude * Restrict type_cache_after_fork visibility and guard type-cache reads type_cache_after_fork is only called from within the crate, so mark it pub(crate) to resolve the unreachable-pub warning. Add debug_assert_current_thread_attached at the has_name_in_mro lock-free type-cache read site, matching lookup_ref_and_version_interned, and gate the function on debug_assertions so it is not compiled unused in release builds where the call sites are elided. Assisted-by: Claude * Guard BinaryOpSubscrGetitem with the cached type version The specialized BINARY_OP_SUBSCR_GETITEM handler used the cached __getitem__ after checking only the function version, unlike the sibling specialized handlers which revalidate the type version tag first. Store the type version in the inline cache at specialization time and revalidate owner.class().tp_version_tag against it before using the cached function, deopting to the generic subscript path on mismatch. Assisted-by: Claude * Call __abstractmethods__ __len__ once in object.__new__ The abstract-method check invoked the user-visible __len__ twice: once via length_opt for the count and again while materializing the method names. Derive the count from the materialized list instead, so __len__ runs once. Assisted-by: Claude * Use i64 fast paths for specialized int add/sub/mul Rewrite execute_binary_op_int to box results through new_int via i64 checked arithmetic instead of raw BigInt ops with new_bigint. Add an int_mul helper and a shared int_fast_op that computes the i64 result and falls back to the BigInt operation on to_i64 or checked-op failure. Wire int_mul into both the specialized BinaryOpMultiplyInt handler and the generic execute_bin_op Multiply/InplaceMultiply path, gated on exact int operands so subclasses keep dispatching through _mul/_imul. Assisted-by: Claude * Use i64 fast paths for exact int floordiv and remainder Add floordiv_i64/mod_i64 computing i64 floor-division and divisor-signed remainder, guarding zero divisor and i64::MIN overflow by returning None. Wire them through int_floordiv/int_mod (shared int_div_fast_op boxes via new_int) into the generic execute_bin_op FloorDivide/Remainder and their Inplace variants, gated on exact int operands so subclasses and zero divisors fall through to the existing _floordiv/_mod/_ifloordiv/_imod slow path. Assisted-by: Claude * Skip trashcan and untrack for non-GC-tracked objects in dealloc default_dealloc read is_gc_tracked() only to decide untracking, and entered the trashcan recursion guard unconditionally. Non-GC objects (int, float, str, ...) own no child references that recurse during deallocation, so they need neither the trashcan nor untracking. Read is_gc_tracked() once and gate both trashcan begin/end and untrack on it, removing three thread-local accesses per non-GC object deallocation. Assisted-by: Claude * Merge trashcan depth and defer queue into one thread-local struct trashcan::end accessed two separate thread-locals (DEALLOC_DEPTH and DEALLOC_QUEUE) at the outermost deallocation. Combine both into a single `Trashcan` thread-local holding Cell-based depth and queue fields, so begin and end each reach their state through one thread-local access. The queue is set back before each deferred dealloc call so reentrant begin/end during draining never holds an outstanding borrow. Assisted-by: Claude * Check object layout compatibility on __bases__ and __class__ assignment Add a shared compatible_for_assignment helper that walks each type to the base that fixes its instance layout and rejects the assignment when the old and new layouts differ (basicsize, itemsize, member count, __dict__, __weakref__, and __slots__ names). Wire it into set_bases, which previously performed no layout check, and replace the inline check in __class__ assignment, which compared the two types directly and over-rejected a subclass that adds no layout. Enable test_descr.test_builtin_bases, which the new set_bases check passes. Assisted-by: Claude * Stop the world around GC pointer-reading phases The cycle collector reads each tracked object's interpreter state during reference subtraction, the reachability walk and the strong-reference snapshot, including the localsplus of frames other threads are executing. Those slots are written without synchronization, so the reads are a data race under threading. collect_inner now stops the world before taking the generation read locks and restarts it after the strong-reference snapshot, before the finalizer/weakref/tp_clear phases. A debug assertion checks that no frame on any thread's call stack was classified unreachable. Automatic collections from maybe_collect are deferred to the next bytecode safepoint via a new eval-breaker GC bit instead of running synchronously: a synchronous collection can hold an internal lock (e.g. the lazy frame locals cell) that another thread is blocked on with no way to reach a safepoint, deadlocking the stop. Hardens the stop-the-world machinery for these frequent, concurrent GC stops: attach_thread now honors a pending stop after re-attaching so a thread doing rapid allow_threads calls cannot run past the requester; stop_the_world completion is level-triggered on all-threads-suspended rather than an edge-triggered countdown; and the thread-start started/ready handshakes detach while waiting so the waiter is parkable. Updates the Frame and PyRwLock/PyMutex traversal SAFETY comments to state the actual invariant. Assisted-by: Claude * Add threaded GC vs executing-frame stress snippet Workers churn frame state (recursion, generators, frame cycles) while a collector loops gc.collect() and an introspector walks live frame objects. Exercises the stop-the-world barrier around GC traversal. Assisted-by: Claude * Serialize fork and GC stop-the-world requesters fork() (posix before/after-fork) and the cycle collector both drive the single StopTheWorldState. With no mutual exclusion their requester word and suspension countdown could interleave and be clobbered, so the completion check never converged and a requester waited on itself forever (reproducible parent-side hang when forking with GC enabled). Add one exclusion held for the whole stop->start span of either requester: stop_the_world acquires it before any stop bookkeeping and start_the_world/reset_after_fork release it. The acquire is park-friendly (poll try-lock and honor a pending suspend between tries) so a requester blocked behind an active stop can still be force-parked instead of deadlocking the active requester. The acquirer holds no other lock while spinning. Also correct the PyRwLock traverse SAFETY note: a failed try_read may mean a force-parked thread holds the write lock; skipping is safe because under-traversal only over-approximates liveness. Document the residual gc.collect()-under-a-non-generation-lock exposure at the collect_inner barrier. Assisted-by: Claude * Add fork under concurrent GC stop-the-world snippet Worker threads allocate cyclic garbage with GC enabled while the main thread forks repeatedly and each child collects. Exercises the fork/GC stop-the-world exclusion; the allocation rate is throttled so a collection stays cheap in unoptimized builds. Assisted-by: Claude * Detach while acquiring the import lock The global import lock is held across bytecode by the importlib bootstrap, so its holder can be parked at a safepoint mid-hold. Acquiring it while attached let another thread block attached on the lock, so a stop-the-world requester could wait forever for that attached thread to suspend while the holder stayed parked. Wrap IMP_LOCK acquisition in allow_threads in both _imp.acquire_lock and the pre-fork acquire_imp_lock_for_fork so the wait honors stop-the-world requests. Correct the acquire_exclusion comment: a spinning requester may hold IMP_LOCK (fork) or the collecting mutex (GC); safety relies on those never being acquired attached-blocking by another thread. Assisted-by: Claude * Add concurrent-import vs GC deadlock snippet Two threads re-import modules (contending the import lock) while a third storms the cycle collector and a fourth allocates cyclic garbage. Regressed as a hang before the import lock acquisition was made park-friendly. Assisted-by: Claude * Create call frames untracked and track generators explicitly Add PyPayload::NEW_REF_UNTRACKED (default false, true for Frame) so PyRef::new_ref skips auto-tracking ordinary call frames in the GC. Generator/coroutine/async-generator frames are tracked explicitly in invoke_with_locals before their generator back-reference is installed. run_frame debug-asserts that a datastack frame is untracked on entry. Assisted-by: Claude * Track escaped call frames lazily at datastack release Rewrite release_datastack_frame around the invariant that a datastack frame is never GC-tracked while it runs: strong_count() == 1 means the frame never escaped, so drop localsplus in place without touching the GC; strong_count() > 1 means it escaped, so materialize localsplus onto the heap and then track the frame. This removes the untrack and the untrack-recheck dance from the common non-escaping path. Assisted-by: Claude * Guard the tracked-frame localsplus invariant Document at Frame::traverse that references to a frame are always recorded as graph edges, so the collector reads a frame's localsplus only when the frame is itself a tracked candidate. Add debug assertions at both frame track sites (escaped datastack frames and generator frames) that a frame is heap-backed before it is tracked, so a collector never reads data-stack-resident, still-mutating storage. Assisted-by: Claude * Reword frame/dealloc invariant comments and drop needless borrow - frame.rs: replace the task codename in the tracked-frame localsplus invariant comment with a description of the collector-vs-executing-frame behavior. - object.rs: remove needless borrow on current_cls in the __class__ assignment compatibility check. - core.rs: rewrite the default_dealloc trashcan/untrack-skip comment to state the actual invariant, which now covers untracked non-escaped frames releasing at interpreter depth with bounded recursion. Assisted-by: Claude * Move type SetAttr slot rewrite inside the type lock and tighten cache reads Run update_slot inside the same with_type_lock transaction as modified_inner and the attributes dict mutation, so the version invalidation, dict change, and slot-table rewrite are published together. Load init_version and getitem_version with Acquire in the specialization cache readers to pair with the Release stores in the writers. Assisted-by: Claude * Capture the type version before reading mutable slots in specializers specialize_load_attr, specialize_store_attr, specialize_to_bool, and the CallAllocAndEnterInit path of specialize_call read the version tag with version_for_specialization before inspecting getattro, setattro, the bool/len slots, and tp_new/tp_alloc, so a concurrent install of the corresponding dunder invalidates the version the specialization is cached against. In the BinaryOpSubscr __getitem__ path, check the HEAPTYPE and eval-frame gates before lookup_ref_and_version_interned, which takes the global type lock and may allocate a version tag. Assisted-by: Claude * Use _get_method_dict for the test_type check in the test runner Replace the direct __func__.__dict__ access with the _get_method_dict helper so plain functions without __func__ do not raise AttributeError. Assisted-by: Claude * Fix lint hooks: cspell dictionary entries, spelling fix, formatting Add qsbr to the rustpython dictionary and reborrows/reparenting to the top-level word list. Rename oldto/newto locals to old_to/new_to. Fix Stabilise -> Stabilize typo in a frame.rs comment. Apply cargo fmt to object.rs, type.rs, and frame.rs, and ruff format/check fixes to two extra_tests snippet files. Assisted-by: Claude * Rebuild all slots for type and descendants on __bases__ reassignment Add PyType::update_all_slots, which invalidates version tags and iterates the full SLOT_DEFS name table calling update_slot::<true> for each distinct name. Unlike init_slots, which is additive and driven only by dunder names present in the current MRO, this resets a slot whose method left the MRO instead of leaving a stale dispatcher pointer. Call it from set_bases in place of the previous modified_inner + init_slots pair, so reassigning __bases__ rebuilds slots for the type and every descendant. Add extra_tests/snippets/type_bases_slot_rebuild.py covering removed-method resets on zelf and deep descendants, the wrong-target switch, __getattr__, the added-method mirror, and a swap-away-and-back round trip. Assisted-by: Claude * Derive the thread-local frame stack from the frame chain Remove the per-VM `frames` Vec. current_frame, sys._getframe, sys._getframemodulename, frame.f_back, sys.monitoring re-instrumentation, gc.get_referrers, faulthandler stack dumps and the post-fork slot rebuild now walk the signal-safe CURRENT_FRAME/`previous` chain instead. Add `Py::from_payload_ptr` to recover a frame object from a chain pointer. The cross-thread `ThreadSlot::frames` registry (sys._current_frames, cross-thread f_back, GC stop-the-world assertion) is unchanged. Assisted-by: Claude * Inherit sub-slot fallback into the field being resolved update_one_slot's number/sequence/mapping fallback inherited via the accessor's default field. Left and right binary ops (add/right_add) share one accessor but occupy distinct fields, so resolving an absent right op or deleting it reset the left op's field, dropping a still-defined __add__ dispatcher. Inherit the exact field under resolution instead. Assisted-by: Claude * Gate type_cache_after_fork to its fork caller and use Self in downcast type_cache_after_fork is only called from the unix fork path, so gate it with all(feature = "host_env", unix) to match the caller and avoid a dead-code error on non-unix targets. Replace the explicit PyType with Self in the modified_inner downcast. Assisted-by: Claude * Unmark test_attr and test_method_call_error in test_monitoring Both TestLoadSuperAttr tests now pass; remove their expectedFailure markers. Assisted-by: Claude * Run tp_new specialization __init__ without a trampoline frame CallAllocAndEnterInit ran __init__ inside a synthetic init-cleanup shim frame whose code carried the __init__ name. Since the thread frame stack is derived from the frame chain, that shim frame was visible to sys._getframe, f_back walks, traceback construction and inspect.stack / inspect.trace. Its code object has empty co_positions, so a stack walk that reached it raised StopIteration inside inspect._get_code_position, cascading into unrelated failures (test_inspect trace/stack/frame, asyncio source traceback). Call __init__ directly via run_frame, enforce the __init__() should return None contract inline, and drop the shim: the init-cleanup code object, its builder, with_frame_untraced, monitoring_disabled_for_code, and the extra-frame datastack/recursion budget. Removing the second frame per construction also cuts the specialization's per-call cost. Assisted-by: Claude * Replace per-call thread-frame mutex with an atomic top-of-stack (unix) On unix threading builds, publish each thread's top Python frame in a single relaxed AtomicPtr store from set_current_frame instead of pushing onto a parking_lot::Mutex<Vec<FramePtr>> per call. Cross-thread readers (sys._current_frames, cross-thread f_back, faulthandler.dump_traceback, the GC unreachable debug-assert) run under stop-the-world and walk the published top frame down the Frame::previous chain; the owning thread is then parked at a safepoint, so the pointer and the frames it reaches are quiescent and alive. The faulthandler watchdog is a plain OS thread that cannot stop-the-world, so it walks the chain lock-free and best-effort. Non-unix threading builds have no stop-the-world and keep the existing mutex-guarded frame stack unchanged. Assisted-by: Claude * Skip exc_info save/restore for callees that never touch the slot with_frame saves and restores the shared exc_info slot around every Python call to contain frames that leave it unbalanced. Cache a has_exc_handling bit on PyCode at creation, set when the bytecode contains any opcode that calls vm.set_exception (PushExcInfo, PopExcept, CheckEgMatch, EndAsyncFor, InstrumentedEndAsyncFor). A callee whose code has none of these cannot mutate the slot, so the save and restore are skipped for it. Generators go through resume_gen_frame and are unaffected. Assisted-by: Claude * Return a write-through FrameLocalsProxy from frame.f_locals Optimized (function) frames now expose `f_locals` as a `FrameLocalsProxy` implementing PEP 667 semantics instead of a cached snapshot dict: - reads go live through the fast-local slots; each access mints a fresh proxy; keys that do not name a fast local are stored in a per-frame `f_extra_locals` side dict and folded into `locals()`. - writes to a fast-local key store into the slot (or its cell) in place; deleting a fast local raises ValueError; extra keys delete normally. - full mapping protocol: keys/values/items (lists), get/pop/setdefault, update (dict or FrameLocalsProxy only), __or__/__ior__/__ror__ (dict result), copy (plain dict), __reduce__ blocks pickling/copy, repr with recursion guard, mapping-pattern and Mapping ABC support. Class/module/exec frames keep returning their namespace mapping directly. Cross-thread access to a frame running on another thread still raises RuntimeError. A closed generator now keeps its frame locals when a durable frame reference escaped (f_locals proxy, sys._getframe, f_back), matching take_ownership; the escape is tracked with a per-frame flag. The snapshot-then-fold locals_to_fast/locals_dirty write-back is retired since proxy writes reach the slots directly. Assisted-by: Claude * Retain the caller frame so f_back resolves after it returns When a frame escapes its execution (referenced through a traceback, `sys._getframe`, `f_locals`, ...), capture a strong reference to its caller at release time. `f_back` consults it once the caller has left the live frame chain, so the Python-visible frame chain survives return. The retained reference is a GC-traversed edge and is cleared by `frame.clear()`, so ancestor chains stay collectable. Assisted-by: Claude * Fix debug-build native stack overflow on deep recursion Make check_c_stack_overflow one-sided (trip whenever the stack pointer is below the soft limit) so a single native frame larger than the margin cannot step past the danger band undetected. Raise the debug STACK_MARGIN_BYTES from 4096 to 16384 words so the margin exceeds a single debug interpreter frame, leaving headroom to raise RecursionError. Release margin unchanged. Clamp the soft-limit margin to half the stack so small explicit thread stacks do not get a soft limit above their stack top. Assisted-by: Claude * Allocate exception instance __dict__ lazily Exception construction went through into_ref_with_type, which eagerly allocated an empty instance dict for every HAS_DICT type. Add into_ref_with_type_lazy_dict, which builds the instance with an unallocated dict slot, and route the four exception construction sites (PyBaseException, PyOSError, OSErrorBuilder, PyBaseExceptionGroup) through it. The dict now materializes on first attribute write or __dict__ access via the existing get_or_insert path. add_note and PyImportError::slot_init now obtain the dict through object_get_dict so they materialize it instead of assuming it exists. A freshly constructed exception no longer reports an empty dict in gc.get_referents, matching the reference interpreter. Assisted-by: Claude * Allocate exception __dict__ lazily in vm.new_exception Route vm.new_exception() through into_ref_with_type_lazy_dict so internally raised exceptions (new_type_error, new_value_error, etc.) start without an instance dict, matching the slot_new path. The dict is materialized on the first attribute write or __dict__ access. Assisted-by: Claude * Validate FrameLocalsProxy.update() arguments Reject keyword arguments and require exactly one positional argument, raising TypeError with the "takes no keyword arguments" and "takes exactly one argument (N given)" messages. Assisted-by: Claude * Address review nits in frame and faulthandler - Drop stale vm.frames reference from the release_datastack_frame uniqueness argument. - Assert has_exc_handling when unwinding an Except-typed stack slot, documenting the invariant that guards the shared exc_info write. - Truncate the specialized __init__ return-type name to 200 chars, matching the unspecialized wrapper. - Reword two comments to describe behavior without prose references to CPython. Assisted-by: Claude * Use scopeguard for start_the_world in faulthandler dump_all_threads Wrap the unix stop-the-world registry walk in a scope with scopeguard::defer! so start_the_world runs on panic, matching the f_back and get_all_current_frames sites. Add scopeguard to the stdlib dependencies. Also correct the restore_exception doc comment to name with_frame after the rename. Assisted-by: Claude * Gitignore docs/superpowers Assisted-by: Claude * Rename type_bases_slot_rebuild.py to builtin_type_bases.py Match the builtin_* naming convention of extra_tests/snippets. Assisted-by: Claude * Apply rustfmt, ruff, and cspell lint fixes Reformat with rustfmt, fix import spacing in builtin_type_bases.py with ruff, and add "pointee" to the cspell word list. Assisted-by: Claude * Gate unix-only QSBR methods to their call sites `online`, `drain_all`, and `reset_after_fork` are called only from unix code (thread attach/detach, post-fork reset), and `offline` from unix code plus a unit test. Gate them with matching cfg so `-D dead_code` does not fire on non-unix targets. Assisted-by: Claude * Visit the function closure tuple as a GC edge PyFunction::traverse visited the cells inside the closure tuple instead of the tuple object itself, so the tuple's reference from the function was never subtracted during cycle collection. A closure tuple that reached back to its function (or, through a frame retained by f_back, to a Thread) was stranded as a false GC root and never collected, leaking the whole cycle. Visit the tuple itself, matching clear(). Assisted-by: Claude * Apply formatting hook fix to builtin_type_bases.py Assisted-by: Claude * Unmark asyncgen finalization-by-gc tests in test_base_events test_asyncgen_finalization_by_gc and test_asyncgen_finalization_by_gc_in_other_thread now pass; GC finalizes the async generators. Assisted-by: Claude * Unmark test_sni_callback_refcycle in test_ssl The servername-callback reference cycle is now collected by GC. Assisted-by: Claude * Widen GC stop-the-world gates from unix-only to all threading builds Change `cfg(all(unix, feature = "threading"))` to `cfg(feature = "threading")` on the stop-the-world machinery so it also compiles and runs on non-unix threading builds: - StopTheWorldState, its stats, stw_trace, and the stop_the_world field - ThreadSlot state/stop_requested/thread fields and their initializers - wait_while_suspended/attach_thread/detach_thread/suspend_if_needed/do_suspend, allow_threads, stop_requested_for_current_thread, and the enter_vm / VmBootstrapGuard / attach_current_thread / release_current_thread / cleanup attach-state wiring - eval_breaker_tripped, check_signals, run_scheduled_gc, signal GC_BIT / schedule_gc / take_gc_scheduled, and the frame.rs safepoint call - CollectStopTheWorld and its use in collect_inner - QSBR::online/offline, now called from attach/detach on all threading builds - debug_assert_current_thread_attached and its type-cache call sites maybe_collect defers auto-collection to the bytecode safepoint on every threading build instead of only unix; non-threading builds keep the inline collect. stw_trace writes to std stderr on non-unix. top_frame publishing (CURRENT_TOP_FRAME_SLOT, set_current_frame), the frame-walk debug assert in collect_inner, and the fork reinit helpers remain unix-only; non-unix keeps ThreadSlot::frames for introspection. Assisted-by: Claude * Detach current thread around blocking _winapi/_overlapped waits Wrap the blocking Windows wait calls in `vm.allow_threads` so the calling thread transitions ATTACHED -> DETACHED for the duration of the wait: - _winapi: WaitForSingleObject, WaitForMultipleObjects, BatchedWaitForMultipleObjects, ConnectNamedPipe, ReadFile, Overlapped.GetOverlappedResult - _overlapped: Overlapped.getresult These previously blocked while ATTACHED, so a stop-the-world requester could never suspend the thread and spun in its wait loop indefinitely. * Format WaitForMultipleObjects allow_threads closure * Treat concurrent sni_callback removal as no-op in invoke_sni_callback * Assert capi refcount on a fresh mortal list instead of the int type object The refcount test asserted exact incref/decref deltas on PyInt's shared type object. That object's reference count is perturbed by the other capi tests running in parallel, and is immortal under some interpreter configurations, so the deltas were not reliably +1/-1. Assert them on a freshly created, uniquely owned list whose reference count is private to the test and mortal. Assisted-by: Claude * Traverse and GC-track PyOSError instances The `#[pyexception]` struct macro now forwards a `traverse` option to the generated `#[pyclass]`, and `ExceptionItemMeta` accepts the `traverse` key. `PyOSError` is marked `traverse = "manual"`, so `HAS_TRAVERSE` is true and OSError-family instances are tracked at creation and traversed by the collector. `PyOSError::traverse` now visits the underlying `PyBaseException` (traceback, cause, context, args) instead of `PyException::try_traverse`, which was a no-op because `PyException` has `HAS_TRAVERSE = false`. * Unmark test_blockingioerror in test_io The BlockingIOError reference cycle is now collected by GC. Assisted-by: Claude	2026-07-08 15:21:41 +09:00
Bas Schoenmaeckers	10591a4ca3	Move type c-api functions (#8233)	2026-07-08 01:26:16 +09:00
Bas Schoenmaeckers	1e0f95ab3c	Add pyframe functions to c-api (#8215)	2026-07-07 00:03:43 +09:00
Bas Schoenmaeckers	80e384e851	Add more abstract functions to c-api (#8202)	2026-07-06 14:12:52 +09:00
Bas Schoenmaeckers	4caf959e58	Use PyO3 main & enable c-api tests (#8000) ... Enable `abi3t` support in PyO3 Add left over c-api functions Fix PyLong_AsUnsignedLongLongMask Update pyo3 to `0.29` Add `PyInt::as_u64_mask` Fix `PyInt::as_u32_mask`	2026-07-05 01:13:45 +09:00
Bas Schoenmaeckers	d1792fd2ce	Add more int functions to the c-api (#8159) ... * Add more int functions to the c-api * Simplify as_mask logic * Add `PyLong_FromNativeBytes` & `PyLong_FromUnsignedNativeBytes`	2026-06-30 23:31:15 +09:00
Bas Schoenmaeckers	4421ac89a1	Add more object c-api's (#8170)	2026-06-26 20:42:01 +09:00
Jeong, YunWon	a1ba6a662c	Align compiler and AST behavior with CPython (#8138) ... * Align compiler and AST behavior with CPython Update the compiler pipeline, AST module support, and related VM compile/eval paths toward CPython 3.14 behavior. Use upstream Ruff parser crates directly and keep RustPython-specific syntax preflight handling in the compile path. Refresh opcode metadata, snapshots, and targeted tests for the aligned bytecode and AST behavior. * fix * Fix PR 8138 CI failures * Fix match mapping wildcard scan complexity * fixes * Use Ruff runtime AST fields * Minimize residual patch diff * refactor error handling * Remove _ast conversion context wrappers * Clean up _ast conversion glue and compiler warning escalation Rename the Node conversion parameter `ctx` to `vm` throughout the _ast module. Remove the unused compile_program, compile_program_single, compile_block_expression, and compile_expression forwarders; compile_top stays as the entry point. Stash an escalated compiler SyntaxWarning so a non-SyntaxWarning category propagates unchanged instead of being rewritten to SyntaxError, matching PyErr_ExceptionMatches(SyntaxWarning) in compiler_warn. Drop the now-unused CompileWarningError::into_codegen_error. Replace the PositionalArguments two-variant enum with a struct holding a shared range and a PositionalArgumentsKind enum. Assisted-by: Claude * Represent compiler flags as a CompilerFlags bitflags Replace the loose PY_CF_*/CO_* i32 constants in compile_mode with a CompilerFlags bitflags covering the PyCF_* flags and __future__ CO_FUTURE_* bits. The Python-visible ast.PyCF_* attribute values are derived from it via bits(); parser input modes stay plain consts. builtins::compile and the compile_string flag handling test flags through CompilerFlags::contains instead of raw bit math. Assisted-by: Claude * Fix PR 8138 test regressions * Fix PR 8138 CI failures * Avoid shell interpolation in Linux deps action * Fix review findings in compiler/AST changes - frame.rs: compute the MATCH_CLASS class name lazily so a successful match no longer takes the type name lock or allocates; the __match_args__-not-a-tuple branch reuses it instead of recomputing. - _ast/constant.rs: add the missing leading space to the with_recursion label ("maximum recursion depth exceeded during compilation"). - sys.set_int_max_str_digits: restore the "maxdigits must be 0 or larger than {n}" error text. - codegen: set CO_MAXBLOCKS to 20 in both ir.rs and compile.rs. - _ast/exception.rs: extract the duplicated excepthandler node guard into ensure_excepthandler_node. - codegen: correct the validate_keywords doc comment. - Drop redundant "CPython" qualifiers from comments added in this branch. Assisted-by: Claude	2026-06-26 18:22:08 +09:00
Bas Schoenmaeckers	473ba7391a	Add more mapping functions to c-api (#8169)	2026-06-25 22:35:04 +09:00
Bas Schoenmaeckers	ab3d050485	Add allocator functions to c-api (#8168)	2026-06-25 22:34:14 +09:00
Bas Schoenmaeckers	64feb29094	Add more number functions to c-api (#8158)	2026-06-25 22:20:44 +09:00
Bas Schoenmaeckers	c57a986c7c	Add remaining float functions to c-apo (#8157)	2026-06-25 14:00:32 +09:00
Bas Schoenmaeckers	2ce6fd0e41	Add stdlib to c-api (#8133)	2026-06-22 16:38:33 +09:00
Shahar Naveh	7da696661b	Reduce number of string clones when compiling python source code (#8103)	2026-06-15 11:20:48 +09:00
Shahar Naveh	f89f415061	Remove redundant to_owned() calls (#8077)	2026-06-12 11:09:22 +09:00
Shahar Naveh	dc81e050c6	more clippy rules (#8050) ... * `checked_conversions` * duration_suboptimal_units * inefficient_to_string * iter_on_single_items * needless_collect * needless_for_each * ptr_offset_by_literal * return_self_not_must_use	2026-06-10 11:13:42 +09:00
Bas Schoenmaeckers	84ba5c25d7	Add more list functions to the c-api (#8045)	2026-06-10 11:05:18 +09:00
Bas Schoenmaeckers	1782dd25ae	Add more unicode functions to c-api (#8044) ... * Add more unicode functions to c-api * Update crates/capi/src/unicodeobject.rs	2026-06-10 11:04:40 +09:00
Bas Schoenmaeckers	cff6f9c2a7	Add more dict functions to c-api (#8043)	2026-06-10 11:03:29 +09:00
Bas Schoenmaeckers	da8e6eef1f	Add more object functions to c-api (#8042)	2026-06-10 11:02:26 +09:00
Bas Schoenmaeckers	a60fe11883	Add iter support to c-api (#8035)	2026-06-04 10:34:28 +09:00
Bas Schoenmaeckers	1a92b4ae07	Add PyDictProxy_New to the c-api (#8033)	2026-06-03 15:45:26 +00:00
Bas Schoenmaeckers	009080dba7	Add basic module functions to c-api (#8032)	2026-06-03 22:52:17 +09:00
Bas Schoenmaeckers	2723ebd81c	Add warning functions to c-api (#8020)	2026-06-03 18:39:24 +09:00
Bas Schoenmaeckers	f3ee5099ac	Add bytesarray support to the c-api (#8009) ... * Add bytesarray support to the c-api * Add SAFETY comment	2026-06-03 18:36:15 +09:00
Bas Schoenmaeckers	6937f3b559	Add slice functions to c-api (#8019)	2026-06-03 18:35:31 +09:00
Bas Schoenmaeckers	7f97f9b2b7	Add number protocol functions (#8005)	2026-06-03 18:34:49 +09:00
Bas Schoenmaeckers	c1aeeef7f8	Add sequence protocol to the c-api (#8016)	2026-06-02 09:33:26 +00:00
Bas Schoenmaeckers	d411a2b9d0	Add weakref support to c-api (#8006)	2026-06-02 15:31:21 +09:00
Bas Schoenmaeckers	8af0f8ab7b	Add mapping c-api functions (#8003)	2026-06-02 15:20:45 +09:00