code.replace() called as_interned_str().unwrap() on its string
arguments, which panics when the caller passes a string that has not
been interned. modulefinder's replace_paths_in_code() builds a fresh
co_filename, so any use of ModuleFinder(replace_paths=...) aborted the
interpreter.
Intern the incoming strings instead, and raise TypeError rather than
panicking when a non-string appears in co_names/co_varnames/
co_cellvars/co_freevars.
Unskips test_modulefinder.test_replace_paths.
Co-authored-by: Pablo Garcia <pablito@MacBook-Neo-de-Pablo.local>
`search_info_literal`'s len>1 arm reset the tail-match cursor by advancing one
character past the matched prefix. That position equals the INFO block's
prefix_skip boundary only when the prefix ends where the skip does; for a
pattern like `ab(cd)`, `_get_literal_prefix` reports prefix_len=4 with
prefix_skip=2, so the tail resumed two characters too far.
`state.cursor` already holds `req.start + skip`, which is what
`sre_lib.h` SRE(search) computes as `ptr - (prefix_len - prefix_skip - 1)`,
and which the len==1 arm above already uses. Take it unconditionally.
Searching "xabcdcd" for `ab(cd)` returned span (1, 7) instead of (1, 5);
over a 600-case corpus stamped with CPython's own answers, 94 cases disagreed
before this change and none after. The wrong resume also dropped matches
outright, not only widened them.
Assisted-by: Claude
According to POSIX, `time_t` should be 64 bits. `musl` changed its
`time_t` to an `i64` over five years ago. `glibc` provides compatibility
features that declare `time_t` as either `i32` or `i64`. Rustix uses the
raw Linux syscall for stat which returns an `i64`. For our purposes, an
`i64` makes sense because it's modern and avoids the year 2038 problem.
It also reduces our dependency on what `libc` defines.
Sources:
* https://www.man7.org/linux/man-pages/man3/time_t.3type.html
* https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/sys_types.h.html
* Allow automated commits through pre-commit hooks
* Fix agent C-API test instructions
Exclude rustpython-capi from root workspace test commands and run its tests from the crate directory so its Cargo configuration applies. Document the project AI policy and required commit trailer.
Assisted-by: Codex:gpt-5
CPython's _PyFrame_GetLocals only syncs fastlocals/cells into locals()
for function scope; module/class scope just returns the namespace dict
as-is. sync_visible_locals_to_mapping did this for every scope, so the
implicit __conditional_annotations__ cell (used for PEP 649/749
deferred annotations) broke two ways: at module scope its cell is
always empty, so syncing overwrote the dict's real value with None,
causing NameError on the next annotated statement; at class scope its
cell is the only real value, so syncing leaked it into locals()/dir(),
unlike CPython.
Skip cell/free slots outside function scope to match CPython, and add
a regression snippet.
Closes#8379
Assisted-by: Claude Code:claude-sonnet-5
* Use type-specialized comparators for homogeneous list sorts
Scan the sort keys once before sorting; when every element is exactly str, int, or float, compare wtf8 bytes / BigInt / f64 directly instead of going through rich_compare_bool dispatch, mirroring CPython's pre-sort check in listsort.c (unsafe_latin_compare and friends). Subclasses and mixed-type lists keep the generic __lt__ path.
* Add edge-case tests for specialized list sorts
* Cache the richcompare slot for homogeneous list sorts
* Specialize tuple sorts on their first elements
* Apply ruff formatting to the new list sort tests
* Deduplicate the reverse swap across sort dispatch arms
The cell type filled neither the richcompare nor the repr slot, so
object's address-based defaults showed through: cell(1) == cell(1) was
False, ordering raised TypeError, and repr rendered
<cell object at 0x...> rather than <cell at 0x...: int object at 0x...>.
Compare cells by contents, with empty cells ordering before everything
else, and render CPython's repr for both the filled and empty cases.
The comparison fills the richcompare slot directly rather than going
through Comparable, whose cmp() can only answer with a bool. CPython
returns whatever PyObject_RichCompare produced, so a contained __eq__
that yields a non-bool must pass through untouched; coercing it would
also call __bool__ and surface exceptions CPython never raises. The
empty-cell branch still answers with a bool, so both arms are needed and
the slot returns Either.
The repr truncates the contained type name the way "%.80s" does: at most
80 bytes, dropping a character the cut would leave incomplete.
Mark the type unhashable. Content-based equality combined with the
inherited identity hash would break the hash/eq contract, and hashing the
contents is not possible either because cell_contents is writable.
CPython gets this implicitly, because defining tp_richcompare suppresses
tp_hash inheritance.
Reference: CPython Objects/cellobject.c, cell_richcompare and cell_repr.
Assisted-by: Claude Code:claude-opus-5
* Factor with_iframe into enter_iframe/exit_iframe helpers
Extract the frame entry (recursion check, TLS link, exception save)
and exit (materialization sync, TLS restore, GC tracking) logic
from with_iframe into standalone enter_iframe/exit_iframe methods.
with_iframe now calls them, with no behavioral change.
This prepares for the trampoline loop where enter/exit are called
individually rather than wrapped around a closure.
Assisted-by: Claude
* Allocate InterpreterFrame and LocalsPlus together on the datastack
Add InterpreterFrame::new_on_datastack() that bump-allocates both the
InterpreterFrame struct and its LocalsPlus data array in a single
datastack push, eliminating one allocation per function call.
Update datastack_frame_size_bytes_for_code() to include InterpreterFrame
size. Convert invoke_prepared_exact_args() and the invoke() fast path to
use the combined allocation.
Add release_datastack_frame() method on InterpreterFrame that drops all
localsplus values, runs field destructors (trace, temporary_refs,
retained_back, etc.), and returns the datastack base pointer for pop.
Assisted-by: Claude
* Implement trampoline loop for Python-to-Python calls
Add ExecutionResult::TailCall variant and a trampoline in
run_frame_fast that flattens Python-to-Python calls into a single
Rust stack frame instead of recursing through the eval loop.
CallPyExactArgs now prepares the callee frame on the datastack and
returns TailCall when tailcall_enabled is set (run_iframe path only).
The trampoline dispatches via a state machine (EnterCallee / ReturnValue
/ Unwind) in a single loop, avoiding mutual recursion between helper
functions that would exhaust the C stack.
Exception propagation through suspended frames uses
trampoline_handle_exception which adds traceback entries and calls
unwind_blocks on each caller.
Assisted-by: Claude
* Optimize trampoline: skip recursion check, avoid temporary_refs mutex, add bound method TailCall
- Add enter_iframe_unchecked for trampoline callee entry (recursion
already checked by specialization_call_recursion_guard)
- Move callable ownership from per-frame temporary_refs mutex to
trampoline-local SuspendedFrame.owned_refs via VM side channel
- Add TailCall support for CallBoundMethodExactArgs
- Move args directly from caller stack to callee fastlocals
- Read materialized pointer once in exit_iframe
Incremental call overhead: ~55 ns -> ~35 ns
Assisted-by: Claude
* Use UnsafeCell for pending_tailcall_refs
The VM is per-thread so RefCell's runtime borrow checking is
unnecessary overhead. Replace with UnsafeCell for direct access.
Assisted-by: Claude
* Remove .claude/settings.json from tracking
Assisted-by: Claude
* Replace SendPtr with Option<SendNonNull> for type-safe pending tailcall
Use NonNull + Option instead of raw *mut T with manual null checks.
The compiler enforces non-null via the type system, and Option<NonNull>
has the same size as a raw pointer thanks to niche optimization.
Also extract take_pending_tailcall helper to deduplicate the pattern.
Assisted-by: Claude
* Restrict PendingFrame to private, expose only set/take methods
Rename SendNonNull to PendingFrame and make it fully private: the
struct, its field, and the pending_tailcall_frame Cell are all
non-pub. External code accesses the side channel only through
set_pending_tailcall (pub(crate)) and take_pending_tailcall (private).
This ensures the unsafe Send+Sync impl cannot be reused elsewhere
without justifying a new safety argument.
Assisted-by: Claude
* Restore C-stack overflow check in trampoline enter_iframe_unchecked
enter_iframe_unchecked was skipping C-stack checks under the
assumption that the trampoline stays in one Rust stack frame. But
each run_iframe call still consumes Rust stack, so deep Python
recursion through the trampoline can exhaust the C stack (observed
as STATUS_STACK_OVERFLOW on Windows CI).
Keep the C-stack check (every 8th call) while still skipping the
Python recursion depth check (already done by
specialization_call_recursion_guard).
Assisted-by: Claude
* Address code review: entry frame double-free, panic safety, cleanup
- Fix double-free: mark entry frame with is_entry flag in
SuspendedFrame so the trampoline skips its datastack release
(the caller owns that cleanup)
- Clear iframe.previous in exit_iframe before unlinking the chain,
matching with_frame and resume_gen_frame behavior
- Add scopeguard in with_iframe for panic safety
- Use saturating_sub(1) for lasti in trampoline_handle_exception
- Extract datastack_iframe_localsplus_offset helper to avoid
duplicated alignment computation
Assisted-by: Claude
Validate decimal integer literals from the parser token stream before
lowering the parsed AST. Reuse the validator for normal compilation and
the `_ast` parsing path so `PyCF_ONLY_AST`, `ast.parse()`, and
`ast.literal_eval()` honor `int_max_str_digits` consistently.
Remove the now-passing `test_literal_eval_str_int_limit` expected-failure
marker. The token-based validator excludes non-decimal integers, floats,
complex literals, strings, and comments without duplicating lexer logic.
Assisted-by: Codex:gpt-5.6-sol
specialization_run_init used prepare_exact_args_frame which heap-allocates
a FrameObject with 4-5 atomic refcount bumps per call. Switch to
invoke_prepared_exact_args which uses InterpreterFrame::new_on_datastack
with zero refcount bumps.
CallAllocAndEnterInit already guards against tracing (eval_frame_active)
and recursion limits before reaching this path.
Assisted-by: Claude
`socket.sethostname` took `PyUtf8StrRef`, so it rejected `bytes` outright and
refused a `str` carrying a surrogate escape. `socketmodule.c
socket_sethostname` accepts a bytes object directly, falls back to
`PyUnicode_FSConverter` for anything else, and hands the syscall the resulting
buffer and its length — the name is never required to be UTF-8.
Take `FsPath`, the converter `if_nametoindex` in this same module already uses,
and pass its bytes down. `host_env::socket::sethostname` correspondingly takes
`&[u8]` and builds the `OsStr` from them; `nix::unistd::sethostname` accepts
`AsRef<OsStr>` and passes pointer and length to the syscall, so nothing on the
path needs a NUL terminator or valid UTF-8.
`Lib/test/test_socket.py test_sethostname` covers this: it calls
`socket.sethostname(b'bar')` and asserts the hostname changed. The test is
skipped unless run as root, which is why the gap went unnoticed.
Assisted-by: Claude
* Fix implementation of bytes.isspace to match the implementation in CPython.
* Fix implementation of bytes.istitle so that test_bigmem now passes.
Also added some extra_test to capture this.
* Fix some clippy issues
Signed-off-by: Hanif Ariffin <hanif.ariffin.4326@gmail.com>
* bytes.istitle returns false when the bytes is empty.
* Fix some edges of istitle with non-alphanumeric characters
* Disable a line because python 3.12 errors on bad escape sequence
* Match CPython's istitle a little closer
* Fix clippy warning
* Rewrite bytes.istitle along _Py_bytes_istitle
The lookahead-based check accepted b"Not--a Titlecase String", which
test_bytes covers. Track whether the previous byte was cased instead.
isspace compares bytes directly rather than converting to char.
Restore the b"omkmok\Xaa" assert: 3.14 reports an invalid escape
sequence as a SyntaxWarning, not an error.
Assisted-by: Claude
---------
Signed-off-by: Hanif Ariffin <hanif.ariffin.4326@gmail.com>
Co-authored-by: Jeong YunWon <jeong@youknowone.org>
Move 10 rarely-used fields (trace, trace_lines, trace_opcodes,
temporary_refs, f_locals_hidden_overlay, f_extra_locals, escaped,
retained_back, pending_stack_pops, pending_unwind_from_stack) from
InterpreterFrame into a lazily-allocated FrameColdData struct.
InterpreterFrame now carries a single UnsafeCell<Option<Box<FrameColdData>>>
(8 bytes) instead of ~200+ bytes of cold fields. The cold() accessor
allocates on first access; frames that never trigger tracing or
debugging pay no allocation cost.
GC traversal skips cold data when it has not been allocated.
Assisted-by: Claude
Nightly miri now flags `fn_ptr as usize` and direct `fn_ptr == fn_ptr`
as UB because both paths go through `FnPtr::addr()`, which attempts to
dereference a function pointer's provenance — function items have no
backing allocation in miri's model.
Add `fn_addr<T>(f: T) -> usize` that uses `transmute_copy` to read
the address as plain integer bytes without triggering provenance
checks. Replace all `f as usize` slot comparison patterns across
the codebase with `fn_addr(f)`.
Also add `-Zmiri-permissive-provenance` to CI MIRIFLAGS as a safety
net for any remaining integer-pointer round-trips elsewhere.
Assisted-by: Claude
* Add powersort implementation for list sorting
Add crates/vm/src/sorting.rs implementing Tim Peters' timsort with powersort's merge-ordering policy (CPython 3.11+): run detection, binary insertion for short runs, galloping merge (merge_lo/merge_hi), and power-based merge ordering (powerloop). Comparison is passed in as a fallible `is_lt` closure, so the algorithm stays generic over the element type and free of interpreter details.
Not yet wired into list.sort(); replaces rust-timsort in a follow-up.
* Use powersort for list.sort() and drop rust-timsort
Wire list sorting through crate::sorting::timsort and remove the
rust-timsort dependency. Fixes the O(N^2) behavior on random input
(1M random floats: ~16min -> ~0.8s), now within ~4x of CPython.
* Fix usize underflow in merge_hi Succeed path
* Fix clippy warnings and formatting in sorting.rs
* Drop redundant test_ prefixes in sorting tests
* Restore buffered elements when a comparison fails mid-merge
* Copy one run-A element per gallop round in merge_lo
`cargo test --workspace ...` segfaults on rustpython-capi when run from
the workspace root; it needs its own config that only applies inside
crates/capi. CI already excludes it from the workspace run and tests it
separately. Document both.
Closes#8415
Assisted-by: Claude Code:claude-sonnet-5
DEVELOPMENT.md was renamed to CONTRIBUTING.md in #7914, but AGENTS.md
wasn't updated. Repoint the two links with a clear target (Linux/macOS
testing section, development guide). Drop the "CPython Version Upgrade
Checklist" reference under venvlauncher, which never pointed to real
content in any revision of DEVELOPMENT.md.
Related to #8356.
Assisted-by: Claude Code:claude-sonnet-5
The "Repository Structure" section listed pre-`crates/` paths (`vm/`,
`compiler/`, `derive/`, etc.) that no longer exist; `CONTRIBUTING.md`
already documents the current layout under "Code organization". Replace
the stale list with a link to that section.
Also generalized the title and intro from "GitHub Copilot Instructions"
to "AI Agent Instructions" — the file was already renamed from
`.github/copilot-instructions.md` to `AGENTS.md` in #6813 to follow the
cross-vendor AGENTS.md standard (https://agents.md/), but the title text
was never updated to match.
Assisted-by: Claude Code:claude-sonnet-5
`method_descriptor.__get__(obj)` raised a TypeError when the owner (the
optional second argument) was omitted, because the METHOD-flag branch
required the owner to be a type. Match CPython: a missing owner binds to
`obj`, while a non-type owner still raises "needs a type, not ...".
Assisted-by: Claude Code:claude-opus-4-8
* csv: port CPython reader state machine
Replace the csv-core reader and per-item quote scanner with one Rust
implementation of CPython's nine-state reader parser. Keep parser state
across iterator items and distinguish virtual item boundaries from true
iterator exhaustion.
Centralize field completion so quote provenance, empty-field None conversion,
float conversion, strict parsing, field limits, blank rows, and escaped or
quoted newlines share one path. Apply the existing reentrant-iterator
generation check to every iterator item consumed by a record.
Also accept empty ASCII line terminators like CPython, report a missing
csv-core writer sentinel as _csv.Error instead of panicking, and name the
FSM entry point process_parser_input.
Remove the expected-failure markers from the eleven reader tests that now
pass.
Assisted-by: Codex:gpt-5.6-sol
* csv: test CRLF boundaries with custom lineterminator
* Add LightFrame for unobserved Python-to-Python calls
Allocate a lightweight LightFrame header on the DataStack instead of a
full Frame PyObject for specialized exact-args call paths. The frame is
materialized lazily only when observed (sys._getframe, traceback, tracing).
Key changes:
- LightFrame struct with borrowed pointers and no refcount overhead
- FrameSource enum (Heavy/Light) replacing the object field on ExecutingFrame
- CURRENT_LIGHT_FRAME TLS for the light frame chain
- Interleaved heavy/light frame walking in frame_at_offset_vm and
current_thread_frame_vm
- invoke_light_slots on PyFunction for the fast call path
Call overhead reduced from ~92ns to ~58ns (~37% reduction).
fib(28) improved from ~152ms to ~111ms (~27% reduction).
Assisted-by: Claude
* Streamline with_frame: inline recursion check, skip tracing dispatch
- Inline check_recursive_call instead of wrapping in with_recursion closure
- Amortize C stack overflow check to every 64th call depth
- Skip dispatch_traced_frame when use_tracing is false (hot path)
- Add recursion_depth_increment/decrement helpers
Assisted-by: Claude
* Address review findings for LightFrame correctness
- Fix materialize_light_frame previous pointer: use payload pointer
(&**frame) instead of Py<Frame> pointer, matching the chain convention
- Fix predecessor ref management: use retained_back instead of
mem::forget to keep materialized predecessors alive
- Fix materialized ref reclaim: use FrameRef::from_raw to drop the
leaked keeper ref without double-incrementing
- Use PyAtomic<u32> for LightFrame.lasti instead of AtomicU32, removing
the unsound cast on non-threading targets
- Add amortized C stack overflow check to light frame path
- Add scopeguard to light frame execution for panic safety
- Remove unnecessary Send/Sync impls from LightFrame
- Make check_c_stack_overflow pub(crate) for light frame path access
Assisted-by: Claude
* Fix CI failures: cspell, stale materialized state, f_back chain gaps
- Fix cspell: 'amortised' → 'amortized'
- Sync lasti, prev_line, and fastlocals when re-observing a materialized
light frame, fixing stale f_locals/f_lineno in test_inspect and others
- In with_frame, materialize any active light frame as the heavy frame's
predecessor and store it in retained_back, so f_back and inspect.stack
see the correct interleaved order (H_new → L_mat → H_old)
- Guard invoke_light_slots on NEWLOCALS | OPTIMIZED code flags
Assisted-by: Claude
* Unify frame chain: replace dual-chain with single FrameChainPtr
Replace the dual-chain frame management (CURRENT_FRAME AtomicPtr<Frame>
+ CURRENT_LIGHT_FRAME Cell<*const LightFrame>) with a single unified
chain using FrameChainPtr, a tagged pointer where bit 0 distinguishes
heavy (*const Frame) from light (*const LightFrame) entries.
Changes:
- Add FrameChainPtr type with tagged pointer encoding
- Change LightFrame: replace previous_light + saved_current_frame with
single previous: FrameChainPtr field
- Change InterpreterFrame.previous from AtomicPtr<Frame> to PyAtomic<usize>
- Change CURRENT_FRAME TLS from AtomicPtr<Frame> to AtomicUsize
- Remove CURRENT_LIGHT_FRAME TLS entirely
- Simplify all chain walkers (frame_at_offset, frame_at_offset_vm,
find_owned_chain_frame, for_each_current_frame, current_thread_frame)
- Remove light-frame materialization block from with_frame
- Update f_back to dispatch on FrameChainPtr tags
- Update faulthandler and gc_state to walk unified chain
- Adaptive C stack check: every call in debug, every 16th in release
Assisted-by: Claude
* Fix light frame materialization: current_frame, f_back, and GC tracking
- current_frame() now uses current_thread_frame_vm() to materialize
light frames, fixing super()/compile()/exec() when called from light
frame context
- materialize_light_frame stores retained_back for heavy predecessors
too, so f_back resolves after the caller returns
- Escaped materialized frames (traceback, sys._getframe) are GC-tracked
at cleanup so reference cycles are collectible
- Restore pre-existing @expectedFailure markers for test_frame proxy tests
Assisted-by: Claude
* Limit light frame materialization to super() only
current_frame() returns heavy-only again to avoid unnecessary
materialization that extends object lifetimes. super() init uses
current_thread_frame_vm() directly since it must inspect the
light frame's code and freevars to find __class__.
Assisted-by: Claude
* Fix remaining CI failures: C stack check, GC tracking threshold, expectedFailure
- Check C stack overflow every call in light frame path (not every
16th) to prevent stack overflow on CI workers with smaller stacks
- Fix GC tracking threshold: escaped materialized frame detection
uses strong_count > 1 (was > 2, missed frames referenced only by
traceback in the result)
- current_frame() materializes light frames so warnings, compile(),
and other callers see the correct frame
- Mark test_futures2 PyFutureTests.test_task_exc_handler_correct_context
as expectedFailure (pre-existing on main: PyTask __del__ timing)
Assisted-by: Claude
* Avoid materialization in hot paths: add light-frame-aware accessors
current_frame() stays heavy-only to avoid costly materialization on
every call. Instead, add current_globals(), current_code(), and
current_builtins() that read directly from light frame raw pointers
without creating frame PyObjects.
Callers migrated:
- PyFunction:🆕 current_builtins() for fallback builtins lookup
- compile()/eval(): current_code() for future feature flags
- type.__new__: current_globals() for __module__ detection
- typevar/typing: current_globals() for caller module name
- _io: current_code() for source path check
- super(): current_thread_frame_vm() for __class__ cell access
- warnings.warn: current_thread_frame_vm() for stack level walk
- import: current_globals() for module resolution
Assisted-by: Claude
* Fix current_locals() to include light frames
current_locals() must materialize light frames so locals() builtin
returns the correct scope. This fixes test_zipimport doctest and
other tests that rely on locals() in light frame context.
Assisted-by: Claude
* Fix clippy, profile tracing, resume_gen_frame, and faulthandler issues
- Fix clippy: collapsible_if in materialize_light_frame, map_unwrap_or in import
- Fix profile breakage: use current_thread_frame_vm in trace_event_inner to
materialize light frames, preventing frame identity mismatch when profiling
is enabled inside a light frame (e.g. profile.Profile.runctx)
- Disable light frames inside trace/profile callbacks (tracing_is_suppressed)
- Fix resume_gen_frame: skip light frames in previous chain to avoid dangling
pointers after generator suspend
- Fix faulthandler: emit '<no Python frame>' when chain has only light frames
Assisted-by: Claude
* Fix materialized frame refcount: transfer localsplus ownership on exit
Remove @expectedFailure from test_futures2: the underlying issue was that
materialized light frames cloned localsplus values (refcount +1 each),
keeping objects alive until the next GC cycle collection.
Fix: after run_light_frame returns, transfer ownership of localsplus
values from the light frame to the materialized frame, replacing the
clones. This eliminates the extra refcount, allowing objects to be
GC'd immediately when their last reference is dropped.
Also stabilize materialized frame's localsplus on the heap in
sync_materialized_on_exit before GC tracking — the data-stack backing
will be popped imminently.
Assisted-by: Claude
* Fix rustfmt, increase stack margin, address review comments
- Fix rustfmt formatting issues (lint CI failure)
- Increase STACK_MARGIN_BYTES from 2048 to 3072 words to prevent C stack
overflow in deep recursion tests on macOS CI (test_functools SIGSEGV)
- Skip dispatch_traced_frame when use_tracing is false (review comment)
- Fix post-fork top_frame initialization: walk past light frames to find
nearest heavy frame (review comment)
Assisted-by: Claude
* Remove redundant test override in test_futures2
The override only called super() without any marker, making it a
no-op that prek flags as a redundant test patch.
Assisted-by: Claude
* Fix test_pdb and test_functools CI failures
- Revert with_frame dispatch_traced_frame skip: tracing can be enabled
mid-execution (pdb.set_trace), so Return events must always be checked.
This fixes 3 test_pdb doctest failures across all platforms.
- Check C stack overflow on every with_frame call instead of every 64th:
the previous sampling approach missed overflows between checks when
native stack frames are large (invoke_light_slots + lru_cache recursion).
- Add #[inline(never)] to invoke_light_slots to prevent the optimizer
from merging its stack frame into callers.
- Add early C stack check before DataStack allocation in invoke_light_slots.
Assisted-by: Claude
* Rename Frame to FrameObject, FrameRef to FrameObjectRef
Prepare for _PyInterpreterFrame/PyFrameObject separation by
renaming the PyObject-backed frame type to FrameObject.
Python-visible name ("frame") is unchanged.
Assisted-by: Claude
* Remove Deref<Target = InterpreterFrame> from FrameObject
Replace implicit field access through Deref with explicit
iframe()/iframe_ref()/iframe_mut() calls. This decouples
FrameObject from InterpreterFrame field layout, enabling
independent modification of InterpreterFrame in future commits.
Assisted-by: Claude
* Move InterpreterFrame identity fields to raw pointers
InterpreterFrame's code/globals/builtins/func_obj fields are now
borrowed raw pointers instead of owned PyRef/PyObjectRef. Ownership
of these references is anchored in FrameObject's new owned_code,
owned_globals, owned_builtins, owned_func_obj fields.
This aligns InterpreterFrame's layout with LightFrame's existing
raw-pointer pattern, enabling future DataStack-based allocation
without PyObject heap allocation.
- Add code()/globals()/builtins()/func_obj() accessor methods
- Add FrameObject::new_ref() for combined create+allocate+patch
- Update ExecutingFrame to use &Py<PyCode> / &Py<PyDict> / &PyObject
- Update monitoring.rs functions to take &Py<PyCode> instead of &PyRef<PyCode>
- Add Send+Sync impls for InterpreterFrame (raw pointers are !Send by default)
Assisted-by: Claude
* Remove LightFrame, FrameChainPtr, and dual-frame-type infrastructure
Delete LightFrame struct, FrameChainPtr tagged pointer, FrameSource enum,
and all supporting functions: materialize_light_frame, sync_light_to_materialized,
transfer_localsplus_to_materialized, sync_materialized_on_exit,
run_light_frame, invoke_light_slots body, LocalsPlus::from_datastack_raw.
All call paths now use a single frame type (FrameObject wrapping
InterpreterFrame). The frame chain is a simple *const FrameObject linked
list through InterpreterFrame.previous.
invoke_light_slots delegates to invoke_exact_args_slots.
ExecutingFrame.frame_source replaced with ExecutingFrame.frame.
Frame chain walking simplified throughout (no more light/heavy dispatch).
Net: -724 lines of dual-frame-type complexity.
Assisted-by: Claude
* Clean up invoke_light_slots and _vm function variants
- Replace invoke_light_slots calls with invoke_exact_args_slots
- Delete the invoke_light_slots delegate method
- Replace current_thread_frame_vm/frame_at_offset_vm with their
non-_vm equivalents (no more light frame materialization needed)
- Remove stale light-frame comment in callable.rs
Assisted-by: Claude
* Fix CI lint issues
- Move github context expressions from run blocks to env blocks
(zizmor/template-injection)
- Add workflow-level permissions: {} to lib-deps-check.yaml
- Suppress zizmor excessive-permissions for lib-deps-check.yaml
(pull_request_target is required for PR comments)
- Apply rustfmt formatting
Assisted-by: Claude
* Stack-allocate InterpreterFrame for non-generator function calls
- Extract InterpreterFrame::new() from FrameObject::new()
- Change frame chain from *const FrameObject to *const InterpreterFrame
- Add InterpreterFrame.materialized field for lazy FrameObject creation
- Add vm.with_iframe()/run_frame_fast() for stack-allocated frame execution
- Move ExecutingFrame.frame to iframe: *const InterpreterFrame
- Implement lazy materialize() for on-demand FrameObject creation
- Update invoke_with_locals/invoke_prepared_exact_args to skip heap allocation
- Update frame chain walking in faulthandler, gc_state, builtins/frame
- super() reads InterpreterFrame directly without materialization
Assisted-by: Claude
* Remove unnecessary refcount and atomic ops from fast call path
- Use raw pointer for func_obj instead of cloning PyObjectRef (saves 2 atomic RMW)
- Skip owner AcqRel swap for stack-allocated frames (always Thread-owned)
Assisted-by: Claude
* Change trace field to Option<PyObjectRef>, remove vm from InterpreterFrame::new
- trace: PyMutex<PyObjectRef> -> PyMutex<Option<PyObjectRef>>
- Eliminates vm.ctx.none() refcount inc/dec per frame init
- Remove vm parameter from InterpreterFrame::new() (no longer needed)
- f_trace getter returns None when trace is unset
- f_trace setter stores None instead of the None singleton
Assisted-by: Claude
* Store current frame pointer on VirtualMachine for fast lookup
- Add current_frame_ptr Cell<usize> field to VirtualMachine
- Hot path (with_iframe) reads/writes vm.current_frame_ptr instead of TLS
- TLS CURRENT_FRAME still maintained for signal-safe traceback walking
- Use set_current_frame_nosave() to skip cross-thread top_frame update
- super(), frame_at_offset, current_thread_frame_materialize use vm Cell
Assisted-by: Claude
* Apply rustfmt
Assisted-by: Claude
* Fix clippy use_self warnings, add inline hints for hot path
Assisted-by: Claude
* Remove code refcount clone from fast call path
Defer code.to_owned() to the generator/coroutine path only. On the fast
path, use &Py<PyCode> directly — the code object is alive via the
PyFunction on the caller's stack.
Assisted-by: Claude
* Remove VM current_frame_ptr Cell (fix SIGSEGV in threading builds)
The Cell<usize> on VirtualMachine could desynchronize from the TLS
CURRENT_FRAME in nested VM scenarios (enter_vm called with different
VMs on the same thread). Revert to TLS-only frame chain management.
Keep set_current_frame_nosave() for the stack-frame fast path: it
skips cross-thread top_frame publication but still does the TLS swap.
Fix clippy not_unsafe_ptr_arg_deref in set_current_frame.
Fix unnecessary Result wrapper in make_generator_or_coro.
Assisted-by: Claude
* Fix CI failures: GC frame assertion, faulthandler Radium import, dead code warning
- Fix SIGSEGV in threading builds: GC debug assertion walked frame
chain via top_frame, which is not updated by set_current_frame_nosave
used by stack-allocated iframes. Walk CURRENT_FRAME TLS chain instead.
- Add InterpreterFrame::get_lasti() accessor, remove direct Radium
import from faulthandler.rs (fixes unused import warning).
- Add cfg_attr allow(dead_code) for from_payload_ptr without threading
(fixes WASM build warning).
Assisted-by: Claude
* Fix materialized frame: copy localsplus, clear stale previous pointers
- materialize_slow: copy fastlocals snapshot to heap instead of empty
localsplus (fixes index-out-of-bounds panic in locals()/f_locals)
- materialize_slow: set previous to null (stack iframes become dangling)
- with_frame/resume_gen_frame: clear iframe.previous on pop (prevents
dangling pointers to freed stack iframes)
- f_back: walk TLS CURRENT_FRAME chain instead of top_frame for
cross-thread lookup (top_frame is stale for stack-allocated iframes)
- check_locals_access: match materialized frames by comparing the
Py<FrameObject> address stored in the stack iframe's materialized field
Assisted-by: Claude
* Fix Windows clippy: gate unix-only imports and variables
FrameObject, Py imports and top_iframe variable are only used in
unix+threading cfg blocks. Gate them properly for non-unix builds.
Assisted-by: Claude
* Fall back to FrameObject when tracing; fix cross-thread frame access
- invoke_with_locals: fall back to heap-allocated FrameObject path
when use_tracing is active, so trace/profile callbacks fire correctly.
This fixes test_trace, test_bdb, test_monitoring regressions.
- set_current_frame: publish top_iframe alongside top_frame in ThreadSlot
so cross-thread readers (sys._current_frames) can materialize
stack-allocated frames from other threads under stop-the-world.
- get_all_current_frames: use TLS CURRENT_FRAME for current thread,
fall back to top_iframe materialization for other threads.
- Use set_current_frame (not nosave) in with_iframe for proper
cross-thread visibility.
Assisted-by: Claude
* Apply rustfmt
Assisted-by: Claude
* Fix Windows: gate unix-only top_iframe_ptr variable
Assisted-by: Claude
* Fix frame chain: f_back, retained_back, owner, faulthandler
- Set materialized frame owner to FrameObject (not Thread) so
frame.clear() works on traceback frames
- Fix f_back for materialized frames: walk TLS chain to find source
iframe and materialize its previous frame
- Fix faulthandler dump_traceback to use dump_live_frames (reads
live lasti from stack iframes instead of stale materialized copies)
- Add retained_back propagation in with_frame cleanup: when a frame
escapes, materialize caller and mark it escaped for chain propagation
- Fix release_datastack_frame: don't overwrite retained_back if
already set by with_frame cleanup
Assisted-by: Claude
* Apply rustfmt
Assisted-by: Claude
* Refine with_frame retained_back: use frame_obj instead of materialize
Avoid materializing caller frames in with_frame cleanup to prevent
creating localsplus snapshots that keep extra refcounts. Only use
existing FrameObjects (frame_obj) for retained_back. Also keep
previous pointer alive in release_datastack_frame since the caller
is still executing at that point.
Assisted-by: Claude
* Fix f_lineno: use Cell<u32> for prev_line, update on every instruction
prev_line was only updated when tracing was active, causing f_lineno
to return stale values when observed mid-CALL (e.g. from warnings.warn
or sys._getframe). Changed prev_line from u32 to Cell<u32> for
interior mutability so it can be read safely while ExecutingFrame
holds a shared reference, and update it on every instruction so
f_lineno always returns the correct line.
Assisted-by: Claude
* Fix f_lineno for materialized frames: read live prev_line from TLS chain
Materialized FrameObjects have a snapshot of prev_line from materialize
time. When f_lineno is called on a materialized frame that is still
executing (its source iframe is on the TLS chain), walk the TLS chain
to find the source iframe and read its live prev_line. This fixes
lineno tracking in warnings, gettext, and other modules that read
frame line numbers during function calls.
Also fixed the pointer comparison: use from_payload_ptr to convert
FrameObject payload address to Py<FrameObject> address for matching
against the materialized field.
Assisted-by: Claude
* Sync live frame state for materialized frames
- Add find_live_source_iframe() to walk TLS chain and find the live
source InterpreterFrame for a materialized FrameObject
- f_lasti: read live lasti from source iframe when available
- f_lineno: use read_volatile for live prev_line from source iframe
- sync_visible_locals_to_mapping: read live localsplus from source iframe
- framelocalsproxy_getval: read live localsplus from source iframe
- with_iframe cleanup: sync localsplus, prev_line, lasti to materialized
FrameObject when the frame returns
- sync_fastlocals_from: new LocalsPlus method to update fastlocals
Fixes test_inspect (stale f_locals, stale lineno in positions),
test_listcomps (stale iteration variable in locals()).
Assisted-by: Claude
* Fix retained_back: use materialize_chain and strong_count check
- Add materialize_chain() that creates a lightweight FrameObject with
empty localsplus for f_back chain building only. This avoids cloning
local variables which would create extra refcounts (fixes test_memoryview).
- with_frame cleanup: use strong_count > 1 instead of escaped flag to
determine if retained_back is needed, ensuring chain propagation.
- with_iframe cleanup: use materialize_chain for retained_back to avoid
extra refcounts from localsplus snapshots.
Fixes test_memoryview (refcount from retained_back chain) and
test_traceback.test_extract_stack (f_back chain depth).
Assisted-by: Claude
* Fix cross-thread f_back chain for sys._current_frames/exceptions
- get_all_current_frames: materialize entire frame chain and link
retained_back during stop-the-world for other threads
- f_back: use stop-the-world to safely materialize cross-thread frame
chains on unix when prev iframe is on another thread
- f_back: check frame_obj() before STW for fast path
Fixes test_sys.test_current_frames and test_sys.test_current_exceptions.
Assisted-by: Claude
* Fix monitoring LINE events and faulthandler thread dump
- Skip all instrumented opcodes in bytecode loop prev_line update to
avoid defeating InstrumentedLine de-duplication
- Add prev_line update in execute_instrumented for non-RESUME/non-LINE
instrumented opcodes
- update_events_mask: walk TLS iframe chain to re-instrument all frames
including stack-allocated ones (fixes missing LINE events for inline
frames already past RESUME)
- faulthandler: use top_iframe fallback for stack-allocated frames in
dump_all_threads and watchdog_thread
- faulthandler: use dump_live_frames for current thread dump
Fixes test_monitoring LINE/CALL tests, test_faulthandler thread dumps.
Assisted-by: Claude
* Fix clippy: remove duplicate Radium import, use previous() helper
Assisted-by: Claude
* Fix materialized frame issues: GC tracking, live locals, f_trace propagation
- Track materialized FrameObjects in GC so cycle collection works
(fixes __del__ not called at exit for exception traceback cycles)
- materialize_chain returns owned PyRef without temporary_refs to
avoid defeating GC cycle detection from non-GC-tracked storage
- framelocalsproxy_setval writes to live source iframe when available
(fixes f_locals proxy writes not affecting executing frame)
- f_lineno uses lasti-based line for non-executing frames instead of
prev_line (fixes PEP 626 wrong line after exception unwind)
- f_trace/f_trace_lines/f_trace_opcodes setters propagate to live
source iframe (fixes pdb set_trace not working through materialized
frames)
- frame.clear() checks find_live_source_iframe to reject clearing a
frame that is backed by a live stack-allocated iframe
- Fix clippy warnings and unused imports in faulthandler
- Remove unsafe top_iframe access from faulthandler watchdog thread
Assisted-by: Claude
* Fix faulthandler crashes, GC tracking timing, and test_generators
- Use top_iframe instead of top_frame for frame chain walking in
faulthandler (dump_all_threads and watchdog), avoiding stale or
GC-cleared FrameObject pointer dereference
- Defer GC tracking of materialized FrameObjects to with_iframe
cleanup (after set_current_frame restores old chain), preventing
premature collection while temporary_refs still holds the only ref
- Add FrameObject::try_iframe() for safe access to potentially
GC-cleared frames
- Remove expectedFailure for test_exhausted_generator_frame_cycle
(now passes with GC tracking fix)
- Add 'noalias' to cspell dictionary
Assisted-by: Claude
* Fix Windows frame dump and remove dead code
- Publish top_iframe on all platforms (not just unix) so Windows
faulthandler and sys._current_frames can see stack-allocated frames
- Use top_iframe instead of frames Mutex in Windows faulthandler
dump and watchdog paths
- Use top_iframe in non-unix get_all_current_frames for sys._current_frames
- Remove unused dump_traceback_thread_chain (replaced by top_iframe walk)
- Mark test_pdb_await_support as expected failure
Assisted-by: Claude
* Fix ThreadSlot init on all platforms, skip test_pdb_await_support
- Add top_iframe to ThreadSlot initializer in init_thread_slot_if_needed
(fixes wasm32, Windows compile errors)
- Skip test_pdb_await_support on RustPython (async pdb exception
callback receives None exc argument)
Assisted-by: Claude
* Remove dead faulthandler code (dump_frame_from_ref, dump_traceback_thread_frames, try_iframe)
These functions are no longer called after switching all frame chain
walking to use top_iframe directly.
Assisted-by: Claude
* Fix Windows sys._current_frames: materialize full frame chain
The non-unix get_all_current_frames path was only materializing the
top iframe without linking retained_back, so f_back chain walking
could not find deeper frames like f123() in test_sys tests.
Replicate the full chain materialization from the unix path.
Assisted-by: Claude
* Use STW-based cross-thread f_back on all platforms
Replace the non-unix frames-mutex fallback with the same
stop-the-world iframe chain materialization used on unix.
Fixes test_current_exceptions on Windows where stack-allocated
iframes were not found in the FrameObject-only frames list.
Assisted-by: Claude
* Fix set_f_lineno to write to live iframe, use STW for cross-thread access
- set_f_lineno: write lasti/pending_stack_pops to the live source
iframe instead of the materialized copy so debugger jumps take
effect on stack-allocated frames
- f_back cross-thread: enter STW before dereferencing the prev
pointer to prevent use-after-free if the owning thread returns
- get_all_current_frames (non-unix): add STW protection when
walking cross-thread iframe chains, matching the unix path
Assisted-by: Claude
* Guard current_location() against lasti==0, always capture retained_back
- current_location(): return first_line_number instead of
panicking when lasti is 0 (before first instruction executes)
- with_frame retained_back: materialize the caller iframe when
needed so f_back always resolves for escaped FrameObjects
Assisted-by: Claude
* Fix format, revert eager retained_back materialization
Remove double blank line in frame.rs (lint failure).
Revert with_frame retained_back to only use already-materialized
caller FrameObjects — eagerly materializing the caller added
refcounts on local variables, preventing timely deallocation and
causing test_io, test_memoryview, and test_futures failures.
Assisted-by: Claude
* Remove rustpython-unicode-isolation-issue.md from gitignore
Assisted-by: Claude
* Remove test_pdb_await_support skip: async pdb now works correctly
Assisted-by: Claude
* Fix STOP_ITERATION monitoring: wrap value in StopIteration instance
fire_stop_iteration was passing the raw iterator return value to
callbacks, but the STOP_ITERATION event callback signature expects
a StopIteration exception instance. Wrap non-StopIteration values
in a new StopIteration(value), matching PyMonitoring_FireStopIterationEvent.
This fixes test_pdb_await_support where bdb's exception_callback
received None instead of a StopIteration instance.
Assisted-by: Claude
* Allow lone-surrogate keyword keys in f(**d)
`f(**{'\udc81': 2})` raised `TypeError: keywords must be strings` even
though the key is a valid `str`, because `KwArgs` stored keys as Rust
`String` (strict UTF-8) and `collect_ex_args` narrowed each key to
`PyUtf8Str` (valid UTF-8 required). CPython only checks that the key is a
`str` (`Py_TPFLAGS_UNICODE_SUBCLASS`), never encoding validity.
Change `KwArgs<T>`'s key type from `String` to `Wtf8Buf` (the WTF-8
storage `PyStr` already uses) and relax the keyword-key downcast from
`PyUtf8Str` to `PyStr` in `collect_ex_args`, `from_vectorcall`/
`from_vectorcall_owned` (which previously panicked on surrogate keys),
the `functools.partial` keyword merge, and the C-API `dict_to_kwargs`.
`Wtf8Buf` borrows only as `Wtf8`, so inherent `get`/`contains_key`/
`swap_remove`/`shift_remove(&str)` on `KwArgs` restore the `&str` lookup
interface `String: Borrow<str>` used to provide (via the zero-cost
`Wtf8::new` cast), and a generic `FromIterator<(K: Into<Wtf8Buf>, T)>`
keeps construction sites unchanged. WTF-8 awareness stays localized to
`function/argument.rs`.
Fixes#8228
Assisted-by: Claude Code:claude-opus-4-8
* Avoid cloning kwargs keys in _ast, borrow via as_ref instead
`new_str` only needs a `&Wtf8`, so pass `key.as_ref()` rather than
cloning the key into an owned `Wtf8Buf`. The key is reused afterwards
(error message, `intern_str`), so a borrow is the right fit.
Assisted-by: Claude Code:claude-opus-4-8
Validate parameter annotations while converting ast.arguments, before
merging positional and keyword-only defaults. This gives invalid annotation
contexts precedence over default-list shape errors, as in CPython.
Merge positional defaults before keyword-only defaults so an excess
positional-default error also takes precedence over a keyword-only default
length mismatch.
Remove the expected-failure markers from the FunctionDef and Lambda AST
validator tests that now pass.
Assisted-by: Codex:gpt-5.6-sol
