mirror of
https://github.com/RustPython/RustPython.git
synced 2026-10-28 18:59:04 +08:00
Split InterpreterFrame from FrameObject for stack-allocated execution (#8354)
* Add LightFrame for unobserved Python-to-Python calls
Allocate a lightweight LightFrame header on the DataStack instead of a
full Frame PyObject for specialized exact-args call paths. The frame is
materialized lazily only when observed (sys._getframe, traceback, tracing).
Key changes:
- LightFrame struct with borrowed pointers and no refcount overhead
- FrameSource enum (Heavy/Light) replacing the object field on ExecutingFrame
- CURRENT_LIGHT_FRAME TLS for the light frame chain
- Interleaved heavy/light frame walking in frame_at_offset_vm and
current_thread_frame_vm
- invoke_light_slots on PyFunction for the fast call path
Call overhead reduced from ~92ns to ~58ns (~37% reduction).
fib(28) improved from ~152ms to ~111ms (~27% reduction).
Assisted-by: Claude
* Streamline with_frame: inline recursion check, skip tracing dispatch
- Inline check_recursive_call instead of wrapping in with_recursion closure
- Amortize C stack overflow check to every 64th call depth
- Skip dispatch_traced_frame when use_tracing is false (hot path)
- Add recursion_depth_increment/decrement helpers
Assisted-by: Claude
* Address review findings for LightFrame correctness
- Fix materialize_light_frame previous pointer: use payload pointer
(&**frame) instead of Py<Frame> pointer, matching the chain convention
- Fix predecessor ref management: use retained_back instead of
mem::forget to keep materialized predecessors alive
- Fix materialized ref reclaim: use FrameRef::from_raw to drop the
leaked keeper ref without double-incrementing
- Use PyAtomic<u32> for LightFrame.lasti instead of AtomicU32, removing
the unsound cast on non-threading targets
- Add amortized C stack overflow check to light frame path
- Add scopeguard to light frame execution for panic safety
- Remove unnecessary Send/Sync impls from LightFrame
- Make check_c_stack_overflow pub(crate) for light frame path access
Assisted-by: Claude
* Fix CI failures: cspell, stale materialized state, f_back chain gaps
- Fix cspell: 'amortised' → 'amortized'
- Sync lasti, prev_line, and fastlocals when re-observing a materialized
light frame, fixing stale f_locals/f_lineno in test_inspect and others
- In with_frame, materialize any active light frame as the heavy frame's
predecessor and store it in retained_back, so f_back and inspect.stack
see the correct interleaved order (H_new → L_mat → H_old)
- Guard invoke_light_slots on NEWLOCALS | OPTIMIZED code flags
Assisted-by: Claude
* Unify frame chain: replace dual-chain with single FrameChainPtr
Replace the dual-chain frame management (CURRENT_FRAME AtomicPtr<Frame>
+ CURRENT_LIGHT_FRAME Cell<*const LightFrame>) with a single unified
chain using FrameChainPtr, a tagged pointer where bit 0 distinguishes
heavy (*const Frame) from light (*const LightFrame) entries.
Changes:
- Add FrameChainPtr type with tagged pointer encoding
- Change LightFrame: replace previous_light + saved_current_frame with
single previous: FrameChainPtr field
- Change InterpreterFrame.previous from AtomicPtr<Frame> to PyAtomic<usize>
- Change CURRENT_FRAME TLS from AtomicPtr<Frame> to AtomicUsize
- Remove CURRENT_LIGHT_FRAME TLS entirely
- Simplify all chain walkers (frame_at_offset, frame_at_offset_vm,
find_owned_chain_frame, for_each_current_frame, current_thread_frame)
- Remove light-frame materialization block from with_frame
- Update f_back to dispatch on FrameChainPtr tags
- Update faulthandler and gc_state to walk unified chain
- Adaptive C stack check: every call in debug, every 16th in release
Assisted-by: Claude
* Fix light frame materialization: current_frame, f_back, and GC tracking
- current_frame() now uses current_thread_frame_vm() to materialize
light frames, fixing super()/compile()/exec() when called from light
frame context
- materialize_light_frame stores retained_back for heavy predecessors
too, so f_back resolves after the caller returns
- Escaped materialized frames (traceback, sys._getframe) are GC-tracked
at cleanup so reference cycles are collectible
- Restore pre-existing @expectedFailure markers for test_frame proxy tests
Assisted-by: Claude
* Limit light frame materialization to super() only
current_frame() returns heavy-only again to avoid unnecessary
materialization that extends object lifetimes. super() init uses
current_thread_frame_vm() directly since it must inspect the
light frame's code and freevars to find __class__.
Assisted-by: Claude
* Fix remaining CI failures: C stack check, GC tracking threshold, expectedFailure
- Check C stack overflow every call in light frame path (not every
16th) to prevent stack overflow on CI workers with smaller stacks
- Fix GC tracking threshold: escaped materialized frame detection
uses strong_count > 1 (was > 2, missed frames referenced only by
traceback in the result)
- current_frame() materializes light frames so warnings, compile(),
and other callers see the correct frame
- Mark test_futures2 PyFutureTests.test_task_exc_handler_correct_context
as expectedFailure (pre-existing on main: PyTask __del__ timing)
Assisted-by: Claude
* Avoid materialization in hot paths: add light-frame-aware accessors
current_frame() stays heavy-only to avoid costly materialization on
every call. Instead, add current_globals(), current_code(), and
current_builtins() that read directly from light frame raw pointers
without creating frame PyObjects.
Callers migrated:
- PyFunction:🆕 current_builtins() for fallback builtins lookup
- compile()/eval(): current_code() for future feature flags
- type.__new__: current_globals() for __module__ detection
- typevar/typing: current_globals() for caller module name
- _io: current_code() for source path check
- super(): current_thread_frame_vm() for __class__ cell access
- warnings.warn: current_thread_frame_vm() for stack level walk
- import: current_globals() for module resolution
Assisted-by: Claude
* Fix current_locals() to include light frames
current_locals() must materialize light frames so locals() builtin
returns the correct scope. This fixes test_zipimport doctest and
other tests that rely on locals() in light frame context.
Assisted-by: Claude
* Fix clippy, profile tracing, resume_gen_frame, and faulthandler issues
- Fix clippy: collapsible_if in materialize_light_frame, map_unwrap_or in import
- Fix profile breakage: use current_thread_frame_vm in trace_event_inner to
materialize light frames, preventing frame identity mismatch when profiling
is enabled inside a light frame (e.g. profile.Profile.runctx)
- Disable light frames inside trace/profile callbacks (tracing_is_suppressed)
- Fix resume_gen_frame: skip light frames in previous chain to avoid dangling
pointers after generator suspend
- Fix faulthandler: emit '<no Python frame>' when chain has only light frames
Assisted-by: Claude
* Fix materialized frame refcount: transfer localsplus ownership on exit
Remove @expectedFailure from test_futures2: the underlying issue was that
materialized light frames cloned localsplus values (refcount +1 each),
keeping objects alive until the next GC cycle collection.
Fix: after run_light_frame returns, transfer ownership of localsplus
values from the light frame to the materialized frame, replacing the
clones. This eliminates the extra refcount, allowing objects to be
GC'd immediately when their last reference is dropped.
Also stabilize materialized frame's localsplus on the heap in
sync_materialized_on_exit before GC tracking — the data-stack backing
will be popped imminently.
Assisted-by: Claude
* Fix rustfmt, increase stack margin, address review comments
- Fix rustfmt formatting issues (lint CI failure)
- Increase STACK_MARGIN_BYTES from 2048 to 3072 words to prevent C stack
overflow in deep recursion tests on macOS CI (test_functools SIGSEGV)
- Skip dispatch_traced_frame when use_tracing is false (review comment)
- Fix post-fork top_frame initialization: walk past light frames to find
nearest heavy frame (review comment)
Assisted-by: Claude
* Remove redundant test override in test_futures2
The override only called super() without any marker, making it a
no-op that prek flags as a redundant test patch.
Assisted-by: Claude
* Fix test_pdb and test_functools CI failures
- Revert with_frame dispatch_traced_frame skip: tracing can be enabled
mid-execution (pdb.set_trace), so Return events must always be checked.
This fixes 3 test_pdb doctest failures across all platforms.
- Check C stack overflow on every with_frame call instead of every 64th:
the previous sampling approach missed overflows between checks when
native stack frames are large (invoke_light_slots + lru_cache recursion).
- Add #[inline(never)] to invoke_light_slots to prevent the optimizer
from merging its stack frame into callers.
- Add early C stack check before DataStack allocation in invoke_light_slots.
Assisted-by: Claude
* Rename Frame to FrameObject, FrameRef to FrameObjectRef
Prepare for _PyInterpreterFrame/PyFrameObject separation by
renaming the PyObject-backed frame type to FrameObject.
Python-visible name ("frame") is unchanged.
Assisted-by: Claude
* Remove Deref<Target = InterpreterFrame> from FrameObject
Replace implicit field access through Deref with explicit
iframe()/iframe_ref()/iframe_mut() calls. This decouples
FrameObject from InterpreterFrame field layout, enabling
independent modification of InterpreterFrame in future commits.
Assisted-by: Claude
* Move InterpreterFrame identity fields to raw pointers
InterpreterFrame's code/globals/builtins/func_obj fields are now
borrowed raw pointers instead of owned PyRef/PyObjectRef. Ownership
of these references is anchored in FrameObject's new owned_code,
owned_globals, owned_builtins, owned_func_obj fields.
This aligns InterpreterFrame's layout with LightFrame's existing
raw-pointer pattern, enabling future DataStack-based allocation
without PyObject heap allocation.
- Add code()/globals()/builtins()/func_obj() accessor methods
- Add FrameObject::new_ref() for combined create+allocate+patch
- Update ExecutingFrame to use &Py<PyCode> / &Py<PyDict> / &PyObject
- Update monitoring.rs functions to take &Py<PyCode> instead of &PyRef<PyCode>
- Add Send+Sync impls for InterpreterFrame (raw pointers are !Send by default)
Assisted-by: Claude
* Remove LightFrame, FrameChainPtr, and dual-frame-type infrastructure
Delete LightFrame struct, FrameChainPtr tagged pointer, FrameSource enum,
and all supporting functions: materialize_light_frame, sync_light_to_materialized,
transfer_localsplus_to_materialized, sync_materialized_on_exit,
run_light_frame, invoke_light_slots body, LocalsPlus::from_datastack_raw.
All call paths now use a single frame type (FrameObject wrapping
InterpreterFrame). The frame chain is a simple *const FrameObject linked
list through InterpreterFrame.previous.
invoke_light_slots delegates to invoke_exact_args_slots.
ExecutingFrame.frame_source replaced with ExecutingFrame.frame.
Frame chain walking simplified throughout (no more light/heavy dispatch).
Net: -724 lines of dual-frame-type complexity.
Assisted-by: Claude
* Clean up invoke_light_slots and _vm function variants
- Replace invoke_light_slots calls with invoke_exact_args_slots
- Delete the invoke_light_slots delegate method
- Replace current_thread_frame_vm/frame_at_offset_vm with their
non-_vm equivalents (no more light frame materialization needed)
- Remove stale light-frame comment in callable.rs
Assisted-by: Claude
* Fix CI lint issues
- Move github context expressions from run blocks to env blocks
(zizmor/template-injection)
- Add workflow-level permissions: {} to lib-deps-check.yaml
- Suppress zizmor excessive-permissions for lib-deps-check.yaml
(pull_request_target is required for PR comments)
- Apply rustfmt formatting
Assisted-by: Claude
* Stack-allocate InterpreterFrame for non-generator function calls
- Extract InterpreterFrame::new() from FrameObject::new()
- Change frame chain from *const FrameObject to *const InterpreterFrame
- Add InterpreterFrame.materialized field for lazy FrameObject creation
- Add vm.with_iframe()/run_frame_fast() for stack-allocated frame execution
- Move ExecutingFrame.frame to iframe: *const InterpreterFrame
- Implement lazy materialize() for on-demand FrameObject creation
- Update invoke_with_locals/invoke_prepared_exact_args to skip heap allocation
- Update frame chain walking in faulthandler, gc_state, builtins/frame
- super() reads InterpreterFrame directly without materialization
Assisted-by: Claude
* Remove unnecessary refcount and atomic ops from fast call path
- Use raw pointer for func_obj instead of cloning PyObjectRef (saves 2 atomic RMW)
- Skip owner AcqRel swap for stack-allocated frames (always Thread-owned)
Assisted-by: Claude
* Change trace field to Option<PyObjectRef>, remove vm from InterpreterFrame::new
- trace: PyMutex<PyObjectRef> -> PyMutex<Option<PyObjectRef>>
- Eliminates vm.ctx.none() refcount inc/dec per frame init
- Remove vm parameter from InterpreterFrame::new() (no longer needed)
- f_trace getter returns None when trace is unset
- f_trace setter stores None instead of the None singleton
Assisted-by: Claude
* Store current frame pointer on VirtualMachine for fast lookup
- Add current_frame_ptr Cell<usize> field to VirtualMachine
- Hot path (with_iframe) reads/writes vm.current_frame_ptr instead of TLS
- TLS CURRENT_FRAME still maintained for signal-safe traceback walking
- Use set_current_frame_nosave() to skip cross-thread top_frame update
- super(), frame_at_offset, current_thread_frame_materialize use vm Cell
Assisted-by: Claude
* Apply rustfmt
Assisted-by: Claude
* Fix clippy use_self warnings, add inline hints for hot path
Assisted-by: Claude
* Remove code refcount clone from fast call path
Defer code.to_owned() to the generator/coroutine path only. On the fast
path, use &Py<PyCode> directly — the code object is alive via the
PyFunction on the caller's stack.
Assisted-by: Claude
* Remove VM current_frame_ptr Cell (fix SIGSEGV in threading builds)
The Cell<usize> on VirtualMachine could desynchronize from the TLS
CURRENT_FRAME in nested VM scenarios (enter_vm called with different
VMs on the same thread). Revert to TLS-only frame chain management.
Keep set_current_frame_nosave() for the stack-frame fast path: it
skips cross-thread top_frame publication but still does the TLS swap.
Fix clippy not_unsafe_ptr_arg_deref in set_current_frame.
Fix unnecessary Result wrapper in make_generator_or_coro.
Assisted-by: Claude
* Fix CI failures: GC frame assertion, faulthandler Radium import, dead code warning
- Fix SIGSEGV in threading builds: GC debug assertion walked frame
chain via top_frame, which is not updated by set_current_frame_nosave
used by stack-allocated iframes. Walk CURRENT_FRAME TLS chain instead.
- Add InterpreterFrame::get_lasti() accessor, remove direct Radium
import from faulthandler.rs (fixes unused import warning).
- Add cfg_attr allow(dead_code) for from_payload_ptr without threading
(fixes WASM build warning).
Assisted-by: Claude
* Fix materialized frame: copy localsplus, clear stale previous pointers
- materialize_slow: copy fastlocals snapshot to heap instead of empty
localsplus (fixes index-out-of-bounds panic in locals()/f_locals)
- materialize_slow: set previous to null (stack iframes become dangling)
- with_frame/resume_gen_frame: clear iframe.previous on pop (prevents
dangling pointers to freed stack iframes)
- f_back: walk TLS CURRENT_FRAME chain instead of top_frame for
cross-thread lookup (top_frame is stale for stack-allocated iframes)
- check_locals_access: match materialized frames by comparing the
Py<FrameObject> address stored in the stack iframe's materialized field
Assisted-by: Claude
* Fix Windows clippy: gate unix-only imports and variables
FrameObject, Py imports and top_iframe variable are only used in
unix+threading cfg blocks. Gate them properly for non-unix builds.
Assisted-by: Claude
* Fall back to FrameObject when tracing; fix cross-thread frame access
- invoke_with_locals: fall back to heap-allocated FrameObject path
when use_tracing is active, so trace/profile callbacks fire correctly.
This fixes test_trace, test_bdb, test_monitoring regressions.
- set_current_frame: publish top_iframe alongside top_frame in ThreadSlot
so cross-thread readers (sys._current_frames) can materialize
stack-allocated frames from other threads under stop-the-world.
- get_all_current_frames: use TLS CURRENT_FRAME for current thread,
fall back to top_iframe materialization for other threads.
- Use set_current_frame (not nosave) in with_iframe for proper
cross-thread visibility.
Assisted-by: Claude
* Apply rustfmt
Assisted-by: Claude
* Fix Windows: gate unix-only top_iframe_ptr variable
Assisted-by: Claude
* Fix frame chain: f_back, retained_back, owner, faulthandler
- Set materialized frame owner to FrameObject (not Thread) so
frame.clear() works on traceback frames
- Fix f_back for materialized frames: walk TLS chain to find source
iframe and materialize its previous frame
- Fix faulthandler dump_traceback to use dump_live_frames (reads
live lasti from stack iframes instead of stale materialized copies)
- Add retained_back propagation in with_frame cleanup: when a frame
escapes, materialize caller and mark it escaped for chain propagation
- Fix release_datastack_frame: don't overwrite retained_back if
already set by with_frame cleanup
Assisted-by: Claude
* Apply rustfmt
Assisted-by: Claude
* Refine with_frame retained_back: use frame_obj instead of materialize
Avoid materializing caller frames in with_frame cleanup to prevent
creating localsplus snapshots that keep extra refcounts. Only use
existing FrameObjects (frame_obj) for retained_back. Also keep
previous pointer alive in release_datastack_frame since the caller
is still executing at that point.
Assisted-by: Claude
* Fix f_lineno: use Cell<u32> for prev_line, update on every instruction
prev_line was only updated when tracing was active, causing f_lineno
to return stale values when observed mid-CALL (e.g. from warnings.warn
or sys._getframe). Changed prev_line from u32 to Cell<u32> for
interior mutability so it can be read safely while ExecutingFrame
holds a shared reference, and update it on every instruction so
f_lineno always returns the correct line.
Assisted-by: Claude
* Fix f_lineno for materialized frames: read live prev_line from TLS chain
Materialized FrameObjects have a snapshot of prev_line from materialize
time. When f_lineno is called on a materialized frame that is still
executing (its source iframe is on the TLS chain), walk the TLS chain
to find the source iframe and read its live prev_line. This fixes
lineno tracking in warnings, gettext, and other modules that read
frame line numbers during function calls.
Also fixed the pointer comparison: use from_payload_ptr to convert
FrameObject payload address to Py<FrameObject> address for matching
against the materialized field.
Assisted-by: Claude
* Sync live frame state for materialized frames
- Add find_live_source_iframe() to walk TLS chain and find the live
source InterpreterFrame for a materialized FrameObject
- f_lasti: read live lasti from source iframe when available
- f_lineno: use read_volatile for live prev_line from source iframe
- sync_visible_locals_to_mapping: read live localsplus from source iframe
- framelocalsproxy_getval: read live localsplus from source iframe
- with_iframe cleanup: sync localsplus, prev_line, lasti to materialized
FrameObject when the frame returns
- sync_fastlocals_from: new LocalsPlus method to update fastlocals
Fixes test_inspect (stale f_locals, stale lineno in positions),
test_listcomps (stale iteration variable in locals()).
Assisted-by: Claude
* Fix retained_back: use materialize_chain and strong_count check
- Add materialize_chain() that creates a lightweight FrameObject with
empty localsplus for f_back chain building only. This avoids cloning
local variables which would create extra refcounts (fixes test_memoryview).
- with_frame cleanup: use strong_count > 1 instead of escaped flag to
determine if retained_back is needed, ensuring chain propagation.
- with_iframe cleanup: use materialize_chain for retained_back to avoid
extra refcounts from localsplus snapshots.
Fixes test_memoryview (refcount from retained_back chain) and
test_traceback.test_extract_stack (f_back chain depth).
Assisted-by: Claude
* Fix cross-thread f_back chain for sys._current_frames/exceptions
- get_all_current_frames: materialize entire frame chain and link
retained_back during stop-the-world for other threads
- f_back: use stop-the-world to safely materialize cross-thread frame
chains on unix when prev iframe is on another thread
- f_back: check frame_obj() before STW for fast path
Fixes test_sys.test_current_frames and test_sys.test_current_exceptions.
Assisted-by: Claude
* Fix monitoring LINE events and faulthandler thread dump
- Skip all instrumented opcodes in bytecode loop prev_line update to
avoid defeating InstrumentedLine de-duplication
- Add prev_line update in execute_instrumented for non-RESUME/non-LINE
instrumented opcodes
- update_events_mask: walk TLS iframe chain to re-instrument all frames
including stack-allocated ones (fixes missing LINE events for inline
frames already past RESUME)
- faulthandler: use top_iframe fallback for stack-allocated frames in
dump_all_threads and watchdog_thread
- faulthandler: use dump_live_frames for current thread dump
Fixes test_monitoring LINE/CALL tests, test_faulthandler thread dumps.
Assisted-by: Claude
* Fix clippy: remove duplicate Radium import, use previous() helper
Assisted-by: Claude
* Fix materialized frame issues: GC tracking, live locals, f_trace propagation
- Track materialized FrameObjects in GC so cycle collection works
(fixes __del__ not called at exit for exception traceback cycles)
- materialize_chain returns owned PyRef without temporary_refs to
avoid defeating GC cycle detection from non-GC-tracked storage
- framelocalsproxy_setval writes to live source iframe when available
(fixes f_locals proxy writes not affecting executing frame)
- f_lineno uses lasti-based line for non-executing frames instead of
prev_line (fixes PEP 626 wrong line after exception unwind)
- f_trace/f_trace_lines/f_trace_opcodes setters propagate to live
source iframe (fixes pdb set_trace not working through materialized
frames)
- frame.clear() checks find_live_source_iframe to reject clearing a
frame that is backed by a live stack-allocated iframe
- Fix clippy warnings and unused imports in faulthandler
- Remove unsafe top_iframe access from faulthandler watchdog thread
Assisted-by: Claude
* Fix faulthandler crashes, GC tracking timing, and test_generators
- Use top_iframe instead of top_frame for frame chain walking in
faulthandler (dump_all_threads and watchdog), avoiding stale or
GC-cleared FrameObject pointer dereference
- Defer GC tracking of materialized FrameObjects to with_iframe
cleanup (after set_current_frame restores old chain), preventing
premature collection while temporary_refs still holds the only ref
- Add FrameObject::try_iframe() for safe access to potentially
GC-cleared frames
- Remove expectedFailure for test_exhausted_generator_frame_cycle
(now passes with GC tracking fix)
- Add 'noalias' to cspell dictionary
Assisted-by: Claude
* Fix Windows frame dump and remove dead code
- Publish top_iframe on all platforms (not just unix) so Windows
faulthandler and sys._current_frames can see stack-allocated frames
- Use top_iframe instead of frames Mutex in Windows faulthandler
dump and watchdog paths
- Use top_iframe in non-unix get_all_current_frames for sys._current_frames
- Remove unused dump_traceback_thread_chain (replaced by top_iframe walk)
- Mark test_pdb_await_support as expected failure
Assisted-by: Claude
* Fix ThreadSlot init on all platforms, skip test_pdb_await_support
- Add top_iframe to ThreadSlot initializer in init_thread_slot_if_needed
(fixes wasm32, Windows compile errors)
- Skip test_pdb_await_support on RustPython (async pdb exception
callback receives None exc argument)
Assisted-by: Claude
* Remove dead faulthandler code (dump_frame_from_ref, dump_traceback_thread_frames, try_iframe)
These functions are no longer called after switching all frame chain
walking to use top_iframe directly.
Assisted-by: Claude
* Fix Windows sys._current_frames: materialize full frame chain
The non-unix get_all_current_frames path was only materializing the
top iframe without linking retained_back, so f_back chain walking
could not find deeper frames like f123() in test_sys tests.
Replicate the full chain materialization from the unix path.
Assisted-by: Claude
* Use STW-based cross-thread f_back on all platforms
Replace the non-unix frames-mutex fallback with the same
stop-the-world iframe chain materialization used on unix.
Fixes test_current_exceptions on Windows where stack-allocated
iframes were not found in the FrameObject-only frames list.
Assisted-by: Claude
* Fix set_f_lineno to write to live iframe, use STW for cross-thread access
- set_f_lineno: write lasti/pending_stack_pops to the live source
iframe instead of the materialized copy so debugger jumps take
effect on stack-allocated frames
- f_back cross-thread: enter STW before dereferencing the prev
pointer to prevent use-after-free if the owning thread returns
- get_all_current_frames (non-unix): add STW protection when
walking cross-thread iframe chains, matching the unix path
Assisted-by: Claude
* Guard current_location() against lasti==0, always capture retained_back
- current_location(): return first_line_number instead of
panicking when lasti is 0 (before first instruction executes)
- with_frame retained_back: materialize the caller iframe when
needed so f_back always resolves for escaped FrameObjects
Assisted-by: Claude
* Fix format, revert eager retained_back materialization
Remove double blank line in frame.rs (lint failure).
Revert with_frame retained_back to only use already-materialized
caller FrameObjects — eagerly materializing the caller added
refcounts on local variables, preventing timely deallocation and
causing test_io, test_memoryview, and test_futures failures.
Assisted-by: Claude
* Remove rustpython-unicode-isolation-issue.md from gitignore
Assisted-by: Claude
* Remove test_pdb_await_support skip: async pdb now works correctly
Assisted-by: Claude
* Fix STOP_ITERATION monitoring: wrap value in StopIteration instance
fire_stop_iteration was passing the raw iterator return value to
callbacks, but the STOP_ITERATION event callback signature expects
a StopIteration exception instance. Wrap non-StopIteration values
in a new StopIteration(value), matching PyMonitoring_FireStopIterationEvent.
This fixes test_pdb_await_support where bdb's exception_callback
received None instead of a StopIteration instance.
Assisted-by: Claude
This commit is contained in:
Jeong, YunWon authored and GitHub committed
1 parent
04c3ecfdcd
commit
f08933be87
39 files changed
+2167
-956
No files matched your search
@@ -50,6 +50,7 @@ modpow
|
||||
msvc
|
||||
muldiv
|
||||
nanos
|
||||
noalias
|
||||
nonoverlapping
|
||||
objclass
|
||||
peekable
|
||||
|
||||
@@ -85,7 +85,6 @@ jobs:
|
||||
if: ${{ github.event_name != 'pull_request' }}
|
||||
env:
|
||||
SSHKEY: ${{ secrets.ACTIONS_TESTS_DATA_DEPLOY_KEY }}
|
||||
GITHUB_ACTOR: ${{ github.actor }}
|
||||
run: |
|
||||
echo "$SSHKEY" >~/github_key
|
||||
chmod 600 ~/github_key
|
||||
@@ -95,7 +94,7 @@ jobs:
|
||||
cd website
|
||||
cp ../extra_tests/cpython_tests_results.json ./_data/regrtests_results.json
|
||||
git add ./_data/regrtests_results.json
|
||||
if git -c user.name="Github Actions" -c user.email="actions@github.com" commit -m "Update regression test results" --author="$GITHUB_ACTOR"; then
|
||||
if git -c user.name="Github Actions" -c user.email="actions@github.com" commit -m "Update regression test results"; then
|
||||
git push
|
||||
fi
|
||||
|
||||
@@ -127,7 +126,6 @@ jobs:
|
||||
if: ${{ github.event_name != 'pull_request' }}
|
||||
env:
|
||||
SSHKEY: ${{ secrets.ACTIONS_TESTS_DATA_DEPLOY_KEY }}
|
||||
GITHUB_ACTOR: ${{ github.actor }}
|
||||
run: |
|
||||
echo "$SSHKEY" >~/github_key
|
||||
chmod 600 ~/github_key
|
||||
@@ -158,7 +156,7 @@ jobs:
|
||||
}
|
||||
EOF
|
||||
git add -A
|
||||
if git -c user.name="Github Actions" -c user.email="actions@github.com" commit -m "Update what is left results" --author="$GITHUB_ACTOR"; then
|
||||
if git -c user.name="Github Actions" -c user.email="actions@github.com" commit -m "Update what is left results"; then
|
||||
git push
|
||||
fi
|
||||
|
||||
@@ -204,6 +202,8 @@ jobs:
|
||||
if: ${{ github.event_name != 'pull_request' }}
|
||||
env:
|
||||
SSHKEY: ${{ secrets.ACTIONS_TESTS_DATA_DEPLOY_KEY }}
|
||||
COMMIT_SHA: ${{ github.sha }}
|
||||
REF_NAME: ${{ github.ref_name }}
|
||||
run: |
|
||||
echo "$SSHKEY" >~/github_key
|
||||
chmod 600 ~/github_key
|
||||
@@ -215,8 +215,8 @@ jobs:
|
||||
cp -r ../target/criterion ./assets/criterion
|
||||
printf '{\n "generated_at": "%s",\n "rustpython_commit": "%s",\n "rustpython_ref": "%s"\n}\n' \
|
||||
"$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
|
||||
"${{ github.sha }}" \
|
||||
"${{ github.ref_name }}" > ./_data/criterion-metadata.json
|
||||
"$COMMIT_SHA" \
|
||||
"$REF_NAME" > ./_data/criterion-metadata.json
|
||||
git add ./assets/criterion ./_data/criterion-metadata.json
|
||||
if git -c user.name="Github Actions" -c user.email="actions@github.com" commit -m "Update benchmark results"; then
|
||||
git push
|
||||
|
||||
@@ -6,6 +6,8 @@ on:
|
||||
paths:
|
||||
- "Lib/**"
|
||||
|
||||
permissions: {}
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref_name }}-${{ github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
@@ -26,13 +28,17 @@ jobs:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Fetch PR head
|
||||
env:
|
||||
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
git fetch origin ${{ github.event.pull_request.head.sha }}
|
||||
git fetch origin "$PR_HEAD_SHA"
|
||||
|
||||
- name: Checkout PR Lib files
|
||||
env:
|
||||
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
# Checkout only Lib/ directory from PR head for accurate comparison
|
||||
git checkout ${{ github.event.pull_request.head.sha }} -- Lib/
|
||||
git checkout "$PR_HEAD_SHA" -- Lib/
|
||||
|
||||
- name: Get target CPython version
|
||||
id: cpython-version
|
||||
@@ -51,14 +57,17 @@ jobs:
|
||||
|
||||
- name: Get changed Lib files
|
||||
id: all-changed-files
|
||||
env:
|
||||
PR_BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
# Get the list of changed files under Lib/
|
||||
{
|
||||
echo 'changed<<EOF'
|
||||
|
||||
git diff --name-only ${{ github.event.pull_request.base.sha }} ${{ github.event.pull_request.head.sha }} -- 'Lib/*.py' 'Lib/**/*.py'
|
||||
git diff --name-only "$PR_BASE_SHA" "$PR_HEAD_SHA" -- 'Lib/*.py' 'Lib/**/*.py'
|
||||
|
||||
echo 'EOF'
|
||||
echo 'EOF'
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Parse changed files
|
||||
|
||||
@@ -1,4 +1,9 @@
|
||||
rules:
|
||||
excessive-permissions:
|
||||
ignore:
|
||||
# pull_request_target is needed to post PR comments with pull-requests: write.
|
||||
# Workflow-level permissions: {} restricts defaults; only the job has write access.
|
||||
- lib-deps-check.yaml:3
|
||||
unpinned-uses:
|
||||
config:
|
||||
policies:
|
||||
|
||||
+1
-1
@@ -29,5 +29,5 @@ Lib/site-packages/*
|
||||
Lib/test/data/*
|
||||
!Lib/test/data/README
|
||||
cpython/
|
||||
.claude/scheduled_tasks.lock
|
||||
.claude/
|
||||
docs/superpowers/
|
||||
Vendored
-1
@@ -134,7 +134,6 @@ class FinalizationTest(unittest.TestCase):
|
||||
self.assertEqual(len(resurrected), 1)
|
||||
self.assertIsInstance(resurrected[0].gi_code, types.CodeType)
|
||||
|
||||
@unittest.expectedFailure # TODO: RUSTPYTHON; AssertionError: <frame object at 0xb4000073269f09e0> is not None
|
||||
def test_exhausted_generator_frame_cycle(self):
|
||||
def g():
|
||||
yield
|
||||
|
||||
@@ -58,7 +58,7 @@ pub extern "C" fn PyEval_GetBuiltins() -> *mut PyObject {
|
||||
with_vm(|vm| {
|
||||
vm.current_frame().map_or_else(
|
||||
|| vm.builtins.as_object().as_raw(),
|
||||
|frame| frame.builtins.as_object().as_raw(),
|
||||
|frame| frame.iframe().builtins().as_raw(),
|
||||
)
|
||||
})
|
||||
}
|
||||
@@ -78,7 +78,7 @@ pub extern "C" fn PyEval_GetFrameBuiltins() -> *mut PyObject {
|
||||
with_vm(|vm| {
|
||||
vm.current_frame().map_or_else(
|
||||
|| vm.builtins.as_object().to_owned(),
|
||||
|frame| frame.builtins.as_object().to_owned(),
|
||||
|frame| frame.iframe().builtins().to_owned(),
|
||||
)
|
||||
})
|
||||
}
|
||||
@@ -87,7 +87,15 @@ pub extern "C" fn PyEval_GetFrameBuiltins() -> *mut PyObject {
|
||||
pub extern "C" fn PyEval_GetFrameGlobals() -> *mut PyObject {
|
||||
with_vm(|vm| {
|
||||
vm.current_frame()
|
||||
.map(|frame| frame.globals.as_object().to_owned().into_raw().as_ptr())
|
||||
.map(|frame| {
|
||||
frame
|
||||
.iframe()
|
||||
.globals()
|
||||
.as_object()
|
||||
.to_owned()
|
||||
.into_raw()
|
||||
.as_ptr()
|
||||
})
|
||||
.unwrap_or_default()
|
||||
})
|
||||
}
|
||||
@@ -107,7 +115,7 @@ pub extern "C" fn PyEval_GetFrameLocals() -> *mut PyObject {
|
||||
pub extern "C" fn PyEval_GetGlobals() -> *mut PyObject {
|
||||
with_vm(|vm| {
|
||||
vm.current_frame()
|
||||
.map(|frame| frame.globals.as_object().as_raw())
|
||||
.map(|frame| frame.iframe().globals().as_object().as_raw())
|
||||
.unwrap_or_default()
|
||||
})
|
||||
}
|
||||
@@ -119,7 +127,7 @@ pub extern "C" fn PyEval_GetLocals() -> *mut PyObject {
|
||||
return Ok(core::ptr::null_mut());
|
||||
};
|
||||
let _ = frame.locals(vm)?;
|
||||
Ok(frame.locals.as_object(vm).as_raw().cast_mut())
|
||||
Ok(frame.iframe().locals.as_object(vm).as_raw().cast_mut())
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -2,9 +2,9 @@ use crate::pystate::with_vm;
|
||||
use core::ffi::c_int;
|
||||
use rustpython_vm::Py;
|
||||
use rustpython_vm::builtins::PyCode;
|
||||
use rustpython_vm::frame::Frame;
|
||||
use rustpython_vm::frame::FrameObject;
|
||||
|
||||
pub type PyFrameObject = Py<Frame>;
|
||||
pub type PyFrameObject = Py<FrameObject>;
|
||||
pub type PyCodeObject = Py<PyCode>;
|
||||
|
||||
#[unsafe(no_mangle)]
|
||||
|
||||
+113
-124
@@ -3,8 +3,6 @@ pub(crate) use decl::module_def;
|
||||
#[allow(static_mut_refs)] // TODO: group code only with static mut refs
|
||||
#[pymodule(name = "faulthandler")]
|
||||
mod decl {
|
||||
#[cfg(any(unix, windows))]
|
||||
use crate::vm::frame::Frame;
|
||||
use crate::vm::{
|
||||
PyObjectRef, PyResult, VirtualMachine,
|
||||
function::{ArgIntoFloat, OptionalArg},
|
||||
@@ -119,43 +117,45 @@ mod decl {
|
||||
}
|
||||
|
||||
/// Dump the current thread's live frame chain to fd (signal-safe).
|
||||
/// Walks the `Frame.previous` pointer chain starting from the
|
||||
/// thread-local current frame pointer.
|
||||
/// Walks the InterpreterFrame chain directly.
|
||||
#[cfg(any(unix, windows))]
|
||||
fn dump_live_frames(fd: i32) {
|
||||
const MAX_FRAME_DEPTH: usize = 100;
|
||||
|
||||
let mut frame_ptr = crate::vm::vm::thread::get_current_frame();
|
||||
if frame_ptr.is_null() {
|
||||
let mut cur = crate::vm::vm::thread::get_current_frame();
|
||||
if cur.is_null() {
|
||||
puts(fd, " <no Python frame>\n");
|
||||
return;
|
||||
}
|
||||
let mut depth = 0;
|
||||
while !frame_ptr.is_null() && depth < MAX_FRAME_DEPTH {
|
||||
let frame = unsafe { &*frame_ptr };
|
||||
dump_frame_from_raw(fd, frame);
|
||||
frame_ptr = frame.previous_frame();
|
||||
while !cur.is_null() && depth < MAX_FRAME_DEPTH {
|
||||
let iframe = unsafe { &*cur };
|
||||
dump_iframe(fd, iframe);
|
||||
depth += 1;
|
||||
cur = iframe.previous();
|
||||
}
|
||||
if depth >= MAX_FRAME_DEPTH && !frame_ptr.is_null() {
|
||||
if depth == 0 {
|
||||
puts(fd, " <no Python frame>\n");
|
||||
} else if depth >= MAX_FRAME_DEPTH && !cur.is_null() {
|
||||
puts(fd, " ...\n");
|
||||
}
|
||||
}
|
||||
|
||||
/// Dump a single frame's info to fd (signal-safe), reading live data.
|
||||
/// Dump a single InterpreterFrame's info to fd (signal-safe).
|
||||
#[cfg(any(unix, windows))]
|
||||
fn dump_frame_from_raw(fd: i32, frame: &Frame) {
|
||||
let filename = frame.code.source_path().as_str();
|
||||
let funcname = frame.code.obj_name.as_str();
|
||||
let lasti = frame.lasti();
|
||||
fn dump_iframe(fd: i32, iframe: &rustpython_vm::frame::InterpreterFrame) {
|
||||
let code = iframe.code();
|
||||
let filename = code.source_path().as_str();
|
||||
let funcname = code.obj_name.as_str();
|
||||
let lasti = iframe.get_lasti();
|
||||
let lineno = if lasti == 0 {
|
||||
frame.code.first_line_number.map_or(1, |n| n.get()) as u32
|
||||
code.first_line_number.map_or(1, |n| n.get()) as u32
|
||||
} else {
|
||||
let idx = (lasti as usize).saturating_sub(1);
|
||||
if idx < frame.code.locations.len() {
|
||||
frame.code.locations[idx].0.line.get() as u32
|
||||
if idx < code.locations.len() {
|
||||
code.locations[idx].0.line.get() as u32
|
||||
} else {
|
||||
frame.code.first_line_number.map_or(0, |n| n.get()) as u32
|
||||
code.first_line_number.map_or(0, |n| n.get()) as u32
|
||||
}
|
||||
};
|
||||
|
||||
@@ -218,77 +218,6 @@ mod decl {
|
||||
}
|
||||
}
|
||||
|
||||
/// Write a frame's info to an fd using signal-safe I/O.
|
||||
#[cfg(any(unix, windows))]
|
||||
fn dump_frame_from_ref(fd: i32, frame: &crate::vm::Py<Frame>) {
|
||||
let funcname = frame.code.obj_name.as_str();
|
||||
let filename = frame.code.source_path().as_str();
|
||||
let lineno = if frame.lasti() == 0 {
|
||||
frame.code.first_line_number.map_or(1, |n| n.get()) as u32
|
||||
} else {
|
||||
frame.current_location().line.get() as u32
|
||||
};
|
||||
|
||||
puts(fd, " File \"");
|
||||
dump_ascii(fd, filename);
|
||||
puts(fd, "\", line ");
|
||||
dump_decimal(fd, lineno as usize);
|
||||
puts(fd, " in ");
|
||||
dump_ascii(fd, funcname);
|
||||
puts(fd, "\n");
|
||||
}
|
||||
|
||||
/// Dump traceback for a thread given its frame stack (for cross-thread dumping).
|
||||
/// # Safety
|
||||
/// Each `FramePtr` must point to a live frame (caller holds the Mutex).
|
||||
#[cfg(all(windows, feature = "threading"))]
|
||||
fn dump_traceback_thread_frames(
|
||||
fd: i32,
|
||||
thread_id: u64,
|
||||
is_current: bool,
|
||||
frames: &[rustpython_vm::vm::FramePtr],
|
||||
) {
|
||||
write_thread_id(fd, thread_id, is_current);
|
||||
|
||||
if frames.is_empty() {
|
||||
puts(fd, " <no Python frame>\n");
|
||||
} else {
|
||||
for fp in frames.iter().rev() {
|
||||
// SAFETY: caller holds the Mutex, so the owning thread can't pop.
|
||||
dump_frame_from_ref(fd, unsafe { fp.as_ref() });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Dump a thread's traceback by walking its published top frame down the
|
||||
/// `previous` chain (most recent first). Signal-safe: only atomic pointer
|
||||
/// loads, no locks. Callers guarantee frame liveness — under stop-the-world
|
||||
/// for `faulthandler.dump_traceback`, or best-effort for the watchdog (like
|
||||
/// `_Py_DumpTracebackThreads`, which walks lock-free while other threads
|
||||
/// may still run).
|
||||
#[cfg(all(unix, feature = "threading"))]
|
||||
fn dump_traceback_thread_chain(fd: i32, thread_id: u64, is_current: bool, top: *const Frame) {
|
||||
const MAX_FRAME_DEPTH: usize = 100;
|
||||
write_thread_id(fd, thread_id, is_current);
|
||||
|
||||
if top.is_null() {
|
||||
puts(fd, " <no Python frame>\n");
|
||||
return;
|
||||
}
|
||||
let mut frame_ptr = top;
|
||||
let mut depth = 0;
|
||||
while !frame_ptr.is_null() && depth < MAX_FRAME_DEPTH {
|
||||
// SAFETY: the frame is alive per the caller's liveness guarantee.
|
||||
let frame = unsafe { &*frame_ptr };
|
||||
dump_frame_from_raw(fd, frame);
|
||||
frame_ptr = frame.previous_frame();
|
||||
depth += 1;
|
||||
}
|
||||
if depth >= MAX_FRAME_DEPTH && !frame_ptr.is_null() {
|
||||
puts(fd, " ...\n");
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(FromArgs)]
|
||||
struct DumpTracebackArgs {
|
||||
#[pyarg(any, default)]
|
||||
@@ -307,9 +236,7 @@ mod decl {
|
||||
dump_all_threads(fd, vm);
|
||||
} else {
|
||||
puts(fd, "Stack (most recent call first):\n");
|
||||
crate::vm::frame::for_each_current_frame(|frame| {
|
||||
dump_frame_from_ref(fd, frame);
|
||||
});
|
||||
dump_live_frames(fd);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -344,21 +271,35 @@ mod decl {
|
||||
if tid == current_tid {
|
||||
continue;
|
||||
}
|
||||
let top = slot.top_frame.load(Ordering::Relaxed) as *const Frame;
|
||||
dump_traceback_thread_chain(fd, tid, false, top);
|
||||
// Under STW, all other threads are suspended so their
|
||||
// stack-allocated iframes are stable. Walk via top_iframe
|
||||
// which covers both FrameObject and stack-allocated paths.
|
||||
let iframe_ptr = slot.top_iframe.load(Ordering::Relaxed)
|
||||
as *const rustpython_vm::frame::InterpreterFrame;
|
||||
write_thread_id(fd, tid, false);
|
||||
if iframe_ptr.is_null() {
|
||||
puts(fd, " <no Python frame>\n");
|
||||
} else {
|
||||
let mut cur = iframe_ptr;
|
||||
let mut depth = 0;
|
||||
while !cur.is_null() && depth < 100 {
|
||||
let iframe = unsafe { &*cur };
|
||||
dump_iframe(fd, iframe);
|
||||
depth += 1;
|
||||
cur = iframe.previous();
|
||||
}
|
||||
if depth >= 100 && !cur.is_null() {
|
||||
puts(fd, " ...\n");
|
||||
}
|
||||
}
|
||||
puts(fd, "\n");
|
||||
}
|
||||
}
|
||||
|
||||
// Now dump current thread from its live frame chain.
|
||||
// Now dump current thread from its live frame chain
|
||||
// (includes stack-allocated iframes without FrameObject).
|
||||
write_thread_id(fd, current_tid, true);
|
||||
if crate::vm::vm::thread::get_current_frame().is_null() {
|
||||
puts(fd, " <no Python frame>\n");
|
||||
} else {
|
||||
crate::vm::frame::for_each_current_frame(|frame| {
|
||||
dump_frame_from_ref(fd, frame);
|
||||
});
|
||||
}
|
||||
dump_live_frames(fd);
|
||||
}
|
||||
|
||||
#[cfg(all(not(unix), feature = "threading"))]
|
||||
@@ -366,7 +307,8 @@ mod decl {
|
||||
let current_tid = rustpython_vm::stdlib::_thread::get_ident();
|
||||
let registry = vm.state.thread_frames.lock();
|
||||
|
||||
// First dump non-current threads, then current thread last
|
||||
// Dump non-current threads using top_iframe, which includes
|
||||
// both FrameObject and stack-allocated frames.
|
||||
#[expect(
|
||||
clippy::iter_over_hash_type,
|
||||
reason = "Iteration order doesn't matter here"
|
||||
@@ -376,29 +318,38 @@ mod decl {
|
||||
continue;
|
||||
}
|
||||
|
||||
let frames_guard = slot.frames.lock();
|
||||
dump_traceback_thread_frames(fd, tid, false, &frames_guard);
|
||||
let iframe_ptr = slot.top_iframe.load(core::sync::atomic::Ordering::Relaxed)
|
||||
as *const rustpython_vm::frame::InterpreterFrame;
|
||||
write_thread_id(fd, tid, false);
|
||||
if iframe_ptr.is_null() {
|
||||
puts(fd, " <no Python frame>\n");
|
||||
} else {
|
||||
let mut cur = iframe_ptr;
|
||||
let mut depth = 0;
|
||||
while !cur.is_null() && depth < 100 {
|
||||
let iframe = unsafe { &*cur };
|
||||
dump_iframe(fd, iframe);
|
||||
depth += 1;
|
||||
cur = iframe.previous();
|
||||
}
|
||||
if depth >= 100 && !cur.is_null() {
|
||||
puts(fd, " ...\n");
|
||||
}
|
||||
}
|
||||
puts(fd, "\n");
|
||||
}
|
||||
|
||||
// Now dump current thread from its live frame chain.
|
||||
// Now dump current thread from its live frame chain
|
||||
// (includes stack-allocated iframes without FrameObject).
|
||||
write_thread_id(fd, current_tid, true);
|
||||
if crate::vm::vm::thread::get_current_frame().is_null() {
|
||||
puts(fd, " <no Python frame>\n");
|
||||
} else {
|
||||
crate::vm::frame::for_each_current_frame(|frame| {
|
||||
dump_frame_from_ref(fd, frame);
|
||||
});
|
||||
}
|
||||
dump_live_frames(fd);
|
||||
}
|
||||
|
||||
#[cfg(not(feature = "threading"))]
|
||||
{
|
||||
let _ = vm;
|
||||
write_thread_id(fd, current_thread_id(), true);
|
||||
crate::vm::frame::for_each_current_frame(|frame| {
|
||||
dump_frame_from_ref(fd, frame);
|
||||
});
|
||||
dump_live_frames(fd);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -724,18 +675,56 @@ mod decl {
|
||||
// the VM, so it cannot stop-the-world. Walk each
|
||||
// published top frame lock-free and best-effort, like
|
||||
// the faulthandler watchdog thread.
|
||||
// Walk via top_iframe for all threads. The watchdog
|
||||
// cannot stop-the-world, so this is best-effort
|
||||
// (like CPython's _Py_DumpTracebackThreads). Stack
|
||||
// frames are still alive because the target thread
|
||||
// is executing (inside a blocking call).
|
||||
for (tid, slot) in &thread_frame_slots {
|
||||
let top = slot
|
||||
.top_frame
|
||||
let iframe_ptr = slot
|
||||
.top_iframe
|
||||
.load(core::sync::atomic::Ordering::Relaxed)
|
||||
as *const Frame;
|
||||
dump_traceback_thread_chain(fd, *tid, false, top);
|
||||
as *const rustpython_vm::frame::InterpreterFrame;
|
||||
write_thread_id(fd, *tid, false);
|
||||
if iframe_ptr.is_null() {
|
||||
puts(fd, " <no Python frame>\n");
|
||||
} else {
|
||||
let mut cur = iframe_ptr;
|
||||
let mut depth = 0;
|
||||
while !cur.is_null() && depth < 100 {
|
||||
let iframe = unsafe { &*cur };
|
||||
dump_iframe(fd, iframe);
|
||||
depth += 1;
|
||||
cur = iframe.previous();
|
||||
}
|
||||
if depth >= 100 && !cur.is_null() {
|
||||
puts(fd, " ...\n");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
all(not(unix), feature = "threading") => {
|
||||
for (tid, slot) in &thread_frame_slots {
|
||||
let frames = slot.frames.lock();
|
||||
dump_traceback_thread_frames(fd, *tid, false, &frames);
|
||||
let iframe_ptr = slot
|
||||
.top_iframe
|
||||
.load(core::sync::atomic::Ordering::Relaxed)
|
||||
as *const rustpython_vm::frame::InterpreterFrame;
|
||||
write_thread_id(fd, *tid, false);
|
||||
if iframe_ptr.is_null() {
|
||||
puts(fd, " <no Python frame>\n");
|
||||
} else {
|
||||
let mut cur = iframe_ptr;
|
||||
let mut depth = 0;
|
||||
while !cur.is_null() && depth < 100 {
|
||||
let iframe = unsafe { &*cur };
|
||||
dump_iframe(fd, iframe);
|
||||
depth += 1;
|
||||
cur = iframe.previous();
|
||||
}
|
||||
if depth >= 100 && !cur.is_null() {
|
||||
puts(fd, " ...\n");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
_ => {
|
||||
|
||||
@@ -5,7 +5,7 @@ use crate::{
|
||||
class::PyClassImpl,
|
||||
common::lock::PyMutex,
|
||||
coroutine::{Coro, warn_deprecated_throw_signature},
|
||||
frame::FrameRef,
|
||||
frame::FrameObjectRef,
|
||||
function::OptionalArg,
|
||||
object::{Traverse, TraverseFn},
|
||||
protocol::PyIterReturn,
|
||||
@@ -50,7 +50,7 @@ impl PyAsyncGen {
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn new(frame: FrameRef, name: PyStrRef, qualname: PyStrRef) -> Self {
|
||||
pub fn new(frame: FrameObjectRef, name: PyStrRef, qualname: PyStrRef) -> Self {
|
||||
Self {
|
||||
inner: Coro::new(frame, name, qualname),
|
||||
running_async: AtomicCell::new(false),
|
||||
@@ -127,7 +127,7 @@ impl PyAsyncGen {
|
||||
self.inner.frame().yield_from_target()
|
||||
}
|
||||
#[pygetset]
|
||||
fn ag_frame(&self, _vm: &VirtualMachine) -> Option<FrameRef> {
|
||||
fn ag_frame(&self, _vm: &VirtualMachine) -> Option<FrameObjectRef> {
|
||||
if self.inner.closed() {
|
||||
None
|
||||
} else {
|
||||
@@ -140,7 +140,7 @@ impl PyAsyncGen {
|
||||
}
|
||||
#[pygetset]
|
||||
fn ag_code(&self, _vm: &VirtualMachine) -> PyRef<PyCode> {
|
||||
self.inner.frame().code.clone()
|
||||
self.inner.frame().iframe().code().to_owned()
|
||||
}
|
||||
|
||||
#[pyclassmethod]
|
||||
@@ -688,7 +688,8 @@ impl PyAnextAwaitable {
|
||||
&& generator
|
||||
.as_coro()
|
||||
.frame()
|
||||
.code
|
||||
.iframe()
|
||||
.code()
|
||||
.flags
|
||||
.contains(crate::bytecode::CodeFlags::ITERABLE_COROUTINE)
|
||||
{
|
||||
|
||||
@@ -3,7 +3,7 @@ use crate::{
|
||||
AsObject, Context, Py, PyObjectRef, PyPayload, PyRef, PyResult, VirtualMachine,
|
||||
class::PyClassImpl,
|
||||
coroutine::{Coro, warn_deprecated_throw_signature},
|
||||
frame::FrameRef,
|
||||
frame::FrameObjectRef,
|
||||
function::OptionalArg,
|
||||
object::{Traverse, TraverseFn},
|
||||
protocol::PyIterReturn,
|
||||
@@ -41,7 +41,7 @@ impl PyCoroutine {
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn new(frame: FrameRef, name: PyStrRef, qualname: PyStrRef) -> Self {
|
||||
pub fn new(frame: FrameObjectRef, name: PyStrRef, qualname: PyStrRef) -> Self {
|
||||
Self {
|
||||
inner: Coro::new(frame, name, qualname),
|
||||
}
|
||||
@@ -80,7 +80,7 @@ impl PyCoroutine {
|
||||
self.inner.frame().yield_from_target()
|
||||
}
|
||||
#[pygetset]
|
||||
fn cr_frame(&self, _vm: &VirtualMachine) -> Option<FrameRef> {
|
||||
fn cr_frame(&self, _vm: &VirtualMachine) -> Option<FrameObjectRef> {
|
||||
if self.inner.closed() {
|
||||
None
|
||||
} else {
|
||||
@@ -93,7 +93,7 @@ impl PyCoroutine {
|
||||
}
|
||||
#[pygetset]
|
||||
fn cr_code(&self, _vm: &VirtualMachine) -> PyRef<PyCode> {
|
||||
self.inner.frame().code.clone()
|
||||
self.inner.frame().iframe().code().to_owned()
|
||||
}
|
||||
// TODO: coroutine origin tracking:
|
||||
// https://docs.python.org/3/library/sys.html#sys.set_coroutine_origin_tracking_depth
|
||||
|
||||
+232
-110
@@ -6,11 +6,14 @@ use super::{PyCode, PyDictRef, PyIntRef, PyStrRef};
|
||||
use crate::{
|
||||
Context, Py, PyObjectRef, PyPayload, PyRef, PyResult, VirtualMachine,
|
||||
class::PyClassImpl,
|
||||
frame::{Frame, FrameOwner, FrameRef},
|
||||
frame::{FrameObject, FrameObjectRef, FrameOwner},
|
||||
function::PySetterValue,
|
||||
types::Representable,
|
||||
};
|
||||
use core::sync::atomic::Ordering::Relaxed;
|
||||
use num_traits::Zero;
|
||||
#[allow(unused_imports)]
|
||||
use rustpython_common::atomic::Radium;
|
||||
use rustpython_compiler_core::bytecode::{self, Constant, Instruction, StackEffect};
|
||||
use stack_analysis::*;
|
||||
|
||||
@@ -426,10 +429,10 @@ pub(crate) mod stack_analysis {
|
||||
}
|
||||
|
||||
pub(crate) fn init(context: &'static Context) {
|
||||
Frame::extend_class(context, context.types.frame_type);
|
||||
FrameObject::extend_class(context, context.types.frame_type);
|
||||
}
|
||||
|
||||
impl Representable for Frame {
|
||||
impl Representable for FrameObject {
|
||||
#[inline]
|
||||
fn repr(_zelf: &Py<Self>, vm: &VirtualMachine) -> PyResult<PyStrRef> {
|
||||
const REPR: &str = "<frame object at .. >";
|
||||
@@ -442,38 +445,86 @@ impl Representable for Frame {
|
||||
}
|
||||
}
|
||||
|
||||
impl FrameObject {
|
||||
/// Find the live source InterpreterFrame on the TLS chain for a
|
||||
/// materialized FrameObject. Returns the raw pointer if found, or null
|
||||
/// if this FrameObject has no live source (already returned or not
|
||||
/// currently executing on this thread).
|
||||
pub(crate) fn find_live_source_iframe(&self) -> *const crate::frame::InterpreterFrame {
|
||||
let self_py_ptr = unsafe { Py::<Self>::from_payload_ptr(self) } as usize;
|
||||
let mut cur = crate::vm::thread::get_current_frame();
|
||||
while !cur.is_null() {
|
||||
let materialized = unsafe { (*cur).materialized.load(Relaxed) };
|
||||
if materialized == self_py_ptr {
|
||||
return cur;
|
||||
}
|
||||
cur = unsafe { &*cur }.previous();
|
||||
}
|
||||
core::ptr::null()
|
||||
}
|
||||
}
|
||||
|
||||
#[pyclass(flags(DISALLOW_INSTANTIATION), with(Py))]
|
||||
impl Frame {
|
||||
impl FrameObject {
|
||||
#[pygetset]
|
||||
fn f_globals(&self) -> PyDictRef {
|
||||
self.globals.clone()
|
||||
self.iframe().globals().to_owned()
|
||||
}
|
||||
|
||||
#[pygetset]
|
||||
fn f_builtins(&self) -> PyObjectRef {
|
||||
self.builtins.clone()
|
||||
self.iframe().builtins().to_owned()
|
||||
}
|
||||
|
||||
#[pygetset]
|
||||
pub fn f_code(&self) -> PyRef<PyCode> {
|
||||
self.code.clone()
|
||||
self.iframe().code().to_owned()
|
||||
}
|
||||
|
||||
#[pygetset]
|
||||
fn f_lasti(&self) -> u32 {
|
||||
// Return byte offset (each instruction is 2 bytes) for compatibility
|
||||
self.lasti() * 2
|
||||
// Return byte offset (each instruction is 2 bytes) for compatibility.
|
||||
// For materialized frames, read live lasti from the source iframe on
|
||||
// the TLS chain so f_lasti reflects the current execution position.
|
||||
let live = self.find_live_source_iframe();
|
||||
let val = if !live.is_null() {
|
||||
unsafe { (*live).lasti.load(Relaxed) }
|
||||
} else {
|
||||
self.lasti()
|
||||
};
|
||||
val * 2
|
||||
}
|
||||
|
||||
#[pygetset]
|
||||
pub fn f_lineno(&self) -> usize {
|
||||
// If lasti is 0, execution hasn't started yet - use first line number
|
||||
// Similar to PyCode_Addr2Line which returns co_firstlineno for addr_q < 0
|
||||
if self.lasti() == 0 {
|
||||
self.code.first_line_number.map_or(1, |n| n.get())
|
||||
} else {
|
||||
self.current_location().line.get()
|
||||
return self
|
||||
.iframe()
|
||||
.code()
|
||||
.first_line_number
|
||||
.map_or(1, |n| n.get());
|
||||
}
|
||||
// For executing frames (on the TLS chain), use prev_line which is
|
||||
// updated at each bytecode instruction *before* the instruction
|
||||
// runs. This gives the correct line even when observed mid-CALL
|
||||
// (where lasti has already advanced past the CALL instruction).
|
||||
let live = self.find_live_source_iframe();
|
||||
if !live.is_null() {
|
||||
// Read live prev_line. Use read_volatile to bypass LLVM noalias
|
||||
// on the &mut InterpreterFrame borrow in with_iframe.
|
||||
let prev = unsafe {
|
||||
let field_ptr = core::ptr::addr_of!((*live).prev_line);
|
||||
core::ptr::read_volatile(field_ptr as *const u32)
|
||||
};
|
||||
if prev > 0 {
|
||||
return prev as usize;
|
||||
}
|
||||
}
|
||||
// For returned frames, use lasti-based location lookup. This is
|
||||
// correct for exception tracebacks where prev_line may have been
|
||||
// updated by cleanup instructions after the exception.
|
||||
self.current_location().line.get()
|
||||
}
|
||||
|
||||
#[pygetset(setter)]
|
||||
@@ -492,7 +543,11 @@ impl Frame {
|
||||
}
|
||||
};
|
||||
|
||||
let first_line = self.code.first_line_number.map_or(1, |n| n.get() as i32);
|
||||
let first_line = self
|
||||
.iframe()
|
||||
.code()
|
||||
.first_line_number
|
||||
.map_or(1, |n| n.get() as i32);
|
||||
|
||||
if l_new_lineno < first_line {
|
||||
return Err(vm.new_value_error(format!(
|
||||
@@ -500,7 +555,7 @@ impl Frame {
|
||||
)));
|
||||
}
|
||||
|
||||
let py_code: &PyCode = &self.code;
|
||||
let py_code: &PyCode = self.iframe().code();
|
||||
let code = &py_code.code;
|
||||
let lines = mark_lines(code);
|
||||
|
||||
@@ -513,13 +568,19 @@ impl Frame {
|
||||
}
|
||||
|
||||
let stacks = mark_stacks(code);
|
||||
let len = self.code.instructions.len();
|
||||
let len = self.iframe().code().instructions.len();
|
||||
|
||||
// lasti points past the current instruction (already incremented).
|
||||
// stacks[lasti - 1] gives the stack state before executing the
|
||||
// instruction that triggered this trace event, which is the current
|
||||
// evaluation stack.
|
||||
let current_lasti = self.lasti() as usize;
|
||||
// evaluation stack. Read from the live iframe when available so the
|
||||
// value reflects the actual execution position.
|
||||
let live = self.find_live_source_iframe();
|
||||
let current_lasti = if !live.is_null() {
|
||||
(unsafe { (*live).lasti.load(Relaxed) }) as usize
|
||||
} else {
|
||||
self.lasti() as usize
|
||||
};
|
||||
let start_idx = current_lasti.saturating_sub(1);
|
||||
let start_stack = if start_idx < stacks.len() {
|
||||
stacks[start_idx]
|
||||
@@ -567,36 +628,61 @@ impl Frame {
|
||||
}
|
||||
}
|
||||
|
||||
// Store the pending unwind for the execution loop to perform.
|
||||
// We cannot pop stack entries here because the execution loop
|
||||
// holds the state mutex, and trying to lock it again would deadlock.
|
||||
self.set_pending_stack_pops(pop_count as u32);
|
||||
self.set_pending_unwind_from_stack(start_stack);
|
||||
|
||||
// Set lasti to best_addr. The executor will read lasti and execute
|
||||
// the instruction at that index next.
|
||||
self.set_lasti(best_addr as u32);
|
||||
// Store the pending unwind and new lasti. When this frame is backed
|
||||
// by a live stack-allocated iframe, write to the live iframe so the
|
||||
// execution loop picks up the jump target. Reuse `live` from above.
|
||||
let target = if !live.is_null() {
|
||||
unsafe { &*live }
|
||||
} else {
|
||||
self.iframe()
|
||||
};
|
||||
target.pending_stack_pops.store(pop_count as u32, Relaxed);
|
||||
target.pending_unwind_from_stack.store(start_stack, Relaxed);
|
||||
target.lasti.store(best_addr as u32, Relaxed);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[pygetset]
|
||||
fn f_trace(&self) -> PyObjectRef {
|
||||
let boxed = self.trace.lock();
|
||||
boxed.clone()
|
||||
fn f_trace(&self, vm: &VirtualMachine) -> PyObjectRef {
|
||||
// Read from live source iframe if available.
|
||||
let live = self.find_live_source_iframe();
|
||||
let trace = if !live.is_null() {
|
||||
unsafe { &*live }.trace.lock().clone()
|
||||
} else {
|
||||
self.iframe().trace.lock().clone()
|
||||
};
|
||||
trace.unwrap_or_else(|| vm.ctx.none())
|
||||
}
|
||||
|
||||
#[pygetset(setter)]
|
||||
fn set_f_trace(&self, value: PySetterValue, vm: &VirtualMachine) {
|
||||
let mut storage = self.trace.lock();
|
||||
*storage = value.unwrap_or_none(vm);
|
||||
let trace = match value {
|
||||
PySetterValue::Assign(v) => {
|
||||
if vm.is_none(&v) {
|
||||
None
|
||||
} else {
|
||||
Some(v)
|
||||
}
|
||||
}
|
||||
PySetterValue::Delete => None,
|
||||
};
|
||||
// Set on the materialized FrameObject.
|
||||
(*self.iframe().trace.lock()).clone_from(&trace);
|
||||
// Also propagate to the live source iframe if this is a
|
||||
// materialized copy of a stack-allocated frame, so pdb's
|
||||
// f_trace assignment takes effect on the executing frame.
|
||||
let live = self.find_live_source_iframe();
|
||||
if !live.is_null() {
|
||||
*unsafe { &*live }.trace.lock() = trace;
|
||||
}
|
||||
}
|
||||
|
||||
#[expect(clippy::unnecessary_wraps, reason = "Needs to comply with a signature")]
|
||||
#[pymember(type = "bool")]
|
||||
fn f_trace_lines(vm: &VirtualMachine, zelf: PyObjectRef) -> PyResult {
|
||||
let zelf: FrameRef = zelf.downcast().unwrap_or_else(|_| unreachable!());
|
||||
let zelf: FrameObjectRef = zelf.downcast().unwrap_or_else(|_| unreachable!());
|
||||
|
||||
let boxed = zelf.trace_lines.lock();
|
||||
let boxed = zelf.iframe().trace_lines.lock();
|
||||
Ok(vm.ctx.new_bool(*boxed).into())
|
||||
}
|
||||
|
||||
@@ -608,14 +694,19 @@ impl Frame {
|
||||
) -> PyResult<()> {
|
||||
match value {
|
||||
PySetterValue::Assign(value) => {
|
||||
let zelf: FrameRef = zelf.downcast().unwrap_or_else(|_| unreachable!());
|
||||
let zelf: FrameObjectRef = zelf.downcast().unwrap_or_else(|_| unreachable!());
|
||||
|
||||
let value: PyIntRef = value
|
||||
.downcast()
|
||||
.map_err(|_| vm.new_type_error("attribute value type must be bool"))?;
|
||||
|
||||
let mut trace_lines = zelf.trace_lines.lock();
|
||||
*trace_lines = !value.as_bigint().is_zero();
|
||||
let val = !value.as_bigint().is_zero();
|
||||
*zelf.iframe().trace_lines.lock() = val;
|
||||
// Propagate to live source iframe.
|
||||
let live = zelf.find_live_source_iframe();
|
||||
if !live.is_null() {
|
||||
*unsafe { &*live }.trace_lines.lock() = val;
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
@@ -626,8 +717,8 @@ impl Frame {
|
||||
#[expect(clippy::unnecessary_wraps, reason = "Needs to comply with a signature")]
|
||||
#[pymember(type = "bool")]
|
||||
fn f_trace_opcodes(vm: &VirtualMachine, zelf: PyObjectRef) -> PyResult {
|
||||
let zelf: FrameRef = zelf.downcast().unwrap_or_else(|_| unreachable!());
|
||||
let trace_opcodes = zelf.trace_opcodes.lock();
|
||||
let zelf: FrameObjectRef = zelf.downcast().unwrap_or_else(|_| unreachable!());
|
||||
let trace_opcodes = zelf.iframe().trace_opcodes.lock();
|
||||
Ok(vm.ctx.new_bool(*trace_opcodes).into())
|
||||
}
|
||||
|
||||
@@ -639,14 +730,19 @@ impl Frame {
|
||||
) -> PyResult<()> {
|
||||
match value {
|
||||
PySetterValue::Assign(value) => {
|
||||
let zelf: FrameRef = zelf.downcast().unwrap_or_else(|_| unreachable!());
|
||||
let zelf: FrameObjectRef = zelf.downcast().unwrap_or_else(|_| unreachable!());
|
||||
|
||||
let value: PyIntRef = value
|
||||
.downcast()
|
||||
.map_err(|_| vm.new_type_error("attribute value type must be bool"))?;
|
||||
|
||||
let mut trace_opcodes = zelf.trace_opcodes.lock();
|
||||
*trace_opcodes = !value.as_bigint().is_zero();
|
||||
let val = !value.as_bigint().is_zero();
|
||||
*zelf.iframe().trace_opcodes.lock() = val;
|
||||
// Propagate to live source iframe.
|
||||
let live = zelf.find_live_source_iframe();
|
||||
if !live.is_null() {
|
||||
*unsafe { &*live }.trace_opcodes.lock() = val;
|
||||
}
|
||||
|
||||
// TODO: Implement the equivalent of _PyEval_SetOpcodeTrace()
|
||||
|
||||
@@ -658,11 +754,15 @@ impl Frame {
|
||||
}
|
||||
|
||||
#[pyclass]
|
||||
impl Py<Frame> {
|
||||
impl Py<FrameObject> {
|
||||
#[pymethod]
|
||||
// = frame_clear_impl
|
||||
fn clear(&self, vm: &VirtualMachine) -> PyResult<()> {
|
||||
let owner = FrameOwner::from_i8(self.owner.load(core::sync::atomic::Ordering::Acquire));
|
||||
let owner = FrameOwner::from_i8(
|
||||
self.iframe()
|
||||
.owner
|
||||
.load(core::sync::atomic::Ordering::Acquire),
|
||||
);
|
||||
match owner {
|
||||
FrameOwner::Generator => {
|
||||
// Generator frame: check if suspended (lasti > 0 means
|
||||
@@ -677,12 +777,16 @@ impl Py<Frame> {
|
||||
return Err(vm.new_runtime_error("cannot clear an executing frame"));
|
||||
}
|
||||
FrameOwner::FrameObject => {
|
||||
// Detached frame: safe to clear.
|
||||
// Check if this materialized frame is backed by a live
|
||||
// stack-allocated iframe — if so, the frame is executing.
|
||||
if !self.find_live_source_iframe().is_null() {
|
||||
return Err(vm.new_runtime_error("cannot clear an executing frame"));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Clear fastlocals
|
||||
// SAFETY: Frame is not executing (detached or stopped).
|
||||
// SAFETY: FrameObject is not executing (detached or stopped).
|
||||
{
|
||||
let fastlocals = unsafe { self.fastlocals_mut() };
|
||||
for slot in fastlocals.iter_mut() {
|
||||
@@ -694,10 +798,10 @@ impl Py<Frame> {
|
||||
self.clear_stack_and_cells();
|
||||
|
||||
// Clear temporary refs
|
||||
self.temporary_refs.lock().clear();
|
||||
self.f_locals_hidden_overlay.lock().take();
|
||||
self.f_extra_locals.lock().take();
|
||||
self.retained_back.lock().take();
|
||||
self.iframe().temporary_refs.lock().clear();
|
||||
self.iframe().f_locals_hidden_overlay.lock().take();
|
||||
self.iframe().f_extra_locals.lock().take();
|
||||
self.iframe().retained_back.lock().take();
|
||||
|
||||
Ok(())
|
||||
}
|
||||
@@ -707,7 +811,12 @@ impl Py<Frame> {
|
||||
// Optimized (function) frames expose a live write-through
|
||||
// FrameLocalsProxy; class/module/exec frames expose their namespace
|
||||
// mapping directly.
|
||||
if self.code.flags.contains(bytecode::CodeFlags::OPTIMIZED) {
|
||||
if self
|
||||
.iframe()
|
||||
.code()
|
||||
.flags
|
||||
.contains(bytecode::CodeFlags::OPTIMIZED)
|
||||
{
|
||||
self.check_locals_access(vm)?;
|
||||
self.mark_escaped();
|
||||
let proxy = crate::builtins::FrameLocalsProxy::new(self.to_owned());
|
||||
@@ -719,85 +828,98 @@ impl Py<Frame> {
|
||||
|
||||
#[pygetset]
|
||||
fn f_generator(&self) -> Option<PyObjectRef> {
|
||||
self.generator.to_owned()
|
||||
self.iframe().generator.to_owned()
|
||||
}
|
||||
|
||||
#[pygetset]
|
||||
pub fn f_back(&self, vm: &VirtualMachine) -> Option<PyRef<Frame>> {
|
||||
#[cfg(not(feature = "threading"))]
|
||||
let _ = vm;
|
||||
let previous = self.previous_frame();
|
||||
if previous.is_null() {
|
||||
return None;
|
||||
pub fn f_back(&self, #[allow(unused)] vm: &VirtualMachine) -> Option<PyRef<FrameObject>> {
|
||||
let mut prev = self.previous_iframe();
|
||||
|
||||
// For materialized frames (previous == 0), find the source iframe on
|
||||
// the TLS chain and use its `previous` instead.
|
||||
if prev.is_null() {
|
||||
// materialized stores `*const Py<FrameObject>` as usize.
|
||||
// `self` is `&Py<FrameObject>` — compare addresses directly.
|
||||
let self_py_ptr = self as *const Self as usize;
|
||||
let mut cur = crate::vm::thread::get_current_frame();
|
||||
while !cur.is_null() {
|
||||
let materialized = unsafe { (*cur).materialized.load(Relaxed) };
|
||||
if materialized == self_py_ptr {
|
||||
// Found the source iframe — use its previous
|
||||
prev = unsafe { (*cur).previous() };
|
||||
break;
|
||||
}
|
||||
cur = unsafe { (*cur).previous() };
|
||||
}
|
||||
if prev.is_null() {
|
||||
// Check retained_back for frames whose callers have returned
|
||||
let retained = self.iframe().retained_back.lock().clone();
|
||||
if let Some(frame) = retained {
|
||||
frame.mark_escaped();
|
||||
return Some(frame);
|
||||
}
|
||||
return None;
|
||||
}
|
||||
}
|
||||
|
||||
// Look for the caller on the current thread's signal-safe frame chain.
|
||||
// Finding it there proves it is still live on this thread.
|
||||
if let Some(frame) = crate::frame::find_owned_chain_frame(previous) {
|
||||
frame.mark_escaped();
|
||||
return Some(frame);
|
||||
// Walk the TLS chain to find the prev iframe and materialize it.
|
||||
// This handles both heap-allocated FrameObjects and stack-allocated
|
||||
// iframes that haven't been observed yet.
|
||||
{
|
||||
let mut cur = crate::vm::thread::get_current_frame();
|
||||
while !cur.is_null() {
|
||||
if core::ptr::eq(cur, prev) {
|
||||
let iframe_ref = unsafe { &*cur };
|
||||
let fo = iframe_ref.materialize(vm);
|
||||
fo.mark_escaped();
|
||||
return Some(fo.to_owned());
|
||||
}
|
||||
cur = unsafe { (*cur).previous() };
|
||||
}
|
||||
}
|
||||
|
||||
// The caller already returned and left the live chain, but this frame
|
||||
// escaped and retained a strong reference to it at release time.
|
||||
let retained = self.retained_back.lock().clone();
|
||||
// The caller already returned — check retained_back
|
||||
let retained = self.iframe().retained_back.lock().clone();
|
||||
if let Some(frame) = retained {
|
||||
frame.mark_escaped();
|
||||
return Some(frame);
|
||||
}
|
||||
|
||||
// The caller lives on another thread. unix: park every thread under
|
||||
// stop-the-world so their frame chains are quiescent and alive, then
|
||||
// walk each published top frame down its `previous` chain looking for
|
||||
// the caller. Request stop-the-world before the registry lock.
|
||||
#[cfg(all(unix, feature = "threading"))]
|
||||
// The caller lives on another thread. Use stop-the-world to
|
||||
// safely materialize the cross-thread frame chain.
|
||||
#[cfg(feature = "threading")]
|
||||
{
|
||||
use core::sync::atomic::Ordering;
|
||||
// Enter STW before dereferencing `prev` — the owning thread may
|
||||
// return and free the stack-allocated iframe at any time.
|
||||
vm.state.stop_the_world.stop_the_world(vm);
|
||||
scopeguard::defer! { vm.state.stop_the_world.start_the_world(vm); }
|
||||
let registry = vm.state.thread_frames.lock();
|
||||
#[expect(
|
||||
clippy::iter_over_hash_type,
|
||||
reason = "Iteration order doesn't matter here"
|
||||
)]
|
||||
for slot in registry.values() {
|
||||
let mut cur = slot.top_frame.load(Ordering::Relaxed) as *const Frame;
|
||||
while !cur.is_null() {
|
||||
if core::ptr::eq(cur, previous) {
|
||||
// SAFETY: world stopped -> this frame is alive on its
|
||||
// owning thread's parked call stack.
|
||||
let f = unsafe { &*Self::from_payload_ptr(cur) };
|
||||
f.mark_escaped();
|
||||
return Some(f.to_owned());
|
||||
let prev_ref = unsafe { &*prev };
|
||||
// Fast path: already materialized.
|
||||
if let Some(fo) = prev_ref.frame_obj() {
|
||||
fo.mark_escaped();
|
||||
return Some(fo.to_owned());
|
||||
}
|
||||
// Slow path: materialize the entire chain and link retained_back.
|
||||
let mut cur = prev;
|
||||
let mut child_fo: Option<crate::PyRef<crate::frame::FrameObject>> = None;
|
||||
while !cur.is_null() {
|
||||
let iframe = unsafe { &*cur };
|
||||
let fo = iframe.materialize(vm).to_owned();
|
||||
if let Some(child) = child_fo.take() {
|
||||
let mut guard = child.iframe().retained_back.lock();
|
||||
if guard.is_none() {
|
||||
*guard = Some(fo.clone());
|
||||
}
|
||||
// SAFETY: chain frames on a parked thread are alive.
|
||||
cur = unsafe { (*cur).previous_frame() };
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(all(not(unix), feature = "threading"))]
|
||||
{
|
||||
let registry = vm.state.thread_frames.lock();
|
||||
#[expect(
|
||||
clippy::iter_over_hash_type,
|
||||
reason = "Iteration order doesn't matter here"
|
||||
)]
|
||||
for slot in registry.values() {
|
||||
let frames = slot.frames.lock();
|
||||
// SAFETY: the owning thread can't pop while we hold the Mutex,
|
||||
// so FramePtr is valid for the duration of the lock.
|
||||
if let Some(frame) = frames.iter().find_map(|fp| {
|
||||
let f = unsafe { fp.as_ref() };
|
||||
let ptr: *const Frame = &**f;
|
||||
core::ptr::eq(ptr, previous).then(|| f.to_owned())
|
||||
}) {
|
||||
frame.mark_escaped();
|
||||
return Some(frame);
|
||||
}
|
||||
child_fo = Some(fo);
|
||||
cur = iframe.previous();
|
||||
}
|
||||
let fo = prev_ref.materialize(vm);
|
||||
fo.mark_escaped();
|
||||
return Some(fo.to_owned());
|
||||
}
|
||||
|
||||
#[allow(unreachable_code)]
|
||||
None
|
||||
}
|
||||
}
|
||||
@@ -7,7 +7,7 @@ use crate::{
|
||||
AsObject, Context, Py, PyObject, PyObjectRef, PyPayload, PyRef, PyResult, VirtualMachine,
|
||||
atomic_func,
|
||||
class::PyClassImpl,
|
||||
frame::FrameRef,
|
||||
frame::FrameObjectRef,
|
||||
function::{FuncArgs, OptionalArg, PyArithmeticValue, PyComparisonValue},
|
||||
object::{Traverse, TraverseFn},
|
||||
protocol::{PyMappingMethods, PyNumberMethods, PySequenceMethods},
|
||||
@@ -23,7 +23,7 @@ use rustpython_common::wtf8::Wtf8Buf;
|
||||
#[pyclass(module = false, name = "FrameLocalsProxy", traverse = "manual")]
|
||||
#[derive(Debug)]
|
||||
pub struct FrameLocalsProxy {
|
||||
frame: FrameRef,
|
||||
frame: FrameObjectRef,
|
||||
}
|
||||
|
||||
unsafe impl Traverse for FrameLocalsProxy {
|
||||
@@ -40,7 +40,7 @@ impl PyPayload for FrameLocalsProxy {
|
||||
}
|
||||
|
||||
impl FrameLocalsProxy {
|
||||
pub(crate) fn new(frame: FrameRef) -> Self {
|
||||
pub(crate) fn new(frame: FrameObjectRef) -> Self {
|
||||
Self { frame }
|
||||
}
|
||||
|
||||
@@ -67,7 +67,7 @@ impl Constructor for FrameLocalsProxy {
|
||||
args.len()
|
||||
)));
|
||||
}
|
||||
let frame: FrameRef = args
|
||||
let frame: FrameObjectRef = args
|
||||
.pop()
|
||||
.unwrap()
|
||||
.downcast()
|
||||
|
||||
@@ -14,7 +14,7 @@ use crate::{
|
||||
bytecode,
|
||||
class::PyClassImpl,
|
||||
common::wtf8::{Wtf8Buf, wtf8_concat},
|
||||
frame::{Frame, FrameRef},
|
||||
frame::{FrameObject, FrameObjectRef},
|
||||
function::{FuncArgs, OptionalArg, PyComparisonValue, PySetterValue},
|
||||
scope::Scope,
|
||||
types::{
|
||||
@@ -182,11 +182,7 @@ impl PyFunction {
|
||||
let module = vm.unwrap_or_none(globals.get_item_opt(identifier!(vm, __name__), vm)?);
|
||||
let builtins = globals.get_item("__builtins__", vm).unwrap_or_else(|_| {
|
||||
// If not in globals, inherit from current execution context
|
||||
if let Some(frame) = vm.current_frame() {
|
||||
frame.builtins.clone()
|
||||
} else {
|
||||
vm.builtins.dict().into()
|
||||
}
|
||||
crate::frame::current_builtins().unwrap_or_else(|| vm.builtins.dict().into())
|
||||
});
|
||||
// If builtins is a module, use its __dict__ instead
|
||||
let builtins = if let Some(module) = builtins.downcast_ref::<PyModule>() {
|
||||
@@ -228,7 +224,28 @@ impl PyFunction {
|
||||
|
||||
fn fill_locals_from_args(
|
||||
&self,
|
||||
frame: &Frame,
|
||||
frame: &FrameObject,
|
||||
func_args: FuncArgs,
|
||||
vm: &VirtualMachine,
|
||||
) -> PyResult<()> {
|
||||
// SAFETY: FrameObject was just created and not yet executing.
|
||||
let fastlocals = unsafe { frame.fastlocals_mut() };
|
||||
self.fill_locals_from_args_inner(fastlocals, func_args, vm)
|
||||
}
|
||||
|
||||
fn fill_locals_from_args_iframe(
|
||||
&self,
|
||||
iframe: &mut crate::frame::InterpreterFrame,
|
||||
func_args: FuncArgs,
|
||||
vm: &VirtualMachine,
|
||||
) -> PyResult<()> {
|
||||
let fastlocals = iframe.localsplus.fastlocals_mut();
|
||||
self.fill_locals_from_args_inner(fastlocals, func_args, vm)
|
||||
}
|
||||
|
||||
fn fill_locals_from_args_inner(
|
||||
&self,
|
||||
fastlocals: &mut [Option<PyObjectRef>],
|
||||
func_args: FuncArgs,
|
||||
vm: &VirtualMachine,
|
||||
) -> PyResult<()> {
|
||||
@@ -236,16 +253,6 @@ impl PyFunction {
|
||||
let nargs = func_args.args.len();
|
||||
let n_expected_args = code.arg_count as usize;
|
||||
let total_args = code.arg_count as usize + code.kwonlyarg_count as usize;
|
||||
// let arg_names = self.code.arg_names();
|
||||
|
||||
// This parses the arguments from args and kwargs into
|
||||
// the proper variables keeping into account default values
|
||||
// and star-args and kwargs.
|
||||
// See also: PyEval_EvalCodeWithName in cpython:
|
||||
// https://github.com/python/cpython/blob/main/Python/ceval.c#L3681
|
||||
|
||||
// SAFETY: Frame was just created and not yet executing.
|
||||
let fastlocals = unsafe { frame.fastlocals_mut() };
|
||||
|
||||
let mut args_iter = func_args.args.into_iter();
|
||||
|
||||
@@ -564,69 +571,114 @@ impl Py<PyFunction> {
|
||||
}
|
||||
}
|
||||
|
||||
let code: PyRef<PyCode> = (*self.code).to_owned();
|
||||
|
||||
let locals = if code.flags.contains(bytecode::CodeFlags::NEWLOCALS) {
|
||||
None
|
||||
} else if let Some(locals) = locals {
|
||||
Some(locals)
|
||||
} else {
|
||||
Some(ArgMapping::from_dict_exact(self.globals.clone()))
|
||||
};
|
||||
let code = &*self.code;
|
||||
|
||||
let is_gen = code.flags.contains(bytecode::CodeFlags::GENERATOR);
|
||||
let is_coro = code.flags.contains(bytecode::CodeFlags::COROUTINE);
|
||||
let is_async_gen = code.flags.contains(bytecode::CodeFlags::ASYNC_GENERATOR);
|
||||
let use_datastack = !(is_gen || is_coro || is_async_gen);
|
||||
|
||||
// Construct frame:
|
||||
let frame = Frame::new(
|
||||
code,
|
||||
Scope::new(locals, self.globals.clone()),
|
||||
self.builtins.clone(),
|
||||
self.closure.as_ref().map_or(&[], |c| c.as_slice()),
|
||||
Some(self.to_owned().into()),
|
||||
use_datastack,
|
||||
vm,
|
||||
)
|
||||
.into_ref(&vm.ctx);
|
||||
let needs_heap_frame = is_gen || is_coro || is_async_gen || vm.use_tracing.get();
|
||||
|
||||
self.fill_locals_from_args(&frame, func_args, vm)?;
|
||||
if use_datastack {
|
||||
let result = vm.run_frame(frame.clone());
|
||||
// Release data stack memory after frame execution completes.
|
||||
crate::frame::release_datastack_frame(&frame, vm);
|
||||
result
|
||||
} else {
|
||||
let obj = if is_async_gen {
|
||||
PyAsyncGen::new(frame.clone(), self.__name__(), self.__qualname__())
|
||||
.into_pyobject(vm)
|
||||
} else if is_gen {
|
||||
PyGenerator::new(frame.clone(), self.__name__(), self.__qualname__())
|
||||
.into_pyobject(vm)
|
||||
if needs_heap_frame {
|
||||
// Heap-allocate FrameObject for generators/coroutines (lifetime
|
||||
// exceeds call stack) or when tracing is active (trace callbacks
|
||||
// need a FrameObject).
|
||||
let code_owned: PyRef<PyCode> = code.to_owned();
|
||||
let locals = if code.flags.contains(bytecode::CodeFlags::NEWLOCALS) {
|
||||
None
|
||||
} else if let Some(locals) = locals {
|
||||
Some(locals)
|
||||
} else {
|
||||
PyCoroutine::new(frame.clone(), self.__name__(), self.__qualname__())
|
||||
.into_pyobject(vm)
|
||||
Some(ArgMapping::from_dict_exact(self.globals.clone()))
|
||||
};
|
||||
// Generator/coroutine frames outlive this call and can join a
|
||||
// reference cycle through their owning generator, so they must
|
||||
// participate in the GC. They were created untracked
|
||||
// (NEW_REF_UNTRACKED); track them now, before the back-reference
|
||||
// is installed. Their localsplus is heap-backed by construction
|
||||
// (use_datastack == false), so a collector never reads data-stack
|
||||
// storage when it traverses them.
|
||||
debug_assert!(
|
||||
!frame.localsplus_is_datastack_backed(),
|
||||
"generator frame is data-stack-backed"
|
||||
let use_datastack = !is_gen && !is_coro && !is_async_gen;
|
||||
let frame = FrameObject::new_ref(
|
||||
code_owned,
|
||||
Scope::new(locals, self.globals.clone()),
|
||||
self.builtins.clone(),
|
||||
self.closure.as_ref().map_or(&[], |c| c.as_slice()),
|
||||
Some(self.to_owned().into()),
|
||||
use_datastack,
|
||||
vm,
|
||||
);
|
||||
// SAFETY: the frame is alive (held by `frame`) and untracked.
|
||||
unsafe {
|
||||
crate::gc_state::gc_state()
|
||||
.track_object(core::ptr::NonNull::from(frame.as_object()));
|
||||
self.fill_locals_from_args(&frame, func_args, vm)?;
|
||||
if is_gen || is_coro || is_async_gen {
|
||||
return Ok(self.make_generator_or_coro(frame, vm));
|
||||
}
|
||||
frame.set_generator(&obj);
|
||||
Ok(obj)
|
||||
// Tracing active: use heap frame with full trace support.
|
||||
let result = vm.run_frame(frame.clone());
|
||||
unsafe {
|
||||
if let Some(base) = frame.iframe_mut().localsplus.release_datastack() {
|
||||
vm.datastack_pop(base);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
// Fast path: stack-allocated InterpreterFrame, no FrameObject.
|
||||
// No refcount inc for code — it's alive via self.code for the call duration.
|
||||
let nlocalsplus = code.localspluskinds.len();
|
||||
let max_stackdepth = code.max_stackdepth as usize;
|
||||
let localsplus =
|
||||
crate::frame::LocalsPlus::new_on_datastack(nlocalsplus, max_stackdepth, vm);
|
||||
|
||||
let locals = if code.flags.contains(bytecode::CodeFlags::NEWLOCALS) {
|
||||
crate::frame::FrameLocals::lazy()
|
||||
} else if let Some(locals) = locals {
|
||||
crate::frame::FrameLocals::with_locals(locals)
|
||||
} else {
|
||||
crate::frame::FrameLocals::with_locals(crate::function::ArgMapping::from_dict_exact(
|
||||
self.globals.clone(),
|
||||
))
|
||||
};
|
||||
|
||||
// Use self.as_object() as raw pointer — no refcount inc/dec.
|
||||
// The function is alive on the caller's stack for the call duration.
|
||||
let mut iframe = crate::frame::InterpreterFrame::new(
|
||||
&self.code,
|
||||
&self.globals,
|
||||
&self.builtins,
|
||||
Some(self.as_object()),
|
||||
localsplus,
|
||||
locals,
|
||||
self.closure.as_ref().map_or(&[], |c| c.as_slice()),
|
||||
crate::frame::FrameOwner::Thread,
|
||||
);
|
||||
let result = self
|
||||
.fill_locals_from_args_iframe(&mut iframe, func_args, vm)
|
||||
.and_then(|()| vm.run_frame_fast(&mut iframe));
|
||||
// Release data stack memory — must happen on both success and error.
|
||||
unsafe {
|
||||
if let Some(base) = iframe.localsplus.release_datastack() {
|
||||
vm.datastack_pop(base);
|
||||
}
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
/// Create generator, coroutine, or async generator from a FrameObject.
|
||||
fn make_generator_or_coro(&self, frame: FrameObjectRef, vm: &VirtualMachine) -> PyObjectRef {
|
||||
let code = frame.iframe().code();
|
||||
let is_async_gen = code.flags.contains(bytecode::CodeFlags::ASYNC_GENERATOR);
|
||||
let is_gen = code.flags.contains(bytecode::CodeFlags::GENERATOR);
|
||||
|
||||
let obj = if is_async_gen {
|
||||
PyAsyncGen::new(frame.clone(), self.__name__(), self.__qualname__()).into_pyobject(vm)
|
||||
} else if is_gen {
|
||||
PyGenerator::new(frame.clone(), self.__name__(), self.__qualname__()).into_pyobject(vm)
|
||||
} else {
|
||||
PyCoroutine::new(frame.clone(), self.__name__(), self.__qualname__()).into_pyobject(vm)
|
||||
};
|
||||
debug_assert!(
|
||||
!frame.localsplus_is_datastack_backed(),
|
||||
"generator frame is data-stack-backed"
|
||||
);
|
||||
// SAFETY: the frame is alive (held by `frame`) and untracked.
|
||||
unsafe {
|
||||
crate::gc_state::gc_state().track_object(core::ptr::NonNull::from(frame.as_object()));
|
||||
}
|
||||
frame.set_generator(&obj);
|
||||
obj
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
@@ -696,7 +748,7 @@ impl Py<PyFunction> {
|
||||
&self,
|
||||
args: impl ExactSizeIterator<Item = PyObjectRef>,
|
||||
vm: &VirtualMachine,
|
||||
) -> FrameRef {
|
||||
) -> FrameObjectRef {
|
||||
let code: PyRef<PyCode> = (*self.code).to_owned();
|
||||
|
||||
debug_assert_eq!(args.len(), code.arg_count as usize);
|
||||
@@ -719,7 +771,7 @@ impl Py<PyFunction> {
|
||||
Some(ArgMapping::from_dict_exact(self.globals.clone()))
|
||||
};
|
||||
|
||||
let frame = Frame::new(
|
||||
let frame = FrameObject::new_ref(
|
||||
code,
|
||||
Scope::new(locals, self.globals.clone()),
|
||||
self.builtins.clone(),
|
||||
@@ -727,8 +779,7 @@ impl Py<PyFunction> {
|
||||
Some(self.to_owned().into()),
|
||||
true, // Exact-args fast path is only used for non-gen/coro functions.
|
||||
vm,
|
||||
)
|
||||
.into_ref(&vm.ctx);
|
||||
);
|
||||
|
||||
{
|
||||
let fastlocals = unsafe { frame.fastlocals_mut() };
|
||||
@@ -745,10 +796,45 @@ impl Py<PyFunction> {
|
||||
args: impl ExactSizeIterator<Item = PyObjectRef>,
|
||||
vm: &VirtualMachine,
|
||||
) -> PyResult {
|
||||
let frame = self.prepare_exact_args_frame(args, vm);
|
||||
let code = &*self.code;
|
||||
let nlocalsplus = code.localspluskinds.len();
|
||||
let max_stackdepth = code.max_stackdepth as usize;
|
||||
let localsplus =
|
||||
crate::frame::LocalsPlus::new_on_datastack(nlocalsplus, max_stackdepth, vm);
|
||||
|
||||
let result = vm.run_frame(frame.clone());
|
||||
crate::frame::release_datastack_frame(&frame, vm);
|
||||
let locals = if code.flags.contains(bytecode::CodeFlags::NEWLOCALS) {
|
||||
crate::frame::FrameLocals::lazy()
|
||||
} else {
|
||||
crate::frame::FrameLocals::with_locals(ArgMapping::from_dict_exact(
|
||||
self.globals.clone(),
|
||||
))
|
||||
};
|
||||
|
||||
let mut iframe = crate::frame::InterpreterFrame::new(
|
||||
code,
|
||||
&self.globals,
|
||||
&self.builtins,
|
||||
Some(self.as_object()),
|
||||
localsplus,
|
||||
locals,
|
||||
self.closure.as_ref().map_or(&[], |c| c.as_slice()),
|
||||
crate::frame::FrameOwner::Thread,
|
||||
);
|
||||
|
||||
// Fill arguments directly into fastlocals
|
||||
{
|
||||
let fastlocals = iframe.localsplus.fastlocals_mut();
|
||||
for (slot, arg) in fastlocals.iter_mut().zip(args) {
|
||||
*slot = Some(arg);
|
||||
}
|
||||
}
|
||||
|
||||
let result = vm.run_frame_fast(&mut iframe);
|
||||
unsafe {
|
||||
if let Some(base) = iframe.localsplus.release_datastack() {
|
||||
vm.datastack_pop(base);
|
||||
}
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ use crate::{
|
||||
AsObject, Context, Py, PyObjectRef, PyPayload, PyRef, PyResult, VirtualMachine,
|
||||
class::PyClassImpl,
|
||||
coroutine::{Coro, warn_deprecated_throw_signature},
|
||||
frame::FrameRef,
|
||||
frame::FrameObjectRef,
|
||||
function::OptionalArg,
|
||||
object::{Traverse, TraverseFn},
|
||||
protocol::PyIterReturn,
|
||||
@@ -43,7 +43,7 @@ impl PyGenerator {
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn new(frame: FrameRef, name: PyStrRef, qualname: PyStrRef) -> Self {
|
||||
pub fn new(frame: FrameObjectRef, name: PyStrRef, qualname: PyStrRef) -> Self {
|
||||
Self {
|
||||
inner: Coro::new(frame, name, qualname),
|
||||
}
|
||||
@@ -70,7 +70,7 @@ impl PyGenerator {
|
||||
}
|
||||
|
||||
#[pygetset]
|
||||
fn gi_frame(&self, _vm: &VirtualMachine) -> Option<FrameRef> {
|
||||
fn gi_frame(&self, _vm: &VirtualMachine) -> Option<FrameObjectRef> {
|
||||
if self.inner.closed() {
|
||||
None
|
||||
} else {
|
||||
@@ -85,7 +85,7 @@ impl PyGenerator {
|
||||
|
||||
#[pygetset]
|
||||
fn gi_code(&self, _vm: &VirtualMachine) -> PyRef<PyCode> {
|
||||
self.inner.frame().code.clone()
|
||||
self.inner.frame().iframe().code().to_owned()
|
||||
}
|
||||
|
||||
#[pygetset]
|
||||
|
||||
@@ -79,22 +79,27 @@ impl Initializer for PySuper {
|
||||
let (typ, obj) = if let OptionalArg::Present(ty) = py_type {
|
||||
(ty, py_obj.unwrap_or_none(vm))
|
||||
} else {
|
||||
let frame = vm
|
||||
.current_frame()
|
||||
.ok_or_else(|| vm.new_runtime_error("super(): no current frame"))?;
|
||||
// Access the InterpreterFrame directly — no need to materialize
|
||||
// a FrameObject just to read code/locals.
|
||||
let iframe_ptr = crate::vm::thread::get_current_frame();
|
||||
if iframe_ptr.is_null() {
|
||||
return Err(vm.new_runtime_error("super(): no current frame"));
|
||||
}
|
||||
let iframe = unsafe { &*iframe_ptr };
|
||||
let code = iframe.code();
|
||||
|
||||
if frame.code.arg_count == 0 {
|
||||
if code.arg_count == 0 {
|
||||
return Err(vm.new_runtime_error("super(): no arguments"));
|
||||
}
|
||||
|
||||
// SAFETY: Frame is current and not concurrently mutated.
|
||||
// SAFETY: InterpreterFrame is current and not concurrently mutated.
|
||||
use rustpython_compiler_core::bytecode::CO_FAST_CELL;
|
||||
let obj = unsafe { frame.fastlocals() }[0]
|
||||
let fastlocals = iframe.localsplus.fastlocals();
|
||||
let obj = fastlocals[0]
|
||||
.clone()
|
||||
.and_then(|val| {
|
||||
// If slot 0 is a merged cell (LOCAL|CELL), extract value from cell
|
||||
if frame
|
||||
.code
|
||||
if code
|
||||
.localspluskinds
|
||||
.first()
|
||||
.is_some_and(|&k| k & CO_FAST_CELL != 0)
|
||||
@@ -108,13 +113,15 @@ impl Initializer for PySuper {
|
||||
|
||||
let mut typ = None;
|
||||
// Search for __class__ in freevars using localspluskinds
|
||||
let nlocalsplus = frame.code.localspluskinds.len();
|
||||
let nfrees = frame.code.freevars.len();
|
||||
let nlocalsplus = code.localspluskinds.len();
|
||||
let nfrees = code.freevars.len();
|
||||
let free_start = nlocalsplus - nfrees;
|
||||
for (i, var) in frame.code.freevars.iter().enumerate() {
|
||||
for (i, var) in code.freevars.iter().enumerate() {
|
||||
if var.as_bytes() == b"__class__" {
|
||||
let class = frame
|
||||
.get_cell_contents(free_start + i)
|
||||
let class = fastlocals[free_start + i]
|
||||
.as_ref()
|
||||
.and_then(|v| v.downcast_ref::<PyCell>())
|
||||
.and_then(|c| c.get())
|
||||
.ok_or_else(|| vm.new_runtime_error("super(): empty __class__ cell"))?;
|
||||
typ = Some(class.downcast().map_err(|o| {
|
||||
vm.new_type_error(format!(
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
use super::{PyList, PyType};
|
||||
use crate::{
|
||||
AsObject, Context, Py, PyPayload, PyRef, PyResult, VirtualMachine, class::PyClassImpl,
|
||||
frame::FrameRef, function::PySetterValue, types::Constructor,
|
||||
frame::FrameObjectRef, function::PySetterValue, types::Constructor,
|
||||
};
|
||||
use rustpython_common::lock::PyMutex;
|
||||
use rustpython_compiler_core::OneIndexed;
|
||||
@@ -10,7 +10,7 @@ use rustpython_compiler_core::OneIndexed;
|
||||
#[derive(Debug)]
|
||||
pub struct PyTraceback {
|
||||
pub next: PyMutex<Option<PyTracebackRef>>,
|
||||
pub frame: FrameRef,
|
||||
pub frame: FrameObjectRef,
|
||||
#[pytraverse(skip)]
|
||||
pub lasti: u32,
|
||||
#[pytraverse(skip)]
|
||||
@@ -31,7 +31,7 @@ impl PyTraceback {
|
||||
#[must_use]
|
||||
pub const fn new(
|
||||
next: Option<PyRef<Self>>,
|
||||
frame: FrameRef,
|
||||
frame: FrameObjectRef,
|
||||
lasti: u32,
|
||||
lineno: OneIndexed,
|
||||
) -> Self {
|
||||
@@ -44,7 +44,7 @@ impl PyTraceback {
|
||||
}
|
||||
|
||||
#[pygetset]
|
||||
fn tb_frame(&self) -> FrameRef {
|
||||
fn tb_frame(&self) -> FrameObjectRef {
|
||||
self.frame.clone()
|
||||
}
|
||||
|
||||
@@ -104,7 +104,7 @@ impl PyTraceback {
|
||||
}
|
||||
|
||||
impl Constructor for PyTraceback {
|
||||
type Args = (Option<PyRef<Self>>, FrameRef, u32, usize);
|
||||
type Args = (Option<PyRef<Self>>, FrameObjectRef, u32, usize);
|
||||
|
||||
fn py_new(_cls: &Py<PyType>, args: Self::Args, vm: &VirtualMachine) -> PyResult<Self> {
|
||||
let (next, frame, lasti, lineno) = args;
|
||||
@@ -130,9 +130,12 @@ impl serde::Serialize for PyTraceback {
|
||||
use serde::ser::SerializeStruct;
|
||||
|
||||
let mut struc = s.serialize_struct("PyTraceback", 3)?;
|
||||
struc.serialize_field("name", self.frame.code.obj_name.as_str())?;
|
||||
struc.serialize_field("name", self.frame.iframe().code().obj_name.as_str())?;
|
||||
struc.serialize_field("lineno", &self.lineno.get())?;
|
||||
struc.serialize_field("filename", self.frame.code.source_path().as_str())?;
|
||||
struc.serialize_field(
|
||||
"filename",
|
||||
self.frame.iframe().code().source_path().as_str(),
|
||||
)?;
|
||||
struc.end()
|
||||
}
|
||||
}
|
||||
@@ -1513,7 +1513,7 @@ impl PyType {
|
||||
// temporary refs so they never see a dangling pointer.
|
||||
let keep_alive = |type_ref: PyTypeRef, retired: &mut Vec<PyObjectRef>| {
|
||||
if let Some(frame) = vm.current_frame() {
|
||||
frame.temporary_refs.lock().push(type_ref.into());
|
||||
frame.iframe().temporary_refs.lock().push(type_ref.into());
|
||||
} else {
|
||||
retired.push(type_ref.into());
|
||||
}
|
||||
@@ -2183,14 +2183,11 @@ impl Constructor for PyType {
|
||||
*f = PyStaticMethod::from(f.clone()).into_pyobject(vm);
|
||||
}
|
||||
|
||||
if let Some(current_frame) = vm.current_frame() {
|
||||
if let Some(globals) = crate::frame::current_globals() {
|
||||
let entry = attributes.entry(identifier!(vm, __module__));
|
||||
if matches!(entry, Entry::Vacant(_)) {
|
||||
let module_name = vm.unwrap_or_none(
|
||||
current_frame
|
||||
.globals
|
||||
.get_item_opt(identifier!(vm, __name__), vm)?,
|
||||
);
|
||||
let module_name =
|
||||
vm.unwrap_or_none(globals.get_item_opt(identifier!(vm, __name__), vm)?);
|
||||
entry.or_insert(module_name);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,7 +3,7 @@ use crate::{
|
||||
builtins::PyStrRef,
|
||||
common::lock::PyMutex,
|
||||
exceptions::types::PyBaseException,
|
||||
frame::{ExecutionResult, Frame, FrameOwner, FrameRef},
|
||||
frame::{ExecutionResult, FrameObject, FrameObjectRef, FrameOwner},
|
||||
function::OptionalArg,
|
||||
object::{PyAtomicRef, Traverse, TraverseFn},
|
||||
protocol::PyIterReturn,
|
||||
@@ -29,7 +29,7 @@ impl ExecutionResult {
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct Coro {
|
||||
frame: FrameRef,
|
||||
frame: FrameObjectRef,
|
||||
pub closed: AtomicCell<bool>, // TODO: https://github.com/RustPython/RustPython/pull/3183#discussion_r720560652
|
||||
running: AtomicCell<bool>,
|
||||
// code
|
||||
@@ -62,7 +62,7 @@ fn gen_name(jen: &PyObject, vm: &VirtualMachine) -> &'static str {
|
||||
}
|
||||
|
||||
impl Coro {
|
||||
pub fn new(frame: FrameRef, name: PyStrRef, qualname: PyStrRef) -> Self {
|
||||
pub fn new(frame: FrameObjectRef, name: PyStrRef, qualname: PyStrRef) -> Self {
|
||||
Self {
|
||||
frame,
|
||||
closed: AtomicCell::new(false),
|
||||
@@ -94,8 +94,8 @@ impl Coro {
|
||||
match res {
|
||||
Ok(ExecutionResult::Return(_)) | Err(_) => {
|
||||
self.closed.store(true);
|
||||
// Frame is no longer suspended; allow frame.clear() to succeed.
|
||||
self.frame.owner.store(
|
||||
// FrameObject is no longer suspended; allow frame.clear() to succeed.
|
||||
self.frame.iframe().owner.store(
|
||||
FrameOwner::FrameObject as i8,
|
||||
core::sync::atomic::Ordering::Release,
|
||||
);
|
||||
@@ -114,7 +114,7 @@ impl Coro {
|
||||
func: F,
|
||||
) -> (PyResult<ExecutionResult>, bool)
|
||||
where
|
||||
F: FnOnce(&Py<Frame>) -> PyResult<ExecutionResult>,
|
||||
F: FnOnce(&Py<FrameObject>) -> PyResult<ExecutionResult>,
|
||||
{
|
||||
if self.running.compare_exchange(false, true).is_err() {
|
||||
return (
|
||||
@@ -290,7 +290,7 @@ impl Coro {
|
||||
self.closed.load()
|
||||
}
|
||||
|
||||
pub fn frame(&self) -> FrameRef {
|
||||
pub fn frame(&self) -> FrameObjectRef {
|
||||
self.frame.clone()
|
||||
}
|
||||
|
||||
@@ -337,7 +337,8 @@ pub(crate) fn get_awaitable_iter(obj: PyObjectRef, vm: &VirtualMachine) -> PyRes
|
||||
|| obj.downcast_ref::<PyGenerator>().is_some_and(|g| {
|
||||
g.as_coro()
|
||||
.frame()
|
||||
.code
|
||||
.iframe()
|
||||
.code()
|
||||
.flags
|
||||
.contains(crate::bytecode::CodeFlags::ITERABLE_COROUTINE)
|
||||
})
|
||||
@@ -352,7 +353,8 @@ pub(crate) fn get_awaitable_iter(obj: PyObjectRef, vm: &VirtualMachine) -> PyRes
|
||||
|| result.downcast_ref::<PyGenerator>().is_some_and(|g| {
|
||||
g.as_coro()
|
||||
.frame()
|
||||
.code
|
||||
.iframe()
|
||||
.code()
|
||||
.flags
|
||||
.contains(crate::bytecode::CodeFlags::ITERABLE_COROUTINE)
|
||||
})
|
||||
|
||||
@@ -401,13 +401,13 @@ fn write_traceback_entry<W: Write>(
|
||||
output: &mut W,
|
||||
tb_entry: &Py<PyTraceback>,
|
||||
) -> Result<(), W::Error> {
|
||||
let filename = tb_entry.frame.code.source_path().as_str();
|
||||
let filename = tb_entry.frame.iframe().code().source_path().as_str();
|
||||
writeln!(
|
||||
output,
|
||||
r##" File "{}", line {}, in {}"##,
|
||||
filename.trim_start_matches(r"\\?\"),
|
||||
tb_entry.lineno,
|
||||
tb_entry.frame.code.obj_name
|
||||
tb_entry.frame.iframe().code().obj_name
|
||||
)?;
|
||||
|
||||
#[cfg(feature = "host_env")]
|
||||
|
||||
+974
-328
File diff suppressed because it is too large.
Load diff
+16
-23
@@ -633,33 +633,26 @@ impl GcState {
|
||||
// classified reachable. A running frame appearing in `unreachable`
|
||||
// would mean the reachability analysis observed its interpreter state
|
||||
// as garbage — the exact hazard the barrier exists to prevent.
|
||||
// Verify no running frame is classified unreachable.
|
||||
// Walk the TLS frame chain (CURRENT_FRAME) instead of top_frame,
|
||||
// because stack-allocated frames update only CURRENT_FRAME (via
|
||||
// set_current_frame_nosave), not top_frame.
|
||||
#[cfg(all(unix, feature = "threading", debug_assertions))]
|
||||
if stw.stopped {
|
||||
let unreachable_set: HashSet<GcPtr> = unreachable.iter().copied().collect();
|
||||
crate::vm::thread::try_with_current_vm(|vm| {
|
||||
let registry = vm.state.thread_frames.lock();
|
||||
#[expect(
|
||||
clippy::iter_over_hash_type,
|
||||
reason = "assertion over every registered thread slot"
|
||||
)]
|
||||
for slot in registry.values() {
|
||||
let mut cur = slot.top_frame.load(core::sync::atomic::Ordering::Relaxed)
|
||||
as *const crate::frame::Frame;
|
||||
while !cur.is_null() {
|
||||
// SAFETY: frames on a thread's active call stack are
|
||||
// alive, and the world is stopped so none can be popped.
|
||||
let obj =
|
||||
unsafe { &*crate::Py::<crate::frame::Frame>::from_payload_ptr(cur) }
|
||||
.as_object();
|
||||
let ptr = GcPtr(NonNull::from(obj));
|
||||
debug_assert!(
|
||||
!unreachable_set.contains(&ptr),
|
||||
"running frame {obj:p} classified unreachable during GC"
|
||||
);
|
||||
cur = unsafe { (*cur).previous_frame() };
|
||||
}
|
||||
let mut cur = crate::vm::thread::get_current_frame();
|
||||
while !cur.is_null() {
|
||||
let iframe = unsafe { &*cur };
|
||||
if let Some(fo) = iframe.frame_obj() {
|
||||
let obj = fo.as_object();
|
||||
let ptr = GcPtr(NonNull::from(obj));
|
||||
debug_assert!(
|
||||
!unreachable_set.contains(&ptr),
|
||||
"running frame {obj:p} classified unreachable during GC"
|
||||
);
|
||||
}
|
||||
});
|
||||
cur = iframe.previous();
|
||||
}
|
||||
}
|
||||
|
||||
if debug.contains(GcDebugFlags::STATS) {
|
||||
|
||||
@@ -221,12 +221,12 @@ fn remove_importlib_frames_inner(
|
||||
return (None, false);
|
||||
};
|
||||
|
||||
let file_name = traceback.frame.code.source_path().as_str();
|
||||
let file_name = traceback.frame.iframe().code().source_path().as_str();
|
||||
|
||||
let (inner_tb, mut now_in_importlib) =
|
||||
remove_importlib_frames_inner(vm, traceback.next.lock().clone(), always_trim);
|
||||
if file_name == "_frozen_importlib" || file_name == "_frozen_importlib_external" {
|
||||
if traceback.frame.code.obj_name.as_str() == "_call_with_frames_removed" {
|
||||
if traceback.frame.iframe().code().obj_name.as_str() == "_call_with_frames_removed" {
|
||||
now_in_importlib = true;
|
||||
}
|
||||
if always_trim || now_in_importlib {
|
||||
|
||||
@@ -211,7 +211,7 @@ pub(super) unsafe fn default_dealloc<T: PyPayload>(obj: *mut PyObject) {
|
||||
}
|
||||
|
||||
// Extract child references to break circular refs (tp_clear), then drop
|
||||
// them. Some payloads (e.g. Frame) drop children in place inside clear_fn
|
||||
// them. Some payloads (e.g. FrameObject) drop children in place inside clear_fn
|
||||
// instead of extracting them, so user code (`__del__`) may run here.
|
||||
let mut edges = Vec::new();
|
||||
if let Some(clear_fn) = vtable.clear {
|
||||
@@ -2191,6 +2191,7 @@ impl<T: PyPayload> Py<T> {
|
||||
/// obtained by dereferencing a `Py<T>`), and the object must outlive the
|
||||
/// returned pointer's use.
|
||||
#[inline]
|
||||
#[cfg_attr(not(feature = "threading"), allow(dead_code))]
|
||||
pub(crate) unsafe fn from_payload_ptr(payload: *const T) -> *const Self {
|
||||
let offset = core::mem::offset_of!(PyInner<T>, payload);
|
||||
// `Py<T>` is a newtype over `PyInner<T>`, so their addresses coincide.
|
||||
|
||||
@@ -333,7 +333,7 @@ impl<T: PyPayload> PyAtomicRef<T> {
|
||||
pub fn swap_to_temporary_refs(&self, pyref: PyRef<T>, vm: &VirtualMachine) {
|
||||
let old = unsafe { self.swap(pyref) };
|
||||
if let Some(frame) = vm.current_frame() {
|
||||
frame.temporary_refs.lock().push(old.into());
|
||||
frame.iframe().temporary_refs.lock().push(old.into());
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -409,7 +409,7 @@ impl<T: PyPayload> PyAtomicRef<Option<T>> {
|
||||
return;
|
||||
};
|
||||
if let Some(frame) = vm.current_frame() {
|
||||
frame.temporary_refs.lock().push(old.into());
|
||||
frame.iframe().temporary_refs.lock().push(old.into());
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -452,7 +452,7 @@ impl PyAtomicRef<PyObject> {
|
||||
pub fn swap_to_temporary_refs(&self, obj: PyObjectRef, vm: &VirtualMachine) {
|
||||
let old = unsafe { self.swap(obj) };
|
||||
if let Some(frame) = vm.current_frame() {
|
||||
frame.temporary_refs.lock().push(old);
|
||||
frame.iframe().temporary_refs.lock().push(old);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -499,7 +499,7 @@ impl PyAtomicRef<Option<PyObject>> {
|
||||
return;
|
||||
};
|
||||
if let Some(frame) = vm.current_frame() {
|
||||
frame.temporary_refs.lock().push(old);
|
||||
frame.iframe().temporary_refs.lock().push(old);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -51,7 +51,7 @@ pub trait PyPayload: MaybeTraverse + PyThreadingConstraint + Sized + 'static {
|
||||
/// Whether `PyRef::new_ref` skips auto-tracking this type in the GC even
|
||||
/// when it would otherwise qualify (has traverse, dict, or heap type).
|
||||
/// Such objects are created untracked and must be tracked explicitly if
|
||||
/// and when they can become part of a reference cycle. Used by `Frame`,
|
||||
/// and when they can become part of a reference cycle. Used by `FrameObject`,
|
||||
/// which is created untracked and tracked lazily only on escape.
|
||||
const NEW_REF_UNTRACKED: bool = false;
|
||||
|
||||
|
||||
@@ -228,12 +228,12 @@ impl VirtualMachine {
|
||||
let is_profile_event = event.is_profile_event();
|
||||
let is_opcode_event = event.is_opcode_event();
|
||||
|
||||
let Some(frame_ref) = self.current_frame() else {
|
||||
let Some(frame_ref) = crate::frame::current_thread_frame_materialize(self) else {
|
||||
return Ok(None);
|
||||
};
|
||||
|
||||
// Opcode events are only dispatched when f_trace_opcodes is set.
|
||||
if is_opcode_event && !*frame_ref.trace_opcodes.lock() {
|
||||
if is_opcode_event && !*frame_ref.iframe().trace_opcodes.lock() {
|
||||
return Ok(None);
|
||||
}
|
||||
|
||||
@@ -261,7 +261,7 @@ impl VirtualMachine {
|
||||
// trace_trampoline behavior: clear per-frame f_trace
|
||||
// and propagate the error.
|
||||
if let Some(frame_ref) = self.current_frame() {
|
||||
*frame_ref.trace.lock() = self.ctx.none();
|
||||
*frame_ref.iframe().trace.lock() = None;
|
||||
}
|
||||
return Err(e);
|
||||
}
|
||||
|
||||
@@ -5393,10 +5393,10 @@ mod _io {
|
||||
if vm.state.config.settings.warn_default_encoding {
|
||||
let mut stacklevel = stacklevel.unwrap_or(2);
|
||||
if stacklevel > 1
|
||||
&& let Some(frame) = vm.current_frame()
|
||||
&& let Some(code) = crate::frame::current_code()
|
||||
&& let Some(stdlib_dir) = vm.state.config.paths.stdlib_dir.as_deref()
|
||||
{
|
||||
let path = frame.code.source_path().as_str();
|
||||
let path = code.source_path().as_str();
|
||||
if !path.starts_with(stdlib_dir) {
|
||||
stacklevel = stacklevel.saturating_sub(1);
|
||||
}
|
||||
|
||||
@@ -25,7 +25,7 @@ pub(crate) mod _thread {
|
||||
PyUtf8StrRef,
|
||||
},
|
||||
common::{lock::PyMutex, wtf8::Wtf8Buf},
|
||||
frame::FrameRef,
|
||||
frame::FrameObjectRef,
|
||||
function::{ArgCallable, FuncArgs, KwArgs, OptionalArg, PySetterValue, TimeoutSeconds},
|
||||
object::{Traverse, TraverseFn},
|
||||
types::{Constructor, GetAttr, Representable, SetAttr},
|
||||
@@ -1161,44 +1161,117 @@ pub(crate) mod _thread {
|
||||
pub(crate) use crate::vm::thread::CurrentFrameSlot;
|
||||
|
||||
/// Get all threads' current (top) frames. Used by sys._current_frames().
|
||||
pub(crate) fn get_all_current_frames(vm: &VirtualMachine) -> Vec<(u64, FrameRef)> {
|
||||
pub(crate) fn get_all_current_frames(vm: &VirtualMachine) -> Vec<(u64, FrameObjectRef)> {
|
||||
// unix: read each thread's published top frame under stop-the-world so
|
||||
// the owning thread is parked at a safepoint and cannot pop or free the
|
||||
// frame while we take a strong reference. Request stop-the-world before
|
||||
// the registry lock to avoid deadlocking a thread parking mid-registry.
|
||||
//
|
||||
// For the current thread, use TLS CURRENT_FRAME directly because
|
||||
// stack-allocated frames only update TLS (not top_frame).
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use core::sync::atomic::Ordering;
|
||||
let current_ident = get_ident();
|
||||
vm.state.stop_the_world.stop_the_world(vm);
|
||||
scopeguard::defer! { vm.state.stop_the_world.start_the_world(vm); }
|
||||
let registry = vm.state.thread_frames.lock();
|
||||
registry
|
||||
.iter()
|
||||
.filter_map(|(id, slot)| {
|
||||
let top = slot.top_frame.load(Ordering::Relaxed);
|
||||
core::ptr::NonNull::new(top).map(|p| {
|
||||
// SAFETY: world stopped -> the owning thread is parked
|
||||
// and cannot pop or free this frame; it is alive on
|
||||
// that thread's call stack.
|
||||
let py =
|
||||
unsafe { &*Py::<crate::frame::Frame>::from_payload_ptr(p.as_ptr()) };
|
||||
(*id, py.to_owned())
|
||||
})
|
||||
if *id == current_ident {
|
||||
// Current thread: materialize from TLS chain
|
||||
crate::frame::current_thread_frame_materialize(vm).map(|frame| (*id, frame))
|
||||
} else {
|
||||
// Other threads: try top_frame first (FrameObject),
|
||||
// fall back to top_iframe (may be a stack-allocated frame).
|
||||
let top = slot.top_frame.load(Ordering::Relaxed);
|
||||
if let Some(p) = core::ptr::NonNull::new(top) {
|
||||
let py = unsafe {
|
||||
&*Py::<crate::frame::FrameObject>::from_payload_ptr(p.as_ptr())
|
||||
};
|
||||
Some((*id, py.to_owned()))
|
||||
} else {
|
||||
// Stack-allocated frame: materialize from top_iframe.
|
||||
// SAFETY: world stopped -> owning thread is parked.
|
||||
let iframe_ptr = slot.top_iframe.load(Ordering::Relaxed)
|
||||
as *const crate::frame::InterpreterFrame;
|
||||
if !iframe_ptr.is_null() {
|
||||
// Materialize the entire frame chain and link
|
||||
// retained_back so f_back works after STW ends.
|
||||
let mut cur = iframe_ptr;
|
||||
let mut child_fo: Option<crate::PyRef<crate::frame::FrameObject>> =
|
||||
None;
|
||||
while !cur.is_null() {
|
||||
let iframe = unsafe { &*cur };
|
||||
let fo = iframe.materialize(vm).to_owned();
|
||||
if let Some(child) = child_fo.take() {
|
||||
let mut guard = child.iframe().retained_back.lock();
|
||||
if guard.is_none() {
|
||||
*guard = Some(fo.clone());
|
||||
}
|
||||
}
|
||||
child_fo = Some(fo);
|
||||
cur = iframe.previous();
|
||||
}
|
||||
let iframe = unsafe { &*iframe_ptr };
|
||||
let fo = iframe.materialize(vm);
|
||||
Some((*id, fo.to_owned()))
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
}
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
#[cfg(not(unix))]
|
||||
{
|
||||
use core::sync::atomic::Ordering;
|
||||
let current_ident = get_ident();
|
||||
vm.state.stop_the_world.stop_the_world(vm);
|
||||
scopeguard::defer! { vm.state.stop_the_world.start_the_world(vm); }
|
||||
let registry = vm.state.thread_frames.lock();
|
||||
registry
|
||||
.iter()
|
||||
.filter_map(|(id, slot)| {
|
||||
let frames = slot.frames.lock();
|
||||
// SAFETY: the owning thread can't pop while we hold the Mutex,
|
||||
// so the FramePtr is valid for the duration of the lock.
|
||||
frames
|
||||
.last()
|
||||
.map(|fp| (*id, unsafe { fp.as_ref() }.to_owned()))
|
||||
if *id == current_ident {
|
||||
// Current thread: materialize from TLS chain
|
||||
crate::frame::current_thread_frame_materialize(vm).map(|frame| (*id, frame))
|
||||
} else {
|
||||
// Other threads: use top_iframe to include
|
||||
// stack-allocated frames. Materialize the entire
|
||||
// chain and link retained_back so f_back works.
|
||||
// SAFETY: world stopped -> owning thread is parked.
|
||||
let iframe_ptr = slot.top_iframe.load(Ordering::Relaxed)
|
||||
as *const crate::frame::InterpreterFrame;
|
||||
if !iframe_ptr.is_null() {
|
||||
let mut cur = iframe_ptr;
|
||||
let mut child_fo: Option<crate::PyRef<crate::frame::FrameObject>> =
|
||||
None;
|
||||
while !cur.is_null() {
|
||||
let iframe = unsafe { &*cur };
|
||||
let fo = iframe.materialize(vm).to_owned();
|
||||
if let Some(child) = child_fo.take() {
|
||||
let mut guard = child.iframe().retained_back.lock();
|
||||
if guard.is_none() {
|
||||
*guard = Some(fo.clone());
|
||||
}
|
||||
}
|
||||
child_fo = Some(fo);
|
||||
cur = iframe.previous();
|
||||
}
|
||||
let iframe = unsafe { &*iframe_ptr };
|
||||
let fo = iframe.materialize(vm);
|
||||
Some((*id, fo.to_owned()))
|
||||
} else {
|
||||
// Fall back to frames stack for FrameObject-only path
|
||||
let frames = slot.frames.lock();
|
||||
frames
|
||||
.last()
|
||||
.map(|fp| (*id, unsafe { fp.as_ref() }.to_owned()))
|
||||
}
|
||||
}
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
@@ -417,9 +417,8 @@ pub(crate) mod decl {
|
||||
};
|
||||
|
||||
// Get caller's module name from frame globals, like typevar.rs caller()
|
||||
let module = vm
|
||||
.current_frame()
|
||||
.and_then(|f| f.globals.get_item("__name__", vm).ok());
|
||||
let module =
|
||||
crate::frame::current_globals().and_then(|g| g.get_item("__name__", vm).ok());
|
||||
|
||||
Ok(Self::new_eager(name, type_params, value, module))
|
||||
}
|
||||
|
||||
@@ -126,12 +126,12 @@ mod builtins {
|
||||
fn merge_compile_future_features(
|
||||
flags: i32,
|
||||
dont_inherit: bool,
|
||||
vm: &VirtualMachine,
|
||||
_vm: &VirtualMachine,
|
||||
) -> bytecode::CodeFlags {
|
||||
let mut future_features = compile_future_features_from_flags(flags);
|
||||
if !dont_inherit && let Some(frame) = vm.current_frame() {
|
||||
if !dont_inherit && let Some(code) = crate::frame::current_code() {
|
||||
future_features |= bytecode::CodeFlags::from_bits_truncate(
|
||||
frame.code.flags.bits() & compile_future_feature_mask().bits(),
|
||||
code.flags.bits() & compile_future_feature_mask().bits(),
|
||||
);
|
||||
}
|
||||
future_features
|
||||
@@ -649,9 +649,9 @@ mod builtins {
|
||||
Either::A(string) => {
|
||||
let source = string.as_str();
|
||||
let mut opts = vm.compile_opts();
|
||||
if let Some(frame) = vm.current_frame() {
|
||||
if let Some(code) = crate::frame::current_code() {
|
||||
opts.future_features = bytecode::CodeFlags::from_bits_truncate(
|
||||
frame.code.flags.bits() & compile_future_feature_mask().bits(),
|
||||
code.flags.bits() & compile_future_feature_mask().bits(),
|
||||
);
|
||||
}
|
||||
vm.compile_with_opts(source, mode, "<string>", opts)
|
||||
|
||||
@@ -44,7 +44,7 @@ pub mod sys {
|
||||
hash::{PyHash, PyUHash},
|
||||
},
|
||||
convert::ToPyObject,
|
||||
frame::FrameRef,
|
||||
frame::FrameObjectRef,
|
||||
function::{FuncArgs, KwArgs, OptionalArg, PosArgs},
|
||||
stdlib::{_warnings::warn, builtins},
|
||||
types::PyStructSequence,
|
||||
@@ -968,9 +968,9 @@ pub mod sys {
|
||||
}
|
||||
|
||||
#[pyfunction]
|
||||
fn _getframe(offset: OptionalArg<usize>, vm: &VirtualMachine) -> PyResult<FrameRef> {
|
||||
fn _getframe(offset: OptionalArg<usize>, vm: &VirtualMachine) -> PyResult<FrameObjectRef> {
|
||||
let offset = offset.into_option().unwrap_or(0);
|
||||
let frame_ref = crate::frame::frame_at_offset(offset)
|
||||
let frame_ref = crate::frame::frame_at_offset(offset, vm)
|
||||
.ok_or_else(|| vm.new_value_error("call stack is not deep enough"))?;
|
||||
frame_ref.mark_escaped();
|
||||
|
||||
@@ -992,8 +992,8 @@ pub mod sys {
|
||||
}
|
||||
|
||||
// Get the frame at the specified depth
|
||||
let func_obj = match crate::frame::frame_at_offset(depth) {
|
||||
Some(frame) => frame.func_obj.clone(),
|
||||
let func_obj = match crate::frame::frame_at_offset(depth, vm) {
|
||||
Some(frame) => frame.iframe().func_obj().map(|o| o.to_owned()),
|
||||
None => return Ok(vm.ctx.none()),
|
||||
};
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
use crate::{
|
||||
AsObject, PyObjectRef, PyPayload, PyRef, PyResult, VirtualMachine,
|
||||
AsObject, Py, PyObjectRef, PyPayload, PyRef, PyResult, VirtualMachine,
|
||||
builtins::{PyCode, PyDictRef, PyNamespace, PyUtf8StrRef, code::CoMonitoringData},
|
||||
function::FuncArgs,
|
||||
};
|
||||
@@ -528,16 +528,23 @@ fn update_events_mask(vm: &VirtualMachine, state: &MonitoringState) {
|
||||
// Each code object gets only the events that apply to it (global + its
|
||||
// own local events), preventing e.g. INSTRUCTION from being applied to
|
||||
// unrelated code objects.
|
||||
crate::frame::for_each_current_frame(|frame| {
|
||||
let code = &frame.code;
|
||||
let code_ver = code.instrumentation_version.load(Ordering::Acquire);
|
||||
if code_ver != new_ver {
|
||||
let code_events = state.events_for_code(code.get_id());
|
||||
instrument_code(code, code_events);
|
||||
code.instrumentation_version
|
||||
.store(new_ver, Ordering::Release);
|
||||
// Re-instrument all frames on the current thread's stack, including
|
||||
// stack-allocated iframes (with_iframe path) that have no FrameObject.
|
||||
{
|
||||
let mut cur = crate::vm::thread::get_current_frame();
|
||||
while !cur.is_null() {
|
||||
let iframe_ref = unsafe { &*cur };
|
||||
let code = iframe_ref.code();
|
||||
let code_ver = code.instrumentation_version.load(Ordering::Acquire);
|
||||
if code_ver != new_ver {
|
||||
let code_events = state.events_for_code(code.get_id());
|
||||
instrument_code(code, code_events);
|
||||
code.instrumentation_version
|
||||
.store(new_ver, Ordering::Release);
|
||||
}
|
||||
cur = iframe_ref.previous();
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
fn use_tool_id(tool_id: i32, name: &str, vm: &VirtualMachine) -> PyResult<()> {
|
||||
@@ -740,7 +747,7 @@ thread_local! {
|
||||
fn fire(
|
||||
vm: &VirtualMachine,
|
||||
event: u32,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
cb_extra: &[PyObjectRef],
|
||||
) -> PyResult<()> {
|
||||
@@ -788,7 +795,7 @@ fn fire(
|
||||
}
|
||||
|
||||
let mut args_vec = Vec::with_capacity(1 + cb_extra.len());
|
||||
args_vec.push(code.clone().into());
|
||||
args_vec.push(code.to_owned().into());
|
||||
args_vec.extend_from_slice(cb_extra);
|
||||
let args = FuncArgs::from(args_vec);
|
||||
|
||||
@@ -823,11 +830,7 @@ fn fire(
|
||||
|
||||
// Public dispatch functions (called from frame.rs)
|
||||
|
||||
pub(crate) fn fire_py_start(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
offset: u32,
|
||||
) -> PyResult<()> {
|
||||
pub(crate) fn fire_py_start(vm: &VirtualMachine, code: &Py<PyCode>, offset: u32) -> PyResult<()> {
|
||||
fire(
|
||||
vm,
|
||||
EVENT_PY_START,
|
||||
@@ -837,11 +840,7 @@ pub(crate) fn fire_py_start(
|
||||
)
|
||||
}
|
||||
|
||||
pub(crate) fn fire_py_resume(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
offset: u32,
|
||||
) -> PyResult<()> {
|
||||
pub(crate) fn fire_py_resume(vm: &VirtualMachine, code: &Py<PyCode>, offset: u32) -> PyResult<()> {
|
||||
fire(
|
||||
vm,
|
||||
EVENT_PY_RESUME,
|
||||
@@ -853,7 +852,7 @@ pub(crate) fn fire_py_resume(
|
||||
|
||||
pub(crate) fn fire_py_return(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
retval: &PyObjectRef,
|
||||
) -> PyResult<()> {
|
||||
@@ -868,7 +867,7 @@ pub(crate) fn fire_py_return(
|
||||
|
||||
pub(crate) fn fire_py_yield(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
retval: &PyObjectRef,
|
||||
) -> PyResult<()> {
|
||||
@@ -883,7 +882,7 @@ pub(crate) fn fire_py_yield(
|
||||
|
||||
pub(crate) fn fire_call(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
callable: &PyObjectRef,
|
||||
arg0: PyObjectRef,
|
||||
@@ -899,7 +898,7 @@ pub(crate) fn fire_call(
|
||||
|
||||
pub(crate) fn fire_c_return(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
callable: &PyObjectRef,
|
||||
arg0: PyObjectRef,
|
||||
@@ -915,7 +914,7 @@ pub(crate) fn fire_c_return(
|
||||
|
||||
pub(crate) fn fire_c_raise(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
callable: &PyObjectRef,
|
||||
arg0: PyObjectRef,
|
||||
@@ -931,7 +930,7 @@ pub(crate) fn fire_c_raise(
|
||||
|
||||
pub(crate) fn fire_line(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
line: u32,
|
||||
) -> PyResult<()> {
|
||||
@@ -940,7 +939,7 @@ pub(crate) fn fire_line(
|
||||
|
||||
pub(crate) fn fire_instruction(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
) -> PyResult<()> {
|
||||
fire(
|
||||
@@ -954,7 +953,7 @@ pub(crate) fn fire_instruction(
|
||||
|
||||
pub(crate) fn fire_raise(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
exception: &PyObjectRef,
|
||||
) -> PyResult<()> {
|
||||
@@ -971,7 +970,7 @@ pub(crate) fn fire_raise(
|
||||
/// preventing duplicate events from chained cleanup handlers.
|
||||
pub(crate) fn fire_reraise(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
exception: &PyObjectRef,
|
||||
) -> PyResult<()> {
|
||||
@@ -994,7 +993,7 @@ pub(crate) fn fire_reraise(
|
||||
|
||||
pub(crate) fn fire_exception_handled(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
exception: &PyObjectRef,
|
||||
) -> PyResult<()> {
|
||||
@@ -1010,7 +1009,7 @@ pub(crate) fn fire_exception_handled(
|
||||
|
||||
pub(crate) fn fire_py_unwind(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
exception: &PyObjectRef,
|
||||
) -> PyResult<()> {
|
||||
@@ -1026,7 +1025,7 @@ pub(crate) fn fire_py_unwind(
|
||||
|
||||
pub(crate) fn fire_py_throw(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
exception: &PyObjectRef,
|
||||
) -> PyResult<()> {
|
||||
@@ -1039,24 +1038,35 @@ pub(crate) fn fire_py_throw(
|
||||
)
|
||||
}
|
||||
|
||||
/// If `value` is already a `StopIteration`, pass it directly; otherwise wrap
|
||||
/// it in a new `StopIteration(value)` — matching `PyMonitoring_FireStopIterationEvent`.
|
||||
pub(crate) fn fire_stop_iteration(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
exception: &PyObjectRef,
|
||||
value: &PyObjectRef,
|
||||
) -> PyResult<()> {
|
||||
let exc: PyObjectRef = if value.fast_isinstance(vm.ctx.exceptions.stop_iteration) {
|
||||
value.clone()
|
||||
} else {
|
||||
vm.ctx
|
||||
.exceptions
|
||||
.stop_iteration
|
||||
.as_object()
|
||||
.call(vec![value.clone()], vm)?
|
||||
};
|
||||
fire(
|
||||
vm,
|
||||
EVENT_STOP_ITERATION,
|
||||
code,
|
||||
offset,
|
||||
&[vm.ctx.new_int(offset).into(), exception.clone()],
|
||||
&[vm.ctx.new_int(offset).into(), exc],
|
||||
)
|
||||
}
|
||||
|
||||
pub(crate) fn fire_jump(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
destination: u32,
|
||||
) -> PyResult<()> {
|
||||
@@ -1074,7 +1084,7 @@ pub(crate) fn fire_jump(
|
||||
|
||||
pub(crate) fn fire_branch_left(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
destination: u32,
|
||||
) -> PyResult<()> {
|
||||
@@ -1092,7 +1102,7 @@ pub(crate) fn fire_branch_left(
|
||||
|
||||
pub(crate) fn fire_branch_right(
|
||||
vm: &VirtualMachine,
|
||||
code: &PyRef<PyCode>,
|
||||
code: &Py<PyCode>,
|
||||
offset: u32,
|
||||
destination: u32,
|
||||
) -> PyResult<()> {
|
||||
|
||||
@@ -47,20 +47,17 @@ pub(crate) mod typevar {
|
||||
///
|
||||
/// Note: CPython's implementation (in typevarobject.c) gets the module from the
|
||||
/// frame's function object using PyFunction_GetModule(f->f_funcobj). However,
|
||||
/// RustPython's Frame doesn't store a reference to the function object, so we
|
||||
/// RustPython's FrameObject doesn't store a reference to the function object, so we
|
||||
/// get the module name from the frame's globals dictionary instead.
|
||||
fn caller(vm: &VirtualMachine) -> Option<PyObjectRef> {
|
||||
let frame = vm.current_frame()?;
|
||||
|
||||
// In RustPython, we get the module name from frame's globals
|
||||
// This is similar to CPython's sys._getframe().f_globals.get('__name__')
|
||||
frame.globals.get_item("__name__", vm).ok()
|
||||
let globals = crate::frame::current_globals()?;
|
||||
globals.get_item("__name__", vm).ok()
|
||||
}
|
||||
|
||||
/// Set __module__ attribute for an object based on the caller's module.
|
||||
/// This follows CPython's behavior for TypeVar and similar objects.
|
||||
fn set_module_from_caller(obj: &PyObject, vm: &VirtualMachine) -> PyResult<()> {
|
||||
// Note: CPython gets module from frame->f_funcobj, but RustPython's Frame
|
||||
// Note: CPython gets module from frame->f_funcobj, but RustPython's FrameObject
|
||||
// architecture is different - we use globals['__name__'] instead
|
||||
let module_value: PyObjectRef = if let Some(module_name) = caller(vm) {
|
||||
// Special handling for certain module names
|
||||
|
||||
@@ -71,17 +71,23 @@ pub fn offer_suggestions(exc: &Py<PyBaseException>, vm: &VirtualMachine) -> Opti
|
||||
let tb = exc.__traceback__()?;
|
||||
let tb = tb.iter().last().unwrap_or(tb);
|
||||
|
||||
let varnames = tb.frame.code.clone().co_varnames(vm);
|
||||
let varnames = tb.frame.iframe().code().to_owned().co_varnames(vm);
|
||||
if let Some(suggestions) = calculate_suggestions(varnames.iter(), &name) {
|
||||
return Some(suggestions);
|
||||
};
|
||||
|
||||
let globals: Vec<_> = tb.frame.globals.as_object().try_to_value(vm).ok()?;
|
||||
let globals: Vec<_> = tb
|
||||
.frame
|
||||
.iframe()
|
||||
.globals()
|
||||
.as_object()
|
||||
.try_to_value(vm)
|
||||
.ok()?;
|
||||
if let Some(suggestions) = calculate_suggestions(globals.iter(), &name) {
|
||||
return Some(suggestions);
|
||||
};
|
||||
|
||||
let builtins: Vec<_> = tb.frame.builtins.try_to_value(vm).ok()?;
|
||||
let builtins: Vec<_> = tb.frame.iframe().builtins().try_to_value(vm).ok()?;
|
||||
calculate_suggestions(builtins.iter(), &name)
|
||||
} else if exc.class().fast_issubclass(vm.ctx.exceptions.import_error) {
|
||||
let mod_name = exc.as_object().get_attr("name", vm).ok()?;
|
||||
|
||||
@@ -178,7 +178,7 @@ impl TypeZoo {
|
||||
dict_itemiterator_type: dict::PyDictItemIterator::init_builtin_type(),
|
||||
dict_reverseitemiterator_type: dict::PyDictReverseItemIterator::init_builtin_type(),
|
||||
ellipsis_type: slice::PyEllipsis::init_builtin_type(),
|
||||
frame_type: crate::frame::Frame::init_builtin_type(),
|
||||
frame_type: crate::frame::FrameObject::init_builtin_type(),
|
||||
frame_locals_proxy_type: frame_locals_proxy::FrameLocalsProxy::init_builtin_type(),
|
||||
function_type: function::PyFunction::init_builtin_type(),
|
||||
generator_type: generator::PyGenerator::init_builtin_type(),
|
||||
|
||||
+274
-86
@@ -33,7 +33,7 @@ use crate::{
|
||||
common::{hash::HashSecret, lock::PyMutex, rc::PyRc},
|
||||
convert::ToPyObject,
|
||||
exceptions::types::PyBaseException,
|
||||
frame::{ExecutionResult, Frame, FrameRef},
|
||||
frame::{ExecutionResult, FrameObject, FrameObjectRef},
|
||||
frozen::FrozenModule,
|
||||
function::{ArgMapping, FuncArgs, PySetterValue},
|
||||
import,
|
||||
@@ -111,23 +111,23 @@ pub struct VirtualMachine {
|
||||
/// Non-owning frame pointer for the non-unix threading frames stack.
|
||||
/// The pointed-to frame is kept alive by the caller of with_frame/resume_gen_frame.
|
||||
/// Unix threading builds publish the top frame through `ThreadSlot::top_frame`
|
||||
/// and walk the rest via `Frame::previous`, so they do not use this type.
|
||||
/// and walk the rest via `FrameObject::previous`, so they do not use this type.
|
||||
#[cfg(all(not(unix), feature = "threading"))]
|
||||
#[derive(Copy, Clone)]
|
||||
pub struct FramePtr(NonNull<Py<Frame>>);
|
||||
pub struct FramePtr(NonNull<Py<FrameObject>>);
|
||||
|
||||
#[cfg(all(not(unix), feature = "threading"))]
|
||||
impl FramePtr {
|
||||
/// # Safety
|
||||
/// The pointed-to frame must still be alive.
|
||||
#[must_use]
|
||||
pub unsafe fn as_ref(&self) -> &Py<Frame> {
|
||||
pub unsafe fn as_ref(&self) -> &Py<FrameObject> {
|
||||
unsafe { self.0.as_ref() }
|
||||
}
|
||||
}
|
||||
|
||||
// SAFETY: FramePtr is only stored in a thread's shared frame stack
|
||||
// (`ThreadSlot::frames`) while the corresponding FrameRef is alive on that
|
||||
// (`ThreadSlot::frames`) while the corresponding FrameObjectRef is alive on that
|
||||
// thread's call stack; readers dereference it under the slot mutex.
|
||||
#[cfg(all(not(unix), feature = "threading"))]
|
||||
unsafe impl Send for FramePtr {}
|
||||
@@ -1332,7 +1332,16 @@ impl VirtualMachine {
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
pub fn run_frame(&self, frame: FrameRef) -> PyResult {
|
||||
/// Run a stack-allocated InterpreterFrame without heap allocation.
|
||||
/// This is the fast path for regular (non-generator) function calls.
|
||||
pub fn run_frame_fast(&self, iframe: &mut crate::frame::InterpreterFrame) -> PyResult {
|
||||
match self.with_iframe(iframe, |iframe| crate::frame::run_iframe(iframe, self))? {
|
||||
ExecutionResult::Return(value) => Ok(value),
|
||||
_ => panic!("Got unexpected result from function"),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn run_frame(&self, frame: FrameObjectRef) -> PyResult {
|
||||
// Only ordinary (datastack) call frames reach `run_frame`; generator
|
||||
// and coroutine frames are resumed through `resume_gen_frame`. A
|
||||
// datastack frame is created untracked and is tracked lazily only when
|
||||
@@ -1580,8 +1589,10 @@ impl VirtualMachine {
|
||||
/// The margin is doubled for debug/sanitized builds because frame
|
||||
/// evaluation consumes more native stack in those configurations.
|
||||
#[cfg_attr(any(miri, target_env = "musl"), allow(dead_code))]
|
||||
// 2× CPython's _PY_STACK_MARGIN_BYTES to account for both heavy and
|
||||
// light frame native stack usage per recursion step.
|
||||
const STACK_MARGIN_BYTES: usize =
|
||||
(if cfg!(debug_assertions) { 16384 } else { 2048 }) * core::mem::size_of::<usize>();
|
||||
(if cfg!(debug_assertions) { 16384 } else { 4096 }) * core::mem::size_of::<usize>();
|
||||
|
||||
/// Get the stack boundaries using platform-specific APIs.
|
||||
/// Returns (base, top) where base is the lowest address and top is the highest.
|
||||
@@ -1667,7 +1678,7 @@ impl VirtualMachine {
|
||||
/// single native frame can exceed the margin and step past it.
|
||||
#[cfg(all(not(miri), not(target_env = "musl")))]
|
||||
#[inline(always)]
|
||||
fn check_c_stack_overflow(&self) -> bool {
|
||||
pub(crate) fn check_c_stack_overflow(&self) -> bool {
|
||||
let current_sp = psm::stack_pointer() as usize;
|
||||
let soft_limit = self.c_stack_soft_limit.get();
|
||||
current_sp < soft_limit
|
||||
@@ -1677,7 +1688,7 @@ impl VirtualMachine {
|
||||
/// the probe during stdlib bootstrap.
|
||||
#[cfg(any(miri, target_env = "musl"))]
|
||||
#[inline(always)]
|
||||
fn check_c_stack_overflow(&self) -> bool {
|
||||
pub(crate) fn check_c_stack_overflow(&self) -> bool {
|
||||
false
|
||||
}
|
||||
|
||||
@@ -1697,68 +1708,230 @@ impl VirtualMachine {
|
||||
f()
|
||||
}
|
||||
|
||||
pub fn with_frame<R, F: FnOnce(FrameRef) -> PyResult<R>>(
|
||||
pub fn with_frame<R, F: FnOnce(FrameObjectRef) -> PyResult<R>>(
|
||||
&self,
|
||||
frame: FrameRef,
|
||||
frame: FrameObjectRef,
|
||||
f: F,
|
||||
) -> PyResult<R> {
|
||||
self.with_recursion("", || {
|
||||
// SAFETY: `frame` (FrameRef) stays alive for the entire closure scope,
|
||||
// keeping the FramePtr valid. We pass a clone to `f` so that `f`
|
||||
// consuming its FrameRef doesn't invalidate our pointer.
|
||||
// Publish the frame for sys._current_frames() and faulthandler.
|
||||
// On unix, set_current_frame below publishes the top frame into the
|
||||
// thread slot; only non-unix builds maintain the mutex-guarded Vec.
|
||||
#[cfg(all(not(unix), feature = "threading"))]
|
||||
crate::vm::thread::push_thread_frame(FramePtr(NonNull::from(&*frame)));
|
||||
// Link frame into the signal-safe frame chain (previous pointer).
|
||||
// This chain is the single source for the current thread's frame
|
||||
// stack (current_frame, sys._getframe, f_back, monitoring).
|
||||
let old_frame = crate::vm::thread::set_current_frame((&**frame) as *const Frame);
|
||||
frame.previous.store(
|
||||
old_frame as *mut Frame,
|
||||
core::sync::atomic::Ordering::Relaxed,
|
||||
);
|
||||
// Normal frame calls share the caller's exc_info slot so that
|
||||
// callees can see the caller's handled exception via sys.exc_info().
|
||||
// Save the current value to restore on exit — this prevents
|
||||
// exc_info pollution from frames with unbalanced
|
||||
// PUSH_EXC_INFO/POP_EXCEPT (e.g., exception escaping an except block
|
||||
// whose cleanup entry is missing from the exception table).
|
||||
// A callee whose bytecode never mutates the slot cannot pollute it,
|
||||
// so the save/restore is skipped for it.
|
||||
let save_exc = frame.code.has_exc_handling;
|
||||
let saved_exc = if save_exc {
|
||||
self.current_exception()
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let old_owner = frame.owner.swap(
|
||||
crate::frame::FrameOwner::Thread as i8,
|
||||
core::sync::atomic::Ordering::AcqRel,
|
||||
);
|
||||
self.check_recursive_call("")?;
|
||||
|
||||
// Ensure cleanup on panic: restore owner, exc_info, and frame chain.
|
||||
scopeguard::defer! {
|
||||
frame.owner.store(old_owner, core::sync::atomic::Ordering::Release);
|
||||
if save_exc {
|
||||
self.restore_exception(saved_exc);
|
||||
// Check the native C stack periodically. The sampling interval
|
||||
// (every 8th call) balances overhead against the risk of missing
|
||||
// an overflow between checks, especially when light and heavy
|
||||
// frames alternate (each recursion step uses different native
|
||||
// stack amounts).
|
||||
let depth = self.recursion_depth.get();
|
||||
if depth & 7 == 0 && self.check_c_stack_overflow() {
|
||||
return Err(self.new_recursion_error(String::new()));
|
||||
}
|
||||
|
||||
self.recursion_depth.update(|d| d + 1);
|
||||
// Decrement on all exit paths (including panic between here and
|
||||
// the explicit decrement at the bottom).
|
||||
let _depth_guard = scopeguard::guard((), |()| {
|
||||
self.recursion_depth.update(|d| d.saturating_sub(1))
|
||||
});
|
||||
|
||||
#[cfg(all(not(unix), feature = "threading"))]
|
||||
crate::vm::thread::push_thread_frame(FramePtr(NonNull::from(&*frame)));
|
||||
let iframe = frame.iframe() as *const crate::frame::InterpreterFrame;
|
||||
let old_chain = crate::vm::thread::set_current_frame(iframe);
|
||||
{
|
||||
#[allow(unused_imports)]
|
||||
use rustpython_common::atomic::Radium;
|
||||
frame
|
||||
.iframe()
|
||||
.previous
|
||||
.store(old_chain as usize, core::sync::atomic::Ordering::Relaxed);
|
||||
}
|
||||
let save_exc = frame.iframe().code().has_exc_handling;
|
||||
let saved_exc = if save_exc {
|
||||
self.current_exception()
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let old_owner = frame.iframe().owner.swap(
|
||||
crate::frame::FrameOwner::Thread as i8,
|
||||
core::sync::atomic::Ordering::AcqRel,
|
||||
);
|
||||
|
||||
let result = self.dispatch_traced_frame(&frame, |frame| f(frame.to_owned()));
|
||||
|
||||
// Capture f_back before clearing previous so code holding a
|
||||
// reference to this FrameObject can walk the chain after return.
|
||||
if !old_chain.is_null() {
|
||||
let strong = frame.as_object().strong_count();
|
||||
// Only set retained_back if someone else holds a reference (escaped)
|
||||
// AND the caller already has a FrameObject. Materializing the caller
|
||||
// here would add refcounts on its local variables, preventing timely
|
||||
// __del__ / ResourceWarning on dealloc. If the caller hasn't been
|
||||
// materialized, f_back will resolve via the TLS chain while the
|
||||
// caller is still executing, or return None after it returns.
|
||||
if strong > 1 {
|
||||
let mut guard = frame.iframe().retained_back.lock();
|
||||
if guard.is_none() {
|
||||
let prev_iframe = unsafe { &*old_chain };
|
||||
if let Some(fo) = prev_iframe.frame_obj() {
|
||||
*guard = Some(fo.to_owned());
|
||||
}
|
||||
}
|
||||
crate::vm::thread::set_current_frame(old_frame);
|
||||
#[cfg(all(not(unix), feature = "threading"))]
|
||||
crate::vm::thread::pop_thread_frame();
|
||||
}
|
||||
}
|
||||
|
||||
self.dispatch_traced_frame(&frame, |frame| f(frame.to_owned()))
|
||||
})
|
||||
frame
|
||||
.iframe()
|
||||
.owner
|
||||
.store(old_owner, core::sync::atomic::Ordering::Release);
|
||||
if save_exc {
|
||||
self.restore_exception(saved_exc);
|
||||
}
|
||||
// Clear previous before popping — it may point to a stack-allocated
|
||||
// iframe that will be freed when the caller's with_iframe exits.
|
||||
{
|
||||
#[allow(unused_imports)]
|
||||
use rustpython_common::atomic::Radium;
|
||||
frame
|
||||
.iframe()
|
||||
.previous
|
||||
.store(0, core::sync::atomic::Ordering::Relaxed);
|
||||
}
|
||||
let _ = crate::vm::thread::set_current_frame(old_chain);
|
||||
#[cfg(all(not(unix), feature = "threading"))]
|
||||
crate::vm::thread::pop_thread_frame();
|
||||
// Disarm the panic guard — normal decrement.
|
||||
scopeguard::ScopeGuard::into_inner(_depth_guard);
|
||||
self.recursion_depth.update(|d| d - 1);
|
||||
|
||||
result
|
||||
}
|
||||
|
||||
/// Frame execution for generator/coroutine resume.
|
||||
/// Execute a stack-allocated InterpreterFrame without heap-allocating
|
||||
/// a FrameObject. This is the fast path for regular function calls.
|
||||
/// The frame is pushed onto the chain as a `*const InterpreterFrame`.
|
||||
#[inline(always)]
|
||||
pub fn with_iframe<R>(
|
||||
&self,
|
||||
iframe: &mut crate::frame::InterpreterFrame,
|
||||
f: impl FnOnce(&mut crate::frame::InterpreterFrame) -> PyResult<R>,
|
||||
) -> PyResult<R> {
|
||||
self.check_recursive_call("")?;
|
||||
|
||||
let depth = self.recursion_depth.get();
|
||||
if depth & 7 == 0 && self.check_c_stack_overflow() {
|
||||
return Err(self.new_recursion_error(String::new()));
|
||||
}
|
||||
|
||||
self.recursion_depth.update(|d| d + 1);
|
||||
|
||||
let iframe_ptr = iframe as *const crate::frame::InterpreterFrame;
|
||||
let old_chain = crate::vm::thread::set_current_frame(iframe_ptr);
|
||||
{
|
||||
#[allow(unused_imports)]
|
||||
use rustpython_common::atomic::Radium;
|
||||
iframe
|
||||
.previous
|
||||
.store(old_chain as usize, core::sync::atomic::Ordering::Relaxed);
|
||||
}
|
||||
let save_exc = iframe.code().has_exc_handling;
|
||||
let saved_exc = if save_exc {
|
||||
self.current_exception()
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
let result = f(iframe);
|
||||
|
||||
// If this iframe was materialized, capture f_back so that code
|
||||
// holding a reference to the FrameObject (e.g. sys._getframe()
|
||||
// return value, traceback frames) can walk the chain after return.
|
||||
//
|
||||
// Read materialized through the raw TLS pointer instead of the
|
||||
// &mut iframe reference. During f(iframe), bytecode can
|
||||
// materialize the frame via the TLS chain (a raw pointer alias);
|
||||
// the &mut borrow lets LLVM assume no aliased writes, which can
|
||||
// cause the store to be invisible through `iframe.materialized`.
|
||||
{
|
||||
// Use read_volatile through the original raw pointer to bypass
|
||||
// LLVM's noalias assumptions on the &mut iframe borrow.
|
||||
let mat_ptr = unsafe {
|
||||
let field_ptr = core::ptr::addr_of!((*iframe_ptr).materialized);
|
||||
core::ptr::read_volatile(field_ptr as *const usize)
|
||||
};
|
||||
if mat_ptr != 0 {
|
||||
let fo = unsafe { &*(mat_ptr as *const crate::Py<crate::frame::FrameObject>) };
|
||||
// Sync localsplus, prev_line, lasti from the live iframe to
|
||||
// the materialized FrameObject so f_locals, f_lineno, f_lasti
|
||||
// reflect the final state after execution.
|
||||
unsafe {
|
||||
let live_iframe = &*iframe_ptr;
|
||||
fo.iframe_mut()
|
||||
.localsplus
|
||||
.sync_fastlocals_from(&live_iframe.localsplus);
|
||||
fo.iframe_mut().prev_line.set(live_iframe.prev_line.get());
|
||||
#[allow(unused_imports)]
|
||||
use rustpython_common::atomic::Radium;
|
||||
fo.iframe_mut().lasti.store(
|
||||
live_iframe
|
||||
.lasti
|
||||
.load(core::sync::atomic::Ordering::Relaxed),
|
||||
core::sync::atomic::Ordering::Relaxed,
|
||||
);
|
||||
}
|
||||
if !old_chain.is_null() {
|
||||
let prev_iframe = unsafe { &*old_chain };
|
||||
// Use materialize_chain to avoid cloning localsplus, which
|
||||
// would create extra refcounts on local variables. The
|
||||
// lightweight frame has empty localsplus; live values are
|
||||
// read through find_live_source_iframe when needed.
|
||||
let back_fo = prev_iframe.materialize_chain(self);
|
||||
*fo.iframe().retained_back.lock() = Some(back_fo);
|
||||
}
|
||||
// Set owner to FrameObject since this frame is no longer
|
||||
// executing on a thread.
|
||||
fo.iframe().owner.store(
|
||||
crate::frame::FrameOwner::FrameObject as i8,
|
||||
core::sync::atomic::Ordering::Release,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if save_exc {
|
||||
self.restore_exception(saved_exc);
|
||||
}
|
||||
// Restore the frame chain BEFORE clearing temporary_refs, so
|
||||
// top_frame no longer points at the materialized FrameObject
|
||||
// when its last strong reference is released.
|
||||
let _ = crate::vm::thread::set_current_frame(old_chain);
|
||||
self.recursion_depth.update(|d| d - 1);
|
||||
|
||||
// Now that the frame is off the chain, track the materialized
|
||||
// FrameObject in the GC and release temporary_refs so cycle
|
||||
// collection can detect and reclaim reference cycles.
|
||||
{
|
||||
let mat_ptr = unsafe {
|
||||
let field_ptr = core::ptr::addr_of!((*iframe_ptr).materialized);
|
||||
core::ptr::read_volatile(field_ptr as *const usize)
|
||||
};
|
||||
if mat_ptr != 0 {
|
||||
let fo = unsafe { &*(mat_ptr as *const crate::Py<crate::frame::FrameObject>) };
|
||||
unsafe {
|
||||
crate::gc_state::gc_state()
|
||||
.track_object(core::ptr::NonNull::from(fo.as_object()));
|
||||
let live_iframe = &*iframe_ptr;
|
||||
live_iframe.temporary_refs.lock().clear();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
result
|
||||
}
|
||||
|
||||
/// FrameObject execution for generator/coroutine resume.
|
||||
/// Pushes a new exc_info slot (gi_exc_state) onto the chain,
|
||||
/// linking the generator's saved handled-exception.
|
||||
pub fn resume_gen_frame<R, F: FnOnce(&Py<Frame>) -> PyResult<R>>(
|
||||
pub fn resume_gen_frame<R, F: FnOnce(&Py<FrameObject>) -> PyResult<R>>(
|
||||
&self,
|
||||
frame: &FrameRef,
|
||||
frame: &FrameObjectRef,
|
||||
exc: Option<PyBaseExceptionRef>,
|
||||
f: F,
|
||||
) -> PyResult<R> {
|
||||
@@ -1768,19 +1941,22 @@ impl VirtualMachine {
|
||||
}
|
||||
self.recursion_depth.update(|d| d + 1);
|
||||
|
||||
// SAFETY: frame (&FrameRef) stays alive for the duration, so NonNull is valid until pop.
|
||||
// SAFETY: frame (&FrameObjectRef) stays alive for the duration, so NonNull is valid until pop.
|
||||
#[cfg(all(not(unix), feature = "threading"))]
|
||||
crate::vm::thread::push_thread_frame(FramePtr(NonNull::from(&**frame)));
|
||||
let old_frame = crate::vm::thread::set_current_frame((&***frame) as *const Frame);
|
||||
frame.previous.store(
|
||||
old_frame as *mut Frame,
|
||||
core::sync::atomic::Ordering::Relaxed,
|
||||
);
|
||||
let iframe = frame.iframe() as *const crate::frame::InterpreterFrame;
|
||||
let old_chain = crate::vm::thread::set_current_frame(iframe);
|
||||
{
|
||||
#[allow(unused_imports)]
|
||||
use rustpython_common::atomic::Radium;
|
||||
frame
|
||||
.iframe()
|
||||
.previous
|
||||
.store(old_chain as usize, core::sync::atomic::Ordering::Relaxed);
|
||||
}
|
||||
// Push generator's exc_info slot onto the chain
|
||||
// (gi_exc_state.previous_item = tstate->exc_info;
|
||||
// tstate->exc_info = &gi_exc_state;)
|
||||
self.push_exception(exc);
|
||||
let old_owner = frame.owner.swap(
|
||||
let old_owner = frame.iframe().owner.swap(
|
||||
crate::frame::FrameOwner::Thread as i8,
|
||||
core::sync::atomic::Ordering::AcqRel,
|
||||
);
|
||||
@@ -1788,9 +1964,16 @@ impl VirtualMachine {
|
||||
// Ensure cleanup on panic: restore owner, pop exc_info slot, frame chain,
|
||||
// frames Vec, and recursion depth.
|
||||
scopeguard::defer! {
|
||||
frame.owner.store(old_owner, core::sync::atomic::Ordering::Release);
|
||||
frame.iframe().owner.store(old_owner, core::sync::atomic::Ordering::Release);
|
||||
self.pop_exception();
|
||||
crate::vm::thread::set_current_frame(old_frame);
|
||||
// Clear previous before popping — it may point to a stack-allocated
|
||||
// iframe that will be freed when the caller's with_iframe exits.
|
||||
{
|
||||
#[allow(unused_imports)]
|
||||
use rustpython_common::atomic::Radium;
|
||||
frame.iframe().previous.store(0, core::sync::atomic::Ordering::Relaxed);
|
||||
}
|
||||
let _ = crate::vm::thread::set_current_frame(old_chain);
|
||||
#[cfg(all(not(unix), feature = "threading"))]
|
||||
crate::vm::thread::pop_thread_frame();
|
||||
|
||||
@@ -1809,9 +1992,9 @@ impl VirtualMachine {
|
||||
/// - Fire `TraceEvent::Return` on both normal return **and** exception
|
||||
/// unwind (`PY_UNWIND` → `PyTrace_RETURN` with `arg = None`).
|
||||
/// Propagate any trace-function error, replacing the original exception.
|
||||
fn dispatch_traced_frame<R, F: FnOnce(&Py<Frame>) -> PyResult<R>>(
|
||||
fn dispatch_traced_frame<R, F: FnOnce(&Py<FrameObject>) -> PyResult<R>>(
|
||||
&self,
|
||||
frame: &Py<Frame>,
|
||||
frame: &Py<FrameObject>,
|
||||
f: F,
|
||||
) -> PyResult<R> {
|
||||
use crate::protocol::TraceEvent;
|
||||
@@ -1819,7 +2002,7 @@ impl VirtualMachine {
|
||||
// Fire 'call' trace event. current_frame() now returns the callee.
|
||||
let trace_result = self.trace_event(TraceEvent::Call, None)?;
|
||||
if let Some(local_trace) = trace_result {
|
||||
*frame.trace.lock() = local_trace;
|
||||
*frame.iframe().trace.lock() = Some(local_trace);
|
||||
}
|
||||
|
||||
let result = f(frame);
|
||||
@@ -1828,7 +2011,7 @@ impl VirtualMachine {
|
||||
// PY_UNWIND fires PyTrace_RETURN with arg=None — so we fire for
|
||||
// both Ok and Err, matching `call_trace_protected` behavior.
|
||||
if self.use_tracing.get()
|
||||
&& (!self.is_none(&frame.trace.lock()) || !self.is_none(&self.profile_func.borrow()))
|
||||
&& (frame.iframe().trace.lock().is_some() || !self.is_none(&self.profile_func.borrow()))
|
||||
{
|
||||
let ret_result = self.trace_event(TraceEvent::Return, None);
|
||||
// call_trace_protected: if trace function raises, its error
|
||||
@@ -1880,21 +2063,23 @@ impl VirtualMachine {
|
||||
}
|
||||
}
|
||||
|
||||
pub fn current_frame(&self) -> Option<FrameRef> {
|
||||
crate::frame::current_thread_frame()
|
||||
pub fn current_frame(&self) -> Option<FrameObjectRef> {
|
||||
crate::frame::current_thread_frame_materialize(self)
|
||||
}
|
||||
|
||||
pub fn current_locals(&self) -> PyResult<ArgMapping> {
|
||||
self.current_frame()
|
||||
// Must include light frames so locals() returns the correct scope.
|
||||
crate::frame::current_thread_frame_materialize(self)
|
||||
.expect("called current_locals but no frames on the stack")
|
||||
.locals(self)
|
||||
}
|
||||
|
||||
pub fn current_globals(&self) -> PyDictRef {
|
||||
self.current_frame()
|
||||
.expect("called current_globals but no frames on the stack")
|
||||
.globals
|
||||
.clone()
|
||||
let ptr = crate::vm::thread::get_current_frame();
|
||||
if !ptr.is_null() {
|
||||
return unsafe { (*ptr).globals().to_owned() };
|
||||
}
|
||||
crate::frame::current_globals().expect("called current_globals but no frames on the stack")
|
||||
}
|
||||
|
||||
pub fn try_class(&self, module: &'static str, class: &'static str) -> PyResult<PyTypeRef> {
|
||||
@@ -1953,11 +2138,14 @@ impl VirtualMachine {
|
||||
.get_attr(identifier!(self, __import__), self)
|
||||
.map_err(|_| self.new_import_error("__import__ not found", module.to_owned()))?;
|
||||
|
||||
let (locals, globals) = if let Some(frame) = self.current_frame() {
|
||||
(
|
||||
Some(frame.locals.clone_mapping(self)),
|
||||
Some(frame.globals.clone()),
|
||||
)
|
||||
let (locals, globals) = if let Some(globals) = crate::frame::current_globals() {
|
||||
// Locals fallback: use the heavy frame if available, otherwise
|
||||
// use globals as locals (light frame locals are on the data stack).
|
||||
let locals_mapping = self.current_frame().map_or_else(
|
||||
|| ArgMapping::from_dict_exact(globals.clone()),
|
||||
|f| f.iframe().locals.clone_mapping(self),
|
||||
);
|
||||
(Some(locals_mapping), Some(globals))
|
||||
} else {
|
||||
(None, None)
|
||||
};
|
||||
|
||||
+92
-33
@@ -5,12 +5,21 @@ use crate::builtins::PyBaseExceptionRef;
|
||||
#[cfg(feature = "threading")]
|
||||
use alloc::sync::Arc;
|
||||
|
||||
use crate::frame::Frame;
|
||||
#[cfg(all(unix, feature = "threading"))]
|
||||
use crate::frame::FrameObject;
|
||||
use crate::frame::InterpreterFrame;
|
||||
#[cfg(all(unix, feature = "threading"))]
|
||||
use crate::{AsObject, Py, PyObject, VirtualMachine};
|
||||
#[cfg(all(not(unix), feature = "threading"))]
|
||||
use crate::{AsObject, PyObject, VirtualMachine};
|
||||
#[cfg(not(feature = "threading"))]
|
||||
use crate::{AsObject, PyObject, VirtualMachine};
|
||||
#[cfg(all(unix, feature = "threading"))]
|
||||
use core::sync::atomic::AtomicPtr;
|
||||
use core::{
|
||||
cell::{Cell, RefCell},
|
||||
ptr::NonNull,
|
||||
sync::atomic::{AtomicPtr, Ordering},
|
||||
sync::atomic::{AtomicUsize, Ordering},
|
||||
};
|
||||
use itertools::Itertools;
|
||||
use std::thread_local;
|
||||
@@ -38,9 +47,13 @@ pub struct ThreadSlot {
|
||||
/// thread at a safepoint and supplies the happens-before edge, so the
|
||||
/// pointer and the frames it reaches are quiescent and alive at read time.
|
||||
#[cfg(unix)]
|
||||
pub top_frame: AtomicPtr<Frame>,
|
||||
pub top_frame: AtomicPtr<FrameObject>,
|
||||
/// Raw InterpreterFrame pointer, published alongside top_frame so
|
||||
/// cross-thread readers (sys._current_frames) can materialize
|
||||
/// stack-allocated frames that have no FrameObject.
|
||||
pub top_iframe: AtomicUsize,
|
||||
/// Raw frame pointers, valid while the owning thread's call stack is active.
|
||||
/// Readers must hold the Mutex and convert to FrameRef inside the lock.
|
||||
/// Readers must hold the Mutex and convert to FrameObjectRef inside the lock.
|
||||
/// Used on non-unix threading builds, which have no stop-the-world.
|
||||
#[cfg(not(unix))]
|
||||
pub frames: parking_lot::Mutex<Vec<FramePtr>>,
|
||||
@@ -82,12 +95,11 @@ thread_local! {
|
||||
static CURRENT_THREAD_SLOT: RefCell<Option<CurrentFrameSlot>> = const { RefCell::new(None) };
|
||||
|
||||
/// Current top frame for signal-safe traceback walking.
|
||||
/// Mirrors `PyThreadState.current_frame`. Read by faulthandler's signal
|
||||
/// handler to dump tracebacks without accessing RefCell or locks.
|
||||
/// Uses AtomicPtr for async-signal-safety (signal handlers may read this
|
||||
/// while the owning thread is writing).
|
||||
pub(crate) static CURRENT_FRAME: AtomicPtr<Frame> =
|
||||
const { AtomicPtr::new(core::ptr::null_mut()) };
|
||||
/// Stores a `*const InterpreterFrame` as `usize`.
|
||||
/// Read by faulthandler's signal handler to dump tracebacks without
|
||||
/// accessing RefCell or locks. Uses AtomicUsize for async-signal-safety.
|
||||
pub(crate) static CURRENT_FRAME: AtomicUsize =
|
||||
const { AtomicUsize::new(0) };
|
||||
|
||||
/// Cached pointer to this thread's `ThreadSlot::top_frame`, so the hot
|
||||
/// push/pop path can publish the top frame with a single relaxed store and
|
||||
@@ -95,7 +107,7 @@ thread_local! {
|
||||
/// initialized; the `Arc<ThreadSlot>` in `CURRENT_THREAD_SLOT` keeps the
|
||||
/// pointee alive until `cleanup_current_thread_frames` clears this.
|
||||
#[cfg(all(unix, feature = "threading"))]
|
||||
static CURRENT_TOP_FRAME_SLOT: Cell<*const AtomicPtr<Frame>> =
|
||||
static CURRENT_TOP_FRAME_SLOT: Cell<*const AtomicPtr<FrameObject>> =
|
||||
const { Cell::new(core::ptr::null()) };
|
||||
|
||||
}
|
||||
@@ -339,6 +351,7 @@ fn init_thread_slot_if_needed(vm: &VirtualMachine) {
|
||||
let new_slot = Arc::new(ThreadSlot {
|
||||
#[cfg(unix)]
|
||||
top_frame: AtomicPtr::new(core::ptr::null_mut()),
|
||||
top_iframe: AtomicUsize::new(0),
|
||||
#[cfg(not(unix))]
|
||||
frames: parking_lot::Mutex::new(Vec::new()),
|
||||
exception: crate::PyAtomicRef::from(None::<PyBaseExceptionRef>),
|
||||
@@ -670,28 +683,56 @@ pub fn pop_thread_frame() {
|
||||
});
|
||||
}
|
||||
|
||||
/// Set the current thread's top frame pointer for signal-safe traceback walking.
|
||||
/// Returns the previous frame pointer so it can be restored on pop.
|
||||
pub fn set_current_frame(frame: *const Frame) -> *const Frame {
|
||||
// Publish the top frame for cross-thread readers. The relaxed store is
|
||||
// ordered by stop-the-world at read time (see `ThreadSlot::top_frame`).
|
||||
#[cfg(all(unix, feature = "threading"))]
|
||||
/// Set the current thread's top InterpreterFrame pointer.
|
||||
/// Returns the previous pointer so it can be restored on pop.
|
||||
#[must_use]
|
||||
#[allow(clippy::not_unsafe_ptr_arg_deref)]
|
||||
pub fn set_current_frame(frame: *const InterpreterFrame) -> *const InterpreterFrame {
|
||||
// Publish the top frame for cross-thread readers (faulthandler,
|
||||
// sys._current_frames).
|
||||
#[cfg(feature = "threading")]
|
||||
{
|
||||
let slot_top = CURRENT_TOP_FRAME_SLOT.with(Cell::get);
|
||||
if !slot_top.is_null() {
|
||||
// SAFETY: points to this thread's `ThreadSlot::top_frame`, kept
|
||||
// alive by the Arc in `CURRENT_THREAD_SLOT` for the thread's life.
|
||||
unsafe { (*slot_top).store(frame as *mut Frame, Ordering::Relaxed) };
|
||||
}
|
||||
CURRENT_THREAD_SLOT.with(|slot| {
|
||||
if let Some(s) = slot.borrow().as_ref() {
|
||||
if !frame.is_null() {
|
||||
#[cfg(unix)]
|
||||
{
|
||||
let frame_obj = unsafe { (*frame).frame_obj() };
|
||||
let fo_ptr = match frame_obj {
|
||||
Some(py) => {
|
||||
py as *const Py<FrameObject> as *const FrameObject
|
||||
as *mut FrameObject
|
||||
}
|
||||
None => core::ptr::null_mut(),
|
||||
};
|
||||
s.top_frame.store(fo_ptr, Ordering::Relaxed);
|
||||
}
|
||||
s.top_iframe.store(frame as usize, Ordering::Relaxed);
|
||||
} else {
|
||||
#[cfg(unix)]
|
||||
s.top_frame.store(core::ptr::null_mut(), Ordering::Relaxed);
|
||||
s.top_iframe.store(0, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
CURRENT_FRAME.with(|c| c.swap(frame as *mut Frame, Ordering::Relaxed) as *const Frame)
|
||||
CURRENT_FRAME.with(|c| c.swap(frame as usize, Ordering::Relaxed)) as *const InterpreterFrame
|
||||
}
|
||||
|
||||
/// Get the current thread's top frame pointer.
|
||||
/// Lightweight version that only writes to TLS CURRENT_FRAME, returning
|
||||
/// the previous value. Does not update cross-thread top_frame (that's
|
||||
/// updated by `set_current_frame` for FrameObject-based calls).
|
||||
#[inline(always)]
|
||||
#[must_use]
|
||||
pub fn set_current_frame_nosave(frame: *const InterpreterFrame) -> *const InterpreterFrame {
|
||||
CURRENT_FRAME.with(|c| c.swap(frame as usize, Ordering::Relaxed)) as *const InterpreterFrame
|
||||
}
|
||||
|
||||
/// Get the current thread's top InterpreterFrame pointer.
|
||||
/// Used by faulthandler's signal handler to start traceback walking.
|
||||
#[must_use]
|
||||
pub fn get_current_frame() -> *const Frame {
|
||||
CURRENT_FRAME.with(|c| c.load(Ordering::Relaxed) as *const Frame)
|
||||
pub fn get_current_frame() -> *const InterpreterFrame {
|
||||
CURRENT_FRAME.with(|c| c.load(Ordering::Relaxed)) as *const InterpreterFrame
|
||||
}
|
||||
|
||||
/// Update the current thread's exception slot atomically (no locks).
|
||||
@@ -784,18 +825,36 @@ pub fn reinit_frame_slot_after_fork(vm: &VirtualMachine) {
|
||||
let mut cur = get_current_frame();
|
||||
while !cur.is_null() {
|
||||
// SAFETY: the forking thread's chain frames are alive.
|
||||
let py = unsafe { crate::Py::<Frame>::from_payload_ptr(cur) };
|
||||
current_frames.push(FramePtr(unsafe { NonNull::new_unchecked(py as *mut _) }));
|
||||
cur = unsafe { (*cur).previous_frame() };
|
||||
let iframe = unsafe { &*cur };
|
||||
if let Some(fo) = iframe.frame_obj() {
|
||||
current_frames.push(FramePtr(unsafe {
|
||||
NonNull::new_unchecked(fo as *const _ as *mut _)
|
||||
}));
|
||||
}
|
||||
cur = iframe.previous.load(Ordering::Relaxed) as *const InterpreterFrame;
|
||||
}
|
||||
current_frames.reverse();
|
||||
current_frames
|
||||
};
|
||||
#[cfg(unix)]
|
||||
let top_fo_ptr = {
|
||||
let top_iframe = get_current_frame();
|
||||
if top_iframe.is_null() {
|
||||
core::ptr::null_mut()
|
||||
} else {
|
||||
match unsafe { (*top_iframe).frame_obj() } {
|
||||
Some(fo) => fo as *const Py<FrameObject> as *const FrameObject as *mut FrameObject,
|
||||
None => core::ptr::null_mut(),
|
||||
}
|
||||
}
|
||||
};
|
||||
let top_iframe_ptr = get_current_frame() as usize;
|
||||
let new_slot = Arc::new(ThreadSlot {
|
||||
// The surviving child thread keeps executing its current frame chain,
|
||||
// whose top is the signal-safe `get_current_frame()`.
|
||||
// The surviving child thread keeps executing its current frame chain.
|
||||
// Only publish heavy frames for signal safety.
|
||||
#[cfg(unix)]
|
||||
top_frame: AtomicPtr::new(get_current_frame() as *mut Frame),
|
||||
top_frame: AtomicPtr::new(top_fo_ptr),
|
||||
top_iframe: AtomicUsize::new(top_iframe_ptr),
|
||||
#[cfg(not(unix))]
|
||||
frames: parking_lot::Mutex::new(current_frames),
|
||||
exception: crate::PyAtomicRef::from(vm.topmost_exception()),
|
||||
|
||||
+18
-8
@@ -513,7 +513,7 @@ fn show_warning(
|
||||
}
|
||||
|
||||
/// Check if a frame's filename starts with any of the given prefixes.
|
||||
fn is_filename_to_skip(frame: &crate::frame::Frame, prefixes: &PyTupleRef) -> bool {
|
||||
fn is_filename_to_skip(frame: &crate::frame::FrameObject, prefixes: &PyTupleRef) -> bool {
|
||||
let filename = frame.f_code().co_filename();
|
||||
let filename_bytes = filename.as_bytes();
|
||||
prefixes.iter().any(|prefix| {
|
||||
@@ -523,15 +523,15 @@ fn is_filename_to_skip(frame: &crate::frame::Frame, prefixes: &PyTupleRef) -> bo
|
||||
})
|
||||
}
|
||||
|
||||
/// Like Frame::next_external_frame but also skips frames matching prefixes.
|
||||
/// Like FrameObject::next_external_frame but also skips frames matching prefixes.
|
||||
fn next_external_frame_with_skip(
|
||||
frame: &crate::frame::FrameRef,
|
||||
frame: &crate::frame::FrameObjectRef,
|
||||
skip_file_prefixes: Option<&PyTupleRef>,
|
||||
vm: &VirtualMachine,
|
||||
) -> Option<crate::frame::FrameRef> {
|
||||
) -> Option<crate::frame::FrameObjectRef> {
|
||||
let mut f = frame.f_back(vm);
|
||||
loop {
|
||||
let current: crate::frame::FrameRef = f.take()?;
|
||||
let current: crate::frame::FrameObjectRef = f.take()?;
|
||||
if current.is_internal_frame()
|
||||
|| skip_file_prefixes.is_some_and(|p| is_filename_to_skip(¤t, p))
|
||||
{
|
||||
@@ -549,7 +549,9 @@ fn setup_context(
|
||||
skip_file_prefixes: Option<&PyTupleRef>,
|
||||
vm: &VirtualMachine,
|
||||
) -> PyResult<(PyStrRef, usize, Option<PyObjectRef>, PyObjectRef)> {
|
||||
let mut f = vm.current_frame();
|
||||
// Materialize the topmost frame (including light frames) so stack
|
||||
// level counting is correct across the full Python frame chain.
|
||||
let mut f = crate::frame::current_thread_frame_materialize(vm);
|
||||
|
||||
// Stack level comparisons to Python code is off by one as there is no
|
||||
// warnings-related stack level to avoid.
|
||||
@@ -576,10 +578,18 @@ fn setup_context(
|
||||
}
|
||||
|
||||
let (globals, filename, lineno) = if let Some(f) = f {
|
||||
(f.globals.clone(), f.code.source_path(), f.f_lineno())
|
||||
(
|
||||
f.iframe().globals().to_owned(),
|
||||
f.iframe().code().source_path(),
|
||||
f.f_lineno(),
|
||||
)
|
||||
} else if let Some(frame) = vm.current_frame() {
|
||||
// We have a frame but it wasn't found during stack walking
|
||||
(frame.globals.clone(), vm.ctx.intern_str("<sys>"), 1)
|
||||
(
|
||||
frame.iframe().globals().to_owned(),
|
||||
vm.ctx.intern_str("<sys>"),
|
||||
1,
|
||||
)
|
||||
} else {
|
||||
// No frames on the stack - use sys.__dict__ (interp->sysdict)
|
||||
let globals = vm
|
||||
|
||||
Reference in new issue
Block a user