* Align codegen CFG cleanup with CPython
* Align codegen bytecode with CPython 3.14
* Remove remove_jump_target_line_nops
The pass dropped a same-line NOP at the start of any jump-targeted block
whose body advanced to a later line, but CPython's basicblock_remove_redundant_nops
only consults the previous/next instruction inside the same block. The
extra pass deleted the else-body line trace anchor exercised by
test_nested_double_async_with.
Ignore test_conditional_break_finally_does_not_keep_break_cleanup_nop;
the break NOP lands in a separate block from the inlined finally body,
so same-line successor elision does not apply here.
* Keep except_handler blocks reachable in eliminate_unreachable_blocks
After convert_pseudo_ops lowers SETUP_FINALLY to a plain NOP, the only
remaining link from the try body to the except_handler block was the
per-instruction except_handler annotation. When earlier passes had
already removed every NOP that carried that annotation (e.g. an empty
try body with `pass`), the handler block became unreachable from the
entry block and its instructions were cleared, dropping the handler
entirely. Seed reachability with blocks already marked except_handler
so handler dispatch survives independent of the in-block annotation.
Also drop two expectedFailure markers in test_patma whose match-tracing
expectations now pass.
* Align bytecode CFG cleanup with CPython
* Add tests for Mode::BlockExpr and VirtualMachine::run_block_expr
test_block_expr_return_const() panics currently.
* Fix "tried to pop from empty stack" with BlockExpr and const
Constant expressions without side effects are optimized away unless in
an interactive mode[1]. This causes panic when last statement in a block
is a constant and Mode::BlockExpr is used as this constant value must be
returned to caller.
Example panic:
---- vm::python_run::tests::test_block_expr_return_const stdout ----
[crates/vm/src/frame.rs:9653:9] self = ExecutingFrame {
code: code: <code object <module> at ??? file "<embedded>", line 1>,
stack_len: 0,
}
thread 'vm::python_run::tests::test_block_expr_return_const' (2640752) panicked at crates/vm/src/frame.rs:9410:18:
tried to pop from empty stack
stack backtrace:
0: __rustc::rust_begin_unwind
at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/panicking.rs:689:5
1: core::panicking::panic_fmt
at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/core/src/panicking.rs:80:14
2: rustpython_vm::frame::ExecutingFrame::fatal
at ./src/frame.rs:9654:9
3: rustpython_vm::frame::ExecutingFrame::pop_stackref_opt
at ./src/frame.rs:9410:18
4: rustpython_vm::frame::ExecutingFrame::pop_stackref
at ./src/frame.rs:9420:18
5: rustpython_vm::frame::ExecutingFrame::pop_value
at ./src/frame.rs:9435:14
6: rustpython_vm::frame::ExecutingFrame::execute_instruction
at ./src/frame.rs:3391:34
7: rustpython_vm::frame::ExecutingFrame::run
at ./src/frame.rs:1623:31
8: rustpython_vm::frame::<impl rustpython_vm::object::core::Py<rustpython_vm::frame::Frame>>::run::{{closure}}
at ./src/frame.rs:1098:44
9: rustpython_vm::frame::<impl rustpython_vm::object::core::Py<rustpython_vm::frame::Frame>>::with_exec
at ./src/frame.rs:1093:9
10: rustpython_vm::frame::<impl rustpython_vm::object::core::Py<rustpython_vm::frame::Frame>>::run
at ./src/frame.rs:1098:14
11: rustpython_vm::vm::VirtualMachine::run_frame::{{closure}}
at ./src/vm/mod.rs:1201:44
12: rustpython_vm::vm::VirtualMachine::with_frame_impl::{{closure}}::{{closure}}
at ./src/vm/mod.rs:1610:60
13: rustpython_vm::vm::VirtualMachine::dispatch_traced_frame
at ./src/vm/mod.rs:1689:22
14: rustpython_vm::vm::VirtualMachine::with_frame_impl::{{closure}}
at ./src/vm/mod.rs:1610:22
15: rustpython_vm::vm::VirtualMachine::with_recursion
at ./src/vm/mod.rs:1547:9
16: rustpython_vm::vm::VirtualMachine::with_frame_impl
at ./src/vm/mod.rs:1572:14
17: rustpython_vm::vm::VirtualMachine::with_frame
at ./src/vm/mod.rs:1555:14
18: rustpython_vm::vm::VirtualMachine::run_frame
at ./src/vm/mod.rs:1201:20
19: rustpython_vm::vm::VirtualMachine::run_code_obj
at ./src/vm/mod.rs:1120:14
20: rustpython_vm::vm::python_run::<impl rustpython_vm::vm::VirtualMachine>::run_block_expr
at ./src/vm/python_run.rs:51:14
21: rustpython_vm::vm::python_run::tests::test_block_expr_return_const::{{closure}}
at ./src/vm/python_run.rs:190:47
22: rustpython_vm::vm::interpreter::Interpreter::enter::{{closure}}
at ./src/vm/interpreter.rs:366:39
23: rustpython_vm::vm::thread::set_current_vm::{{closure}}
at ./src/vm/thread.rs:107:9
24: std::thread::local::LocalKey<T>::try_with
at /home/im/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:513:12
25: std::thread::local::LocalKey<T>::with
at /home/im/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:477:20
26: rustpython_vm::vm::thread::set_current_vm
at ./src/vm/thread.rs:102:14
27: rustpython_vm::vm::thread::enter_vm
at ./src/vm/thread.rs:132:5
28: rustpython_vm::vm::interpreter::Interpreter::enter
at ./src/vm/interpreter.rs:366:9
29: rustpython_vm::vm::python_run::tests::test_block_expr_return_const
at ./src/vm/python_run.rs:188:23
30: rustpython_vm::vm::python_run::tests::test_block_expr_return_const::{{closure}}
at ./src/vm/python_run.rs:187:38
31: core::ops::function::FnOnce::call_once
at /home/im/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
32: <fn() -> core::result::Result<(), alloc::string::String> as core::ops::function::FnOnce<()>>::call_once
at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/core/src/ops/function.rs:250:5
[1] https://github.com/RustPython/RustPython/blob/883ce9d273db98b435eb3f51128925a1255ba88d/crates/codegen/src/compile.rs#L2990-L2997
* Align LOAD_FAST_BORROW analysis with CPython chain shape
Three changes that bring optimize_load_fast_borrow closer to CPython's
optimize_load_fast in flowgraph.c:
* ir.rs: split mark_cold into the CPython-style two passes. Phase 1
propagates "warm" from the entry block, phase 2 propagates "cold" from
except_handler blocks. Blocks reached by neither phase keep cold=false
and stay in their original b_next position, matching CPython's handling
of empty placeholders left by remove_unreachable (e.g. the inner_end of
a nested try/except whose incoming jumps were re-routed by optimize_cfg).
* ir.rs: in optimize_load_fast_borrow, push the fall-through successor
only when the current block has a last instruction (is_some_and).
Empty blocks now terminate fall-through propagation, matching the
`term != NULL` check in optimize_load_fast.
* compile.rs: add switch_to_new_or_reuse_empty() helper and use it in
compile_while. The helper reuses the current block when it is empty
and unlinked, mirroring USE_LABEL absorption in
cfg_builder_maybe_start_new_block. This stops a stray empty block
from appearing between e.g. a try/except end_block and the following
while loop header.
Four codegen tests that depended on the previous fall-through-through-
empty behavior are marked #[ignore] with TODO comments.
Also includes a handful of dictionary entries in .cspell.dict picked up
during the work.
* Interleave const fold passes per-block to match CPython
Mirror CPython's optimize_basic_block() (flowgraph.c) by walking each
block once in instruction order and trying tuple, list, set, unary, and
binop folding at each position before advancing. This replaces the
previous global-pass sequence where every fold_unary_constants pattern
in the whole CFG was registered before any tuple constant, leaving
negated literals like `-1` at co_consts positions earlier than CPython
produces (e.g. snippets.py: -1 at idx 280 vs CPython idx 726).
Changes:
- Extract `fold_unary_constant_at` and `fold_binop_constant_at` per-
position helpers from the existing global passes; the global passes
now call the helpers in a loop.
- Add `fold_constants_per_block` that walks each block to a fixed point,
trying all five folds at each instruction position.
- Call the new walker before the legacy global passes in
optimize_finalize so co_consts insertion order matches CPython's.
Measured on the full Lib tree: differing files 270 → 269; the only
newly matching file is `test/test_ast/snippets.py`, the case raised in
youknowone/RustPython#28.
* Inline small fast-return blocks only through unconditional jumps
`inline_small_fast_return_blocks` previously appended the target
`LOAD_FAST(_BORROW)/RETURN_VALUE` block's instructions onto any
predecessor whose fall-through eventually reached it, in addition to the
unconditional-jump case CPython handles in
`inline_small_or_no_lineno_blocks` (flowgraph.c:1210). CPython only
inlines through unconditional jumps, leaving fall-through predecessors
to reach the shared return block via the natural CFG layout. The extra
fall-through branch duplicated the return tail (e.g. `if/elif/return`
emitted two adjacent `LOAD_FAST_BORROW x; RETURN_VALUE` sequences).
Remove the fall-through inlining branch and keep only the
unconditional-jump path.
Measured on the full Lib tree: differing files 270 → 239 (-31), no new
regressions. Files newly matching include copy.py, argparse.py,
dataclasses.py, logging/__init__.py, pathlib/__init__.py, etc.
* Allow scope-exit/jump-back reorder within a shared except handler
`reorder_conditional_scope_exit_and_jump_back_blocks` previously skipped
any reorder where the conditional, scope-exit, or jump-back block had
an `except_handler` attached, even when all three shared the same
handler. CPython reorders these regardless of try/except context, as
long as the blocks stay within the same protected region. The over-
conservative guard left patterns like `try: for: if cond: return` with
the loop body's scope-exit ahead of the backedge, while CPython places
the backedge first and inverts the conditional.
Replace the `block_is_protected` triple-check with a single
`mismatched_protection` test: skip only when the three blocks do not
share the same `except_handler`. Same-handler reorders preserve the
protected range because every instruction's `except_handler` field
stays attached as `.next` pointers shift.
Measured on the full Lib tree: differing files 239 → 237; no new
regressions.
* Skip jump-over-cleanup reorder when target restarts an exception scope
reorder_jump_over_exception_cleanup_blocks was swapping a small
scope-exit target with a preceding cold cleanup chain even when the
target block began a fresh try (SETUP_FINALLY/SETUP_CLEANUP/SETUP_WITH).
The swap moved the next try's setup ahead of the prior handler's
cleanup_end/next_handler/cleanup_block, making the cleanup_body's
JUMP_FORWARD fall through directly to the cleanup_end and get elided as
redundant. The bytecode then lacked the JUMP_FORWARD that skips the
cleanup blocks and matched the prior handler's borrow tail incorrectly.
Skip the reorder when the target block contains any block-push pseudo
op so a new try's setup stays in source order. Re-enables the four
named/typed except-cleanup borrow tests that were marked #[ignore] in
commit 7481459ea.
* fix if block_idx == BlockIdx::NULL
* Remove unused rust impl for formatting dis output
* remove `examples/dis.rs`
* Added tests
* Update lock
* Try to set snapshot dir
* Remove verbose flag
* Regenerate snapshots after #7711
* Revert "Bump insta from 1.46.3 to 1.47.2 (#7706)"
This reverts commit e6d9ea6bfe.
* Debug info
* Show diff as well
* Show debug faster
* CI: true env
* Recert CI
* Add `CI: true` in ci emv
* Reapply "Bump insta from 1.46.3 to 1.47.2 (#7706)"
This reverts commit 693ca8cbe4d7885a81162a9be31e8bb567db885a.
* simplify macro
* trim on function side
* Force insta workspace root
* fix merge
CPython convention: top-level module / interactive / expression code
does not carry CO_NEWLOCALS or CO_OPTIMIZED. The per-scope mapping at
enter_scope::CompilerScope::Module already returns empty flags, but
Compiler::new seeded the root CodeInfo with CodeFlags::NEWLOCALS,
forcing module code into the NEWLOCALS arm of frame.rs:725-731 so
locals were allocated as a fresh empty dict instead of being bound to
globals (the correct semantics for exec(code, globals)).
Restore the seed to empty() so it matches the per-scope mapping and
CPython's compiler_enter_scope for module scope.
* Align CFG cleanup bytecode with CPython
* Bytecode parity: fblock unwind, fstring join, folding, scope
- compile.rs: unwind_fblock_stack returns whether a finally ran so
return-statement emission can adjust location handling; restructure
try/except/finally cleanup to preserve or drop boundary NOPs based on
whether the body falls through; rework f-string lowering with
count/join helpers; remove the per-collection-type heuristic for
AST-level folding and defer to flowgraph passes; add several folding
helpers and a ComprehensionLoopControl enum.
- ir.rs: re-run unary/binop folding around tuple folding, add
reorder_conditional_scope_exit_and_jump_back_blocks and several block
classification helpers, add MAX_STR_SIZE, change is_exit_without_lineno
to take the block list.
- symboltable.rs: in analyze_cells, remove names owned as cells in
function-like scopes from the parent's free set; mark lambda scope
type explicitly.
* Refine CFG scope-exit backedge ordering
* typealias reviews
* Bytecode parity - try/except block order, CFG reorder
Reorder try/except/else/finally to emit else+finally before
except handlers matching CPython layout. Add set_no_location
for cleanup blocks. Extend CFG reorder pass to handle true-path
jump-back for generators, break/continue, and assert in loops.
Add stop-iteration error handler awareness to block protection.
* Align CFG cleanup bytecode with CPython
* Unmark test_dis.test_findlabels expected failure
* Compute target predecessor flags in single pass
remove_nops and remove_redundant_nops_in_blocks repeated
has_jump_predecessor / has_plain_jump_predecessor / target lookups
per block, scanning all blocks each time. With ~200,000 if blocks
this became O(B^2 * I) and timed out test_compile.test_stack_overflow.
Fold the three flags into one O(B * I) pass via
compute_target_predecessor_flags.
* Address review feedback on slice folding, fallthrough, attr-chain
- try_fold_constant_slice now delegates to try_fold_constant_expr so
slice bounds accept the same constants other folding paths do
(unary-folded values, __debug__, etc.).
- remove_nops resolves fallthrough predecessors through empty blocks
via next_nonempty_block before checking ends_with_for_cleanup.
- should_deopt_borrowed_attr_chain ReturnIter matcher now accepts
Instruction::CallKw alongside Instruction::Call, matching the
Call/CallKw treatment in the surrounding deopt trigger check.
* remove test
Constant folding:
- Add string/bytes multiply and bytes concat folding in IR
- Add constant subscript folding (str, bytes, tuple indexing)
- Delegate list/set constant folding to IR passes
- Stream big non-const list/set via BUILD+LIST_APPEND
Class/generic compilation:
- Reorder class body prologue: __type_params__ before __classdict__
- Build class function before .generic_base in generic classes
- Register .type_params/.generic_base symbols in proper scopes
- Use load_name/store_name helpers for synthetic variables
Return block handling:
- Only duplicate return-None epilogues, not arbitrary returns
- Add inline_pop_except_return_blocks pass
- Add duplicate_named_except_cleanup_returns pass
Other fixes:
- Fix eliminate_dead_stores to only collapse adjacent duplicates
- Skip STORE_FAST_LOAD_FAST superinstruction in generators after FOR_ITER
- Thread jumps through NOP-only blocks
- Transfer NOP line info to following unconditional jumps
- Extract scope_needs_conditional_annotations_cell helper
- Register __conditional_annotations__ for module future annotations
- Remove unnecessary NOP between FOR_ITER and unpack/store
by compiling loop target directly on target range
- Fix t-string compilation to match stack order: build
strings tuple first, then evaluate interpolations
- Split compile_tstring_into into collect_tstring_strings
and compile_tstring_interpolations
- Handle debug text literals and default repr conversion
for debug specifier in t-strings
- Always set bit 1 in BUILD_INTERPOLATION oparg encoding
- Add inline_with_suppress_return_blocks pass to inline return
epilogues after with-suppress cleanup sequences
- Extend duplicate_end_returns to handle conditional jumps to the
final return block, not just unconditional ones
- Process jump targets in reverse order to preserve indices
- Add extra deoptimize_store_fast_store_fast pass after superinstructions
- Add tests for listcomp cleanup tail and with-suppress tail
* Align bytecode codegen structure with CPython 3.14
* Bytecode parity - constant folding, annotation ordering, superinstruction alignment
- Add BoolOp constant folding with short-circuit semantics in compile_expression
- Add constant truthiness evaluation for assert statement optimization
- Disable const collection/boolop folding in starred unpack and assignment contexts
- Move annotation block generation after body with AnnotationsPlaceholder splicing
- Reorder insert_superinstructions to run before push_cold_blocks (matching flowgraph.c)
- Lower LOAD_CLOSURE after superinstructions to avoid false LOAD_FAST_LOAD_FAST
- Add ToBool before PopJumpIf in comparisons and chained compare cleanup blocks
- Unify annotation dict building to always use incremental BuildMap + StoreSubscr
- Add TrueDivide constant folding for integer operands
- Fold constant sets to Frozenset (not Tuple) in try_fold_constant_collection
- Add PyVmBag for frozenset constant materialization in code objects
- Add remove_redundant_const_pop_top_pairs pass and peephole const+branch folding
- Emit Nop for skipped constant expressions and constant-true asserts
- Preserve comprehension local ordering by source-order bound name collection
- Simplify annotation scanning in symboltable (remove simple-name gate)
* Fix CI regressions in marshal and fast-local ops
* impl more
* Align bytecode codegen with CPython structure
* Bytecode parity - comprehension/except scope ordering, load_fast_borrow fixes
- Reorder comprehension symbol-table walk so the outermost iterator
registers its sub_tables in the enclosing scope before the comp
scope, and rescan elt/ifs in CPython's order. Codegen peeks past the
outermost iterator's nested scopes to find the comprehension table.
- For plain try/except, emit handler sub_tables before the else block
so codegen's linear sub_table cursor stays aligned.
- Rename `collect_simple_annotations` to `collect_annotations` and
evaluate non-simple annotations during __annotate__ compilation to
preserve source-order side effects while keeping the simple-name
index stable.
- Dedupe equivalent code constants in `arg_constant` and add a
structural equality check on `CodeObject`.
- Disable LOAD_FAST_BORROW for the tail end block when a try has a
bare `except:` clause, and have `new_block` inherit the flag from
the current block.
- Remove `cfg!(debug_assertions)` guard around the
`optimize_load_fast_borrow` start-depth check so mismatches are
handled (return instead of assert) in release builds.
- Collapse nop-only blocks that precede a return epilogue and hoist
the prior line number into the next real instruction so the
line table matches.
- Unmark now-passing `test_consts_in_conditionals`,
`test_load_fast_unknown_simple`,
`test_load_fast_known_because_already_loaded`, and PEP 646 f3/f4
annotation checks.
* Bytecode parity - try/except line tracking, assert 0 shape
- In `compile_try_except`, drop the leading Nop and set the end
block's source range from the last orelse/body statement so line
events after the try fall on the right line.
- Recognise constant-false asserts as the direct-raise shape (no
ToBool/PopJumpIfFalse) and flip the test assertion accordingly.
- Extend `remove_redundant_nops_in_blocks` to also look through a
trailing nop before a return-epilogue pair (LoadConst/ReturnValue
or LoadSmallInt/ReturnValue) so the epilogue keeps the correct
line number.
- Rename `preds` to `predecessor_blocks` in the LOAD_FAST_BORROW
disable pass and add a test-only `debug_late_cfg_trace` helper.
- Regenerate the `nested_double_async_with` snapshot: the tail
reference to `stop_exc` now emits LOAD_FAST instead of
LOAD_FAST_BORROW.
* Bytecode parity - iter folding, break/continue line, cold inlining
- Fold a constant list iterable into a constant tuple in for-loop
iterable position, matching the CPython optimizer, and strip a
redundant LIST_TO_TUPLE immediately before GET_ITER in the IR
peephole pass.
- Emit a Nop at the break/continue source range before unwinding
so line events land on the break/continue statement instead of
the following instruction.
- Drop `propagate_disable_load_fast_borrow`; the forward propagation
was over-zealous and the per-block inheritance in `new_block` plus
the bare-except marker are enough.
- Relax `inline_small_or_no_lineno_blocks` so small exit blocks at
the tail of a cold block are always inlined, not just return
epilogues.
- Add codegen tests covering the LIST_TO_TUPLE/GET_ITER peephole and
the late-CFG trace helper for a for-loop list-literal iterable.
* Add InstructionMetadata::stack_effect_jump for branch stack effects
CPython's compile.c provides stack_effect(opcode, oparg, jump) where the
jump parameter selects between fallthrough and branch effects. The existing
stack_effect() only returns the fallthrough effect.
Add stack_effect_jump() that returns the branch effect. Most instructions
have identical fallthrough/branch effects; ForIter and Send are the
exceptions (ForIter: fallthrough=+1, branch=-1; Send: fallthrough=0,
branch=-1).
* apply review
- Use POP_TOP instead of POP_ITER for for-loop break/return cleanup
- Expand duplicate_end_returns to clone final return for jump predecessors
- Restrict late jump threading pass to unconditional jumps only
- Skip exception blocks in inline/reorder passes
- Simplify threaded_jump_instr NoInterrupt handling
* Fix exception handling: except* chaining, finally cleanup, RERAISE
- Align except* bytecode chaining
- Fix exception state model and finally handler cleanup
- Fix RERAISE to only pop exception, preserve values below
* Port IR optimization passes from flowgraph.c
- BUILD_TUPLE n + UNPACK_SEQUENCE n elimination
- Dead store elimination within basic blocks
- apply_static_swaps for SWAP reduction
* Add bytecode comparison and disassembly dump scripts
- compare_bytecode.py: compare CPython vs RustPython bytecode output
- dis_dump.py: extract disassembly in normalized JSON format
* Base resume context
* Fixes for api change
* Align codegen
* Align `frame.rs` to the api changes
* fix jit
* Use new oparg
* Fix doc
* let `ir` to decide exception depth
* Bytecode parity phase 3
Compiler changes:
- Emit TO_BOOL in and/or short-circuit evaluation (COPY+TO_BOOL+JUMP)
- Add module-level __conditional_annotations__ cell (PEP 649)
- Only set conditional annotations for AnnAssign, not function params
- Skip __classdict__ cell when future annotations are active
- Convert list literals to tuples in for-loop iterables
- Fix cell variable ordering: parameters first, then alphabetical
- Fix RESUME DEPTH1 flag for yield-from/await
- Don't propagate __classdict__/__conditional_annotations__ freevar
through regular functions — only annotation/type-param scopes
- Inline string compilation path
* Skip test_thread_safety in _test_multiprocessing
SIGSEGV in _finalizer_registry dict access under aggressive GC
and thread switching. Root cause is dict thread-safety in VM.
* Skip list→tuple optimization for async for; propagate future_annotations to nested scopes
* Match CPython LOAD_SPECIAL stack semantics for with/async-with
LOAD_SPECIAL now pushes (callable, self_or_null) matching CPython's
CALL convention, instead of a single bound method:
- Function descriptors: push (func, self)
- Plain attributes: push (bound, NULL)
Updated all with-statement paths:
- Entry: add SWAP 3 after SWAP 2, remove PUSH_NULL before CALL 0
- Normal exit: remove PUSH_NULL before CALL 3
- Exception handler (WITH_EXCEPT_START): read exit_func at TOS-4
and self_or_null at TOS-3
- Suppress block: 3 POP_TOPs after POP_EXCEPT (was 2)
- FBlock exit (preserve_tos): SWAP 3 + SWAP 2 rotation
- UnwindAction::With: remove PUSH_NULL
Stack effects updated: LoadSpecial (2,1), WithExceptStart (7,6)
* Normalize LOAD_FAST_CHECK and JUMP_BACKWARD_NO_INTERRUPT
Add LOAD_FAST_CHECK → LOAD_FAST and JUMP_BACKWARD_NO_INTERRUPT →
JUMP_BACKWARD to opname normalization in dis_dump.py. These are
optimization variants with identical semantics.
* Add EXTENDED_ARG to SKIP_OPS, normalize LOAD_FAST_CHECK and JUMP_BACKWARD_NO_INTERRUPT
* Remove duplicate return-None when block already has return
Skip duplicate_end_returns for blocks that already end with
LOAD_CONST + RETURN_VALUE. Run DCE + unreachable elimination
after duplication to remove the now-unreachable original return
block.
* Improve __static_attributes__ collection accuracy
- Support tuple/list unpacking targets: (self.x, self.y) = val
- Skip @staticmethod and @classmethod decorated methods
- Use scan_target_for_attrs helper for recursive target scanning
* Use method mode for function-local import attribute calls
Function-local imports (scope is Local+IMPORTED) should use method
mode LOAD_ATTR like regular names, not plain mode. Only module/class
scope imports use plain LOAD_ATTR + PUSH_NULL.
* Optimize constant iterable before GET_ITER to LOAD_CONST tuple
Convert BUILD_LIST/SET 0 + LOAD_CONST + LIST_EXTEND/SET_UPDATE + GET_ITER
to just LOAD_CONST (tuple) + GET_ITER, matching CPython's optimization
for constant list/set literals in for-loop iterables.
Also fix is_name_imported to use method mode for function-local imports,
and improve __static_attributes__ accuracy (skip @classmethod/@staticmethod,
handle tuple/list unpacking targets).
* Fix cell variable ordering: parameters first, then alphabetical
CPython orders cell variables with parameter cells first (in
parameter definition order), then non-parameter cells sorted
alphabetically. Previously all cells were sorted alphabetically.
Also add for-loop iterable optimization: constant BUILD_LIST/SET
before GET_ITER is folded to just LOAD_CONST tuple.
* Emit COPY_FREE_VARS before MAKE_CELL matching CPython order
CPython emits COPY_FREE_VARS first, then MAKE_CELL instructions.
Previously RustPython emitted them in reverse order.
* Fix RESUME AfterYield encoding to match CPython 3.14 (value 5)
CPython 3.14 uses RESUME arg=5 for after-yield, not 1.
Also reorder COPY_FREE_VARS before MAKE_CELL and fix cell
variable ordering (parameters first, then alphabetical).
* Address code review feedback from #7481
- Set is_generator flag for generator expressions in scan_comprehension
- Fix posonlyargs priority in collect_static_attributes first param
- Add match statement support to scan_store_attrs
- Fix stale decorator stack comment
- Reorder NOP removal after fold_unary_negative for better collection folding
* Fold constant list/set/tuple literals in compiler
When all elements of a list/set/tuple literal are constants and
there are 3+ elements, fold them into a single constant:
- list: BUILD_LIST 0 + LOAD_CONST (tuple) + LIST_EXTEND 1
- set: BUILD_SET 0 + LOAD_CONST (tuple) + SET_UPDATE 1
- tuple: LOAD_CONST (tuple)
This matches CPython's compiler optimization and fixes the most
common bytecode difference (92/200 sampled files).
Also add bytecode comparison scripts (dis_dump.py, compare_bytecode.py)
for systematic parity tracking.
* Use BUILD_MAP 0 + MAP_ADD for large dicts (>= 16 pairs)
Match CPython's compiler behavior: dicts with 16+ key-value pairs
use BUILD_MAP 0 followed by MAP_ADD for each pair, instead of
pushing all keys/values on the stack and calling BUILD_MAP N.
* Fix clippy warnings and cargo fmt
* fix surrogate
* Bytecode parity
Compiler changes:
- Remove PUSH_NULL from decorator cal
ls, use CALL 0
- Collect __static_attributes__ from self.xxx = patterns
- Sort __static_attributes__ alphabetically
- Move __classdict__ init before __doc__ in class prologue
- Fold unary negative constants
- Fold constant list/set literals (3+ elements)
- Use BUILD_MAP 0 + MAP_ADD for 16+ dict pairs
- Always run peephole optimizer for s
uperinstructions
- Emit RETURN_GENERATOR for generator
functions
- Add is_generator flag to SymbolTabl
e
* Fix formatting and collapsible_if clippy warnings in compile.rs
* Fix clippy, fold_unary_negative chaining, and generator line tracing
- Replace irrefutable if-let with let for ExceptHandler
- Remove folded UNARY_NEGATIVE instead of replacing with NOP,
enabling chained negation folding
- Initialize prev_line to def line for generators/coroutines
to suppress spurious LINE events from preamble instructions
- Remove expectedFailure markers for now-passing tests
* Fix JIT StoreFastStoreFast, format, and remove expectedFailure markers
- Add StoreFastStoreFast handling in JIT instructions
- Fix cargo fmt in frame.rs
- Remove 11 expectedFailure markers for async jump tests in
test_sys_settrace that now pass
* Fix peephole optimizer: use NOP replacement instead of remove()
Using remove() shifts instruction indices and corrupts subsequent
references, causing "pop stackref but null found" panics at runtime.
Replace folded/combined instructions with NOP instead, which are
cleaned up by the existing remove_nops pass.
* Revert peephole_optimize to use remove() for chaining support
NOP replacement broke chaining of peephole optimizations (e.g.
LOAD_CONST+TO_BOOL then LOAD_CONST+UNARY_NOT for 'not True').
The remove() approach is used by upstream and works correctly here;
fold_unary_negative keeps NOP replacement since it doesn't need chaining.
* Fix StoreFastStoreFast to handle NULL from LoadFastAndClear
StoreFast uses pop_value_opt() to allow NULL values from
LoadFastAndClear in inlined comprehension cleanup paths.
StoreFastStoreFast must do the same, otherwise the peephole
optimizer's fusion of two StoreFast instructions panics when
restoring unbound locals after an inlined comprehension.
* Emit TO_BOOL before conditional jumps, fix class/module prologue
- Emit TO_BOOL before POP_JUMP_IF_TRUE/FALSE in the general case
of compile_jump_if (Compare expressions excluded since they
already produce a bool)
- Module-level __doc__: use STORE_NAME instead of STORE_GLOBAL
- Class body __module__: use LOAD_NAME instead of LOAD_GLOBAL
- Class body: store __firstlineno__ before __doc__
* Emit MAKE_CELL and COPY_FREE_VARS before RESUME
Emit MAKE_CELL for each cell variable and COPY_FREE_VARS N for
free variables at the start of each code object, before RESUME.
These instructions are no-ops in the VM but align the bytecode
with CPython 3.14's output.
* Emit __static_attributes__ at end of class bodies
Store a tuple of attribute names (currently always empty) as
__static_attributes__ in the class namespace, matching CPython
3.14's class body epilogue. Attribute name collection from
self.xxx accesses is a follow-up task.
* Remove expectedFailure from DictProxyTests iter tests
test_iter_keys, test_iter_values, test_iter_items now pass
because class bodies emit __static_attributes__ and
__firstlineno__, matching the expected dict key set.
* Use 1-based stack indexing for LIST_EXTEND, SET_UPDATE, etc.
Switch LIST_APPEND, LIST_EXTEND, SET_ADD, SET_UPDATE, MAP_ADD
from 0-based to 1-based stack depth argument, matching CPython's
PEEK(oparg) convention. Adjust the VM to subtract 1 before
calling nth_value.
* Use plain LOAD_ATTR + PUSH_NULL for calls on imported names
When the call target is an attribute of an imported name (e.g.,
logging.getLogger()), use plain LOAD_ATTR (method_flag=0) with
a separate PUSH_NULL instead of method-mode LOAD_ATTR. This
matches CPython 3.14's behavior which avoids the method call
optimization for module attribute access.
* Duplicate return-None epilogue for fall-through blocks
When the last block in a code object is exactly LOAD_CONST None +
RETURN_VALUE (the implicit return), duplicate these instructions
into blocks that would otherwise fall through to it. This matches
CPython 3.14's behavior of giving each code path its own explicit
return instruction.
* Run cargo fmt on ir.rs
* Remove expectedFailure from test_intrinsic_1 in test_dis
* Emit TO_BOOL before conditional jumps for all expressions including Compare
* Add __classdict__ cell for classes with function definitions
Set needs_classdict=true for class scopes that contain function
definitions (def/async def), matching CPython 3.14's behavior for
PEP 649 deferred annotation support. Also restore the Compare
expression check in compile_jump_if to skip TO_BOOL for comparison
operations.
* Emit __classdictcell__ store in class body epilogue
Store the __classdict__ cell reference as __classdictcell__ in
the class namespace when the class has __classdict__ as a cell
variable. Uses LOAD_DEREF (RustPython separates cell vars from
fast locals unlike CPython's unified array).
* Always run DCE to remove dead code after terminal instructions
Run basic dead code elimination (truncating instructions after
RETURN_VALUE/RAISE/JUMP within blocks) at all optimization
levels, not just optimize > 0. CPython always removes this dead
code during assembly.
* Restrict LOAD_ATTR plain mode to module/class scope imports
Only use plain LOAD_ATTR + PUSH_NULL for imports at module or
class scope. Function-local imports use method call mode LOAD_ATTR,
matching CPython 3.14's behavior.
* Eliminate unreachable blocks after jump normalization
Split DCE into two phases: (1) within-block truncation after
terminal instructions (always runs), (2) whole-block elimination
for blocks only reachable via fall-through from terminal blocks
(runs after normalize_jumps when dead jump instructions exist).
* Fold BUILD_TUPLE 0 into LOAD_CONST empty tuple
Convert BUILD_TUPLE with size 0 to LOAD_CONST () during constant
folding, matching CPython's optimization for empty tuple literals.
* Handle __classcell__ and __classdictcell__ in type.__new__
- Remove __classcell__ from class dict after setting the cell value
- Add __classdictcell__ handling: set cell to class namespace dict,
then remove from class dict
- Register __classdictcell__ identifier
- Use LoadClosure instead of LoadDeref for __classdictcell__ emission
- Reorder MakeFunctionFlag bits to match CPython
- Run ruff format on scripts
* Revert __classdict__ cell and __classdictcell__ changes
The __classdict__ cell addition (for classes with function defs)
and __classdictcell__ store caused cell initialization failures
in importlib. These require deeper VM changes to properly support
the cell variable lifecycle. Reverted for stability.
* Fix unreachable block elimination with fixpoint reachability
Use fixpoint iteration to properly determine block reachability:
only mark jump targets of already-reachable blocks, preventing
orphaned blocks from falsely marking their targets as reachable.
Also add a final DCE pass after assembly NOP removal to catch
dead code created by normalize_jumps.
* Check enclosing scopes for IMPORTED flag in LOAD_ATTR mode
When deciding whether to use plain LOAD_ATTR for attribute calls,
check if the name is imported in any enclosing scope (not just
the current scope). This handles the common pattern where a module
is imported at module level but used inside functions.
* Add __classdict__ cell for classes with function definitions
Set needs_classdict=true when a class scope contains function
definitions (def/async def), matching CPython 3.14 which always
creates a __classdict__ cell for PEP 649 support in such classes.
* Store __classdictcell__ in class body epilogue
Store the __classdict__ cell reference as __classdictcell__ in
the class namespace using LoadClosure (which loads the cell
object itself, not the value inside). This matches CPython 3.14's
class body epilogue.
* Fix clippy collapsible_if warnings and cargo fmt
* Revert __classdict__ and __classdictcell__ changes (cause import failures)
* Revert type.__new__ __classcell__ removal and __classdictcell__ handling
Revert the class cell cleanup changes from e6975f973 that cause
import failures when frozen module bytecode is stale. The original
behavior (not removing __classcell__ from class dict) is restored.
* Re-add __classdict__ cell and __classdictcell__ store
Restore the __classdict__ cell for classes with function
definitions and __classdictcell__ store in class body epilogue.
Previous failure was caused by stale .pyc cache files containing
bytecode from an intermediate MakeFunctionFlag reorder attempt,
not by these changes themselves.
* Reorder MakeFunctionFlag to match CPython's SET_FUNCTION_ATTRIBUTE
Reorder discriminants: Defaults=0, KwOnlyDefaults=1, Annotations=2,
Closure=3, Annotate=4, TypeParams=5. This aligns the oparg values
with CPython 3.14's convention.
Note: after this change, stale .pyc cache files must be deleted
(find . -name '*.pyc' -delete) to avoid bytecode mismatch errors.
* Use CPython-compatible power-of-two encoding for SET_FUNCTION_ATTRIBUTE
Override From/TryFrom for MakeFunctionFlag to use power-of-two
values (1,2,4,8,16,32) matching CPython's SET_FUNCTION_ATTRIBUTE
oparg encoding, instead of sequential discriminants (0,1,2,3,4,5).
* Remove expectedFailure from test_elim_jump_after_return1 and test_no_jump_over_return_out_of_finally_block
* Remove __classcell__ and __classdictcell__ from class dict in type.__new__
* Remove expectedFailure from test___classcell___expected_behaviour, cargo fmt
* Handle MakeCell and CopyFreeVars as no-ops in JIT
These prologue instructions are handled at frame creation time
by the VM. The JIT operates on already-initialized frames, so
these can be safely skipped during compilation.
* Remove expectedFailure from test_load_fast_known_simple
* Restore expectedFailure for test_load_fast_known_simple
The test expects LOAD_FAST_BORROW_LOAD_FAST_BORROW superinstruction
which RustPython does not emit yet.
When an inlined comprehension's first iterator expression contains
nested scopes (such as a lambda), those scopes' sub_tables appear at the
current position in the parent's sub_table list. The previous code
spliced the comprehension's own child sub_tables (e.g. inner inlined
comprehensions) into that same position before compiling the iterator,
which shifted the iterator's sub_tables to wrong indices.
Move the splice after the first iterator is compiled so its sub_tables
are consumed at their original positions.
Fixes nested list comprehensions like:
```python
[[x for _, x in g] for _, g in itertools.groupby(..., lambda x: ...)]
```
Disclosure: I used AI to develop the patch though I was heavily
involved.
* Enable PEP 709 inlined comprehensions for function-like scopes
Activate the existing compile_inlined_comprehension() implementation
by fixing 6 bugs that prevented it from working:
- LoadFastAndClear: push NULL (not None) when slot is empty so
StoreFast can restore empty state after comprehension
- StoreFast: accept NULL from stack for the restore path
- sub_tables.remove(0) replaced with next_sub_table cursor to
match the pattern used elsewhere in the compiler
- in_inlined_comp flag moved from non-inlined to inlined path
- is_inlined_comprehension_context() now checks comp_inlined flag
and restricts inlining to function-like scopes
- comp_inlined set only when parent scope uses fastlocals
Symbol table analysis handles conflict detection:
- Nested scopes in comprehension → skip inlining
- Bound name conflicts with parent symbol → skip inlining
- Cross-comprehension reference conflicts → skip inlining
- Splice comprehension sub_tables into parent for nested scope tracking
* Add localspluskinds, unify DEREF to localsplus index
- Add CO_FAST_LOCAL/CELL/FREE/HIDDEN constants and
localspluskinds field to CodeObject for per-slot metadata
- Change DEREF instruction opargs from cell-relative indices
(NameIdx) to localsplus absolute indices (oparg::VarNum)
- Add fixup_deref_opargs pass in ir.rs to convert cell-relative
indices to localsplus indices after finalization
- Replace get_cell_name with get_localsplus_name in
InstrDisplayContext trait
- Update VM cell_ref/get_cell_contents/set_cell_contents to use
localsplus indices directly (no nlocals offset)
- Update function.rs cell2arg, super.rs __class__ lookup with
explicit nlocals offsets
* Fix clippy warnings, formatting, restore _opcode_metadata.py
Fix cast_possible_truncation, nonminimal_bool, collapsible_if,
manual_contains clippy lints. Restore _opcode_metadata.py to
upstream/main version (3.14 aligned).
Pre-copy closure cells in Frame::new for coroutine locals().
Handle raw values in merged cell slots during inlined comps.
Exclude async comprehensions from inlining path.
* Exclude async/await comprehensions from PEP 709 inlining in symboltable
Async comprehensions and comprehensions with await in the element
expression need their own coroutine scope and cannot be inlined.
The symboltable builder was not checking these conditions, causing
incorrect symbol scope resolution when an async comprehension was
nested inside an inlined comprehension (e.g. [[x async for x in g]
for j in items]).
Jeong, YunWon