Commit Graph
25 Commits
Author SHA1 Message Date
youknowdot 93918795b8 Align required canonical test expectations with Python 3.15
Take only demonstrated contract changes from CPython v3.15.0, d220409a13b77de7a6c30abe16ea90db01528edc: removed module __cached__, Struct format exception classes, and decimal-format diagnostic. Preserve unrelated tests and expectations.

Assisted-by: OpenAI Codex:model identifier unavailable
2026-10-10 09:40:09 +09:00
Jeong, YunWon 7bc3ef6c0a ExceptionGroup fields, OSError use-init, and live f_lineno (#8738)
* Add ExceptionGroup fields and OSError use-init

BaseExceptionGroup stores msg, excs, and excs_str; args keep
the original sequence. OSError defers parse when a subclass
overrides only __init__. characters_written can be deleted.
Exception args/traceback/cause/context reject invalid
setattr/delattr. UnicodeError.object is a deletable field.
except* remainder projection copies traceback/context/cause.
SyntaxError.__init__ clears omitted location attributes.

Assisted-by: Grok:grok-4.6

* Fix OSError payload init and UnicodeError construction

new_payload_exception now runs slot_new so BlockingIOError
parses characters_written. new_exception falls back to
invoke_exception for extra-payload types. Mixed tab/space
UnexpectedIndentation is TabError. AttributeError pickle
omits obj.

Assisted-by: Grok:grok-4.6

* address review: classify TabError from error line

Assisted-by: Grok:grok-4.6

* Add new_simple_exception; fill SyntaxError and f_lineno

new_exception constructs extra-payload types; BaseException-sized
types use new_simple_exception. Non-UTF-8 SyntaxError sets location
and text. Codegen errors load text via ProgramText. Keep Missing
parentheses capitalization. f-string adjacent atoms report a missing
comma. f_lineno may be None; lasti is not restored on unwind.

Assisted-by: Grok:grok-4.6

* address review: skip f-string continuation keywords

Assisted-by: Grok:grok-4.6

* Fix clippy map_unwrap_or in Non-UTF-8 offset

Assisted-by: Grok:grok-4.6

* Rename encode error ctor and exception setters

new_unicode_encode_error_real is new_unicode_encode_error.
set___traceback__/__cause__/__context__ are pygetset setters;
Rust uses set_traceback/set_cause/set_context.

Assisted-by: Grok:grok-4.6

* Use set_context in native pickle

Assisted-by: Grok:grok-4.6

* Collapse nested if in asyncio traceback restore

Assisted-by: Grok:grok-4.6

* Use current instruction for f_lineno addr2line

lasti is stored as the next instruction index. addr2line of
that byte offset is past RETURN_VALUE, so f_lineno was None
and stdlib_threading_current_frames hung on leftover workers.

Assisted-by: Grok:grok-4.6

* Look up format args, dir class, and frame builtins

Consume printf args before rejecting unknown conversions.
object.__dir__ uses getattr of __class__/__dict__.
_PyEval_GetBuiltin looks up iter/__import__ on the frame.

Assisted-by: Grok:grok-4.6

* Fix builtin lookups after rebase and unxfail passing tests

Assisted-by: Grok:grok-4.6

* Restore RERAISE lasti and SyntaxError locations

RERAISE with oparg restores lasti from the exception table
before unwind, so leftover f_lineno matches the original raise.
Handler lookup still uses the raising instruction.

Indentation errors outrank print/exec missing-parentheses.
BOM-with-non-utf8, bytes literals, and global+nonlocal now
report the same offsets as the parser/symtable.

object.__dir__ keeps string keys; type.__type_params__ skips
the getset wrapper stored on builtin types.

Assisted-by: Grok:grok-4.6

* Unxfail syntax/import tests and bind jump locals

IMPORT_NAME passes fromlist None instead of an empty tuple.
Jumping f_lineno binds None into unbound localsplus slots.
str lstrip/rstrip copy subclasses to exact str when unchanged.

Assisted-by: Grok:grok-4.6

* Fix import_from callers and rename frame lineno

sqlite3.iterdump now passes fromlist as a PyObject.
FrameObject::lineno is the i32 helper; f_lineno is the getset.

Assisted-by: Grok:grok-4.6

* Pass sqlite fromlist without Into conversion

from_list.into() could not infer Into<PyObjectRef> at the
import_from call. The typed tuple already converts.

Assisted-by: Grok:grok-4.6
2026-09-20 14:22:58 +09:00
Jeong, YunWon a1ba6a662c Align compiler and AST behavior with CPython (#8138)
* Align compiler and AST behavior with CPython

Update the compiler pipeline, AST module support, and related VM compile/eval paths toward CPython 3.14 behavior.

Use upstream Ruff parser crates directly and keep RustPython-specific syntax preflight handling in the compile path.

Refresh opcode metadata, snapshots, and targeted tests for the aligned bytecode and AST behavior.

* fix

* Fix PR 8138 CI failures

* Fix match mapping wildcard scan complexity

* fixes

* Use Ruff runtime AST fields

* Minimize residual patch diff

* refactor error handling

* Remove _ast conversion context wrappers

* Clean up _ast conversion glue and compiler warning escalation

Rename the Node conversion parameter `ctx` to `vm` throughout the _ast
module.

Remove the unused compile_program, compile_program_single,
compile_block_expression, and compile_expression forwarders; compile_top
stays as the entry point.

Stash an escalated compiler SyntaxWarning so a non-SyntaxWarning category
propagates unchanged instead of being rewritten to SyntaxError, matching
PyErr_ExceptionMatches(SyntaxWarning) in compiler_warn. Drop the now-unused
CompileWarningError::into_codegen_error.

Replace the PositionalArguments two-variant enum with a struct holding a
shared range and a PositionalArgumentsKind enum.

Assisted-by: Claude

* Represent compiler flags as a CompilerFlags bitflags

Replace the loose PY_CF_*/CO_* i32 constants in compile_mode with a
CompilerFlags bitflags covering the PyCF_* flags and __future__
CO_FUTURE_* bits. The Python-visible ast.PyCF_* attribute values are
derived from it via bits(); parser input modes stay plain consts.

builtins::compile and the compile_string flag handling test flags
through CompilerFlags::contains instead of raw bit math.

Assisted-by: Claude

* Fix PR 8138 test regressions

* Fix PR 8138 CI failures

* Avoid shell interpolation in Linux deps action

* Fix review findings in compiler/AST changes

- frame.rs: compute the MATCH_CLASS class name lazily so a successful
  match no longer takes the type name lock or allocates; the
  __match_args__-not-a-tuple branch reuses it instead of recomputing.
- _ast/constant.rs: add the missing leading space to the with_recursion
  label ("maximum recursion depth exceeded during compilation").
- sys.set_int_max_str_digits: restore the "maxdigits must be 0 or larger
  than {n}" error text.
- codegen: set CO_MAXBLOCKS to 20 in both ir.rs and compile.rs.
- _ast/exception.rs: extract the duplicated excepthandler node guard
  into ensure_excepthandler_node.
- codegen: correct the validate_keywords doc comment.
- Drop redundant "CPython" qualifiers from comments added in this branch.

Assisted-by: Claude
2026-06-26 18:22:08 +09:00
Jeong, YunWon edd7b4d45d Align bytecode CFG cleanup with CPython (#7781) 2026-05-12 22:14:24 +09:00
Shahar Naveh 39ff3af119 Update test_peepholer.py to 3.14.5 (#7864) 2026-05-12 20:56:36 +09:00
Jeong, YunWon 12c58d242c Bytecode parity: align borrow optimization with CFG traversal (#7773)
* Align CFG cleanup bytecode with CPython

* Bytecode parity: fblock unwind, fstring join, folding, scope

- compile.rs: unwind_fblock_stack returns whether a finally ran so
  return-statement emission can adjust location handling; restructure
  try/except/finally cleanup to preserve or drop boundary NOPs based on
  whether the body falls through; rework f-string lowering with
  count/join helpers; remove the per-collection-type heuristic for
  AST-level folding and defer to flowgraph passes; add several folding
  helpers and a ComprehensionLoopControl enum.
- ir.rs: re-run unary/binop folding around tuple folding, add
  reorder_conditional_scope_exit_and_jump_back_blocks and several block
  classification helpers, add MAX_STR_SIZE, change is_exit_without_lineno
  to take the block list.
- symboltable.rs: in analyze_cells, remove names owned as cells in
  function-like scopes from the parent's free set; mark lambda scope
  type explicitly.

* Refine CFG scope-exit backedge ordering
2026-05-05 14:04:12 +09:00
Shahar Naveh 66b03dbe7e Align _opcode.has_* functions with 3.14.4 (#7711)
* Align `_opcode.has*` with 3.14.4

* Unmark passing tests

* Unmark more passing tests
2026-04-28 12:15:23 +00:00
Jeong, YunWon 9ee32a1b38 Bytecode parity - boolop, comprehension, CFG passes (#7631)
* Bytecode parity - boolop, comprehension, CFG passes

- Flatten nested same-op BoolOp and add IfExpr to jump_if
- Simplify is_name_imported to module-level only
- Enable inlined comprehensions in module/class scope
- Add TweakInlinedComprehensionScopes with fast_hidden tracking
- Track fb_range in FBlockInfo for with-statement line info
- Add constant subscript folding and unary Not folding
- Add emit_return_const_no_location for implicit returns
- Use JumpNoInterrupt for ternary/except jumps
- Add STACK_USE_GUIDELINE threshold for collection building
- Reorder CFG pipeline: inline small blocks earlier, resolve
  line numbers before cold block extraction
- Add redirect_empty_unconditional_jump_targets,
  reorder_conditional_chain_and_jump_back_blocks,
  materialize_empty_conditional_exit_targets,
  duplicate_shared_jump_back_targets passes
- Add borrow deoptimization for multi-handler, named-except,
  protected conditional tail, and protected import joins
- Run folding/optimization passes twice around peephole

* Bytecode parity - slice augassign, async comp inline

- Augmented assignment with two-part slices uses BINARY_SLICE/STORE_SLICE
- Inline async comprehensions (remove await/async guards)
- Inlined comprehension cleanup jump uses JumpNoInterrupt
- Class firstlineno uses first decorator line when decorated
- Recurse into nested functions for static attribute collection
- Fold unary positive complex constants (+0.0j)
- Add deoptimize_borrow_for_folded_nonliteral_exprs pass
- Add inline_single_predecessor_artificial_expr_exit_blocks pass
- Skip shared artificial expr-stmt exit blocks in small-block inlining
- Mark folded boolop tail as folded_from_nonliteral_expr

* apply reviews
2026-04-21 00:10:00 +09:00
Jeong, YunWon ce0e41de9d Bytecode parity - folding, class prologue, except cleanup
Constant folding:
- Add string/bytes multiply and bytes concat folding in IR
- Add constant subscript folding (str, bytes, tuple indexing)
- Delegate list/set constant folding to IR passes
- Stream big non-const list/set via BUILD+LIST_APPEND

Class/generic compilation:
- Reorder class body prologue: __type_params__ before __classdict__
- Build class function before .generic_base in generic classes
- Register .type_params/.generic_base symbols in proper scopes
- Use load_name/store_name helpers for synthetic variables

Return block handling:
- Only duplicate return-None epilogues, not arbitrary returns
- Add inline_pop_except_return_blocks pass
- Add duplicate_named_except_cleanup_returns pass

Other fixes:
- Fix eliminate_dead_stores to only collapse adjacent duplicates
- Skip STORE_FAST_LOAD_FAST superinstruction in generators after FOR_ITER
- Thread jumps through NOP-only blocks
- Transfer NOP line info to following unconditional jumps
- Extract scope_needs_conditional_annotations_cell helper
- Register __conditional_annotations__ for module future annotations
2026-04-18 15:17:56 +09:00
Jeong, YunWon a04189fa34 Align bytecode codegen structure with CPython 3.14 (#7588)
* Align bytecode codegen structure with CPython 3.14

* Bytecode parity - constant folding, annotation ordering, superinstruction alignment

- Add BoolOp constant folding with short-circuit semantics in compile_expression
- Add constant truthiness evaluation for assert statement optimization
- Disable const collection/boolop folding in starred unpack and assignment contexts
- Move annotation block generation after body with AnnotationsPlaceholder splicing
- Reorder insert_superinstructions to run before push_cold_blocks (matching flowgraph.c)
- Lower LOAD_CLOSURE after superinstructions to avoid false LOAD_FAST_LOAD_FAST
- Add ToBool before PopJumpIf in comparisons and chained compare cleanup blocks
- Unify annotation dict building to always use incremental BuildMap + StoreSubscr
- Add TrueDivide constant folding for integer operands
- Fold constant sets to Frozenset (not Tuple) in try_fold_constant_collection
- Add PyVmBag for frozenset constant materialization in code objects
- Add remove_redundant_const_pop_top_pairs pass and peephole const+branch folding
- Emit Nop for skipped constant expressions and constant-true asserts
- Preserve comprehension local ordering by source-order bound name collection
- Simplify annotation scanning in symboltable (remove simple-name gate)

* Fix CI regressions in marshal and fast-local ops

* impl more

* Align bytecode codegen with CPython structure

* Bytecode parity - comprehension/except scope ordering, load_fast_borrow fixes

- Reorder comprehension symbol-table walk so the outermost iterator
  registers its sub_tables in the enclosing scope before the comp
  scope, and rescan elt/ifs in CPython's order. Codegen peeks past the
  outermost iterator's nested scopes to find the comprehension table.
- For plain try/except, emit handler sub_tables before the else block
  so codegen's linear sub_table cursor stays aligned.
- Rename `collect_simple_annotations` to `collect_annotations` and
  evaluate non-simple annotations during __annotate__ compilation to
  preserve source-order side effects while keeping the simple-name
  index stable.
- Dedupe equivalent code constants in `arg_constant` and add a
  structural equality check on `CodeObject`.
- Disable LOAD_FAST_BORROW for the tail end block when a try has a
  bare `except:` clause, and have `new_block` inherit the flag from
  the current block.
- Remove `cfg!(debug_assertions)` guard around the
  `optimize_load_fast_borrow` start-depth check so mismatches are
  handled (return instead of assert) in release builds.
- Collapse nop-only blocks that precede a return epilogue and hoist
  the prior line number into the next real instruction so the
  line table matches.
- Unmark now-passing `test_consts_in_conditionals`,
  `test_load_fast_unknown_simple`,
  `test_load_fast_known_because_already_loaded`, and PEP 646 f3/f4
  annotation checks.

* Bytecode parity - try/except line tracking, assert 0 shape

- In `compile_try_except`, drop the leading Nop and set the end
  block's source range from the last orelse/body statement so line
  events after the try fall on the right line.
- Recognise constant-false asserts as the direct-raise shape (no
  ToBool/PopJumpIfFalse) and flip the test assertion accordingly.
- Extend `remove_redundant_nops_in_blocks` to also look through a
  trailing nop before a return-epilogue pair (LoadConst/ReturnValue
  or LoadSmallInt/ReturnValue) so the epilogue keeps the correct
  line number.
- Rename `preds` to `predecessor_blocks` in the LOAD_FAST_BORROW
  disable pass and add a test-only `debug_late_cfg_trace` helper.
- Regenerate the `nested_double_async_with` snapshot: the tail
  reference to `stop_exc` now emits LOAD_FAST instead of
  LOAD_FAST_BORROW.

* Bytecode parity - iter folding, break/continue line, cold inlining

- Fold a constant list iterable into a constant tuple in for-loop
  iterable position, matching the CPython optimizer, and strip a
  redundant LIST_TO_TUPLE immediately before GET_ITER in the IR
  peephole pass.
- Emit a Nop at the break/continue source range before unwinding
  so line events land on the break/continue statement instead of
  the following instruction.
- Drop `propagate_disable_load_fast_borrow`; the forward propagation
  was over-zealous and the per-block inheritance in `new_block` plus
  the bare-except marker are enough.
- Relax `inline_small_or_no_lineno_blocks` so small exit blocks at
  the tail of a cold block are always inlined, not just return
  epilogues.
- Add codegen tests covering the LIST_TO_TUPLE/GET_ITER peephole and
  the late-CFG trace helper for a for-loop list-literal iterable.
2026-04-18 09:19:11 +09:00
Jeong, YunWon 0b7e0f007b Bytecode parity - exception (#7557)
* Fix exception handling: except* chaining, finally cleanup, RERAISE

- Align except* bytecode chaining
- Fix exception state model and finally handler cleanup
- Fix RERAISE to only pop exception, preserve values below

* Port IR optimization passes from flowgraph.c

- BUILD_TUPLE n + UNPACK_SEQUENCE n elimination
- Dead store elimination within basic blocks
- apply_static_swaps for SWAP reduction

* Add bytecode comparison and disassembly dump scripts

- compare_bytecode.py: compare CPython vs RustPython bytecode output
- dis_dump.py: extract disassembly in normalized JSON format
2026-04-10 12:51:48 +09:00
Jeong, YunWon 844fdf3a8d Bytecode parity (#7536)
* Add CFG block splitting, jump threading, backward jump normalization, genexpr StopIteration wrapper

- split_blocks_at_jumps: split blocks at branch points so each has one exit
- jump_threading: thread jumps through single-jump blocks (flowgraph.c jump_thread)
- Backward conditional jump normalization: invert and create NOT_TAKEN+JUMP block
- Follow empty blocks in jump-to-return optimization (next_nonempty_block)
- Add PEP 479 StopIteration handler to compile_comprehension for generators

* Add ConstantData::Slice and constant slice folding

- Add Slice variant to ConstantData and BorrowedConstant
- Fold constant slices (x[:3], x[1:4]) into LOAD_CONST(slice(...))
- Marshal serialization/deserialization for Slice type
- Box::leak in borrow_obj_constant for PySlice roundtrip

* Add duplicate_exits_without_lineno (disabled) and Block: Clone

Prepare infrastructure for exit block duplication optimization.
Currently disabled pending stackdepth integration.

* Improve codegen bytecode parity
2026-03-30 18:50:58 +09:00
Jeong, YunWon cbdc7bb587 Compiler parity: docstring dedent, StopIteration wrapper, constant folding (#7530) 2026-03-29 22:36:20 +09:00
Jeong, YunWon 5433a3d08c Bytecode parity (#7507)
* Bytecode parity phase 3

Compiler changes:
- Emit TO_BOOL in and/or short-circuit evaluation (COPY+TO_BOOL+JUMP)
- Add module-level __conditional_annotations__ cell (PEP 649)
- Only set conditional annotations for AnnAssign, not function params
- Skip __classdict__ cell when future annotations are active
- Convert list literals to tuples in for-loop iterables
- Fix cell variable ordering: parameters first, then alphabetical
- Fix RESUME DEPTH1 flag for yield-from/await
- Don't propagate __classdict__/__conditional_annotations__ freevar
  through regular functions — only annotation/type-param scopes
- Inline string compilation path

* Skip test_thread_safety in _test_multiprocessing

SIGSEGV in _finalizer_registry dict access under aggressive GC
and thread switching. Root cause is dict thread-safety in VM.

* Skip list→tuple optimization for async for; propagate future_annotations to nested scopes
2026-03-27 12:42:29 +09:00
Jeong, YunWon a293e11238 Bytecode parity (#7481)
* Bytecode parity

 Compiler changes:
    - Remove PUSH_NULL from decorator cal
ls, use CALL 0
    - Collect __static_attributes__ from self.xxx = patterns
    - Sort __static_attributes__ alphabetically
    - Move __classdict__ init before __doc__ in class prologue
    - Fold unary negative constants
    - Fold constant list/set literals (3+ elements)
    - Use BUILD_MAP 0 + MAP_ADD for 16+ dict pairs
    - Always run peephole optimizer for s
uperinstructions
    - Emit RETURN_GENERATOR for generator
 functions
    - Add is_generator flag to SymbolTabl
e

* Fix formatting and collapsible_if clippy warnings in compile.rs

* Fix clippy, fold_unary_negative chaining, and generator line tracing

- Replace irrefutable if-let with let for ExceptHandler
- Remove folded UNARY_NEGATIVE instead of replacing with NOP,
  enabling chained negation folding
- Initialize prev_line to def line for generators/coroutines
  to suppress spurious LINE events from preamble instructions
- Remove expectedFailure markers for now-passing tests

* Fix JIT StoreFastStoreFast, format, and remove expectedFailure markers

- Add StoreFastStoreFast handling in JIT instructions
- Fix cargo fmt in frame.rs
- Remove 11 expectedFailure markers for async jump tests in
  test_sys_settrace that now pass

* Fix peephole optimizer: use NOP replacement instead of remove()

Using remove() shifts instruction indices and corrupts subsequent
references, causing "pop stackref but null found" panics at runtime.
Replace folded/combined instructions with NOP instead, which are
cleaned up by the existing remove_nops pass.

* Revert peephole_optimize to use remove() for chaining support

NOP replacement broke chaining of peephole optimizations (e.g.
LOAD_CONST+TO_BOOL then LOAD_CONST+UNARY_NOT for 'not True').
The remove() approach is used by upstream and works correctly here;
fold_unary_negative keeps NOP replacement since it doesn't need chaining.

* Fix StoreFastStoreFast to handle NULL from LoadFastAndClear

StoreFast uses pop_value_opt() to allow NULL values from
LoadFastAndClear in inlined comprehension cleanup paths.
StoreFastStoreFast must do the same, otherwise the peephole
optimizer's fusion of two StoreFast instructions panics when
restoring unbound locals after an inlined comprehension.
2026-03-25 16:10:19 +09:00
Jeong, YunWon bef2e41295 Bytecode parity (#7475)
* Emit TO_BOOL before conditional jumps, fix class/module prologue

- Emit TO_BOOL before POP_JUMP_IF_TRUE/FALSE in the general case
  of compile_jump_if (Compare expressions excluded since they
  already produce a bool)
- Module-level __doc__: use STORE_NAME instead of STORE_GLOBAL
- Class body __module__: use LOAD_NAME instead of LOAD_GLOBAL
- Class body: store __firstlineno__ before __doc__

* Emit MAKE_CELL and COPY_FREE_VARS before RESUME

Emit MAKE_CELL for each cell variable and COPY_FREE_VARS N for
free variables at the start of each code object, before RESUME.
These instructions are no-ops in the VM but align the bytecode
with CPython 3.14's output.

* Emit __static_attributes__ at end of class bodies

Store a tuple of attribute names (currently always empty) as
__static_attributes__ in the class namespace, matching CPython
3.14's class body epilogue. Attribute name collection from
self.xxx accesses is a follow-up task.

* Remove expectedFailure from DictProxyTests iter tests

test_iter_keys, test_iter_values, test_iter_items now pass
because class bodies emit __static_attributes__ and
__firstlineno__, matching the expected dict key set.

* Use 1-based stack indexing for LIST_EXTEND, SET_UPDATE, etc.

Switch LIST_APPEND, LIST_EXTEND, SET_ADD, SET_UPDATE, MAP_ADD
from 0-based to 1-based stack depth argument, matching CPython's
PEEK(oparg) convention. Adjust the VM to subtract 1 before
calling nth_value.

* Use plain LOAD_ATTR + PUSH_NULL for calls on imported names

When the call target is an attribute of an imported name (e.g.,
logging.getLogger()), use plain LOAD_ATTR (method_flag=0) with
a separate PUSH_NULL instead of method-mode LOAD_ATTR. This
matches CPython 3.14's behavior which avoids the method call
optimization for module attribute access.

* Duplicate return-None epilogue for fall-through blocks

When the last block in a code object is exactly LOAD_CONST None +
RETURN_VALUE (the implicit return), duplicate these instructions
into blocks that would otherwise fall through to it. This matches
CPython 3.14's behavior of giving each code path its own explicit
return instruction.

* Run cargo fmt on ir.rs

* Remove expectedFailure from test_intrinsic_1 in test_dis

* Emit TO_BOOL before conditional jumps for all expressions including Compare

* Add __classdict__ cell for classes with function definitions

Set needs_classdict=true for class scopes that contain function
definitions (def/async def), matching CPython 3.14's behavior for
PEP 649 deferred annotation support. Also restore the Compare
expression check in compile_jump_if to skip TO_BOOL for comparison
operations.

* Emit __classdictcell__ store in class body epilogue

Store the __classdict__ cell reference as __classdictcell__ in
the class namespace when the class has __classdict__ as a cell
variable. Uses LOAD_DEREF (RustPython separates cell vars from
fast locals unlike CPython's unified array).

* Always run DCE to remove dead code after terminal instructions

Run basic dead code elimination (truncating instructions after
RETURN_VALUE/RAISE/JUMP within blocks) at all optimization
levels, not just optimize > 0. CPython always removes this dead
code during assembly.

* Restrict LOAD_ATTR plain mode to module/class scope imports

Only use plain LOAD_ATTR + PUSH_NULL for imports at module or
class scope. Function-local imports use method call mode LOAD_ATTR,
matching CPython 3.14's behavior.

* Eliminate unreachable blocks after jump normalization

Split DCE into two phases: (1) within-block truncation after
terminal instructions (always runs), (2) whole-block elimination
for blocks only reachable via fall-through from terminal blocks
(runs after normalize_jumps when dead jump instructions exist).

* Fold BUILD_TUPLE 0 into LOAD_CONST empty tuple

Convert BUILD_TUPLE with size 0 to LOAD_CONST () during constant
folding, matching CPython's optimization for empty tuple literals.

* Handle __classcell__ and __classdictcell__ in type.__new__

- Remove __classcell__ from class dict after setting the cell value
- Add __classdictcell__ handling: set cell to class namespace dict,
  then remove from class dict
- Register __classdictcell__ identifier
- Use LoadClosure instead of LoadDeref for __classdictcell__ emission
- Reorder MakeFunctionFlag bits to match CPython
- Run ruff format on scripts

* Revert __classdict__ cell and __classdictcell__ changes

The __classdict__ cell addition (for classes with function defs)
and __classdictcell__ store caused cell initialization failures
in importlib. These require deeper VM changes to properly support
the cell variable lifecycle. Reverted for stability.

* Fix unreachable block elimination with fixpoint reachability

Use fixpoint iteration to properly determine block reachability:
only mark jump targets of already-reachable blocks, preventing
orphaned blocks from falsely marking their targets as reachable.
Also add a final DCE pass after assembly NOP removal to catch
dead code created by normalize_jumps.

* Check enclosing scopes for IMPORTED flag in LOAD_ATTR mode

When deciding whether to use plain LOAD_ATTR for attribute calls,
check if the name is imported in any enclosing scope (not just
the current scope). This handles the common pattern where a module
is imported at module level but used inside functions.

* Add __classdict__ cell for classes with function definitions

Set needs_classdict=true when a class scope contains function
definitions (def/async def), matching CPython 3.14 which always
creates a __classdict__ cell for PEP 649 support in such classes.

* Store __classdictcell__ in class body epilogue

Store the __classdict__ cell reference as __classdictcell__ in
the class namespace using LoadClosure (which loads the cell
object itself, not the value inside). This matches CPython 3.14's
class body epilogue.

* Fix clippy collapsible_if warnings and cargo fmt

* Revert __classdict__ and __classdictcell__ changes (cause import failures)

* Revert type.__new__ __classcell__ removal and __classdictcell__ handling

Revert the class cell cleanup changes from e6975f973 that cause
import failures when frozen module bytecode is stale. The original
behavior (not removing __classcell__ from class dict) is restored.

* Re-add __classdict__ cell and __classdictcell__ store

Restore the __classdict__ cell for classes with function
definitions and __classdictcell__ store in class body epilogue.
Previous failure was caused by stale .pyc cache files containing
bytecode from an intermediate MakeFunctionFlag reorder attempt,
not by these changes themselves.

* Reorder MakeFunctionFlag to match CPython's SET_FUNCTION_ATTRIBUTE

Reorder discriminants: Defaults=0, KwOnlyDefaults=1, Annotations=2,
Closure=3, Annotate=4, TypeParams=5. This aligns the oparg values
with CPython 3.14's convention.

Note: after this change, stale .pyc cache files must be deleted
(find . -name '*.pyc' -delete) to avoid bytecode mismatch errors.

* Use CPython-compatible power-of-two encoding for SET_FUNCTION_ATTRIBUTE

Override From/TryFrom for MakeFunctionFlag to use power-of-two
values (1,2,4,8,16,32) matching CPython's SET_FUNCTION_ATTRIBUTE
oparg encoding, instead of sequential discriminants (0,1,2,3,4,5).

* Remove expectedFailure from test_elim_jump_after_return1 and test_no_jump_over_return_out_of_finally_block

* Remove __classcell__ and __classdictcell__ from class dict in type.__new__

* Remove expectedFailure from test___classcell___expected_behaviour, cargo fmt

* Handle MakeCell and CopyFreeVars as no-ops in JIT

These prologue instructions are handled at frame creation time
by the VM. The JIT operates on already-initialized frames, so
these can be safely skipped during compilation.

* Remove expectedFailure from test_load_fast_known_simple

* Restore expectedFailure for test_load_fast_known_simple

The test expects LOAD_FAST_BORROW_LOAD_FAST_BORROW superinstruction
which RustPython does not emit yet.
2026-03-23 11:31:30 +09:00
Jeong, YunWon 1b89b8f793 instruction CACHE (#7251)
* Emit CACHE code units in bytecode to match CPython 3.14

- Add cache_entries() method to Instruction enum
- Emit CACHE code units after opcodes in finalize_code
- Handle NO_LOCATION (line=-1) in linetable for CACHE entries
- Account for CACHE entries in exception table generation
- Skip CACHE entries in VM execution loop (with jump detection)
- Handle CACHE in InstrumentedLine/InstrumentedInstruction/InstrumentedForIter/InstrumentedNotTaken
- Skip CACHE in monitoring instrumentation phases
- Update co_branches() for cache-adjusted offsets
- Restore _cache_format in Lib/opcode.py
- Remove expectedFailure from test_c_call, test_start_offset

* Use relative jump offsets and fix bytecode layout

- Convert jump arguments from absolute to relative offsets
  in frame.rs, monitoring.rs, and stack_analysis
- Add jump_relative_forward/backward helpers to ExecutingFrame
- Resolve pseudo jump instructions before offset fixpoint loop
- Emit NOP for break, continue, pass to match line-tracing
- Fix async for: emit EndAsyncFor with correct target, add NotTaken
- Fix comprehension if-cleanup to use separate block
- Fix super() source range for multi-line calls
- Fix NOP removal to preserve line-marker NOPs
- Fix InstrumentedLine cache skipping after re-dispatch
- Match InstrumentedResume/YieldValue in yield_from_target
- Remove CALL_FUNCTION_EX cache entry from opcode.py
- Remove resolved expectedFailure markers

* Align CPython 3.14 LOAD_GLOBAL null-bit and RERAISE semantics

* Remove redundant CPython-referencing comments

Clean up comments that unnecessarily mention CPython per project
convention. Replace with concise descriptions of the behavior itself.
2026-03-01 14:51:54 +09:00
Jeong, YunWon f54f226f84 Unmark successful tests
test_peepholer
test_bdb
2026-02-27 09:07:23 +09:00
Jeong, YunWon e123b9c073 set_f_lineno, set_f_lasti, PyAtomic refactor
- Implement set_f_lineno with stack analysis and deferred unwinding
- Add Frame::set_lasti() for trace callback line jumps
- Implement co_branches() on code objects
- Clear _cache_format in opcode.py (no inline caches)
- Fix getattro slot inheritance: preserve native slot from inherit_slots
- Fix BRANCH_RIGHT src_offset in InstrumentedPopJumpIf*
- Move lasti increment before line event for correct f_lineno
- Skip RESUME instruction from generating line events
- Defer stack pops via pending_stack_pops/pending_unwind_from_stack
  to avoid deadlock with state mutex
- Fix ForIter exhaust target in mark_stacks to skip END_FOR
- Prevent exception handler paths from overwriting normal-flow stacks
- Replace #[cfg(feature = "threading")] duplication with PyAtomic<T>
  from rustpython_common::atomic (Radium-based unified API)
- Remove expectedFailure from 31 now-passing jump tests
2026-02-27 09:07:23 +09:00
Jeong, YunWon 061c709434 cold block reordering and jump normalization (#7210)
* cold block reordering and jump normalization

Add mark_cold, push_cold_blocks_to_end, and normalize_jumps
passes to the codegen CFG pipeline. Use JumpNoInterrupt for
exception handler exit paths in try-except-finally compilation.

* mark test_peepholer
2026-02-24 08:53:07 +09:00
Jeong, YunWon 95abec4c21 More Load instructions including LoadLocals, LoadFastBorrow (#6886)
* fix stdlib_io

* more scpell dict

* LoadLocals

* LoadFastBorrow

* more ops
2026-01-29 19:33:46 +09:00
ShaharNaveh 7cc4f43779 Update test_peepholer from 3.14.2 2026-01-27 10:49:04 +09:00
Jeong YunWon 8c73f3cb30 emit RESUME 2026-01-26 10:40:22 +09:00
Jeong, YunWon e0cceaf31c Remove ReturnConst/Break/Continue ops (#6816)
* Remove ReturnConst

* Remove break/continue ops
2026-01-21 00:03:43 +09:00
Shahar Naveh 892116c009 Add some compiler tests from 3.13.11 (#6752)
* Add some compiler tests from 3.13.11

* Unmark passing test
2026-01-18 00:26:06 +09:00