mirror of
https://github.com/RustPython/RustPython.git
synced 2026-10-29 03:04:38 +08:00
main
22
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jeong, YunWonanddot
|
2a09ee02bd |
Align compiler and VM bytecode contracts with CPython 3.15 (#9105)
* Align compiler and VM with final CPython 3.15 bytecode Reconstruct the compiler-only baseline from validated487ae47 on the current main and frozen mapping prerequisitec78d811. Preserve the full source delta while dropping obsolete prerequisite ancestry; detailed old-to-new provenance is retained in the recovery artifact. Assisted-by: OpenAI Codex: model identifier unavailable * Fix final script validation, exception cleanup and private cache contracts Replay reviewed compiler CI repairs |
||
|
Jeong, YunWon
|
f87cfe7e7a |
Match CPython on code objects, ast, symtable and syntax error positions (#8580)
* Guard all/any/tuple(genexpr) calls when the generator is a coroutine
`maybe_optimize_function_call()` reserves a `skip_optimization` label for
every `name(genexpr)` call and, for `all`, `any` and `tuple`, emits an
identity guard against the builtin plus an inlined loop. The port skipped
both whenever the generator expression's symbol table was a coroutine, so
`all(await x for x in xs)` and `all(x async for x in xs)` compiled to a
plain call.
An `await` or an `async for` in the generator does not disqualify the
shape: the inlined `FOR_ITER` raises the same `TypeError` that calling the
builtin on an async generator raises, which is what
test_builtin test_builtin_call_async_genexpr_no_crash asserts.
This was the last code-level difference against CPython across `Lib/`:
comparing every file's code tree by structure, opcode and constant value
(qualnames aside, which moved in 3.14.3+, and set constants compared by
value rather than by hash order) now matches on 1720 of 1720 comparable
files, up from 1719.
Assisted-by: Claude Code:claude-opus-5
* Name the ellipsis type `ellipsis`
`type(...).__name__`, its `__qualname__`, and `types.EllipsisType.__name__`
read `EllipsisType`.
Assisted-by: Claude Code:claude-opus-5
* Align compiler front end and instruction positions with CPython
Reject a bare generator expression in a class header and a format spec
nested more than two deep. Both parse here; the reference grammar has no
rule for either, since a class header only takes `arguments` and the
tokenizer runs out of nesting levels at the third format spec.
Scan the interpolations a format spec carries inside another format spec,
so `f"{x:{y:{z}}}"` reaches `z` when building the symbol table.
Give every generic function's type params scope a `.defaults` slot,
whether or not the signature has any default, so `co_varnames` and
`co_nlocals` match. Classes and type aliases keep theirs empty.
Decide an unparenthesized sole generator argument by the parser's
`parenthesized` flag instead of scanning the source for the shape of its
element. `_sum((d := x - c) * d for x in data)` opens with a parenthesized
group the rest of the expression continues, which the scan read as the
generator's own parentheses and left the call's out of its range.
Drop empty f-string fragments unconditionally. A concatenation whose
fragments are all empty now loads its empty string at the whole
concatenation instead of at whichever fragments happened to be plain
string literals.
Keep a byte order mark in source handed over as text, and report it as an
invalid non-printable character. Reading a file to freeze strips it, as
the encoded-bytes path already did.
Carry the filename into the errors `compile(..., PyCF_ONLY_AST)` raises;
they reported `<unknown>` or an empty name.
Across `Lib/`, the 1729 files that compile on both sides now agree on
opcodes, constants, flags, variable names, names, and exception tables,
and on instruction positions in all but 7 files, where a compound
statement whose body ends in a trailing `;` ends one column short.
Assisted-by: Claude Code:claude-opus-5
* Align ast module output with CPython
- Keep lone surrogates in string, f-string and t-string literal values by
reading such a literal from its source; the literal-value helpers move
out of `Compiler` into `rustpython_codegen` and `ConstantLiteral::Str`
holds `Wtf8Buf`.
- Give `{expr=}` interpolations their leading text as a Constant and
default their conversion to `repr`, in both f-strings and t-strings.
- Span folded consecutive literals from the first of them to the last.
- Take the `:` into a format spec's range only when the parser left it
out, instead of always widening by one.
- Report a node end that lands on a line start as that line, column 0.
- Give a generator expression written straight into a call's parentheses
the range of those parentheses, in the preprocess pass, replacing
codegen's source scan and `_ast`'s one-column adjustment.
- Drop the expectedFailure on test_ast_line_numbers_with_parentheses.
Assisted-by: Claude Code:claude-opus-5
* Align symbol table output with CPython
- Register an import as `DEF_IMPORT` alone, and an annotated parameter as
a plain parameter, so `import x; global x` is accepted and annotated
parameters are not reported as annotated names.
- Record a `global` declaration in the module block as well, and resolve a
name a `global` in an enclosing scope covers as an implicit global, so a
class body reading such a name emits `LOAD_NAME`.
- Resolve a name bound nowhere as an implicit global instead of leaving
its scope unknown.
- Require a binding only for a name an explicit `nonlocal` declared;
a free variable that reached the scope any other way already resolved.
- Name a generator expression's block `genexpr`; the code object keeps
`<genexpr>`.
- Locate a `def`/`class` block at its keyword rather than at the first
decorator. `decorated_definition_range()` moves from `Compiler` to
`rustpython_codegen` for that.
- Carry a comprehension symbol's flags over when inlining it into a parent
entry that only holds a free variable propagated from a child.
- Leave `__conditional_annotations__` out of an annotation block's
symbols and have the compiler cook up the free variable, which places
it after the ones the symbol table supplies.
- Check `from __future__` placement while building the symbol table, and
give `symtable.symtable()` the same docstring stripping and future
feature validation the compiler does.
- Floor `co_stacksize` at 1.
Assisted-by: Claude Code:claude-opus-5
* Locate the eval return and the interactive print like CPython
Leave `eval` mode's `RETURN_VALUE` without a location, so the exit block
is duplicated per reaching path and each copy carries the location of the
branch that jumps to it. `a and b` now ends with two returns as it does
in CPython, and `a if b else c` gives the first one the `a` it returns.
Emit an interactive `CALL_INTRINSIC_1(Print)` at the expression statement
rather than wherever the expression left the location, which differ for
`(a := 10)` and `f'{0:fz}'`.
Assisted-by: Claude Code:claude-opus-5
* Give interactive mode the conditional annotations cell
`compile_program_single()` emitted the `__conditional_annotations__` set
and its name accesses but never added the cell that holds it, so a module
compiled in `single` mode was missing the `MAKE_CELL` and the cellvar that
the same source gets in `exec` mode.
Assisted-by: Claude Code:claude-opus-5
* Give compiler-raised SyntaxErrors an end position
`CodegenError` and `SymbolTableError` now carry an `end_location`
alongside `location`, and `CompileError::python_end_location()` returns
it, so errors such as "'await' outside function" set `end_lineno` and
`end_offset` instead of leaving them None.
`SymbolTableBuilder::error_ranged()` builds both positions from a range,
replacing the inline `source_location()` calls at the error sites.
Drop the `expectedFailure` on
`test_exceptions.SyntaxErrorTests.test_file_source`, which now passes.
Assisted-by: Claude Code:claude-opus-5
* Report parser diagnostics with CPython's wording
`InvalidAnnotatedAssignmentTarget` becomes "illegal target for
annotation", `UnexpectedExpressionToken` becomes plain "invalid syntax"
rather than prefixing the parser's text, and the list-recovery
diagnostics ("Expected an expression or a '}'" and its siblings) join the
two that already mapped to "invalid syntax".
Assisted-by: Claude Code:claude-opus-5
* Punctuate the code object repr like CPython
`repr(code)` was missing the comma after the address and escaped the
filename through Rust's Debug formatting.
Drop the `expectedFailure` on the six `test_dis` tests that compare
disassembly text, which now pass.
Assisted-by: Claude Code:claude-opus-5
* Match CPython's parser error positions
A syntax error the parser reports counts its columns in characters, so
`source_location()` converts with UTF-32 rather than UTF-8; an offset
inside a character walks back to where that character starts.
Spans the parser reported differently:
- an error between two tokens covered nothing, and now covers one
character, since the narrowest token still covers one
- a "was never closed" bracket ends at column 0
- an unterminated string literal ends where it starts
- "leading zeros in decimal integer literals" spans the run of zeros
- an exponent with no digits and no sign is blamed on the digit before
the exponent letter, which the tokenizer puts back
Assisted-by: Claude Code:claude-opus-5
* codegen: seek to the finally body's own scopes before copying it
An early exit from a try block emits an extra copy of the finally body.
Nested scopes are handed out by position, and the cursors still sit inside
the try block's own run of scopes there, so the copy took those instead of
the ones the finally body opens: a generator expression got a sibling's
symbol table and raised "the symbol 'k' must be present in the symbol
table", or, where the sibling was a named scope, built a code object with
no `.0` argument that raised UnboundLocalError when it ran.
Seek the cursors to the first scope beginning on or after the finally
body's first line before compiling the copy. The existing restore then
leaves the try statement's own copies starting from the same place.
Assisted-by: Claude
|
||
|
Jeong, YunWon
|
5add3d6a69 |
Align codegen passes and opcode metadata with CPython (#7987)
* Share marshal ref table between code object and its internals
read_marshal_bytes, _str, _str_vec, _name_tuple, and _const_tuple now
take a shared ref table and resolve TYPE_REF / register FLAG_REF
entries. deserialize_code is split into a public wrapper and an inner
function that receives the ref table; deserialize_value_depth opens a
fresh inner ref space when it hits Type::Code, mirroring CPython's
behaviour of putting the code object itself at ref slot 0. Nested code
objects inside const tuples reuse the surrounding code's ref space via
the new read_const_value helper.
* Align PYC magic number, FORMAT_VERSION, and header check with CPython 3.14
PYC_MAGIC_NUMBER changes from 2994 to 3627, matching CPython 3.14's
pyc_magic_number_token (0x0a0d0e2b). marshal FORMAT_VERSION drops from
5 to 4 (the encoder/marshal.version value; the decoder already accepts
both). check_pyc_magic_number_bytes now compares all four magic bytes
instead of the first two.
* Accept CPython-tagged .pyc as read-only bytecode source
SourceFileLoader.get_code now also looks for .pyc files using
_RP_FALLBACK_CACHE_TAGS (currently ('cpython-314',)) in addition to
sys.implementation.cache_tag. The matched .pyc is only used for
reading; recompilation still writes to the RustPython-tagged path, so
CPython's .pyc is never overwritten. Source-stat / hash / timestamp
validation logic is unchanged.
* Apply rustfmt to marshal helpers
* Marshal PySlice from format version 4 instead of 5
CPython's marshal supports TYPE_SLICE from format version 4 onwards
and that is the default version. Rejecting slice dumps below version
5 made marshal.dumps(slice(...)) fail with the default version and
broke test.test_marshal.SliceTestCase.test_slice.
* Revert "Accept CPython-tagged .pyc as read-only bytecode source"
Lib/importlib/_bootstrap_external.py is CPython's own code copied
verbatim; local patches here defeat compatibility tracking. The
cpython-XX cache_tag fallback needs to live on the RustPython side
(Rust code or sys.implementation.cache_tag policy), not as edits to
the imported standard library.
This reverts commit 1fc426d0fb5fcdb50d35cad13bbb43e8f6ce1c7f.
* Set marshal FORMAT_VERSION to 5 to match CPython 3.14.5
Py_MARSHAL_VERSION is 5 in CPython 3.14.5 (Include/marshal.h:16) and
TYPE_SLICE serialization rejects version < 5 (Python/marshal.c:720).
Restore the same threshold and constant so marshal.version and the
slice-marshal gate match CPython.
* Thread marshal recursion depth through nested code objects
Code objects embedded in const-tuples reset the depth budget on each
recursion, so a hostile or pathological marshal stream of code-in-tuple-
in-code can blow the stack despite MAX_MARSHAL_STACK_DEPTH. Pass the
current depth through deserialize_code_inner and read_marshal_const_tuple
and decrement at each code-object/tuple boundary.
Also route dict keys through deserialize_value_after_header so TYPE_CODE
keys decode instead of failing with BadType.
* align compiler to CPython
* Align codegen with CPython compile.c
Rename CFG helpers and accessors to the names used in CPython's
compile.c (basicblock_next_instr, basicblock_last_instr,
basicblock_append_instructions, bb_has_fallthrough, is_jump,
make_cfg_traversal_stack, mark_warm/mark_cold, etc.). Drop the unused
boolop-folding gate, mark_cpython_cfg_label_block helper, and
ComprehensionLoopControl::iter_range field.
Track an is_coroutine flag on SymbolTable, set in async def, await, and
async comprehensions, and propagate it through non-generator
comprehensions per symtable_handle_comprehension().
Mark SetupCleanup/SetupFinally/SetupWith as has_arg pseudo-ops, mark
ForIter as a terminator, and add has_arg/has_const on AnyInstruction.
Fix Instruction::stack_effect_jump to delegate to the opcode's
stack_effect_jump rather than stack_effect.
* Align codegen IR with CPython CFG structures
* Match CPython CFG annotation offset arithmetic
* Propagate CPython CFG label translation errors
* Align CPython exception target labeling flow
* Propagate CPython CFG traversal stack allocation errors
* Match CPython optimize_load_fast allocation flow
* Propagate CPython basicblock allocation errors
* Propagate CPython redundant NOP cleanup errors
* Propagate CPython unused const cleanup errors
* Propagate CPython const folding errors
* Propagate CPython swaptimize allocation errors
* Match CPython list-to-tuple fold allocation
* Skip const folding on CPython allocation failures
* Skip subscript folding on CPython allocation failures
* Propagate CPython assembler allocation errors
* Propagate CPython localsplus allocation errors
* Propagate CPython localsplus setup allocation errors
* Propagate CPython jump label map allocation errors
* Propagate CPython instruction sequence allocation errors
* Propagate CPython instruction label allocation errors
* Guard CPython codegen block allocation
* Guard CPython label shadow allocation
* Propagate CPython label shadow allocation errors
* Align CPython c-array allocation updates
* Align CPython ref stack growth
* Match CPython CFG builder debug check
* Avoid Rust-only CFG append clone
* Reuse CPython cleared block slots
* Use CPython block append in copy_basicblock
* Match CPython cfg builder creation order
* Clear label map after CPython apply pass
* Match CPython cfg builder allocation check
* Propagate CPython c-array size errors
* Drop Rust-only label uniqueness check
* Drop Rust-only label shadow debug checks
* Propagate CFG block index overflow
* Propagate CFG label oparg overflow
* Model CPython basicblock instruction storage
* Drop Rust-only recorded CFG precheck
* Model CPython instruction sequence storage
* Propagate instruction sequence offset overflow
* Model CPython instruction sequence labels
* Match CPython jump offset arithmetic
* Match CPython exception table arithmetic
* Match CPython label index arithmetic
* Match CPython instruction offset casts
* Match CPython jump offset indexing
* Match CPython oparg locals casts
* Match CPython localsplus offset arithmetic
* Match CPython cell prefix indexing
* Match CPython C array growth arithmetic
* Match CPython label map allocation arithmetic
* Match CPython label map size tracking
* Use CPython label map size in sequence passes
* Assert CPython label map clearing invariants
* Match CPython label oparg assignment
* Match CPython compiler direct arithmetic
* Match CPython load fast local casts
* Match CPython load fast depth assert
* Match CPython resume depth flagging
* Match CPython stack depth arithmetic
* Drop Rust-only stack overflow error
* Return CPython stack depth directly
* Match CPython C array growth errors
* Match CPython instruction insert asserts
* Match CPython unreachable pseudo jump
* Match CPython CFG size guard
* Match CPython superinstruction assert
* Match CPython redundant jump assert
* Match CPython stackdepth errors
* Match CPython jump offset flow
* Match CPython assembler buffer defaults
* Match CPython bytecode emit growth
* Match CPython assembler entry growth
* Match CPython assembler growth overflow check
* Match CPython remove_unreachable structure
* Match CPython static swap flow
* Inline CPython code unit preprocessing
* Match CPython C array growth checks
* Match CPython label map size guard
* Match CPython load-fast flow
* Simplify CPython CFG condition flow
* Align exception fallthrough propagation
* Match CPython pseudo target table
* Match CPython annotations CFG assert
* Match CPython inverted op assert
* Match CPython many-locals guard
* Reject deopt opcodes in CFG stack effects
* Match CPython invalid stack effect error
* Test CPython deopt stack effect guard
* Match CPython load-fast extended-arg assert
* Match CPython instruction allocation asserts
* Match CPython basicblock last-instr asserts
* Match CPython opcode range asserts
* Assert CPython fallthrough line propagation invariant
* Assert CPython CFG target offset sign
* Assert CPython exception fallthrough invariant
* Assert CPython exception stack bounds
* Assert CPython traversal stack allocation
* Match CPython label-map allocation in shadow
* Mirror CPython label-map sentinel fill
* Match CPython CFG builder allocation asserts
* Match CPython exception stack structure
* Match CPython ref stack structure
* Match CPython CFG traversal stack structure
* Mirror CPython CFG traversal stack pointer
* Use CPython fixed exception handler stack
* Mirror CPython ref stack capacity field
* Match CPython swap optimizer scratch stack
* Align static swap helpers with CPython blocks
* Align swaptimize signature with CPython
* Match CPython redundant pair pass result
* Match CPython inline pass result
* Match CPython redundant NOP pass results
* Fix bytecode metadata after upstream rebase
* Match CPython opcode stack metadata
* Add CPython identifiers to cspell dictionary
Add CNOTAB, LNOTAB, ialloc, ioffset, iused, nblocks, ncellsused,
ncellvars, nextop, noffsets, nvars, swaptimize, untargeted to
.cspell.dict/cpython.txt for the new CFG/assembler code in
crates/codegen/src/ir.rs.
* Fix CI failures from bytecode-parity work
- clippy: drop redundant `test_` prefix on three test functions and
remove an unnecessary `u32` cast in basicblock_clear_reuses_cpython_spare_slots_in_offset_order
- insta: regenerate nested_double_async_with snapshot to match the new
CFG output that drops unreferenced labels after the redundant-NOP pass
- regrtest: drop `@expectedFailure` markers from test_func_args,
test_meth_args (test_compile), test_disassemble_with,
test_disassemble_try_finally (test_dis), and test_except_star
(test_monitoring) which now pass
* Resync generated opcode metadata
Empty conf.toml since WithExceptStart and Setup{Cleanup,Finally,With}
stack effects already match CPython, so the TODO override entries are
stale and only cause CI hook diffs.
Regenerate opcode_metadata.rs and drop the matching SetupCleanup/
SetupFinally/SetupWith assertions on PseudoOpcode::has_arg(); their
`HAS_ARG` flag comes from pseudo definitions in bytecodes.c that the
upstream analyzer does not propagate through PseudoInstruction.properties,
so the generated has_arg() excludes them. has_target() still covers
these block-push pseudos via is_block_push().
* Drop is_block_push has_arg invariant
The CPython invariant `assert(OPCODE_HAS_ARG(op) || !IS_BLOCK_PUSH(op))`
relies on SETUP_{FINALLY,CLEANUP,WITH} carrying `HAS_ARG_FLAG` in
CPython's metadata. The autogen tool reads pseudo-opcode properties from
target instructions and does not propagate the pseudo's own
HAS_ARG flag, so PseudoOpcode::has_arg() omits these three opcodes.
Drop the debug_assert that fired inside py_freeze proc-macro expansion.
* Auto-generate has_eval_break and route AnyInstruction has_arg/has_const via macro
Add fn_has_eval_break to generate_rs_opcode_metadata.py using CPython's
Properties.eval_breaker, removing the hand-written matches! body for
Opcode::has_eval_break and PseudoOpcode::has_eval_break.
Forward has_arg/has_const from Instruction and PseudoInstruction to
their opcode, so AnyInstruction can use either_real_pseudo! like the
other has_* accessors instead of an open-coded match.
|
||
|
Jeong, YunWon
|
6440e64355 | Align codegen metadata with CPython (#7952) | ||
|
Jeong, YunWon
|
c19af81e05 |
Bytecode parity CFG normalization and jump cleanup (#7721)
* typealias reviews * Bytecode parity - try/except block order, CFG reorder Reorder try/except/else/finally to emit else+finally before except handlers matching CPython layout. Add set_no_location for cleanup blocks. Extend CFG reorder pass to handle true-path jump-back for generators, break/continue, and assert in loops. Add stop-iteration error handler awareness to block protection. * Align CFG cleanup bytecode with CPython * Unmark test_dis.test_findlabels expected failure * Compute target predecessor flags in single pass remove_nops and remove_redundant_nops_in_blocks repeated has_jump_predecessor / has_plain_jump_predecessor / target lookups per block, scanning all blocks each time. With ~200,000 if blocks this became O(B^2 * I) and timed out test_compile.test_stack_overflow. Fold the three flags into one O(B * I) pass via compute_target_predecessor_flags. * Address review feedback on slice folding, fallthrough, attr-chain - try_fold_constant_slice now delegates to try_fold_constant_expr so slice bounds accept the same constants other folding paths do (unary-folded values, __debug__, etc.). - remove_nops resolves fallthrough predecessors through empty blocks via next_nonempty_block before checking ends_with_for_cleanup. - should_deopt_borrowed_attr_chain ReturnIter matcher now accepts Instruction::CallKw alongside Instruction::Call, matching the Call/CallKw treatment in the surrounding deopt trigger check. * remove test |
||
|
Shahar Naveh
|
66b03dbe7e |
Align _opcode.has_* functions with 3.14.4 (#7711)
* Align `_opcode.has*` with 3.14.4 * Unmark passing tests * Unmark more passing tests |
||
|
Jeong, YunWon
|
7abf8cc247 |
Bytecode parity - for-loop cleanup, return duplication, late jump threading fix (#7580)
- Use POP_TOP instead of POP_ITER for for-loop break/return cleanup - Expand duplicate_end_returns to clone final return for jump predecessors - Restrict late jump threading pass to unconditional jumps only - Skip exception blocks in inline/reorder passes - Simplify threaded_jump_instr NoInterrupt handling |
||
|
Jeong, YunWon
|
bef2e41295 |
Bytecode parity (#7475)
* Emit TO_BOOL before conditional jumps, fix class/module prologue - Emit TO_BOOL before POP_JUMP_IF_TRUE/FALSE in the general case of compile_jump_if (Compare expressions excluded since they already produce a bool) - Module-level __doc__: use STORE_NAME instead of STORE_GLOBAL - Class body __module__: use LOAD_NAME instead of LOAD_GLOBAL - Class body: store __firstlineno__ before __doc__ * Emit MAKE_CELL and COPY_FREE_VARS before RESUME Emit MAKE_CELL for each cell variable and COPY_FREE_VARS N for free variables at the start of each code object, before RESUME. These instructions are no-ops in the VM but align the bytecode with CPython 3.14's output. * Emit __static_attributes__ at end of class bodies Store a tuple of attribute names (currently always empty) as __static_attributes__ in the class namespace, matching CPython 3.14's class body epilogue. Attribute name collection from self.xxx accesses is a follow-up task. * Remove expectedFailure from DictProxyTests iter tests test_iter_keys, test_iter_values, test_iter_items now pass because class bodies emit __static_attributes__ and __firstlineno__, matching the expected dict key set. * Use 1-based stack indexing for LIST_EXTEND, SET_UPDATE, etc. Switch LIST_APPEND, LIST_EXTEND, SET_ADD, SET_UPDATE, MAP_ADD from 0-based to 1-based stack depth argument, matching CPython's PEEK(oparg) convention. Adjust the VM to subtract 1 before calling nth_value. * Use plain LOAD_ATTR + PUSH_NULL for calls on imported names When the call target is an attribute of an imported name (e.g., logging.getLogger()), use plain LOAD_ATTR (method_flag=0) with a separate PUSH_NULL instead of method-mode LOAD_ATTR. This matches CPython 3.14's behavior which avoids the method call optimization for module attribute access. * Duplicate return-None epilogue for fall-through blocks When the last block in a code object is exactly LOAD_CONST None + RETURN_VALUE (the implicit return), duplicate these instructions into blocks that would otherwise fall through to it. This matches CPython 3.14's behavior of giving each code path its own explicit return instruction. * Run cargo fmt on ir.rs * Remove expectedFailure from test_intrinsic_1 in test_dis * Emit TO_BOOL before conditional jumps for all expressions including Compare * Add __classdict__ cell for classes with function definitions Set needs_classdict=true for class scopes that contain function definitions (def/async def), matching CPython 3.14's behavior for PEP 649 deferred annotation support. Also restore the Compare expression check in compile_jump_if to skip TO_BOOL for comparison operations. * Emit __classdictcell__ store in class body epilogue Store the __classdict__ cell reference as __classdictcell__ in the class namespace when the class has __classdict__ as a cell variable. Uses LOAD_DEREF (RustPython separates cell vars from fast locals unlike CPython's unified array). * Always run DCE to remove dead code after terminal instructions Run basic dead code elimination (truncating instructions after RETURN_VALUE/RAISE/JUMP within blocks) at all optimization levels, not just optimize > 0. CPython always removes this dead code during assembly. * Restrict LOAD_ATTR plain mode to module/class scope imports Only use plain LOAD_ATTR + PUSH_NULL for imports at module or class scope. Function-local imports use method call mode LOAD_ATTR, matching CPython 3.14's behavior. * Eliminate unreachable blocks after jump normalization Split DCE into two phases: (1) within-block truncation after terminal instructions (always runs), (2) whole-block elimination for blocks only reachable via fall-through from terminal blocks (runs after normalize_jumps when dead jump instructions exist). * Fold BUILD_TUPLE 0 into LOAD_CONST empty tuple Convert BUILD_TUPLE with size 0 to LOAD_CONST () during constant folding, matching CPython's optimization for empty tuple literals. * Handle __classcell__ and __classdictcell__ in type.__new__ - Remove __classcell__ from class dict after setting the cell value - Add __classdictcell__ handling: set cell to class namespace dict, then remove from class dict - Register __classdictcell__ identifier - Use LoadClosure instead of LoadDeref for __classdictcell__ emission - Reorder MakeFunctionFlag bits to match CPython - Run ruff format on scripts * Revert __classdict__ cell and __classdictcell__ changes The __classdict__ cell addition (for classes with function defs) and __classdictcell__ store caused cell initialization failures in importlib. These require deeper VM changes to properly support the cell variable lifecycle. Reverted for stability. * Fix unreachable block elimination with fixpoint reachability Use fixpoint iteration to properly determine block reachability: only mark jump targets of already-reachable blocks, preventing orphaned blocks from falsely marking their targets as reachable. Also add a final DCE pass after assembly NOP removal to catch dead code created by normalize_jumps. * Check enclosing scopes for IMPORTED flag in LOAD_ATTR mode When deciding whether to use plain LOAD_ATTR for attribute calls, check if the name is imported in any enclosing scope (not just the current scope). This handles the common pattern where a module is imported at module level but used inside functions. * Add __classdict__ cell for classes with function definitions Set needs_classdict=true when a class scope contains function definitions (def/async def), matching CPython 3.14 which always creates a __classdict__ cell for PEP 649 support in such classes. * Store __classdictcell__ in class body epilogue Store the __classdict__ cell reference as __classdictcell__ in the class namespace using LoadClosure (which loads the cell object itself, not the value inside). This matches CPython 3.14's class body epilogue. * Fix clippy collapsible_if warnings and cargo fmt * Revert __classdict__ and __classdictcell__ changes (cause import failures) * Revert type.__new__ __classcell__ removal and __classdictcell__ handling Revert the class cell cleanup changes from e6975f973 that cause import failures when frozen module bytecode is stale. The original behavior (not removing __classcell__ from class dict) is restored. * Re-add __classdict__ cell and __classdictcell__ store Restore the __classdict__ cell for classes with function definitions and __classdictcell__ store in class body epilogue. Previous failure was caused by stale .pyc cache files containing bytecode from an intermediate MakeFunctionFlag reorder attempt, not by these changes themselves. * Reorder MakeFunctionFlag to match CPython's SET_FUNCTION_ATTRIBUTE Reorder discriminants: Defaults=0, KwOnlyDefaults=1, Annotations=2, Closure=3, Annotate=4, TypeParams=5. This aligns the oparg values with CPython 3.14's convention. Note: after this change, stale .pyc cache files must be deleted (find . -name '*.pyc' -delete) to avoid bytecode mismatch errors. * Use CPython-compatible power-of-two encoding for SET_FUNCTION_ATTRIBUTE Override From/TryFrom for MakeFunctionFlag to use power-of-two values (1,2,4,8,16,32) matching CPython's SET_FUNCTION_ATTRIBUTE oparg encoding, instead of sequential discriminants (0,1,2,3,4,5). * Remove expectedFailure from test_elim_jump_after_return1 and test_no_jump_over_return_out_of_finally_block * Remove __classcell__ and __classdictcell__ from class dict in type.__new__ * Remove expectedFailure from test___classcell___expected_behaviour, cargo fmt * Handle MakeCell and CopyFreeVars as no-ops in JIT These prologue instructions are handled at frame creation time by the VM. The JIT operates on already-initialized frames, so these can be safely skipped during compilation. * Remove expectedFailure from test_load_fast_known_simple * Restore expectedFailure for test_load_fast_known_simple The test expects LOAD_FAST_BORROW_LOAD_FAST_BORROW superinstruction which RustPython does not emit yet. |
||
|
Shahar Naveh
|
3069d06851 |
Align _opcode_metadata.py to 3.14.3 (#7456)
* Align `_opcode_metadata.py` to 3.14.3 * Unmark passing test * Ensure python 3.14 runs on CI * Update banner * Fix `test__opcode.py` * Adjust generate script * Fix docs |
||
|
Jeong, YunWon
|
8a05c5ca94 |
Fix test_code: compiler and code object improvements
- Add CO_NESTED flag (0x10) for nested function scopes - Emit LOAD_SMALL_INT for integers 0..=255 instead of LOAD_CONST - Eliminate dead constant expression statements (no side effects) - Ensure None in co_consts for functions with no other constants - Add code.__replace__() for copy.replace() support - Mark test_co_lnotab and test_invalid_bytecode as expectedFailure |
||
|
Jeong, YunWon
|
1b89b8f793 |
instruction CACHE (#7251)
* Emit CACHE code units in bytecode to match CPython 3.14 - Add cache_entries() method to Instruction enum - Emit CACHE code units after opcodes in finalize_code - Handle NO_LOCATION (line=-1) in linetable for CACHE entries - Account for CACHE entries in exception table generation - Skip CACHE entries in VM execution loop (with jump detection) - Handle CACHE in InstrumentedLine/InstrumentedInstruction/InstrumentedForIter/InstrumentedNotTaken - Skip CACHE in monitoring instrumentation phases - Update co_branches() for cache-adjusted offsets - Restore _cache_format in Lib/opcode.py - Remove expectedFailure from test_c_call, test_start_offset * Use relative jump offsets and fix bytecode layout - Convert jump arguments from absolute to relative offsets in frame.rs, monitoring.rs, and stack_analysis - Add jump_relative_forward/backward helpers to ExecutingFrame - Resolve pseudo jump instructions before offset fixpoint loop - Emit NOP for break, continue, pass to match line-tracing - Fix async for: emit EndAsyncFor with correct target, add NotTaken - Fix comprehension if-cleanup to use separate block - Fix super() source range for multi-line calls - Fix NOP removal to preserve line-marker NOPs - Fix InstrumentedLine cache skipping after re-dispatch - Match InstrumentedResume/YieldValue in yield_from_target - Remove CALL_FUNCTION_EX cache entry from opcode.py - Remove resolved expectedFailure markers * Align CPython 3.14 LOAD_GLOBAL null-bit and RERAISE semantics * Remove redundant CPython-referencing comments Clean up comments that unnecessarily mention CPython per project convention. Replace with concise descriptions of the behavior itself. |
||
|
Jeong, YunWon
|
e123b9c073 |
set_f_lineno, set_f_lasti, PyAtomic refactor
- Implement set_f_lineno with stack analysis and deferred unwinding - Add Frame::set_lasti() for trace callback line jumps - Implement co_branches() on code objects - Clear _cache_format in opcode.py (no inline caches) - Fix getattro slot inheritance: preserve native slot from inherit_slots - Fix BRANCH_RIGHT src_offset in InstrumentedPopJumpIf* - Move lasti increment before line event for correct f_lineno - Skip RESUME instruction from generating line events - Defer stack pops via pending_stack_pops/pending_unwind_from_stack to avoid deadlock with state mutex - Fix ForIter exhaust target in mark_stacks to skip END_FOR - Prevent exception handler paths from overwriting normal-flow stacks - Replace #[cfg(feature = "threading")] duplication with PyAtomic<T> from rustpython_common::atomic (Radium-based unified API) - Remove expectedFailure from 31 now-passing jump tests |
||
|
Jeong YunWon
|
4b823ebaf5 | mark test_dis failures | ||
|
CPython Developers
|
131c296bb4 | Update dis from v3.14.2 | ||
|
Jeong, YunWon
|
40a43f3210 |
instruction improvements (#6829)
New Features Direct small-integer loading (0–255) and locals-loading for faster execution Async-generator wrapping and improved generator resume behavior Performance Faster integer loads and streamlined jump/loop handling for better runtime performance Bug Fixes More robust StopIteration handling and stricter init return checks Corrected iterator cleanup for async and sync loops Improvements Aligns loop and jump semantics with CPython 3.14 patterns |
||
|
Jeong, YunWon
|
e0cceaf31c |
Remove ReturnConst/Break/Continue ops (#6816)
* Remove ReturnConst * Remove break/continue ops |
||
|
Shahar Naveh
|
a400386774 |
Remove _dis module (#6690)
* Remove `_dis` module * Adjust CodeFlag |
||
|
Shahar Naveh
|
c2bfdf30bd | Assign opcode ids (#6637) | ||
|
Jeong, YunWon
|
eee360822c |
Rework compiler about exception handling (#6638)
* asyncgen * fix coroutine * rewrite compiler * tests * set pyc magic number |
||
|
Shahar Naveh
|
8d9002800e |
Update test_dis.py from 3.13.11 (#6528)
|
||
|
Bolun Thompson
|
5ecb288819 |
Rewrote dis tests to start a new process to capture stdout (and stderr)
This was chaned because stdout is being written to by rust code, which currently can't be redirected by reassigning sys.stdout. This caused the tests to fail because they couldn't see the output of what was being tested. |
||
|
Bolun Thompson
|
b066b94196 |
Tests for dis module.
As of this commit, these tests fail. |