mirrors/RustPython
0
0 Fork 0
mirror of https://github.com/RustPython/RustPython.git synced 2026-10-28 18:59:04 +08:00
Code Activity

Bytecode parity (#7475)

Browse Source 
* Emit TO_BOOL before conditional jumps, fix class/module prologue

- Emit TO_BOOL before POP_JUMP_IF_TRUE/FALSE in the general case
  of compile_jump_if (Compare expressions excluded since they
  already produce a bool)
- Module-level __doc__: use STORE_NAME instead of STORE_GLOBAL
- Class body __module__: use LOAD_NAME instead of LOAD_GLOBAL
- Class body: store __firstlineno__ before __doc__

* Emit MAKE_CELL and COPY_FREE_VARS before RESUME

Emit MAKE_CELL for each cell variable and COPY_FREE_VARS N for
free variables at the start of each code object, before RESUME.
These instructions are no-ops in the VM but align the bytecode
with CPython 3.14's output.

* Emit __static_attributes__ at end of class bodies

Store a tuple of attribute names (currently always empty) as
__static_attributes__ in the class namespace, matching CPython
3.14's class body epilogue. Attribute name collection from
self.xxx accesses is a follow-up task.

* Remove expectedFailure from DictProxyTests iter tests

test_iter_keys, test_iter_values, test_iter_items now pass
because class bodies emit __static_attributes__ and
__firstlineno__, matching the expected dict key set.

* Use 1-based stack indexing for LIST_EXTEND, SET_UPDATE, etc.

Switch LIST_APPEND, LIST_EXTEND, SET_ADD, SET_UPDATE, MAP_ADD
from 0-based to 1-based stack depth argument, matching CPython's
PEEK(oparg) convention. Adjust the VM to subtract 1 before
calling nth_value.

* Use plain LOAD_ATTR + PUSH_NULL for calls on imported names

When the call target is an attribute of an imported name (e.g.,
logging.getLogger()), use plain LOAD_ATTR (method_flag=0) with
a separate PUSH_NULL instead of method-mode LOAD_ATTR. This
matches CPython 3.14's behavior which avoids the method call
optimization for module attribute access.

* Duplicate return-None epilogue for fall-through blocks

When the last block in a code object is exactly LOAD_CONST None +
RETURN_VALUE (the implicit return), duplicate these instructions
into blocks that would otherwise fall through to it. This matches
CPython 3.14's behavior of giving each code path its own explicit
return instruction.

* Run cargo fmt on ir.rs

* Remove expectedFailure from test_intrinsic_1 in test_dis

* Emit TO_BOOL before conditional jumps for all expressions including Compare

* Add __classdict__ cell for classes with function definitions

Set needs_classdict=true for class scopes that contain function
definitions (def/async def), matching CPython 3.14's behavior for
PEP 649 deferred annotation support. Also restore the Compare
expression check in compile_jump_if to skip TO_BOOL for comparison
operations.

* Emit __classdictcell__ store in class body epilogue

Store the __classdict__ cell reference as __classdictcell__ in
the class namespace when the class has __classdict__ as a cell
variable. Uses LOAD_DEREF (RustPython separates cell vars from
fast locals unlike CPython's unified array).

* Always run DCE to remove dead code after terminal instructions

Run basic dead code elimination (truncating instructions after
RETURN_VALUE/RAISE/JUMP within blocks) at all optimization
levels, not just optimize > 0. CPython always removes this dead
code during assembly.

* Restrict LOAD_ATTR plain mode to module/class scope imports

Only use plain LOAD_ATTR + PUSH_NULL for imports at module or
class scope. Function-local imports use method call mode LOAD_ATTR,
matching CPython 3.14's behavior.

* Eliminate unreachable blocks after jump normalization

Split DCE into two phases: (1) within-block truncation after
terminal instructions (always runs), (2) whole-block elimination
for blocks only reachable via fall-through from terminal blocks
(runs after normalize_jumps when dead jump instructions exist).

* Fold BUILD_TUPLE 0 into LOAD_CONST empty tuple

Convert BUILD_TUPLE with size 0 to LOAD_CONST () during constant
folding, matching CPython's optimization for empty tuple literals.

* Handle __classcell__ and __classdictcell__ in type.__new__

- Remove __classcell__ from class dict after setting the cell value
- Add __classdictcell__ handling: set cell to class namespace dict,
  then remove from class dict
- Register __classdictcell__ identifier
- Use LoadClosure instead of LoadDeref for __classdictcell__ emission
- Reorder MakeFunctionFlag bits to match CPython
- Run ruff format on scripts

* Revert __classdict__ cell and __classdictcell__ changes

The __classdict__ cell addition (for classes with function defs)
and __classdictcell__ store caused cell initialization failures
in importlib. These require deeper VM changes to properly support
the cell variable lifecycle. Reverted for stability.

* Fix unreachable block elimination with fixpoint reachability

Use fixpoint iteration to properly determine block reachability:
only mark jump targets of already-reachable blocks, preventing
orphaned blocks from falsely marking their targets as reachable.
Also add a final DCE pass after assembly NOP removal to catch
dead code created by normalize_jumps.

* Check enclosing scopes for IMPORTED flag in LOAD_ATTR mode

When deciding whether to use plain LOAD_ATTR for attribute calls,
check if the name is imported in any enclosing scope (not just
the current scope). This handles the common pattern where a module
is imported at module level but used inside functions.

* Add __classdict__ cell for classes with function definitions

Set needs_classdict=true when a class scope contains function
definitions (def/async def), matching CPython 3.14 which always
creates a __classdict__ cell for PEP 649 support in such classes.

* Store __classdictcell__ in class body epilogue

Store the __classdict__ cell reference as __classdictcell__ in
the class namespace using LoadClosure (which loads the cell
object itself, not the value inside). This matches CPython 3.14's
class body epilogue.

* Fix clippy collapsible_if warnings and cargo fmt

* Revert __classdict__ and __classdictcell__ changes (cause import failures)

* Revert type.__new__ __classcell__ removal and __classdictcell__ handling

Revert the class cell cleanup changes from e6975f973 that cause
import failures when frozen module bytecode is stale. The original
behavior (not removing __classcell__ from class dict) is restored.

* Re-add __classdict__ cell and __classdictcell__ store

Restore the __classdict__ cell for classes with function
definitions and __classdictcell__ store in class body epilogue.
Previous failure was caused by stale .pyc cache files containing
bytecode from an intermediate MakeFunctionFlag reorder attempt,
not by these changes themselves.

* Reorder MakeFunctionFlag to match CPython's SET_FUNCTION_ATTRIBUTE

Reorder discriminants: Defaults=0, KwOnlyDefaults=1, Annotations=2,
Closure=3, Annotate=4, TypeParams=5. This aligns the oparg values
with CPython 3.14's convention.

Note: after this change, stale .pyc cache files must be deleted
(find . -name '*.pyc' -delete) to avoid bytecode mismatch errors.

* Use CPython-compatible power-of-two encoding for SET_FUNCTION_ATTRIBUTE

Override From/TryFrom for MakeFunctionFlag to use power-of-two
values (1,2,4,8,16,32) matching CPython's SET_FUNCTION_ATTRIBUTE
oparg encoding, instead of sequential discriminants (0,1,2,3,4,5).

* Remove expectedFailure from test_elim_jump_after_return1 and test_no_jump_over_return_out_of_finally_block

* Remove __classcell__ and __classdictcell__ from class dict in type.__new__

* Remove expectedFailure from test___classcell___expected_behaviour, cargo fmt

* Handle MakeCell and CopyFreeVars as no-ops in JIT

These prologue instructions are handled at frame creation time
by the VM. The JIT operates on already-initialized frames, so
these can be safely skipped during compilation.

* Remove expectedFailure from test_load_fast_known_simple

* Restore expectedFailure for test_load_fast_known_simple

The test expects LOAD_FAST_BORROW_LOAD_FAST_BORROW superinstruction
which RustPython does not emit yet.
This commit is contained in:
Jeong, YunWon authored and GitHub committed 2026-03-23 11:31:30 +09:00
1 parent df79c0e3a9
commit bef2e41295
17 files changed
+511 -246
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

Lib/test/test_descr.py
Vendored
-3
View File
@@ -5179,7 +5179,6 @@ class DictProxyTests(unittest.TestCase):
pass
self.C = C
@unittest.expectedFailure # TODO: RUSTPYTHON
@unittest.skipIf(hasattr(sys, 'gettrace') and sys.gettrace(),
'trace function introduces __local__')
def test_iter_keys(self):
@@ -5193,7 +5192,6 @@ class DictProxyTests(unittest.TestCase):
'__static_attributes__', '__weakref__',
'meth'])
@unittest.expectedFailure # TODO: RUSTPYTHON; AssertionError: 5 != 7
@unittest.skipIf(hasattr(sys, 'gettrace') and sys.gettrace(),
'trace function introduces __local__')
def test_iter_values(self):
@@ -5203,7 +5201,6 @@ class DictProxyTests(unittest.TestCase):
values = list(it)
self.assertEqual(len(values), 7)
@unittest.expectedFailure # TODO: RUSTPYTHON
@unittest.skipIf(hasattr(sys, 'gettrace') and sys.gettrace(),
'trace function introduces __local__')
def test_iter_items(self):
Lib/test/test_dis.py
Vendored
-1
View File
@@ -1134,7 +1134,6 @@ class DisTests(DisTestBase):
# Test that value is displayed for keyword argument names:
self.do_disassembly_test(wrap_func_w_kwargs, dis_kw_names)
@unittest.expectedFailure # TODO: RUSTPYTHON
def test_intrinsic_1(self):
# Test that argrepr is displayed for CALL_INTRINSIC_1
self.do_disassembly_test("from math import *", dis_intrinsic_1_2)
Lib/test/test_peepholer.py
Vendored
+1 -2
View File
@@ -612,7 +612,6 @@ class TestTranforms(BytecodeTestCase):
print(i)
self.check_jump_targets(f)
@unittest.expectedFailure # TODO: RUSTPYTHON; 611 JUMP_BACKWARD 16
def test_elim_jump_after_return1(self):
# Eliminate dead code: jumps immediately after returns can't be reached
def f(cond1, cond2):
@@ -863,7 +862,7 @@ class TestMarkingVariablesAsUnKnown(BytecodeTestCase):
self.addCleanup(sys.settrace, sys.gettrace())
sys.settrace(None)
@unittest.expectedFailure # TODO: RUSTPYTHON; BINARY_OP 0 (+)
@unittest.expectedFailure # TODO: RUSTPYTHON; no LOAD_FAST_BORROW_LOAD_FAST_BORROW superinstruction
def test_load_fast_known_simple(self):
def f():
x = 1
Lib/test/test_super.py
Vendored
-1
View File
@@ -209,7 +209,6 @@ class TestSuper(unittest.TestCase):
self.assertIs(test_class, A)
@unittest.expectedFailure # TODO: RUSTPYTHON
def test___classcell___expected_behaviour(self):
# See issue #23722
class Meta(type):
Lib/test/test_sys_settrace.py
Vendored
-2
View File
@@ -2063,8 +2063,6 @@ class JumpTestCase(unittest.TestCase):
async with asynctracecontext(output, 4):
output.append(5)
# TODO: RUSTPYTHON
@unittest.expectedFailure
@jump_test(5, 7, [2, 4], (ValueError, "after"))
def test_no_jump_over_return_out_of_finally_block(output):
try:
crates/codegen/src/compile.rs
+108 -37
View File
@@ -610,13 +610,13 @@ impl Compiler {
self.compile_expression(value)?;
match collection_type {
CollectionType::List => {
emit!(self, Instruction::ListExtend { i: 0 });
emit!(self, Instruction::ListExtend { i: 1 });
}
CollectionType::Set => {
emit!(self, Instruction::SetUpdate { i: 0 });
emit!(self, Instruction::SetUpdate { i: 1 });
}
CollectionType::Tuple => {
emit!(self, Instruction::ListExtend { i: 0 });
emit!(self, Instruction::ListExtend { i: 1 });
}
}
} else {
@@ -627,13 +627,13 @@ impl Compiler {
// Sequence already exists, append to it
match collection_type {
CollectionType::List => {
emit!(self, Instruction::ListAppend { i: 0 });
emit!(self, Instruction::ListAppend { i: 1 });
}
CollectionType::Set => {
emit!(self, Instruction::SetAdd { i: 0 });
emit!(self, Instruction::SetAdd { i: 1 });
}
CollectionType::Tuple => {
emit!(self, Instruction::ListAppend { i: 0 });
emit!(self, Instruction::ListAppend { i: 1 });
}
}
} else {
@@ -692,6 +692,23 @@ impl Compiler {
.expect("symbol_table_stack is empty! This is a compiler bug.")
}
/// Check if a name is imported in current scope or any enclosing scope.
fn is_name_imported(&self, name: &str) -> bool {
if let Some(sym) = self.current_symbol_table().symbols.get(name) {
if sym.flags.contains(SymbolFlags::IMPORTED) {
return true;
} else if sym.scope == SymbolScope::Local {
return false;
}
}
self.symbol_table_stack.iter().rev().skip(1).any(|table| {
table
.symbols
.get(name)
.is_some_and(|sym| sym.flags.contains(SymbolFlags::IMPORTED))
})
}
/// Get the cell-relative index of a free variable.
/// Returns ncells + freevar_idx. Fixed up to localsplus index during finalize.
fn get_free_var_index(&mut self, name: &str) -> CompileResult<oparg::VarNum> {
@@ -1151,7 +1168,16 @@ impl Compiler {
self.set_qualname();
}
// Emit COPY_FREE_VARS and MAKE_CELL prolog before RESUME
// Emit MAKE_CELL for each cell variable (before RESUME)
{
let ncells = self.code_stack.last().unwrap().metadata.cellvars.len();
for i in 0..ncells {
let i_varnum: oparg::VarNum = u32::try_from(i).expect("too many cellvars").into();
emit!(self, Instruction::MakeCell { i: i_varnum });
}
}
// Emit COPY_FREE_VARS if there are free variables (before RESUME)
{
let nfrees = self.code_stack.last().unwrap().metadata.freevars.len();
if nfrees > 0 {
@@ -1162,11 +1188,6 @@ impl Compiler {
}
);
}
let ncells = self.code_stack.last().unwrap().metadata.cellvars.len();
for i in 0..ncells {
let i_varnum: oparg::VarNum = u32::try_from(i).expect("too many cellvars").into();
emit!(self, Instruction::MakeCell { i: i_varnum });
}
}
// Emit RESUME (handles async preamble and module lineno 0)
@@ -1739,7 +1760,7 @@ impl Compiler {
value: value.into(),
});
let doc = self.name("__doc__");
emit!(self, Instruction::StoreGlobal { namei: doc })
emit!(self, Instruction::StoreName { namei: doc })
}
// Handle annotations based on future_annotations flag
@@ -3424,7 +3445,7 @@ impl Compiler {
if n == 0 {
// Empty handlers (invalid AST) - append rest to list and proceed
// Stack: [prev_exc, orig, list, rest]
emit!(self, Instruction::ListAppend { i: 0 });
emit!(self, Instruction::ListAppend { i: 1 });
// Stack: [prev_exc, orig, list]
emit!(
self,
@@ -3542,7 +3563,7 @@ impl Compiler {
// After pop: [prev_exc, orig, list, new_rest, lasti] (len=5)
// nth_value(i) = stack[len - i - 1], we need stack[2] = list
// stack[5 - i - 1] = 2 -> i = 2
emit!(self, Instruction::ListAppend { i: 2 });
emit!(self, Instruction::ListAppend { i: 3 });
// Stack: [prev_exc, orig, list, new_rest, lasti]
// POP_TOP - pop lasti
@@ -3571,7 +3592,7 @@ impl Compiler {
// PEEK(1) = stack[len-1] after pop
// RustPython nth_value(i) = stack[len-i-1] after pop
// For LIST_APPEND 1: stack[len-1] = stack[len-i-1] -> i = 0
emit!(self, Instruction::ListAppend { i: 0 });
emit!(self, Instruction::ListAppend { i: 1 });
// Stack: [prev_exc, orig, list]
emit!(
self,
@@ -4561,9 +4582,9 @@ impl Compiler {
// 2. Set up class namespace
let (doc_str, body) = split_doc(body, &self.opts);
// Load (global) __name__ and store as __module__
// Load __name__ and store as __module__
let dunder_name = self.name("__name__");
self.emit_load_global(dunder_name, false);
emit!(self, Instruction::LoadName { namei: dunder_name });
let dunder_module = self.name("__module__");
emit!(
self,
@@ -4584,14 +4605,7 @@ impl Compiler {
}
);
// Store __doc__ only if there's an explicit docstring
if let Some(doc) = doc_str {
self.emit_load_const(ConstantData::Str { value: doc.into() });
let doc_name = self.name("__doc__");
emit!(self, Instruction::StoreName { namei: doc_name });
}
// Store __firstlineno__ (new in Python 3.12+)
// Store __firstlineno__ before __doc__
self.emit_load_const(ConstantData::Integer {
value: BigInt::from(firstlineno),
});
@@ -4603,6 +4617,13 @@ impl Compiler {
}
);
// Store __doc__ only if there's an explicit docstring
if let Some(doc) = doc_str {
self.emit_load_const(ConstantData::Str { value: doc.into() });
let doc_name = self.name("__doc__");
emit!(self, Instruction::StoreName { namei: doc_name });
}
// Set __type_params__ if we have type parameters
if type_params.is_some() {
// Load .type_params from enclosing scope
@@ -4661,6 +4682,44 @@ impl Compiler {
.iter()
.position(|var| *var == "__class__");
// Emit __static_attributes__ tuple
{
let attrs: Vec<String> = self
.code_stack
.last()
.unwrap()
.static_attributes
.as_ref()
.map(|s| s.iter().cloned().collect())
.unwrap_or_default();
self.emit_load_const(ConstantData::Tuple {
elements: attrs
.into_iter()
.map(|s| ConstantData::Str { value: s.into() })
.collect(),
});
let static_attrs_name = self.name("__static_attributes__");
emit!(
self,
Instruction::StoreName {
namei: static_attrs_name
}
);
}
// Store __classdictcell__ if __classdict__ is a cell variable
if self.current_symbol_table().needs_classdict {
let classdict_idx = u32::from(self.get_cell_var_index("__classdict__")?);
emit!(self, PseudoInstruction::LoadClosure { i: classdict_idx });
let classdictcell = self.name("__classdictcell__");
emit!(
self,
Instruction::StoreName {
namei: classdictcell
}
);
}
if let Some(classcell_idx) = classcell_idx {
emit!(
self,
@@ -4810,11 +4869,11 @@ impl Compiler {
if let ast::Expr::Starred(ast::ExprStarred { value, .. }) = arg {
// Starred: compile and extend
self.compile_expression(value)?;
emit!(self, Instruction::ListExtend { i: 0 });
emit!(self, Instruction::ListExtend { i: 1 });
} else {
// Non-starred: compile and append
self.compile_expression(arg)?;
emit!(self, Instruction::ListAppend { i: 0 });
emit!(self, Instruction::ListAppend { i: 1 });
}
}
}
@@ -4826,7 +4885,7 @@ impl Compiler {
namei: dot_generic_base
}
);
emit!(self, Instruction::ListAppend { i: 0 });
emit!(self, Instruction::ListAppend { i: 1 });
// Convert list to tuple
emit!(
@@ -6495,7 +6554,7 @@ impl Compiler {
self.emit_load_const(ConstantData::Integer {
value: annotation_index.into(),
});
emit!(self, Instruction::SetAdd { i: 0 });
emit!(self, Instruction::SetAdd { i: 1 });
emit!(self, Instruction::PopTop);
}
}
@@ -6742,6 +6801,10 @@ impl Compiler {
_ => {
// Fall back case which always will work!
self.compile_expression(expression)?;
// Compare already produces a bool; everything else needs TO_BOOL
if !matches!(expression, ast::Expr::Compare(_)) {
emit!(self, Instruction::ToBool);
}
if condition {
emit!(
self,
@@ -7240,7 +7303,7 @@ impl Compiler {
emit!(
compiler,
Instruction::ListAppend {
i: generators.len().to_u32(),
i: (generators.len() + 1).to_u32(),
}
);
Ok(())
@@ -7266,7 +7329,7 @@ impl Compiler {
emit!(
compiler,
Instruction::SetAdd {
i: generators.len().to_u32(),
i: (generators.len() + 1).to_u32(),
}
);
Ok(())
@@ -7298,7 +7361,7 @@ impl Compiler {
emit!(
compiler,
Instruction::MapAdd {
i: generators.len().to_u32(),
i: (generators.len() + 1).to_u32(),
}
);
@@ -7516,11 +7579,19 @@ impl Compiler {
// CALL at .method( line (not the full expression line)
self.codegen_call_helper(0, args, attr.range())?;
} else {
// Normal method call: compile object, then LOAD_ATTR with method flag
// LOAD_ATTR(method=1) pushes [method, self_or_null] on stack
self.compile_expression(value)?;
let idx = self.name(attr.as_str());
self.emit_load_attr_method(idx);
// Imported names use plain LOAD_ATTR + PUSH_NULL;
// other names use method call mode LOAD_ATTR.
// Check current scope and enclosing scopes for IMPORTED flag.
let is_import = matches!(value.as_ref(), ast::Expr::Name(ast::ExprName { id, .. })
if self.is_name_imported(id.as_str()));
if is_import {
self.emit_load_attr(idx);
emit!(self, Instruction::PushNull);
} else {
self.emit_load_attr_method(idx);
}
self.codegen_call_helper(0, args, call_range)?;
}
} else {
@@ -7558,7 +7629,7 @@ impl Compiler {
self.compile_expression(&kw.value)?;
if big {
emit!(self, Instruction::MapAdd { i: 0 });
emit!(self, Instruction::MapAdd { i: 1 });
}
}
crates/codegen/src/ir.rs
+139 -2
View File
@@ -195,8 +195,9 @@ impl CodeInfo {
self.remove_unused_consts();
self.remove_nops();
// DCE always runs (removes dead code after terminal instructions)
self.dce();
if opts.optimize > 0 {
self.dce();
self.peephole_optimize();
}
@@ -208,6 +209,9 @@ impl CodeInfo {
label_exception_targets(&mut self.blocks);
push_cold_blocks_to_end(&mut self.blocks);
normalize_jumps(&mut self.blocks);
self.dce(); // re-run within-block DCE after normalize_jumps creates new instructions
self.eliminate_unreachable_blocks();
duplicate_end_returns(&mut self.blocks);
self.optimize_load_global_push_null();
let max_stackdepth = self.max_stackdepth()?;
@@ -331,6 +335,18 @@ impl CodeInfo {
blocks[bi].instructions = kept;
}
// Final DCE: truncate instructions after terminal ops in linearized blocks.
// This catches dead code created by normalize_jumps after the initial DCE.
for block in blocks.iter_mut() {
if let Some(pos) = block
.instructions
.iter()
.position(|ins| ins.instr.is_scope_exit() || ins.instr.is_unconditional_jump())
{
block.instructions.truncate(pos + 1);
}
}
// Pre-compute cache_entries for real (non-pseudo) instructions
for block in blocks.iter_mut() {
for instr in &mut block.instructions {
@@ -546,6 +562,7 @@ impl CodeInfo {
}
fn dce(&mut self) {
// Truncate instructions after terminal instructions within each block
for block in &mut self.blocks {
let mut last_instr = None;
for (i, ins) in block.instructions.iter().enumerate() {
@@ -560,6 +577,54 @@ impl CodeInfo {
}
}
/// Clear blocks that are unreachable (not entry, not a jump target,
/// and only reachable via fall-through from a terminal block).
fn eliminate_unreachable_blocks(&mut self) {
let mut reachable = vec![false; self.blocks.len()];
reachable[0] = true;
// Fixpoint: only mark targets of already-reachable blocks
let mut changed = true;
while changed {
changed = false;
for i in 0..self.blocks.len() {
if !reachable[i] {
continue;
}
// Mark jump targets and exception handlers
for ins in &self.blocks[i].instructions {
if ins.target != BlockIdx::NULL && !reachable[ins.target.idx()] {
reachable[ins.target.idx()] = true;
changed = true;
}
if let Some(eh) = &ins.except_handler
&& !reachable[eh.handler_block.idx()]
{
reachable[eh.handler_block.idx()] = true;
changed = true;
}
}
// Mark fall-through
let next = self.blocks[i].next;
if next != BlockIdx::NULL
&& !reachable[next.idx()]
&& !self.blocks[i].instructions.last().is_some_and(|ins| {
ins.instr.is_scope_exit() || ins.instr.is_unconditional_jump()
})
{
reachable[next.idx()] = true;
changed = true;
}
}
}
for (i, block) in self.blocks.iter_mut().enumerate() {
if !reachable[i] {
block.instructions.clear();
}
}
}
/// Constant folding: fold LOAD_CONST/LOAD_SMALL_INT + BUILD_TUPLE into LOAD_CONST tuple
/// fold_tuple_of_constants
fn fold_tuple_constants(&mut self) {
@@ -574,7 +639,20 @@ impl CodeInfo {
};
let tuple_size = u32::from(instr.arg) as usize;
if tuple_size == 0 || i < tuple_size {
if tuple_size == 0 {
// BUILD_TUPLE 0 → LOAD_CONST ()
let (const_idx, _) = self.metadata.consts.insert_full(ConstantData::Tuple {
elements: Vec::new(),
});
block.instructions[i].instr = Instruction::LoadConst {
consti: Arg::marker(),
}
.into();
block.instructions[i].arg = OpArg::new(const_idx as u32);
i += 1;
continue;
}
if i < tuple_size {
i += 1;
continue;
}
@@ -1647,6 +1725,65 @@ fn normalize_jumps(blocks: &mut [Block]) {
}
}
/// Duplicate `LOAD_CONST None + RETURN_VALUE` for blocks that fall through
/// to the final return block. Matches CPython's behavior of ensuring every
/// code path that reaches the end of a function/module has its own explicit
/// return instruction.
fn duplicate_end_returns(blocks: &mut [Block]) {
// Walk the block chain to find the last block
let mut last_block = BlockIdx(0);
let mut current = BlockIdx(0);
while current != BlockIdx::NULL {
last_block = current;
current = blocks[current.idx()].next;
}
// Check if the last block ends with LOAD_CONST + RETURN_VALUE (the implicit return)
let last_insts = &blocks[last_block.idx()].instructions;
// Only apply when the last block is EXACTLY a return-None epilogue
let is_return_block = last_insts.len() == 2
&& matches!(
last_insts[0].instr,
AnyInstruction::Real(Instruction::LoadConst { .. })
)
&& matches!(
last_insts[1].instr,
AnyInstruction::Real(Instruction::ReturnValue)
);
if !is_return_block {
return;
}
// Get the return instructions to clone
let return_insts: Vec<InstructionInfo> = last_insts[last_insts.len() - 2..].to_vec();
// Find non-cold blocks that fall through to the last block
let mut blocks_to_fix = Vec::new();
current = BlockIdx(0);
while current != BlockIdx::NULL {
let block = &blocks[current.idx()];
if current != last_block && block.next == last_block && !block.cold && !block.except_handler
{
let has_fallthrough = block
.instructions
.last()
.map(|ins| !ins.instr.is_scope_exit() && !ins.instr.is_unconditional_jump())
.unwrap_or(true);
if has_fallthrough {
blocks_to_fix.push(current);
}
}
current = blocks[current.idx()].next;
}
// Duplicate the return instructions at the end of fall-through blocks
for block_idx in blocks_to_fix {
blocks[block_idx.idx()]
.instructions
.extend_from_slice(&return_insts);
}
}
/// Label exception targets: walk CFG with except stack, set per-instruction
/// handler info and block preserve_lasti flag. Converts POP_BLOCK to NOP.
/// flowgraph.c label_exception_targets + push_except_block
crates/codegen/src/snapshots/rustpython_codegen__compile__tests__if_ands.snap
Generated
+27 -13
View File
@@ -1,21 +1,35 @@
---
source: crates/codegen/src/compile.rs
assertion_line: 9100
assertion_line: 9317
expression: "compile_exec(\"\\\nif True and False and False:\n pass\n\")"
---
1 0 RESUME (0)
>> 1 LOAD_CONST (True)
2 POP_JUMP_IF_FALSE (9)
3 CACHE
4 NOT_TAKEN
>> 5 LOAD_CONST (False)
6 POP_JUMP_IF_FALSE (5)
1 LOAD_CONST (True)
2 TO_BOOL
>> 3 CACHE
4 CACHE
5 CACHE
6 POP_JUMP_IF_FALSE (19)
7 CACHE
8 NOT_TAKEN
>> 9 LOAD_CONST (False)
10 POP_JUMP_IF_FALSE (1)
11 CACHE
12 NOT_TAKEN
9 LOAD_CONST (False)
10 TO_BOOL
>> 11 CACHE
12 CACHE
13 CACHE
14 POP_JUMP_IF_FALSE (11)
15 CACHE
16 NOT_TAKEN
17 LOAD_CONST (False)
18 TO_BOOL
>> 19 CACHE
20 CACHE
21 CACHE
22 POP_JUMP_IF_FALSE (3)
23 CACHE
24 NOT_TAKEN
2 13 LOAD_CONST (None)
14 RETURN_VALUE
2 25 LOAD_CONST (None)
26 RETURN_VALUE
27 LOAD_CONST (None)
28 RETURN_VALUE
crates/codegen/src/snapshots/rustpython_codegen__compile__tests__if_mixed.snap
Generated
+32 -14
View File
@@ -1,25 +1,43 @@
---
source: crates/codegen/src/compile.rs
assertion_line: 9110
assertion_line: 9327
expression: "compile_exec(\"\\\nif (True and False) or (False and True):\n pass\n\")"
---
1 0 RESUME (0)
>> 1 LOAD_CONST (True)
2 POP_JUMP_IF_FALSE (5)
3 CACHE
4 NOT_TAKEN
>> 5 LOAD_CONST (False)
6 POP_JUMP_IF_TRUE (9)
1 LOAD_CONST (True)
2 TO_BOOL
>> 3 CACHE
4 CACHE
5 CACHE
6 POP_JUMP_IF_FALSE (9)
7 CACHE
8 NOT_TAKEN
>> 9 LOAD_CONST (False)
10 POP_JUMP_IF_FALSE (5)
11 CACHE
12 NOT_TAKEN
13 LOAD_CONST (True)
14 POP_JUMP_IF_FALSE (1)
10 TO_BOOL
>> 11 CACHE
12 CACHE
13 CACHE
14 POP_JUMP_IF_TRUE (17)
15 CACHE
16 NOT_TAKEN
>> 17 LOAD_CONST (False)
18 TO_BOOL
19 CACHE
20 CACHE
21 CACHE
22 POP_JUMP_IF_FALSE (11)
23 CACHE
24 NOT_TAKEN
25 LOAD_CONST (True)
26 TO_BOOL
27 CACHE
28 CACHE
29 CACHE
30 POP_JUMP_IF_FALSE (3)
31 CACHE
32 NOT_TAKEN
2 17 LOAD_CONST (None)
18 RETURN_VALUE
2 33 LOAD_CONST (None)
34 RETURN_VALUE
35 LOAD_CONST (None)
36 RETURN_VALUE
crates/codegen/src/snapshots/rustpython_codegen__compile__tests__if_ors.snap
Generated
+25 -11
View File
@@ -1,21 +1,35 @@
---
source: crates/codegen/src/compile.rs
assertion_line: 9090
assertion_line: 9307
expression: "compile_exec(\"\\\nif True or False or False:\n pass\n\")"
---
1 0 RESUME (0)
>> 1 LOAD_CONST (True)
2 POP_JUMP_IF_TRUE (9)
3 CACHE
4 NOT_TAKEN
>> 5 LOAD_CONST (False)
6 POP_JUMP_IF_TRUE (5)
1 LOAD_CONST (True)
2 TO_BOOL
>> 3 CACHE
4 CACHE
5 CACHE
6 POP_JUMP_IF_TRUE (17)
7 CACHE
8 NOT_TAKEN
>> 9 LOAD_CONST (False)
10 POP_JUMP_IF_FALSE (1)
10 TO_BOOL
11 CACHE
12 NOT_TAKEN
12 CACHE
13 CACHE
14 POP_JUMP_IF_TRUE (9)
15 CACHE
16 NOT_TAKEN
>> 17 LOAD_CONST (False)
18 TO_BOOL
19 CACHE
20 CACHE
21 CACHE
22 POP_JUMP_IF_FALSE (3)
23 CACHE
24 NOT_TAKEN
2 13 LOAD_CONST (None)
14 RETURN_VALUE
2 25 LOAD_CONST (None)
26 RETURN_VALUE
27 LOAD_CONST (None)
28 RETURN_VALUE
crates/codegen/src/snapshots/rustpython_codegen__compile__tests__nested_double_async_with.snap
Generated
+115 -137
View File
@@ -1,6 +1,6 @@
---
source: crates/codegen/src/compile.rs
assertion_line: 9089
assertion_line: 9362
expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIteration('spam'), StopAsyncIteration('ham')):\n with self.subTest(type=type(stop_exc)):\n try:\n async with egg():\n raise stop_exc\n except Exception as ex:\n self.assertIs(ex, stop_exc)\n else:\n self.fail(f'{stop_exc} was suppressed')\n\")"
---
1 0 RESUME (0)
@@ -32,7 +32,7 @@ expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIter
24 GET_ITER
25 FOR_ITER (71)
26 CACHE
>> 27 STORE_FAST (0, stop_exc)
27 STORE_FAST (0, stop_exc)
3 >> 28 LOAD_GLOBAL (4, self)
29 CACHE
@@ -115,162 +115,140 @@ expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIter
5 102 CLEANUP_THROW
103 JUMP_BACKWARD_NO_INTERRUPT(10)
6 104 NOP
5 105 PUSH_NULL
106 LOAD_CONST (None)
104 PUSH_EXC_INFO
105 WITH_EXCEPT_START
106 GET_AWAITABLE (2)
107 LOAD_CONST (None)
108 LOAD_CONST (None)
109 CALL (3)
110 CACHE
111 CACHE
112 CACHE
113 GET_AWAITABLE (2)
114 LOAD_CONST (None)
115 SEND (4)
108 SEND (4)
109 CACHE
110 YIELD_VALUE (1)
111 RESUME (3)
112 JUMP_BACKWARD_NO_INTERRUPT(5)
113 CLEANUP_THROW
114 END_SEND
115 TO_BOOL
116 CACHE
117 YIELD_VALUE (1)
118 RESUME (3)
119 JUMP_BACKWARD_NO_INTERRUPT(5)
120 CLEANUP_THROW
121 END_SEND
122 POP_TOP
123 JUMP_FORWARD (27)
124 PUSH_EXC_INFO
125 WITH_EXCEPT_START
126 GET_AWAITABLE (2)
127 LOAD_CONST (None)
128 SEND (4)
129 CACHE
130 YIELD_VALUE (1)
131 RESUME (3)
132 JUMP_BACKWARD_NO_INTERRUPT(5)
133 CLEANUP_THROW
134 END_SEND
135 TO_BOOL
117 CACHE
118 CACHE
119 POP_JUMP_IF_TRUE (2)
120 CACHE
121 NOT_TAKEN
122 RERAISE (2)
123 POP_TOP
124 POP_EXCEPT
125 POP_TOP
126 POP_TOP
127 JUMP_FORWARD (3)
128 COPY (3)
129 POP_EXCEPT
130 RERAISE (1)
131 JUMP_FORWARD (47)
132 PUSH_EXC_INFO
7 133 LOAD_GLOBAL (12, Exception)
134 CACHE
135 CACHE
136 CACHE
137 CACHE
138 CACHE
139 POP_JUMP_IF_TRUE (2)
138 CHECK_EXC_MATCH
139 POP_JUMP_IF_FALSE (34)
140 CACHE
141 NOT_TAKEN
142 RERAISE (2)
143 POP_TOP
144 POP_EXCEPT
145 POP_TOP
146 POP_TOP
147 JUMP_FORWARD (3)
148 COPY (3)
149 POP_EXCEPT
150 RERAISE (1)
151 JUMP_FORWARD (47)
152 PUSH_EXC_INFO
142 STORE_FAST (1, ex)
7 153 LOAD_GLOBAL (12, Exception)
8 143 LOAD_GLOBAL (4, self)
144 CACHE
145 CACHE
146 CACHE
147 CACHE
148 LOAD_ATTR (15, assertIs, method=true)
149 CACHE
150 CACHE
151 CACHE
152 CACHE
153 CACHE
154 CACHE
155 CACHE
156 CACHE
157 CACHE
158 CHECK_EXC_MATCH
159 POP_JUMP_IF_FALSE (34)
160 CACHE
161 NOT_TAKEN
162 STORE_FAST (1, ex)
158 LOAD_FAST (1, ex)
159 LOAD_FAST (0, stop_exc)
160 CALL (2)
161 CACHE
162 CACHE
163 CACHE
164 POP_TOP
165 JUMP_FORWARD (4)
166 LOAD_CONST (None)
167 STORE_FAST (1, ex)
168 DELETE_FAST (1, ex)
169 RERAISE (1)
170 POP_EXCEPT
171 LOAD_CONST (None)
172 STORE_FAST (1, ex)
173 DELETE_FAST (1, ex)
174 JUMP_FORWARD (28)
175 RERAISE (0)
176 COPY (3)
177 POP_EXCEPT
178 RERAISE (1)
8 163 LOAD_GLOBAL (4, self)
164 CACHE
165 CACHE
166 CACHE
167 CACHE
168 LOAD_ATTR (15, assertIs, method=true)
169 CACHE
170 CACHE
171 CACHE
172 CACHE
173 CACHE
174 CACHE
175 CACHE
176 CACHE
177 CACHE
178 LOAD_FAST (1, ex)
179 LOAD_FAST (0, stop_exc)
180 CALL (2)
10 179 LOAD_GLOBAL (4, self)
180 CACHE
181 CACHE
182 CACHE
183 CACHE
184 POP_TOP
185 JUMP_FORWARD (4)
186 LOAD_CONST (None)
187 STORE_FAST (1, ex)
188 DELETE_FAST (1, ex)
189 RERAISE (1)
190 POP_EXCEPT
191 LOAD_CONST (None)
192 STORE_FAST (1, ex)
193 DELETE_FAST (1, ex)
194 JUMP_FORWARD (28)
195 RERAISE (0)
196 COPY (3)
197 POP_EXCEPT
198 RERAISE (1)
10 199 LOAD_GLOBAL (4, self)
184 LOAD_ATTR (17, fail, method=true)
185 CACHE
186 CACHE
187 CACHE
188 CACHE
189 CACHE
190 CACHE
191 CACHE
192 CACHE
193 CACHE
194 LOAD_FAST_BORROW (0, stop_exc)
195 FORMAT_SIMPLE
196 LOAD_CONST (" was suppressed")
197 BUILD_STRING (2)
198 CALL (1)
199 CACHE
200 CACHE
201 CACHE
202 CACHE
203 CACHE
204 LOAD_ATTR (17, fail, method=true)
205 CACHE
206 CACHE
207 CACHE
208 CACHE
209 CACHE
202 POP_TOP
203 NOP
3 204 PUSH_NULL
205 LOAD_CONST (None)
206 LOAD_CONST (None)
207 LOAD_CONST (None)
208 CALL (3)
>> 209 CACHE
210 CACHE
211 CACHE
212 CACHE
213 CACHE
214 LOAD_FAST_BORROW (0, stop_exc)
215 FORMAT_SIMPLE
216 LOAD_CONST (" was suppressed")
217 BUILD_STRING (2)
218 CALL (1)
212 POP_TOP
213 JUMP_FORWARD (18)
214 PUSH_EXC_INFO
215 WITH_EXCEPT_START
216 TO_BOOL
217 CACHE
218 CACHE
219 CACHE
220 CACHE
220 POP_JUMP_IF_TRUE (2)
221 CACHE
222 POP_TOP
223 NOP
3 224 PUSH_NULL
225 LOAD_CONST (None)
226 LOAD_CONST (None)
227 LOAD_CONST (None)
228 CALL (3)
>> 229 CACHE
230 CACHE
231 CACHE
232 POP_TOP
233 JUMP_FORWARD (18)
234 PUSH_EXC_INFO
235 WITH_EXCEPT_START
236 TO_BOOL
237 CACHE
238 CACHE
239 CACHE
240 POP_JUMP_IF_TRUE (2)
241 CACHE
242 NOT_TAKEN
243 RERAISE (2)
244 POP_TOP
245 POP_EXCEPT
246 POP_TOP
247 POP_TOP
248 JUMP_FORWARD (3)
249 COPY (3)
250 POP_EXCEPT
251 RERAISE (1)
252 JUMP_BACKWARD (229)
253 CACHE
222 NOT_TAKEN
223 RERAISE (2)
224 POP_TOP
225 POP_EXCEPT
226 POP_TOP
227 POP_TOP
228 JUMP_FORWARD (3)
229 COPY (3)
230 POP_EXCEPT
231 RERAISE (1)
232 JUMP_BACKWARD (209)
233 CACHE
2 MAKE_FUNCTION
3 STORE_NAME (0, test)
crates/codegen/src/symboltable.rs
+13
View File
@@ -292,6 +292,19 @@ fn drop_class_free(symbol_table: &mut SymbolTable, newfree: &mut IndexSet<String
symbol_table.needs_classdict = true;
}
// Classes with function definitions need __classdict__ for PEP 649
if !symbol_table.needs_classdict {
let has_functions = symbol_table.sub_tables.iter().any(|t| {
matches!(
t.typ,
CompilerScope::Function | CompilerScope::AsyncFunction
)
});
if has_functions {
symbol_table.needs_classdict = true;
}
}
// Check if __conditional_annotations__ is in the free variables collected from children
// Remove it from free set - it's handled specially in class scope
if newfree.shift_remove("__conditional_annotations__") {
crates/compiler-core/src/bytecode/oparg.rs
+18 -8
View File
@@ -385,13 +385,13 @@ bitflagset::bitflag! {
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash)]
#[repr(u8)]
pub enum MakeFunctionFlag {
Closure = 0,
Annotations = 1,
KwOnlyDefaults = 2,
Defaults = 3,
TypeParams = 4,
Defaults = 0,
KwOnlyDefaults = 1,
Annotations = 2,
Closure = 3,
/// PEP 649: __annotate__ function closure (instead of __annotations__ dict)
Annotate = 5,
Annotate = 4,
TypeParams = 5,
}
}
@@ -403,14 +403,24 @@ bitflagset::bitflagset! {
impl TryFrom<u32> for MakeFunctionFlag {
type Error = MarshalError;
/// Decode from CPython-compatible power-of-two value
fn try_from(value: u32) -> Result<Self, Self::Error> {
Self::try_from(value as u8).map_err(|_| MarshalError::InvalidBytecode)
match value {
0x01 => Ok(Self::Defaults),
0x02 => Ok(Self::KwOnlyDefaults),
0x04 => Ok(Self::Annotations),
0x08 => Ok(Self::Closure),
0x10 => Ok(Self::Annotate),
0x20 => Ok(Self::TypeParams),
_ => Err(MarshalError::InvalidBytecode),
}
}
}
impl From<MakeFunctionFlag> for u32 {
/// Encode as CPython-compatible power-of-two value
fn from(flag: MakeFunctionFlag) -> Self {
flag as u32
1u32 << (flag as u32)
}
}
crates/jit/src/instructions.rs
+4 -1
View File
@@ -624,7 +624,10 @@ impl<'a, 'b> FunctionCompiler<'a, 'b> {
_ => Err(JitCompileError::NotSupported),
}
}
Instruction::ExtendedArg | Instruction::Cache => Ok(()),
Instruction::ExtendedArg
| Instruction::Cache
| Instruction::MakeCell { .. }
| Instruction::CopyFreeVars { .. } => Ok(()),
Instruction::JumpBackward { .. }
| Instruction::JumpBackwardNoInterrupt { .. }
crates/vm/src/builtins/type.rs
+23 -9
View File
@@ -2133,15 +2133,29 @@ impl Constructor for PyType {
}
}
if let Some(cell) = typ.attributes.write().get(identifier!(vm, __classcell__)) {
let cell = PyCellRef::try_from_object(vm, cell.clone()).map_err(|_| {
vm.new_type_error(format!(
"__classcell__ must be a nonlocal cell, not {}",
cell.class().name()
))
})?;
cell.set(Some(typ.clone().into()));
};
{
let mut attrs = typ.attributes.write();
if let Some(cell) = attrs.get(identifier!(vm, __classcell__)) {
let cell = PyCellRef::try_from_object(vm, cell.clone()).map_err(|_| {
vm.new_type_error(format!(
"__classcell__ must be a nonlocal cell, not {}",
cell.class().name()
))
})?;
cell.set(Some(typ.clone().into()));
attrs.shift_remove(identifier!(vm, __classcell__));
}
if let Some(cell) = attrs.get(identifier!(vm, __classdictcell__)) {
let cell = PyCellRef::try_from_object(vm, cell.clone()).map_err(|_| {
vm.new_type_error(format!(
"__classdictcell__ must be a nonlocal cell, not {}",
cell.class().name()
))
})?;
cell.set(Some(dict.clone().into()));
attrs.shift_remove(identifier!(vm, __classdictcell__));
}
}
// All *classes* should have a dict. Exceptions are *instances* of
// classes that define __slots__ and instances of built-in classes
crates/vm/src/frame.rs
+5 -5
View File
@@ -2623,7 +2623,7 @@ impl ExecutingFrame<'_> {
}
Instruction::ListAppend { i } => {
let item = self.pop_value();
let obj = self.nth_value(i.get(arg));
let obj = self.nth_value(i.get(arg) - 1);
let list: &Py<PyList> = unsafe {
// SAFETY: trust compiler
obj.downcast_unchecked_ref()
@@ -2633,7 +2633,7 @@ impl ExecutingFrame<'_> {
}
Instruction::ListExtend { i } => {
let iterable = self.pop_value();
let obj = self.nth_value(i.get(arg));
let obj = self.nth_value(i.get(arg) - 1);
let list: &Py<PyList> = unsafe {
// SAFETY: compiler guarantees correct type
obj.downcast_unchecked_ref()
@@ -2977,7 +2977,7 @@ impl ExecutingFrame<'_> {
Instruction::MapAdd { i } => {
let value = self.pop_value();
let key = self.pop_value();
let obj = self.nth_value(i.get(arg));
let obj = self.nth_value(i.get(arg) - 1);
let dict: &Py<PyDict> = unsafe {
// SAFETY: trust compiler
obj.downcast_unchecked_ref()
@@ -3308,7 +3308,7 @@ impl ExecutingFrame<'_> {
}
Instruction::SetAdd { i } => {
let item = self.pop_value();
let obj = self.nth_value(i.get(arg));
let obj = self.nth_value(i.get(arg) - 1);
let set: &Py<PySet> = unsafe {
// SAFETY: trust compiler
obj.downcast_unchecked_ref()
@@ -3318,7 +3318,7 @@ impl ExecutingFrame<'_> {
}
Instruction::SetUpdate { i } => {
let iterable = self.pop_value();
let obj = self.nth_value(i.get(arg));
let obj = self.nth_value(i.get(arg) - 1);
let set: &Py<PySet> = unsafe {
// SAFETY: compiler guarantees correct type
obj.downcast_unchecked_ref()
crates/vm/src/vm/context.rs
+1
View File
@@ -118,6 +118,7 @@ declare_const_name! {
__class__,
__class_getitem__,
__classcell__,
__classdictcell__,
__complex__,
__contains__,
__copy__,