mirrors/RustPython
0
0 Fork 0
mirror of https://github.com/RustPython/RustPython.git synced 2026-10-29 03:04:38 +08:00
Code Activity

Bytecode parity (#7504)

Browse Source 
* Match CPython LOAD_SPECIAL stack semantics for with/async-with

LOAD_SPECIAL now pushes (callable, self_or_null) matching CPython's
CALL convention, instead of a single bound method:
- Function descriptors: push (func, self)
- Plain attributes: push (bound, NULL)

Updated all with-statement paths:
- Entry: add SWAP 3 after SWAP 2, remove PUSH_NULL before CALL 0
- Normal exit: remove PUSH_NULL before CALL 3
- Exception handler (WITH_EXCEPT_START): read exit_func at TOS-4
  and self_or_null at TOS-3
- Suppress block: 3 POP_TOPs after POP_EXCEPT (was 2)
- FBlock exit (preserve_tos): SWAP 3 + SWAP 2 rotation
- UnwindAction::With: remove PUSH_NULL

Stack effects updated: LoadSpecial (2,1), WithExceptStart (7,6)

* Normalize LOAD_FAST_CHECK and JUMP_BACKWARD_NO_INTERRUPT

Add LOAD_FAST_CHECK → LOAD_FAST and JUMP_BACKWARD_NO_INTERRUPT →
JUMP_BACKWARD to opname normalization in dis_dump.py. These are
optimization variants with identical semantics.

* Add EXTENDED_ARG to SKIP_OPS, normalize LOAD_FAST_CHECK and JUMP_BACKWARD_NO_INTERRUPT

* Remove duplicate return-None when block already has return

Skip duplicate_end_returns for blocks that already end with
LOAD_CONST + RETURN_VALUE. Run DCE + unreachable elimination
after duplication to remove the now-unreachable original return
block.

* Improve __static_attributes__ collection accuracy

- Support tuple/list unpacking targets: (self.x, self.y) = val
- Skip @staticmethod and @classmethod decorated methods
- Use scan_target_for_attrs helper for recursive target scanning

* Use method mode for function-local import attribute calls

Function-local imports (scope is Local+IMPORTED) should use method
mode LOAD_ATTR like regular names, not plain mode. Only module/class
scope imports use plain LOAD_ATTR + PUSH_NULL.

* Optimize constant iterable before GET_ITER to LOAD_CONST tuple

Convert BUILD_LIST/SET 0 + LOAD_CONST + LIST_EXTEND/SET_UPDATE + GET_ITER
to just LOAD_CONST (tuple) + GET_ITER, matching CPython's optimization
for constant list/set literals in for-loop iterables.

Also fix is_name_imported to use method mode for function-local imports,
and improve __static_attributes__ accuracy (skip @classmethod/@staticmethod,
handle tuple/list unpacking targets).

* Fix cell variable ordering: parameters first, then alphabetical

CPython orders cell variables with parameter cells first (in
parameter definition order), then non-parameter cells sorted
alphabetically. Previously all cells were sorted alphabetically.

Also add for-loop iterable optimization: constant BUILD_LIST/SET
before GET_ITER is folded to just LOAD_CONST tuple.

* Emit COPY_FREE_VARS before MAKE_CELL matching CPython order

CPython emits COPY_FREE_VARS first, then MAKE_CELL instructions.
Previously RustPython emitted them in reverse order.

* Fix RESUME AfterYield encoding to match CPython 3.14 (value 5)

CPython 3.14 uses RESUME arg=5 for after-yield, not 1.
Also reorder COPY_FREE_VARS before MAKE_CELL and fix cell
variable ordering (parameters first, then alphabetical).

* Address code review feedback from #7481

- Set is_generator flag for generator expressions in scan_comprehension
- Fix posonlyargs priority in collect_static_attributes first param
- Add match statement support to scan_store_attrs
- Fix stale decorator stack comment
- Reorder NOP removal after fold_unary_negative for better collection folding

* Fold constant list/set/tuple literals in compiler

When all elements of a list/set/tuple literal are constants and
there are 3+ elements, fold them into a single constant:
- list: BUILD_LIST 0 + LOAD_CONST (tuple) + LIST_EXTEND 1
- set:  BUILD_SET 0  + LOAD_CONST (tuple) + SET_UPDATE 1
- tuple: LOAD_CONST (tuple)

This matches CPython's compiler optimization and fixes the most
common bytecode difference (92/200 sampled files).

Also add bytecode comparison scripts (dis_dump.py, compare_bytecode.py)
for systematic parity tracking.

* Use BUILD_MAP 0 + MAP_ADD for large dicts (>= 16 pairs)

Match CPython's compiler behavior: dicts with 16+ key-value pairs
use BUILD_MAP 0 followed by MAP_ADD for each pair, instead of
pushing all keys/values on the stack and calling BUILD_MAP N.

* Fix clippy warnings and cargo fmt

* fix surrogate
This commit is contained in:
Jeong, YunWon authored and GitHub committed 2026-03-25 22:25:21 +09:00
1 parent 44cd8d53a1
commit 1bf829a456
10 files changed
+371 -200
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

crates/codegen/src/compile.rs
+211 -106
View File
@@ -564,6 +564,31 @@ impl Compiler {
_ => n > 4,
};
// Fold all-constant collections (>= 3 elements) regardless of size
if !seen_star
&& pushed == 0
&& n >= 3
&& elts.iter().all(|e| e.is_constant())
&& let Some(folded) = self.try_fold_constant_collection(elts)?
{
match collection_type {
CollectionType::Tuple => {
self.emit_load_const(folded);
}
CollectionType::List => {
emit!(self, Instruction::BuildList { count: 0 });
self.emit_load_const(folded);
emit!(self, Instruction::ListExtend { i: 1 });
}
CollectionType::Set => {
emit!(self, Instruction::BuildSet { count: 0 });
self.emit_load_const(folded);
emit!(self, Instruction::SetUpdate { i: 1 });
}
}
return Ok(());
}
// If no stars and not too big, compile all elements and build once
if !seen_star && !big {
for elt in elts {
@@ -694,13 +719,21 @@ impl Compiler {
/// Check if a name is imported in current scope or any enclosing scope.
fn is_name_imported(&self, name: &str) -> bool {
if let Some(sym) = self.current_symbol_table().symbols.get(name) {
let current = self.current_symbol_table();
if let Some(sym) = current.symbols.get(name) {
if sym.flags.contains(SymbolFlags::IMPORTED) {
return true;
} else if sym.scope == SymbolScope::Local {
// Module/class scope imports use plain LOAD_ATTR
// Function-local imports use method mode (scope is Local)
return !matches!(
current.typ,
CompilerScope::Function | CompilerScope::AsyncFunction | CompilerScope::Lambda
);
}
if sym.scope == SymbolScope::Local {
return false;
}
}
// Check enclosing scopes for module-level imports accessed as globals
self.symbol_table_stack.iter().rev().skip(1).any(|table| {
table
.symbols
@@ -1033,7 +1066,9 @@ impl Compiler {
// Build cellvars using dictbytype (CELL scope or COMP_CELL flag, sorted)
let mut cellvar_cache = IndexSet::default();
let mut cell_names: Vec<_> = ste
// CPython ordering: parameter cells first (in parameter order),
// then non-parameter cells (alphabetically sorted)
let cell_symbols: Vec<_> = ste
.symbols
.iter()
.filter(|(_, s)| {
@@ -1041,8 +1076,22 @@ impl Compiler {
})
.map(|(name, _)| name.clone())
.collect();
cell_names.sort();
for name in cell_names {
let mut param_cells = Vec::new();
let mut nonparam_cells = Vec::new();
for name in cell_symbols {
if varname_cache.contains(&name) {
param_cells.push(name);
} else {
nonparam_cells.push(name);
}
}
// param_cells are already in parameter order (from varname_cache insertion order)
param_cells.sort_by_key(|n| varname_cache.get_index_of(n.as_str()).unwrap_or(usize::MAX));
nonparam_cells.sort();
for name in param_cells {
cellvar_cache.insert(name);
}
for name in nonparam_cells {
cellvar_cache.insert(name);
}
@@ -1168,16 +1217,7 @@ impl Compiler {
self.set_qualname();
}
// Emit MAKE_CELL for each cell variable (before RESUME)
{
let ncells = self.code_stack.last().unwrap().metadata.cellvars.len();
for i in 0..ncells {
let i_varnum: oparg::VarNum = u32::try_from(i).expect("too many cellvars").into();
emit!(self, Instruction::MakeCell { i: i_varnum });
}
}
// Emit COPY_FREE_VARS if there are free variables (before RESUME)
// Emit COPY_FREE_VARS first, then MAKE_CELL (CPython order)
{
let nfrees = self.code_stack.last().unwrap().metadata.freevars.len();
if nfrees > 0 {
@@ -1189,6 +1229,13 @@ impl Compiler {
);
}
}
{
let ncells = self.code_stack.last().unwrap().metadata.cellvars.len();
for i in 0..ncells {
let i_varnum: oparg::VarNum = u32::try_from(i).expect("too many cellvars").into();
emit!(self, Instruction::MakeCell { i: i_varnum });
}
}
// Emit RESUME (handles async preamble and module lineno 0)
// CPython: LOCATION(lineno, lineno, 0, 0), then loc.lineno = 0 for module
@@ -1476,25 +1523,20 @@ impl Compiler {
}
FBlockType::With | FBlockType::AsyncWith => {
// Stack when entering: [..., __exit__, return_value (if preserve_tos)]
// Need to call __exit__(None, None, None)
// Stack: [..., exit_func, self_exit, return_value (if preserve_tos)]
emit!(self, PseudoInstruction::PopBlock);
// If preserving return value, swap it below __exit__
if preserve_tos {
emit!(self, Instruction::Swap { i: 2 });
// Rotate return value below the exit pair
// [exit_func, self_exit, value] → [value, exit_func, self_exit]
emit!(self, Instruction::Swap { i: 3 }); // [value, self_exit, exit_func]
emit!(self, Instruction::Swap { i: 2 }); // [value, exit_func, self_exit]
}
// Stack after swap: [..., return_value, __exit__] or [..., __exit__]
// Call __exit__(None, None, None)
// Call protocol: [callable, self_or_null, arg1, arg2, arg3]
emit!(self, Instruction::PushNull);
// Stack: [..., __exit__, NULL]
// Call exit_func(self_exit, None, None, None)
self.emit_load_const(ConstantData::None);
self.emit_load_const(ConstantData::None);
self.emit_load_const(ConstantData::None);
// Stack: [..., __exit__, NULL, None, None, None]
emit!(self, Instruction::Call { argc: 3 });
// For async with, await the result
@@ -2756,7 +2798,7 @@ impl Compiler {
}
/// Push decorators onto the stack in source order.
/// For @dec1 @dec2 def foo(): stack becomes [dec1, NULL, dec2, NULL]
/// For @dec1 @dec2 def foo(): stack becomes [dec1, dec2]
fn prepare_decorators(&mut self, decorator_list: &[ast::Decorator]) -> CompileResult<()> {
for decorator in decorator_list {
self.compile_expression(&decorator.expression)?;
@@ -4514,22 +4556,55 @@ impl Compiler {
fn collect_static_attributes(body: &[ast::Stmt], attrs: Option<&mut IndexSet<String>>) {
let Some(attrs) = attrs else { return };
for stmt in body {
// Only scan def/async def at class body level
let (params, func_body) = match stmt {
ast::Stmt::FunctionDef(f) => (&f.parameters, &f.body),
let f = match stmt {
ast::Stmt::FunctionDef(f) => f,
_ => continue,
};
// Get first parameter name (usually "self" or "cls")
let first_param = params
.args
// Skip @staticmethod and @classmethod decorated functions
let dominated_by_special = f.decorator_list.iter().any(|d| {
matches!(&d.expression, ast::Expr::Name(n)
if n.id.as_str() == "staticmethod" || n.id.as_str() == "classmethod")
});
if dominated_by_special {
continue;
}
let first_param = f
.parameters
.posonlyargs
.first()
.or(params.posonlyargs.first())
.or(f.parameters.args.first())
.map(|p| &p.parameter.name);
let Some(self_name) = first_param else {
continue;
};
// Scan function body for self.xxx = ... (STORE_ATTR on first param)
Self::scan_store_attrs(func_body, self_name.as_str(), attrs);
Self::scan_store_attrs(&f.body, self_name.as_str(), attrs);
}
}
/// Extract self.attr patterns from an assignment target expression.
fn scan_target_for_attrs(target: &ast::Expr, name: &str, attrs: &mut IndexSet<String>) {
match target {
ast::Expr::Attribute(ast::ExprAttribute { value, attr, .. }) => {
if let ast::Expr::Name(n) = value.as_ref()
&& n.id.as_str() == name
{
attrs.insert(attr.to_string());
}
}
ast::Expr::Tuple(t) => {
for elt in &t.elts {
Self::scan_target_for_attrs(elt, name, attrs);
}
}
ast::Expr::List(l) => {
for elt in &l.elts {
Self::scan_target_for_attrs(elt, name, attrs);
}
}
ast::Expr::Starred(s) => {
Self::scan_target_for_attrs(&s.value, name, attrs);
}
_ => {}
}
}
@@ -4539,22 +4614,11 @@ impl Compiler {
match stmt {
ast::Stmt::Assign(a) => {
for target in &a.targets {
if let ast::Expr::Attribute(ast::ExprAttribute { value, attr, .. }) = target
&& let ast::Expr::Name(n) = value.as_ref()
&& n.id.as_str() == name
{
attrs.insert(attr.to_string());
}
Self::scan_target_for_attrs(target, name, attrs);
}
}
ast::Stmt::AnnAssign(a) => {
if let ast::Expr::Attribute(ast::ExprAttribute { value, attr, .. }) =
a.target.as_ref()
&& let ast::Expr::Name(n) = value.as_ref()
&& n.id.as_str() == name
{
attrs.insert(attr.to_string());
}
Self::scan_target_for_attrs(&a.target, name, attrs);
}
ast::Stmt::AugAssign(a) => {
if let ast::Expr::Attribute(ast::ExprAttribute { value, attr, .. }) =
@@ -4591,6 +4655,11 @@ impl Compiler {
ast::Stmt::With(s) => {
Self::scan_store_attrs(&s.body, name, attrs);
}
ast::Stmt::Match(s) => {
for case in &s.cases {
Self::scan_store_attrs(&case.body, name, attrs);
}
}
_ => {}
}
}
@@ -5180,16 +5249,16 @@ impl Compiler {
Instruction::LoadSpecial {
method: SpecialMethod::AExit
}
); // [cm, bound_aexit]
emit!(self, Instruction::Swap { i: 2 }); // [bound_aexit, cm]
); // [cm, aexit_func, self_ae]
emit!(self, Instruction::Swap { i: 2 }); // [cm, self_ae, aexit_func]
emit!(self, Instruction::Swap { i: 3 }); // [aexit_func, self_ae, cm]
emit!(
self,
Instruction::LoadSpecial {
method: SpecialMethod::AEnter
}
); // [bound_aexit, bound_aenter]
emit!(self, Instruction::PushNull);
emit!(self, Instruction::Call { argc: 0 }); // [bound_aexit, awaitable]
); // [aexit_func, self_ae, aenter_func, self_an]
emit!(self, Instruction::Call { argc: 0 }); // [aexit_func, self_ae, awaitable]
emit!(self, Instruction::GetAwaitable { r#where: 1 });
self.emit_load_const(ConstantData::None);
let _ = self.compile_yield_from_sequence(true)?;
@@ -5200,16 +5269,16 @@ impl Compiler {
Instruction::LoadSpecial {
method: SpecialMethod::Exit
}
); // [cm, bound_exit]
emit!(self, Instruction::Swap { i: 2 }); // [bound_exit, cm]
); // [cm, exit_func, self_exit]
emit!(self, Instruction::Swap { i: 2 }); // [cm, self_exit, exit_func]
emit!(self, Instruction::Swap { i: 3 }); // [exit_func, self_exit, cm]
emit!(
self,
Instruction::LoadSpecial {
method: SpecialMethod::Enter
}
); // [bound_exit, bound_enter]
emit!(self, Instruction::PushNull);
emit!(self, Instruction::Call { argc: 0 }); // [bound_exit, enter_result]
); // [exit_func, self_exit, enter_func, self_enter]
emit!(self, Instruction::Call { argc: 0 }); // [exit_func, self_exit, result]
}
// Stack: [..., __exit__, enter_result]
@@ -5263,10 +5332,9 @@ impl Compiler {
});
// ===== Normal exit path =====
// Stack: [..., bound_exit]
// Call bound_exit(None, None, None)
// Stack: [..., exit_func, self_exit]
// Call exit_func(self_exit, None, None, None)
self.set_source_range(with_range);
emit!(self, Instruction::PushNull);
self.emit_load_const(ConstantData::None);
self.emit_load_const(ConstantData::None);
self.emit_load_const(ConstantData::None);
@@ -5280,11 +5348,10 @@ impl Compiler {
emit!(self, PseudoInstruction::Jump { delta: after_block });
// ===== Exception handler path =====
// Stack at entry (after unwind): [..., __exit__, lasti, exc]
// PUSH_EXC_INFO -> [..., __exit__, lasti, prev_exc, exc]
// Stack at entry: [..., exit_func, self_exit, lasti, exc]
// PUSH_EXC_INFO -> [..., exit_func, self_exit, lasti, prev_exc, exc]
self.switch_to_block(exc_handler_block);
// Create blocks for exception handling
let cleanup_block = self.new_block();
let suppress_block = self.new_block();
@@ -5296,12 +5363,10 @@ impl Compiler {
);
self.push_fblock(FBlockType::ExceptionHandler, exc_handler_block, after_block)?;
// PUSH_EXC_INFO: [exc] -> [prev_exc, exc]
emit!(self, Instruction::PushExcInfo);
// WITH_EXCEPT_START: call __exit__(type, value, tb)
// Stack: [..., __exit__, lasti, prev_exc, exc]
// __exit__ is at TOS-3, call with exception info
// WITH_EXCEPT_START: call exit_func(self_exit, type, value, tb)
// Stack: [..., exit_func, self_exit, lasti, prev_exc, exc]
emit!(self, Instruction::WithExceptStart);
if is_async {
@@ -5310,7 +5375,6 @@ impl Compiler {
let _ = self.compile_yield_from_sequence(true)?;
}
// TO_BOOL + POP_JUMP_IF_TRUE: check if exception is suppressed
emit!(self, Instruction::ToBool);
emit!(
self,
@@ -5319,25 +5383,19 @@ impl Compiler {
}
);
// Pop the nested fblock BEFORE RERAISE so that RERAISE's exception
// handler points to the outer handler (try-except), not cleanup_block.
// This is critical: when RERAISE propagates the exception, the exception
// table should route it to the outer try-except, not back to cleanup.
emit!(self, PseudoInstruction::PopBlock);
self.pop_fblock(FBlockType::ExceptionHandler);
// Not suppressed: RERAISE 2
emit!(self, Instruction::Reraise { depth: 2 });
// ===== Suppress block =====
// Exception was suppressed, clean up stack
// Stack: [..., __exit__, lasti, prev_exc, exc, True]
// Need to pop: True, exc, prev_exc, __exit__
// Stack: [..., exit_func, self_exit, lasti, prev_exc, exc, True]
self.switch_to_block(suppress_block);
emit!(self, Instruction::PopTop); // pop True (TO_BOOL result)
emit!(self, Instruction::PopExcept); // pop exc and restore prev_exc
emit!(self, Instruction::PopTop); // pop __exit__
emit!(self, Instruction::PopTop); // pop True
emit!(self, Instruction::PopExcept); // pop exc, restore prev_exc
emit!(self, Instruction::PopTop); // pop lasti
emit!(self, Instruction::PopTop); // pop self_exit
emit!(self, Instruction::PopTop); // pop exit_func
emit!(self, PseudoInstruction::Jump { delta: after_block });
// ===== Cleanup block (for nested exception during __exit__) =====
@@ -6989,10 +7047,7 @@ impl Compiler {
let has_unpacking = items.iter().any(|item| item.key.is_none());
if !has_unpacking {
// STACK_USE_GUIDELINE: for large dicts (16+ pairs), use
// BUILD_MAP 0 + MAP_ADD to avoid excessive stack usage
let big = items.len() * 2 > 30; // ~15 pairs threshold
if big {
if items.len() >= 16 {
emit!(self, Instruction::BuildMap { count: 0 });
for item in items {
self.compile_expression(item.key.as_ref().unwrap())?;
@@ -7567,23 +7622,8 @@ impl Compiler {
self.compile_expr_tstring(tstring)?;
}
ast::Expr::StringLiteral(string) => {
let value = string.value.to_str();
if value.contains(char::REPLACEMENT_CHARACTER) {
let value = string
.value
.iter()
.map(|lit| {
let source = self.source_file.slice(lit.range);
crate::string_parser::parse_string_literal(source, lit.flags.into())
})
.collect();
// might have a surrogate literal; should reparse to be sure
self.emit_load_const(ConstantData::Str { value });
} else {
self.emit_load_const(ConstantData::Str {
value: value.into(),
});
}
let value = self.compile_string_value(string);
self.emit_load_const(ConstantData::Str { value });
}
ast::Expr::BytesLiteral(bytes) => {
let iter = bytes.value.iter().flat_map(|x| x.iter().copied());
@@ -8514,11 +8554,78 @@ impl Compiler {
// fn block_done()
/// Convert a string literal AST node to Wtf8Buf, handling surrogates correctly.
fn compile_string_value(&self, string: &ast::ExprStringLiteral) -> Wtf8Buf {
let value = string.value.to_str();
if value.contains(char::REPLACEMENT_CHARACTER) {
// Might have a surrogate literal; reparse from source to preserve them
string
.value
.iter()
.map(|lit| {
let source = self.source_file.slice(lit.range);
crate::string_parser::parse_string_literal(source, lit.flags.into())
})
.collect()
} else {
value.into()
}
}
fn arg_constant(&mut self, constant: ConstantData) -> oparg::ConstIdx {
let info = self.current_code_info();
info.metadata.consts.insert_full(constant).0.to_u32().into()
}
/// Try to fold a collection of constant expressions into a single ConstantData::Tuple.
/// Returns None if any element cannot be folded.
fn try_fold_constant_collection(
&mut self,
elts: &[ast::Expr],
) -> CompileResult<Option<ConstantData>> {
let mut constants = Vec::with_capacity(elts.len());
for elt in elts {
match elt {
ast::Expr::NumberLiteral(num) => match &num.value {
ast::Number::Int(int) => {
let value = ruff_int_to_bigint(int).map_err(|e| self.error(e))?;
constants.push(ConstantData::Integer { value });
}
ast::Number::Float(f) => {
constants.push(ConstantData::Float { value: *f });
}
ast::Number::Complex { real, imag } => {
constants.push(ConstantData::Complex {
value: Complex::new(*real, *imag),
});
}
},
ast::Expr::StringLiteral(s) => {
let value = self.compile_string_value(s);
constants.push(ConstantData::Str { value });
}
ast::Expr::BytesLiteral(b) => {
constants.push(ConstantData::Bytes {
value: b.value.bytes().collect(),
});
}
ast::Expr::BooleanLiteral(b) => {
constants.push(ConstantData::Boolean { value: b.value });
}
ast::Expr::NoneLiteral(_) => {
constants.push(ConstantData::None);
}
ast::Expr::EllipsisLiteral(_) => {
constants.push(ConstantData::Ellipsis);
}
_ => return Ok(None),
}
}
Ok(Some(ConstantData::Tuple {
elements: constants,
}))
}
fn emit_load_const(&mut self, constant: ConstantData) {
let idx = self.arg_constant(constant);
self.emit_arg(idx, |consti| Instruction::LoadConst { consti })
@@ -8740,10 +8847,8 @@ impl Compiler {
for action in unwind_actions {
match action {
UnwindAction::With { is_async } => {
// codegen_unwind_fblock(WITH/ASYNC_WITH)
// Stack: [..., exit_func, self_exit]
emit!(self, PseudoInstruction::PopBlock);
// compiler_call_exit_with_nones
emit!(self, Instruction::PushNull);
self.emit_load_const(ConstantData::None);
self.emit_load_const(ConstantData::None);
self.emit_load_const(ConstantData::None);
crates/codegen/src/ir.rs
+63 -4
View File
@@ -191,9 +191,12 @@ impl CodeInfo {
) -> crate::InternalResult<CodeObject> {
// Constant folding passes
self.fold_unary_negative();
self.remove_nops(); // remove NOPs from unary folding so tuple/list/set see contiguous LOADs
self.fold_tuple_constants();
self.fold_list_constants();
self.fold_set_constants();
self.remove_nops(); // remove NOPs from collection folding
self.fold_const_iterable_for_iter();
self.convert_to_load_small_int();
self.remove_unused_consts();
self.remove_nops();
@@ -214,6 +217,8 @@ impl CodeInfo {
self.dce(); // re-run within-block DCE after normalize_jumps creates new instructions
self.eliminate_unreachable_blocks();
duplicate_end_returns(&mut self.blocks);
self.dce(); // truncate after terminal in blocks that got return duplicated
self.eliminate_unreachable_blocks(); // remove now-unreachable last block
self.optimize_load_global_push_null();
let max_stackdepth = self.max_stackdepth()?;
@@ -876,6 +881,49 @@ impl CodeInfo {
}
}
/// Convert constant list/set construction before GET_ITER to just LOAD_CONST tuple.
/// BUILD_LIST 0 + LOAD_CONST (tuple) + LIST_EXTEND 1 + GET_ITER
/// → LOAD_CONST (tuple) + GET_ITER
/// Also handles BUILD_SET 0 + LOAD_CONST + SET_UPDATE 1 + GET_ITER.
fn fold_const_iterable_for_iter(&mut self) {
for block in &mut self.blocks {
let mut i = 0;
while i + 3 < block.instructions.len() {
let is_build = matches!(
block.instructions[i].instr.real(),
Some(Instruction::BuildList { .. } | Instruction::BuildSet { .. })
) && u32::from(block.instructions[i].arg) == 0;
let is_const = matches!(
block.instructions[i + 1].instr.real(),
Some(Instruction::LoadConst { .. })
);
let is_extend = matches!(
block.instructions[i + 2].instr.real(),
Some(Instruction::ListExtend { .. } | Instruction::SetUpdate { .. })
) && u32::from(block.instructions[i + 2].arg) == 1;
let is_iter = matches!(
block.instructions[i + 3].instr.real(),
Some(Instruction::GetIter)
);
if is_build && is_const && is_extend && is_iter {
// Replace: BUILD_X 0 → NOP, keep LOAD_CONST, LIST_EXTEND → NOP
let loc = block.instructions[i].location;
block.instructions[i].instr = Instruction::Nop.into();
block.instructions[i].location = loc;
block.instructions[i + 2].instr = Instruction::Nop.into();
block.instructions[i + 2].location = loc;
i += 4;
} else {
i += 1;
}
}
}
}
/// Fold constant set literals: LOAD_CONST* + BUILD_SET N →
/// BUILD_SET 0 + LOAD_CONST (frozenset-as-tuple) + SET_UPDATE 1
fn fold_set_constants(&mut self) {
@@ -1987,6 +2035,7 @@ fn duplicate_end_returns(blocks: &mut [Block]) {
// Check if the last block ends with LOAD_CONST + RETURN_VALUE (the implicit return)
let last_insts = &blocks[last_block.idx()].instructions;
// Only apply when the last block is EXACTLY a return-None epilogue
// AND the return instructions have no explicit line number (lineno <= 0)
let is_return_block = last_insts.len() == 2
&& matches!(
last_insts[0].instr,
@@ -2010,12 +2059,22 @@ fn duplicate_end_returns(blocks: &mut [Block]) {
let block = &blocks[current.idx()];
if current != last_block && block.next == last_block && !block.cold && !block.except_handler
{
let has_fallthrough = block
.instructions
.last()
let last_ins = block.instructions.last();
let has_fallthrough = last_ins
.map(|ins| !ins.instr.is_scope_exit() && !ins.instr.is_unconditional_jump())
.unwrap_or(true);
if has_fallthrough {
// Don't duplicate if block already ends with the same return pattern
let already_has_return = block.instructions.len() >= 2 && {
let n = block.instructions.len();
matches!(
block.instructions[n - 2].instr,
AnyInstruction::Real(Instruction::LoadConst { .. })
) && matches!(
block.instructions[n - 1].instr,
AnyInstruction::Real(Instruction::ReturnValue)
)
};
if has_fallthrough && !already_has_return {
blocks_to_fix.push(current);
}
}
crates/codegen/src/snapshots/rustpython_codegen__compile__tests__if_ands.snap
Generated
+1 -1
View File
@@ -1,6 +1,6 @@
---
source: crates/codegen/src/compile.rs
assertion_line: 9458
assertion_line: 9553
expression: "compile_exec(\"\\\nif True and False and False:\n pass\n\")"
---
1 0 RESUME (0)
crates/codegen/src/snapshots/rustpython_codegen__compile__tests__if_mixed.snap
Generated
+1 -1
View File
@@ -1,6 +1,6 @@
---
source: crates/codegen/src/compile.rs
assertion_line: 9468
assertion_line: 9563
expression: "compile_exec(\"\\\nif (True and False) or (False and True):\n pass\n\")"
---
1 0 RESUME (0)
crates/codegen/src/snapshots/rustpython_codegen__compile__tests__if_ors.snap
Generated
+1 -1
View File
@@ -1,6 +1,6 @@
---
source: crates/codegen/src/compile.rs
assertion_line: 9448
assertion_line: 9543
expression: "compile_exec(\"\\\nif True or False or False:\n pass\n\")"
---
1 0 RESUME (0)
crates/codegen/src/snapshots/rustpython_codegen__compile__tests__nested_double_async_with.snap
Generated
+66 -65
View File
@@ -1,6 +1,6 @@
---
source: crates/codegen/src/compile.rs
assertion_line: 9496
assertion_line: 9591
expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIteration('spam'), StopAsyncIteration('ham')):\n with self.subTest(type=type(stop_exc)):\n try:\n async with egg():\n raise stop_exc\n except Exception as ex:\n self.assertIs(ex, stop_exc)\n else:\n self.fail(f'{stop_exc} was suppressed')\n\")"
---
1 0 RESUME (0)
@@ -23,8 +23,8 @@ expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIter
15 CACHE
16 CACHE
17 CACHE
>> 18 LOAD_CONST ("ham")
19 CALL (1)
18 LOAD_CONST ("ham")
>> 19 CALL (1)
20 CACHE
21 CACHE
22 CACHE
@@ -67,8 +67,8 @@ expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIter
58 COPY (1)
59 LOAD_SPECIAL (__exit__)
60 SWAP (2)
61 LOAD_SPECIAL (__enter__)
62 PUSH_NULL
61 SWAP (3)
62 LOAD_SPECIAL (__enter__)
63 CALL (0)
64 CACHE
65 CACHE
@@ -89,8 +89,8 @@ expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIter
78 COPY (1)
79 LOAD_SPECIAL (__aexit__)
80 SWAP (2)
81 LOAD_SPECIAL (__aenter__)
82 PUSH_NULL
81 SWAP (3)
82 LOAD_SPECIAL (__aenter__)
83 CALL (0)
84 CACHE
85 CACHE
@@ -138,31 +138,31 @@ expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIter
124 POP_EXCEPT
125 POP_TOP
126 POP_TOP
127 JUMP_FORWARD (3)
128 COPY (3)
129 POP_EXCEPT
130 RERAISE (1)
131 JUMP_FORWARD (46)
132 PUSH_EXC_INFO
127 POP_TOP
128 JUMP_FORWARD (3)
129 COPY (3)
130 POP_EXCEPT
131 RERAISE (1)
132 JUMP_FORWARD (46)
133 PUSH_EXC_INFO
7 133 LOAD_GLOBAL (12, Exception)
134 CACHE
7 134 LOAD_GLOBAL (12, Exception)
135 CACHE
136 CACHE
137 CACHE
138 CHECK_EXC_MATCH
139 POP_JUMP_IF_FALSE (33)
140 CACHE
141 NOT_TAKEN
142 STORE_FAST (1, ex)
138 CACHE
139 CHECK_EXC_MATCH
140 POP_JUMP_IF_FALSE (33)
141 CACHE
142 NOT_TAKEN
143 STORE_FAST (1, ex)
8 143 LOAD_GLOBAL (4, self)
144 CACHE
8 144 LOAD_GLOBAL (4, self)
145 CACHE
146 CACHE
147 CACHE
148 LOAD_ATTR (15, assertIs, method=true)
149 CACHE
148 CACHE
149 LOAD_ATTR (15, assertIs, method=true)
150 CACHE
151 CACHE
152 CACHE
@@ -171,34 +171,34 @@ expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIter
155 CACHE
156 CACHE
157 CACHE
158 LOAD_FAST_LOAD_FAST (ex, stop_exc)
159 CALL (2)
160 CACHE
158 CACHE
159 LOAD_FAST_LOAD_FAST (ex, stop_exc)
160 CALL (2)
161 CACHE
162 CACHE
163 POP_TOP
164 JUMP_FORWARD (4)
165 LOAD_CONST (None)
166 STORE_FAST (1, ex)
167 DELETE_FAST (1, ex)
168 RERAISE (1)
169 POP_EXCEPT
170 LOAD_CONST (None)
171 STORE_FAST (1, ex)
172 DELETE_FAST (1, ex)
173 JUMP_FORWARD (28)
174 RERAISE (0)
175 COPY (3)
176 POP_EXCEPT
177 RERAISE (1)
163 CACHE
164 POP_TOP
165 JUMP_FORWARD (4)
166 LOAD_CONST (None)
167 STORE_FAST (1, ex)
168 DELETE_FAST (1, ex)
169 RERAISE (1)
170 POP_EXCEPT
171 LOAD_CONST (None)
172 STORE_FAST (1, ex)
173 DELETE_FAST (1, ex)
174 JUMP_FORWARD (28)
175 RERAISE (0)
176 COPY (3)
177 POP_EXCEPT
178 RERAISE (1)
10 178 LOAD_GLOBAL (4, self)
179 CACHE
10 179 LOAD_GLOBAL (4, self)
180 CACHE
181 CACHE
182 CACHE
183 LOAD_ATTR (17, fail, method=true)
184 CACHE
183 CACHE
184 LOAD_ATTR (17, fail, method=true)
185 CACHE
186 CACHE
187 CACHE
@@ -207,27 +207,27 @@ expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIter
190 CACHE
191 CACHE
192 CACHE
193 LOAD_FAST_BORROW (0, stop_exc)
194 FORMAT_SIMPLE
195 LOAD_CONST (" was suppressed")
196 BUILD_STRING (2)
197 CALL (1)
198 CACHE
193 CACHE
194 LOAD_FAST_BORROW (0, stop_exc)
195 FORMAT_SIMPLE
196 LOAD_CONST (" was suppressed")
197 BUILD_STRING (2)
198 CALL (1)
199 CACHE
200 CACHE
201 POP_TOP
202 NOP
201 CACHE
202 POP_TOP
203 NOP
3 203 PUSH_NULL
204 LOAD_CONST (None)
3 204 LOAD_CONST (None)
205 LOAD_CONST (None)
206 LOAD_CONST (None)
207 CALL (3)
>> 208 CACHE
209 CACHE
208 CACHE
>> 209 CACHE
210 CACHE
211 POP_TOP
212 JUMP_FORWARD (18)
212 JUMP_FORWARD (19)
213 PUSH_EXC_INFO
214 WITH_EXCEPT_START
215 TO_BOOL
@@ -242,12 +242,13 @@ expression: "compile_exec(\"\\\nasync def test():\n for stop_exc in (StopIter
224 POP_EXCEPT
225 POP_TOP
226 POP_TOP
227 JUMP_FORWARD (3)
228 COPY (3)
229 POP_EXCEPT
230 RERAISE (1)
231 JUMP_BACKWARD (208)
232 CACHE
227 POP_TOP
228 JUMP_FORWARD (3)
229 COPY (3)
230 POP_EXCEPT
231 RERAISE (1)
232 JUMP_BACKWARD (209)
233 CACHE
2 MAKE_FUNCTION
3 STORE_NAME (0, test)
crates/codegen/src/symboltable.rs
+2
View File
@@ -2136,6 +2136,8 @@ impl SymbolTableBuilder {
CompilerScope::Comprehension,
self.line_index_start(range),
);
// Generator expressions need the is_generator flag
self.tables.last_mut().unwrap().is_generator = is_generator;
// PEP 709: Mark non-generator comprehensions for inlining,
// but only inside function-like scopes (fastlocals).
crates/compiler-core/src/bytecode/instruction.rs
+2 -2
View File
@@ -1020,7 +1020,7 @@ impl InstructionMetadata for Instruction {
Self::LoadLocals => (1, 0),
Self::LoadName { .. } => (1, 0),
Self::LoadSmallInt { .. } => (1, 0),
Self::LoadSpecial { .. } => (1, 1),
Self::LoadSpecial { .. } => (2, 1),
Self::LoadSuperAttr { .. } => (1 + (oparg & 1), 3),
Self::LoadSuperAttrAttr => (1, 3),
Self::LoadSuperAttrMethod => (2, 3),
@@ -1085,7 +1085,7 @@ impl InstructionMetadata for Instruction {
Self::UnpackSequenceList => (oparg, 1),
Self::UnpackSequenceTuple => (oparg, 1),
Self::UnpackSequenceTwoTuple => (2, 1),
Self::WithExceptStart => (6, 5),
Self::WithExceptStart => (7, 6),
Self::YieldValue { .. } => (1, 1),
};
crates/compiler-core/src/bytecode/oparg.rs
+2 -2
View File
@@ -290,7 +290,7 @@ impl From<u32> for ResumeType {
fn from(value: u32) -> Self {
match value {
0 => Self::AtFuncStart,
1 => Self::AfterYield,
5 => Self::AfterYield,
2 => Self::AfterYieldFrom,
3 => Self::AfterAwait,
_ => Self::Other(value),
@@ -302,7 +302,7 @@ impl From<ResumeType> for u32 {
fn from(typ: ResumeType) -> Self {
match typ {
ResumeType::AtFuncStart => 0,
ResumeType::AfterYield => 1,
ResumeType::AfterYield => 5,
ResumeType::AfterYieldFrom => 2,
ResumeType::AfterAwait => 3,
ResumeType::Other(v) => v,
crates/vm/src/frame.rs
+22 -18
View File
@@ -2946,22 +2946,23 @@ impl ExecutingFrame<'_> {
Ok(None)
}
Instruction::LoadSpecial { method } => {
// Stack effect: 0 (replaces TOS with bound method)
// Input: [..., obj]
// Output: [..., bound_method]
// Pops obj, pushes (callable, self_or_null) for CALL convention.
// Push order: callable first (deeper), self_or_null on top.
use crate::vm::PyMethod;
let obj = self.pop_value();
let oparg = method.get(arg);
let method_name = get_special_method_name(oparg, vm);
let bound = match vm.get_special_method(&obj, method_name)? {
match vm.get_special_method(&obj, method_name)? {
Some(PyMethod::Function { target, func }) => {
crate::builtins::PyBoundMethod::new(target, func)
.into_ref(&vm.ctx)
.into()
self.push_value(func); // callable (deeper)
self.push_value(target); // self (TOS)
}
Some(PyMethod::Attribute(bound)) => {
self.push_value(bound); // callable (deeper)
self.push_null(); // NULL (TOS)
}
Some(PyMethod::Attribute(bound)) => bound,
None => {
return Err(vm.new_type_error(get_special_method_error_msg(
oparg,
@@ -2970,7 +2971,6 @@ impl ExecutingFrame<'_> {
)));
}
};
self.push_value(bound);
Ok(None)
}
Instruction::MakeFunction => self.execute_make_function(vm),
@@ -3522,24 +3522,28 @@ impl ExecutingFrame<'_> {
self.unpack_sequence(expected, vm)
}
Instruction::WithExceptStart => {
// Stack: [..., __exit__, lasti, prev_exc, exc]
// Call __exit__(type, value, tb) and push result
// __exit__ is at TOS-3 (below lasti, prev_exc, and exc)
// Stack: [..., exit_func, self_or_null, lasti, prev_exc, exc]
// exit_func at TOS-4, self_or_null at TOS-3
let exc = vm.current_exception();
let stack_len = self.localsplus.stack_len();
let exit = expect_unchecked(
self.localsplus.stack_index(stack_len - 4).clone(),
"WithExceptStart: __exit__ is NULL",
let exit_func = expect_unchecked(
self.localsplus.stack_index(stack_len - 5).clone(),
"WithExceptStart: exit_func is NULL",
);
let self_or_null = self.localsplus.stack_index(stack_len - 4).clone();
let args = if let Some(ref exc) = exc {
let (tp, val, tb) = if let Some(ref exc) = exc {
vm.split_exception(exc.clone())
} else {
(vm.ctx.none(), vm.ctx.none(), vm.ctx.none())
};
let exit_res = exit.call(args, vm)?;
// Push result on top of stack
let exit_res = if let Some(self_exit) = self_or_null {
exit_func.call((self_exit.to_pyobj(), tp, val, tb), vm)?
} else {
exit_func.call((tp, val, tb), vm)?
};
self.push_value(exit_res);
Ok(None)