* Add ExceptionGroup fields and OSError use-init
BaseExceptionGroup stores msg, excs, and excs_str; args keep
the original sequence. OSError defers parse when a subclass
overrides only __init__. characters_written can be deleted.
Exception args/traceback/cause/context reject invalid
setattr/delattr. UnicodeError.object is a deletable field.
except* remainder projection copies traceback/context/cause.
SyntaxError.__init__ clears omitted location attributes.
Assisted-by: Grok:grok-4.6
* Fix OSError payload init and UnicodeError construction
new_payload_exception now runs slot_new so BlockingIOError
parses characters_written. new_exception falls back to
invoke_exception for extra-payload types. Mixed tab/space
UnexpectedIndentation is TabError. AttributeError pickle
omits obj.
Assisted-by: Grok:grok-4.6
* address review: classify TabError from error line
Assisted-by: Grok:grok-4.6
* Add new_simple_exception; fill SyntaxError and f_lineno
new_exception constructs extra-payload types; BaseException-sized
types use new_simple_exception. Non-UTF-8 SyntaxError sets location
and text. Codegen errors load text via ProgramText. Keep Missing
parentheses capitalization. f-string adjacent atoms report a missing
comma. f_lineno may be None; lasti is not restored on unwind.
Assisted-by: Grok:grok-4.6
* address review: skip f-string continuation keywords
Assisted-by: Grok:grok-4.6
* Fix clippy map_unwrap_or in Non-UTF-8 offset
Assisted-by: Grok:grok-4.6
* Rename encode error ctor and exception setters
new_unicode_encode_error_real is new_unicode_encode_error.
set___traceback__/__cause__/__context__ are pygetset setters;
Rust uses set_traceback/set_cause/set_context.
Assisted-by: Grok:grok-4.6
* Use set_context in native pickle
Assisted-by: Grok:grok-4.6
* Collapse nested if in asyncio traceback restore
Assisted-by: Grok:grok-4.6
* Use current instruction for f_lineno addr2line
lasti is stored as the next instruction index. addr2line of
that byte offset is past RETURN_VALUE, so f_lineno was None
and stdlib_threading_current_frames hung on leftover workers.
Assisted-by: Grok:grok-4.6
* Look up format args, dir class, and frame builtins
Consume printf args before rejecting unknown conversions.
object.__dir__ uses getattr of __class__/__dict__.
_PyEval_GetBuiltin looks up iter/__import__ on the frame.
Assisted-by: Grok:grok-4.6
* Fix builtin lookups after rebase and unxfail passing tests
Assisted-by: Grok:grok-4.6
* Restore RERAISE lasti and SyntaxError locations
RERAISE with oparg restores lasti from the exception table
before unwind, so leftover f_lineno matches the original raise.
Handler lookup still uses the raising instruction.
Indentation errors outrank print/exec missing-parentheses.
BOM-with-non-utf8, bytes literals, and global+nonlocal now
report the same offsets as the parser/symtable.
object.__dir__ keeps string keys; type.__type_params__ skips
the getset wrapper stored on builtin types.
Assisted-by: Grok:grok-4.6
* Unxfail syntax/import tests and bind jump locals
IMPORT_NAME passes fromlist None instead of an empty tuple.
Jumping f_lineno binds None into unbound localsplus slots.
str lstrip/rstrip copy subclasses to exact str when unchanged.
Assisted-by: Grok:grok-4.6
* Fix import_from callers and rename frame lineno
sqlite3.iterdump now passes fromlist as a PyObject.
FrameObject::lineno is the i32 helper; f_lineno is the getset.
Assisted-by: Grok:grok-4.6
* Pass sqlite fromlist without Into conversion
from_list.into() could not infer Into<PyObjectRef> at the
import_from call. The typed tuple already converts.
Assisted-by: Grok:grok-4.6
* Add inline values flags and special-method lookup
lookup_maybe_method clears AttributeError from descr_get.
slot_tp_richcompare treats a miss as NotImplemented.
hackcheck rejects object.__setattr__ on types.
Heap types with a managed dict and itemsize 0 get
INLINE_VALUES; _testinternalcapi.has_inline_values
reports the valid bit.
super is a base type; subclass __get__ binds the instance.
__classcell__/__qualname__ may share a name with a slot.
Unset member access uses the type's fully qualified name.
Assisted-by: Grok:grok-4.6
* Call custom mro and descr_get for type.__new__
type.mro() computes C3 from the current bases.
A metaclass other than type supplies the MRO via mro().
super() compares obj.__class__ to the instance type.
slot_tp_new looks up __new__ through descr_get.
is_subtype walks the stored MRO without an identity shortcut.
__slots__ names are sorted so __class__ assignment matches.
Assisted-by: Grok:grok-4.6
* Expose _testinternalcapi recursion and rare events
get_recursion_depth feeds test.support.infinite_recursion.
SPECIALIZATION_* and SHARED_KEYS_MAX_SIZE match the
adaptive counters. Rare-event hooks increment on
__class__/__bases__/function/builtins writes.
Assisted-by: Grok:grok-4.6
* Add remaining _testinternalcapi code and locale APIs
Expose get_co_localskinds, get_code_var_counts, verify_stateless_code,
code_returns_only_none, normalize_path, EncodeLocaleEx, DecodeLocaleEx,
and run_in_subinterp_with_config. Mark unimplemented compiler IR and
specialization monitoring tests as expectedFailure.
Assisted-by: Grok:grok-4.6
* Fix lint for _testinternalcapi identifiers
Add CPython identifier spellings to the cspell dictionary and apply
the rustfmt signature that CI rewrote.
Assisted-by: Grok:grok-4.6
* Expand _testinternalcapi with remaining test APIs
Add constants, compiler_cleandoc, dict_getitem_knownhash, time
helpers, interpreter/frame accessors, configs, static types, a
persistent hamt map, and GC/size counters used by the test suite.
Assisted-by: Grok:grok-4.6
* Expose compiler IR APIs in _testinternalcapi
Add InstructionSequence and compiler_codegen, optimize_cfg, and
assemble_code_object. Drop expectedFailure marks on tests that pass.
Assisted-by: Grok:grok-4.6
* Match optimize_cfg consts and addop locations
Write optimized consts back into the caller's list, mark SETUP_*
pseudo-ops as having an argument, and store addop locations in
instruction-sequence field order. Drop expectedFailure marks on
tests that pass.
Assisted-by: Grok:grok-4.6
* Fix hamt API, SETUP_* HAS_ARG, and FOR_ITER events
Honor explicit pseudo flags when generating opcode metadata so
SETUP_FINALLY/CLEANUP/WITH keep HAS_ARG after regeneration.
Complete the _testinternalcapi hamt mapping: identity on
set/delete, richcompare, weakrefs, subscript, contains, and
length-bearing iterators.
Fire RAISE/STOP_ITERATION for FOR_ITER StopIteration, and skip
both END_FOR and POP_ITER for the exhausted-branch destination.
Assisted-by: Grok:grok-4.6
* Skip FOR_ITER RAISE for sequence iterators
List/range/tuple exhaustion matches FOR_ITER_LIST/RANGE/TUPLE and
must not emit a monitoring RAISE. That event stopped pdb's
monitoring backend on for-over-list with Internal StopIteration.
Assisted-by: Grok:grok-4.6
* Fix generator FOR_ITER events and review issues
Restore settrace StopIteration for generator for-loops so
test_bdb next/return cases pass, while sequence iterators still
skip RAISE.
Honor review notes: addop oparg bounds, linetable locations,
AST instance errors, MRO subclass lock, builtin-dict notify,
direct-base layout flags, and leftover _testinternalcapi args.
Assisted-by: Grok:grok-4.6
Pin the RustPython Ruff fork to the patch commit based on Ruff 0.16.5 and adapt compiler and AST integration to the updated APIs.
Assisted-by: Codex:GPT-5
* Guard all/any/tuple(genexpr) calls when the generator is a coroutine
`maybe_optimize_function_call()` reserves a `skip_optimization` label for
every `name(genexpr)` call and, for `all`, `any` and `tuple`, emits an
identity guard against the builtin plus an inlined loop. The port skipped
both whenever the generator expression's symbol table was a coroutine, so
`all(await x for x in xs)` and `all(x async for x in xs)` compiled to a
plain call.
An `await` or an `async for` in the generator does not disqualify the
shape: the inlined `FOR_ITER` raises the same `TypeError` that calling the
builtin on an async generator raises, which is what
test_builtin test_builtin_call_async_genexpr_no_crash asserts.
This was the last code-level difference against CPython across `Lib/`:
comparing every file's code tree by structure, opcode and constant value
(qualnames aside, which moved in 3.14.3+, and set constants compared by
value rather than by hash order) now matches on 1720 of 1720 comparable
files, up from 1719.
Assisted-by: Claude Code:claude-opus-5
* Name the ellipsis type `ellipsis`
`type(...).__name__`, its `__qualname__`, and `types.EllipsisType.__name__`
read `EllipsisType`.
Assisted-by: Claude Code:claude-opus-5
* Align compiler front end and instruction positions with CPython
Reject a bare generator expression in a class header and a format spec
nested more than two deep. Both parse here; the reference grammar has no
rule for either, since a class header only takes `arguments` and the
tokenizer runs out of nesting levels at the third format spec.
Scan the interpolations a format spec carries inside another format spec,
so `f"{x:{y:{z}}}"` reaches `z` when building the symbol table.
Give every generic function's type params scope a `.defaults` slot,
whether or not the signature has any default, so `co_varnames` and
`co_nlocals` match. Classes and type aliases keep theirs empty.
Decide an unparenthesized sole generator argument by the parser's
`parenthesized` flag instead of scanning the source for the shape of its
element. `_sum((d := x - c) * d for x in data)` opens with a parenthesized
group the rest of the expression continues, which the scan read as the
generator's own parentheses and left the call's out of its range.
Drop empty f-string fragments unconditionally. A concatenation whose
fragments are all empty now loads its empty string at the whole
concatenation instead of at whichever fragments happened to be plain
string literals.
Keep a byte order mark in source handed over as text, and report it as an
invalid non-printable character. Reading a file to freeze strips it, as
the encoded-bytes path already did.
Carry the filename into the errors `compile(..., PyCF_ONLY_AST)` raises;
they reported `<unknown>` or an empty name.
Across `Lib/`, the 1729 files that compile on both sides now agree on
opcodes, constants, flags, variable names, names, and exception tables,
and on instruction positions in all but 7 files, where a compound
statement whose body ends in a trailing `;` ends one column short.
Assisted-by: Claude Code:claude-opus-5
* Align ast module output with CPython
- Keep lone surrogates in string, f-string and t-string literal values by
reading such a literal from its source; the literal-value helpers move
out of `Compiler` into `rustpython_codegen` and `ConstantLiteral::Str`
holds `Wtf8Buf`.
- Give `{expr=}` interpolations their leading text as a Constant and
default their conversion to `repr`, in both f-strings and t-strings.
- Span folded consecutive literals from the first of them to the last.
- Take the `:` into a format spec's range only when the parser left it
out, instead of always widening by one.
- Report a node end that lands on a line start as that line, column 0.
- Give a generator expression written straight into a call's parentheses
the range of those parentheses, in the preprocess pass, replacing
codegen's source scan and `_ast`'s one-column adjustment.
- Drop the expectedFailure on test_ast_line_numbers_with_parentheses.
Assisted-by: Claude Code:claude-opus-5
* Align symbol table output with CPython
- Register an import as `DEF_IMPORT` alone, and an annotated parameter as
a plain parameter, so `import x; global x` is accepted and annotated
parameters are not reported as annotated names.
- Record a `global` declaration in the module block as well, and resolve a
name a `global` in an enclosing scope covers as an implicit global, so a
class body reading such a name emits `LOAD_NAME`.
- Resolve a name bound nowhere as an implicit global instead of leaving
its scope unknown.
- Require a binding only for a name an explicit `nonlocal` declared;
a free variable that reached the scope any other way already resolved.
- Name a generator expression's block `genexpr`; the code object keeps
`<genexpr>`.
- Locate a `def`/`class` block at its keyword rather than at the first
decorator. `decorated_definition_range()` moves from `Compiler` to
`rustpython_codegen` for that.
- Carry a comprehension symbol's flags over when inlining it into a parent
entry that only holds a free variable propagated from a child.
- Leave `__conditional_annotations__` out of an annotation block's
symbols and have the compiler cook up the free variable, which places
it after the ones the symbol table supplies.
- Check `from __future__` placement while building the symbol table, and
give `symtable.symtable()` the same docstring stripping and future
feature validation the compiler does.
- Floor `co_stacksize` at 1.
Assisted-by: Claude Code:claude-opus-5
* Locate the eval return and the interactive print like CPython
Leave `eval` mode's `RETURN_VALUE` without a location, so the exit block
is duplicated per reaching path and each copy carries the location of the
branch that jumps to it. `a and b` now ends with two returns as it does
in CPython, and `a if b else c` gives the first one the `a` it returns.
Emit an interactive `CALL_INTRINSIC_1(Print)` at the expression statement
rather than wherever the expression left the location, which differ for
`(a := 10)` and `f'{0:fz}'`.
Assisted-by: Claude Code:claude-opus-5
* Give interactive mode the conditional annotations cell
`compile_program_single()` emitted the `__conditional_annotations__` set
and its name accesses but never added the cell that holds it, so a module
compiled in `single` mode was missing the `MAKE_CELL` and the cellvar that
the same source gets in `exec` mode.
Assisted-by: Claude Code:claude-opus-5
* Give compiler-raised SyntaxErrors an end position
`CodegenError` and `SymbolTableError` now carry an `end_location`
alongside `location`, and `CompileError::python_end_location()` returns
it, so errors such as "'await' outside function" set `end_lineno` and
`end_offset` instead of leaving them None.
`SymbolTableBuilder::error_ranged()` builds both positions from a range,
replacing the inline `source_location()` calls at the error sites.
Drop the `expectedFailure` on
`test_exceptions.SyntaxErrorTests.test_file_source`, which now passes.
Assisted-by: Claude Code:claude-opus-5
* Report parser diagnostics with CPython's wording
`InvalidAnnotatedAssignmentTarget` becomes "illegal target for
annotation", `UnexpectedExpressionToken` becomes plain "invalid syntax"
rather than prefixing the parser's text, and the list-recovery
diagnostics ("Expected an expression or a '}'" and its siblings) join the
two that already mapped to "invalid syntax".
Assisted-by: Claude Code:claude-opus-5
* Punctuate the code object repr like CPython
`repr(code)` was missing the comma after the address and escaped the
filename through Rust's Debug formatting.
Drop the `expectedFailure` on the six `test_dis` tests that compare
disassembly text, which now pass.
Assisted-by: Claude Code:claude-opus-5
* Match CPython's parser error positions
A syntax error the parser reports counts its columns in characters, so
`source_location()` converts with UTF-32 rather than UTF-8; an offset
inside a character walks back to where that character starts.
Spans the parser reported differently:
- an error between two tokens covered nothing, and now covers one
character, since the narrowest token still covers one
- a "was never closed" bracket ends at column 0
- an unterminated string literal ends where it starts
- "leading zeros in decimal integer literals" spans the run of zeros
- an exponent with no digits and no sign is blamed on the digit before
the exponent letter, which the tokenizer puts back
Assisted-by: Claude Code:claude-opus-5
* codegen: seek to the finally body's own scopes before copying it
An early exit from a try block emits an extra copy of the finally body.
Nested scopes are handed out by position, and the cursors still sit inside
the try block's own run of scopes there, so the copy took those instead of
the ones the finally body opens: a generator expression got a sibling's
symbol table and raised "the symbol 'k' must be present in the symbol
table", or, where the sibling was a named scope, built a code object with
no `.0` argument that raised UnboundLocalError when it ran.
Seek the cursors to the first scope beginning on or after the finally
body's first line before compiling the copy. The existing restore then
leaves the try statement's own copies starting from the same place.
Assisted-by: Claude
* Align compiler futures and annotations with CPython
Handle Barry parsing consistently across AST, type-comment, REPL, and WASM paths. Preserve deferred annotation source and scope metadata, and make _symtable conversion cached and linear-time.
Assisted-by: Codex:gpt-5.4
* Fix f-string debug ranges and update Barry/annotate test expectations
Compute the debug f-string source range from the raw leading and
trailing text instead of the comment-stripped text, so a comment inside
a multi-line replacement field no longer shifts the emitted LOAD_CONST
location.
Restrict the obsolete-operator location shift to `<>` by also requiring
a `>` at the reported offset; without it any `ExpectedExpression` that
followed a `<` (`2 <;`) was reported one column to the left.
Add `CodeFlags::FUTURE_MASK` and use it in builtins, the shell, and the
WASM VM instead of repeating the eight-flag union, and collapse the
duplicated Barry diagnostic sequences into `BarrySource::diagnostic`.
Update the `__annotate__` disassembly snapshot to LOAD_FAST_BORROW and
replace the RustPython-only `barry_as_FLUFL` co_flags assertion in
extra_tests with the CPython-matching one.
Assisted-by: Claude Code:claude-opus-5
* Restore the annotated function in __annotate__ qualnames
Signature annotation scopes are compiled in the scope enclosing the
annotated function, so the function name has to be folded back into the
qualname: `f.__annotate__`, `C.m.__annotate__`,
`outer.<locals>.inner.__annotate__`. test_type_annotations
test_annotate_qualname covers this.
test_pyrepl test_future_barry_as_flufl now passes, so drop its
expectedFailure marker.
Assisted-by: Claude Code:claude-opus-5
* Pass safepoints in the attached-thread interpreter tests
`main_and_subinterpreter_run_sections_overlap` parked its workers on a
condvar while attached, and `busy_main_interpreter_does_not_block_subinterpreter`
looped over protocol calls that never reach a safepoint. An attached
thread that never reaches a safepoint cannot be suspended, so a
concurrent process-wide stop-the-world (from the gc test running in
parallel) never completes and the already-stopped sibling interpreter
stays stopped; the second worker then waits in `wait_while_suspended`
until the 30s deadline.
Both loops now call `vm.check_signals()` each iteration, and the condvar
wait uses `wait_timeout` so the lock is released between safepoints.
Reproduced on Linux (4 CPUs): the two tests with `--test-threads=2`
failed 2/40 before and 0/40 after; the full `rustpython-vm` lib binary
failed 1/30 before and 0/30 after.
Assisted-by: Claude Code:claude-opus-5
* Report the obsolete `<>` operator from one place
`<>` was located twice: `BarrySource` recorded the token ranges it
rewrote to `!=`, and `cpython_parse_diagnostic_override` separately
peeked at the bytes around an `ExpectedExpression` location. Both
produced "invalid syntax" spanning the whole operator.
`prepare_barry_as_flufl_source` now records the operator ranges in the
non-Barry case too -- by plain text search, since nothing is rewritten
there -- so `BarrySource::invalid_legacy_operator` covers both, and the
byte-peeking helper is gone. It also defers to an unclosed bracket
earlier in the source, which the diagnostic chain it used to sit in did
for it.
A `<>` that starts a statement now spans both characters: the parser
stops at the `<` rather than one character in, which the byte peek did
not handle.
Assisted-by: Claude Code:claude-opus-5
* Drop a py object's payload before its class
`PyInner` declares `typ` before `payload`, so `drop_in_place` dropped the
class first and `PyAtomicRef::drop` left it null. A `PyWeak` payload is
still linked into the weakref list of the object it points at until its
own `Drop` unlinks it under the stripe lock, which happens after that.
`WeakRefList::add` walks the same list looking for a proxy to reuse and
reads `node.class()` off every node it passes, so it could dereference
the null class of a node on its way out.
`PyInner::dealloc` now runs the two destructors in the other order at all
three of its exit paths, with a compile-time check that no other field
has one.
Reproduced by the new test in about 0.1s, as the panic CI hit in the
c-api `weakrefobject::tests::new_proxy` run:
panicked at crates/vm/src/object/ext.rs:312:
unsafe precondition(s) violated: hint::unreachable_unchecked must
never be reached
Assisted-by: Claude Code:claude-opus-5
* Remove the private assignment-expression marker
Why:
extend_namedexpr_scope() already propagates assignment-expression
targets and records DEF_GLOBAL or DEF_NONLOCAL in comprehension scopes.
ASSIGNED_IN_COMPREHENSION remained only for conflict checks. Its private
bit overlaps CPython's packed LOCAL scope bit.
Changes:
- Register assignment-expression targets as ordinary assignments.
- Detect later target conflicts from existing declaration flags.
Assisted-by: Codex:gpt-5.6-sol
* Remove the private comprehension iterator flag
Why:
ITER duplicates the DEF_LOCAL and DEF_COMP_ITER facts already recorded
for comprehension targets. Its private bit also overlaps CPython's
packed scope field.
Changes:
- Use those flags for rebinding checks and local restoration around
inlined comprehensions.
Assisted-by: Codex:gpt-5.6-sol
* Preserve source names in named-expression diagnostics
Why:
Assignment-expression diagnostics exposed the mangled keys used for
symbol-table lookup. CPython reports identifiers as written in source.
Changes:
- Use source names in both comprehension conflict diagnostics.
- Enable test_named_expression_invalid_mangled_class_variables.
Assisted-by: Codex:gpt-5.6-sol
* Fix SymbolUsge::Iter
* fix os.rename
* fix sre
* Re-export rename from crate::posix on Windows
posix_windows.rs is mounted as the `posix` module on Windows, so
`crate::posix_windows` does not resolve.
Assisted-by: Claude
* Apply rustfmt to sre_engine tests
Assisted-by: Claude
* Update test expectations for the sre and tokenize fixes
- test_re: drop expectedFailure from test_word_boundaries,
test_possessive_quantifiers and test_bug_gh101955
- test_inspect, test_pydoc: drop expectedFailure from the signature
tests that now pass
- test_pdb: drop expectedFailure from the two file-modification tests
and +EXPECTED_FAILURE from the test_post_mortem_chained and
test_pdb_asynctask doctests
- test_unittest: mark test_autospec_on_bound_builtin_function as
expectedFailure; inspect.signature() now succeeds on time.ctime
because of its auto-generated __text_signature__
Assisted-by: Claude
* Fix clippy warning
* Fix clippy warnings surfaced by newer clippy
- exceptions.rs: {b:#02x} -> {b:#04x} (unused_format_specs); output
unchanged for decode-error bytes (always >= 0x80)
- binascii.rs: [b'\r', b'\n'] -> b"\r\n" (byte_str_slice)
- host_env, _io.rs, ssl.rs, pyexpat.rs: expect std_instead_of_core where
the suggested core::io items (ErrorKind, Cursor) are unstable (core_io);
use expect so the suppression is flagged for removal once core::io
stabilizes. build_posix_spawn_attrs co-gates the expect with the cfg
block so it is not left unfulfilled on platforms compiling it out.
Assisted-by: Claude
* Fix more
---------
Co-authored-by: Jeong YunWon <jeong@youknowone.org>
* Extract unicodedata core into rustpython-unicode crate
Move the vm-free Unicode character database access — the generated UCD 3.2.0 /
latest tables, their build-time generation, and the icu4x/unicode_names2 lookups
for category/bidirectional/combining/east_asian_width/mirrored/decomposition/
normalize/is_normalized/digit/decimal/numeric/name/lookup — into a new leaf crate
over char/CodePoint/&Wtf8. stdlib/unicodedata.rs keeps the UCD pyclass binding: it
extracts the code point, boxes results, and maps errors. The unicode data files,
build.rs table generation, and the icu_properties/icu_normalizer/unicode_names2
dependencies move out of stdlib.
Assisted-by: Claude
* Route str predicates, casefold, and printable through rustpython-unicode
Add classify/case/identifier modules to the shared crate over char/&str/&Wtf8:
isalpha/isalnum/isdecimal/isdigit/isnumeric/isspace/isprintable classification,
XID identifier predicates, and full-mapping casefold. vm/builtins/str.rs and
literal/char.rs now call these instead of icu_properties/icu_casemap directly.
String-level iteration with final-sigma handling (lower/upper/title/capitalize,
islower/isupper) stays in the runtime.
Assisted-by: Claude
* Route sre_engine char classes through rustpython-unicode
Add the regex module with the SRE character-class and case predicates
(is_word/is_space/is_digit, is_uni_* Unicode variants, ascii/locale/unicode case
folding). sre_engine/string.rs now forwards to it, keeping its public API and
behavior identical — including the ASCII-only is_uni_digit and the hardcoded
is_uni_space table.
Assisted-by: Claude
* Route \N{} name lookups through rustpython-unicode
codegen/string_parser.rs (\N{...} escapes) and common/encodings.rs (the
namereplace error handler) now resolve character names via rustpython-unicode
instead of depending on unicode_names2 directly, leaving the crate as the sole
owner of the name database.
Assisted-by: Claude
* Add differential Unicode sweep and shared-crate snippet test
tests/differential.rs sweeps the full 0..0x110000 range and compares each str
classification predicate against a committed CPython 3.14 reference dataset
(tests/data/cpython3.14_predicates.txt, produced by generate_reference.py). Code
points that differ only because the Rust std / icu4x build ships a later Unicode
release than CPython 3.14's 16.0.0 are recorded in
tests/data/version_skew_cpython3.14.txt, regenerable via
RUSTPYTHON_UNICODE_REGEN_SKEW=1; the regen refuses to record any
cpython=true/crate=false divergence, so only newly-assigned code points are
allowed. Any other divergence fails. Both data files use a run-length
`predicate start:end,...` encoding.
extra_tests/snippets/stdlib_unicode_shared.py exercises the routed surface end to
end (str predicates, casefold, identifiers, unicodedata, normalize, \N{}, and re
character classes) and passes identically on CPython 3.14 and RustPython.
Assisted-by: Claude
* Match regex \d to Unicode decimal digits
is_uni_digit previously matched only ASCII 0-9, so re's \d in Unicode mode
missed decimal digits like ٥ and ५. SRE_UNI_IS_DIGIT matches Py_UNICODE_ISDECIMAL
(category Nd), so route it through classify::is_decimal. Unmasks test_bug_6561 in
test_re and extends the shared-crate snippet with the Nd/Nl/No cases.
Assisted-by: Claude
* Replace shallow Unicode wrappers with re-exports
Address review feedback on the extraction.
Functions that only forwarded to the shared crate become `use ... as`
re-exports rather than hand-written wrappers:
- literal: is_printable is dropped; escape.rs calls
rustpython_unicode::classify::is_repr_printable directly.
- unicode::identifier::is_continue re-exports is_xid_continue.
- unicode::data::lookup_character re-exports unicode_names2::character.
- Drop the unused repr(u8) on DecompositionType.
The SRE character-class and case predicates move back into
sre_engine::string (their pre-extraction home) instead of living in a
unicode::regex module that only sre_engine and the vm _sre binding used;
is_uni_digit/is_uni_alnum still delegate to rustpython_unicode::classify.
engine.rs and _sre consume them from sre_engine::string again.
The unicode crate's tests move from a single root tests module in lib.rs
into per-module test submodules (case, classify, data, identifier,
normalize).
Assisted-by: Claude
* no-std
* Address review feedback: test module, dead variant, WTF-8 normalization
- differential.rs: move the sweep tests into a `mod tests` block and drop
the file-level allow of clippy::tests_outside_test_module and
std_instead_of_alloc; the test uses alloc collections instead.
- data.rs: remove the never-constructed DecompositionType::Canonical variant
and its allow(unused); compatibility decomposition never produces it and
canonical decomposition is handled through icu4x.
- normalize.rs: is_normalized now takes &Wtf8 and checks each UTF-8 run,
skipping lone surrogates, matching normalize's run-wise behavior.
- unicodedata.rs: pass as_wtf8() to is_normalized.
Assisted-by: Claude
* Align compiler and AST behavior with CPython
Update the compiler pipeline, AST module support, and related VM compile/eval paths toward CPython 3.14 behavior.
Use upstream Ruff parser crates directly and keep RustPython-specific syntax preflight handling in the compile path.
Refresh opcode metadata, snapshots, and targeted tests for the aligned bytecode and AST behavior.
* fix
* Fix PR 8138 CI failures
* Fix match mapping wildcard scan complexity
* fixes
* Use Ruff runtime AST fields
* Minimize residual patch diff
* refactor error handling
* Remove _ast conversion context wrappers
* Clean up _ast conversion glue and compiler warning escalation
Rename the Node conversion parameter `ctx` to `vm` throughout the _ast
module.
Remove the unused compile_program, compile_program_single,
compile_block_expression, and compile_expression forwarders; compile_top
stays as the entry point.
Stash an escalated compiler SyntaxWarning so a non-SyntaxWarning category
propagates unchanged instead of being rewritten to SyntaxError, matching
PyErr_ExceptionMatches(SyntaxWarning) in compiler_warn. Drop the now-unused
CompileWarningError::into_codegen_error.
Replace the PositionalArguments two-variant enum with a struct holding a
shared range and a PositionalArgumentsKind enum.
Assisted-by: Claude
* Represent compiler flags as a CompilerFlags bitflags
Replace the loose PY_CF_*/CO_* i32 constants in compile_mode with a
CompilerFlags bitflags covering the PyCF_* flags and __future__
CO_FUTURE_* bits. The Python-visible ast.PyCF_* attribute values are
derived from it via bits(); parser input modes stay plain consts.
builtins::compile and the compile_string flag handling test flags
through CompilerFlags::contains instead of raw bit math.
Assisted-by: Claude
* Fix PR 8138 test regressions
* Fix PR 8138 CI failures
* Avoid shell interpolation in Linux deps action
* Fix review findings in compiler/AST changes
- frame.rs: compute the MATCH_CLASS class name lazily so a successful
match no longer takes the type name lock or allocates; the
__match_args__-not-a-tuple branch reuses it instead of recomputing.
- _ast/constant.rs: add the missing leading space to the with_recursion
label ("maximum recursion depth exceeded during compilation").
- sys.set_int_max_str_digits: restore the "maxdigits must be 0 or larger
than {n}" error text.
- codegen: set CO_MAXBLOCKS to 20 in both ir.rs and compile.rs.
- _ast/exception.rs: extract the duplicated excepthandler node guard
into ensure_excepthandler_node.
- codegen: correct the validate_keywords doc comment.
- Drop redundant "CPython" qualifiers from comments added in this branch.
Assisted-by: Claude
* General nitpicks
* Move Compiler::constant_truthiness to a ConstantData method
* More code cleanup
* bytecode::CodeFlags -> CodeFlags
* More code clenaups
* Share marshal ref table between code object and its internals
read_marshal_bytes, _str, _str_vec, _name_tuple, and _const_tuple now
take a shared ref table and resolve TYPE_REF / register FLAG_REF
entries. deserialize_code is split into a public wrapper and an inner
function that receives the ref table; deserialize_value_depth opens a
fresh inner ref space when it hits Type::Code, mirroring CPython's
behaviour of putting the code object itself at ref slot 0. Nested code
objects inside const tuples reuse the surrounding code's ref space via
the new read_const_value helper.
* Align PYC magic number, FORMAT_VERSION, and header check with CPython 3.14
PYC_MAGIC_NUMBER changes from 2994 to 3627, matching CPython 3.14's
pyc_magic_number_token (0x0a0d0e2b). marshal FORMAT_VERSION drops from
5 to 4 (the encoder/marshal.version value; the decoder already accepts
both). check_pyc_magic_number_bytes now compares all four magic bytes
instead of the first two.
* Accept CPython-tagged .pyc as read-only bytecode source
SourceFileLoader.get_code now also looks for .pyc files using
_RP_FALLBACK_CACHE_TAGS (currently ('cpython-314',)) in addition to
sys.implementation.cache_tag. The matched .pyc is only used for
reading; recompilation still writes to the RustPython-tagged path, so
CPython's .pyc is never overwritten. Source-stat / hash / timestamp
validation logic is unchanged.
* Apply rustfmt to marshal helpers
* Marshal PySlice from format version 4 instead of 5
CPython's marshal supports TYPE_SLICE from format version 4 onwards
and that is the default version. Rejecting slice dumps below version
5 made marshal.dumps(slice(...)) fail with the default version and
broke test.test_marshal.SliceTestCase.test_slice.
* Revert "Accept CPython-tagged .pyc as read-only bytecode source"
Lib/importlib/_bootstrap_external.py is CPython's own code copied
verbatim; local patches here defeat compatibility tracking. The
cpython-XX cache_tag fallback needs to live on the RustPython side
(Rust code or sys.implementation.cache_tag policy), not as edits to
the imported standard library.
This reverts commit 1fc426d0fb5fcdb50d35cad13bbb43e8f6ce1c7f.
* Set marshal FORMAT_VERSION to 5 to match CPython 3.14.5
Py_MARSHAL_VERSION is 5 in CPython 3.14.5 (Include/marshal.h:16) and
TYPE_SLICE serialization rejects version < 5 (Python/marshal.c:720).
Restore the same threshold and constant so marshal.version and the
slice-marshal gate match CPython.
* Thread marshal recursion depth through nested code objects
Code objects embedded in const-tuples reset the depth budget on each
recursion, so a hostile or pathological marshal stream of code-in-tuple-
in-code can blow the stack despite MAX_MARSHAL_STACK_DEPTH. Pass the
current depth through deserialize_code_inner and read_marshal_const_tuple
and decrement at each code-object/tuple boundary.
Also route dict keys through deserialize_value_after_header so TYPE_CODE
keys decode instead of failing with BadType.
* align compiler to CPython
* Align codegen with CPython compile.c
Rename CFG helpers and accessors to the names used in CPython's
compile.c (basicblock_next_instr, basicblock_last_instr,
basicblock_append_instructions, bb_has_fallthrough, is_jump,
make_cfg_traversal_stack, mark_warm/mark_cold, etc.). Drop the unused
boolop-folding gate, mark_cpython_cfg_label_block helper, and
ComprehensionLoopControl::iter_range field.
Track an is_coroutine flag on SymbolTable, set in async def, await, and
async comprehensions, and propagate it through non-generator
comprehensions per symtable_handle_comprehension().
Mark SetupCleanup/SetupFinally/SetupWith as has_arg pseudo-ops, mark
ForIter as a terminator, and add has_arg/has_const on AnyInstruction.
Fix Instruction::stack_effect_jump to delegate to the opcode's
stack_effect_jump rather than stack_effect.
* Align codegen IR with CPython CFG structures
* Match CPython CFG annotation offset arithmetic
* Propagate CPython CFG label translation errors
* Align CPython exception target labeling flow
* Propagate CPython CFG traversal stack allocation errors
* Match CPython optimize_load_fast allocation flow
* Propagate CPython basicblock allocation errors
* Propagate CPython redundant NOP cleanup errors
* Propagate CPython unused const cleanup errors
* Propagate CPython const folding errors
* Propagate CPython swaptimize allocation errors
* Match CPython list-to-tuple fold allocation
* Skip const folding on CPython allocation failures
* Skip subscript folding on CPython allocation failures
* Propagate CPython assembler allocation errors
* Propagate CPython localsplus allocation errors
* Propagate CPython localsplus setup allocation errors
* Propagate CPython jump label map allocation errors
* Propagate CPython instruction sequence allocation errors
* Propagate CPython instruction label allocation errors
* Guard CPython codegen block allocation
* Guard CPython label shadow allocation
* Propagate CPython label shadow allocation errors
* Align CPython c-array allocation updates
* Align CPython ref stack growth
* Match CPython CFG builder debug check
* Avoid Rust-only CFG append clone
* Reuse CPython cleared block slots
* Use CPython block append in copy_basicblock
* Match CPython cfg builder creation order
* Clear label map after CPython apply pass
* Match CPython cfg builder allocation check
* Propagate CPython c-array size errors
* Drop Rust-only label uniqueness check
* Drop Rust-only label shadow debug checks
* Propagate CFG block index overflow
* Propagate CFG label oparg overflow
* Model CPython basicblock instruction storage
* Drop Rust-only recorded CFG precheck
* Model CPython instruction sequence storage
* Propagate instruction sequence offset overflow
* Model CPython instruction sequence labels
* Match CPython jump offset arithmetic
* Match CPython exception table arithmetic
* Match CPython label index arithmetic
* Match CPython instruction offset casts
* Match CPython jump offset indexing
* Match CPython oparg locals casts
* Match CPython localsplus offset arithmetic
* Match CPython cell prefix indexing
* Match CPython C array growth arithmetic
* Match CPython label map allocation arithmetic
* Match CPython label map size tracking
* Use CPython label map size in sequence passes
* Assert CPython label map clearing invariants
* Match CPython label oparg assignment
* Match CPython compiler direct arithmetic
* Match CPython load fast local casts
* Match CPython load fast depth assert
* Match CPython resume depth flagging
* Match CPython stack depth arithmetic
* Drop Rust-only stack overflow error
* Return CPython stack depth directly
* Match CPython C array growth errors
* Match CPython instruction insert asserts
* Match CPython unreachable pseudo jump
* Match CPython CFG size guard
* Match CPython superinstruction assert
* Match CPython redundant jump assert
* Match CPython stackdepth errors
* Match CPython jump offset flow
* Match CPython assembler buffer defaults
* Match CPython bytecode emit growth
* Match CPython assembler entry growth
* Match CPython assembler growth overflow check
* Match CPython remove_unreachable structure
* Match CPython static swap flow
* Inline CPython code unit preprocessing
* Match CPython C array growth checks
* Match CPython label map size guard
* Match CPython load-fast flow
* Simplify CPython CFG condition flow
* Align exception fallthrough propagation
* Match CPython pseudo target table
* Match CPython annotations CFG assert
* Match CPython inverted op assert
* Match CPython many-locals guard
* Reject deopt opcodes in CFG stack effects
* Match CPython invalid stack effect error
* Test CPython deopt stack effect guard
* Match CPython load-fast extended-arg assert
* Match CPython instruction allocation asserts
* Match CPython basicblock last-instr asserts
* Match CPython opcode range asserts
* Assert CPython fallthrough line propagation invariant
* Assert CPython CFG target offset sign
* Assert CPython exception fallthrough invariant
* Assert CPython exception stack bounds
* Assert CPython traversal stack allocation
* Match CPython label-map allocation in shadow
* Mirror CPython label-map sentinel fill
* Match CPython CFG builder allocation asserts
* Match CPython exception stack structure
* Match CPython ref stack structure
* Match CPython CFG traversal stack structure
* Mirror CPython CFG traversal stack pointer
* Use CPython fixed exception handler stack
* Mirror CPython ref stack capacity field
* Match CPython swap optimizer scratch stack
* Align static swap helpers with CPython blocks
* Align swaptimize signature with CPython
* Match CPython redundant pair pass result
* Match CPython inline pass result
* Match CPython redundant NOP pass results
* Fix bytecode metadata after upstream rebase
* Match CPython opcode stack metadata
* Add CPython identifiers to cspell dictionary
Add CNOTAB, LNOTAB, ialloc, ioffset, iused, nblocks, ncellsused,
ncellvars, nextop, noffsets, nvars, swaptimize, untargeted to
.cspell.dict/cpython.txt for the new CFG/assembler code in
crates/codegen/src/ir.rs.
* Fix CI failures from bytecode-parity work
- clippy: drop redundant `test_` prefix on three test functions and
remove an unnecessary `u32` cast in basicblock_clear_reuses_cpython_spare_slots_in_offset_order
- insta: regenerate nested_double_async_with snapshot to match the new
CFG output that drops unreferenced labels after the redundant-NOP pass
- regrtest: drop `@expectedFailure` markers from test_func_args,
test_meth_args (test_compile), test_disassemble_with,
test_disassemble_try_finally (test_dis), and test_except_star
(test_monitoring) which now pass
* Resync generated opcode metadata
Empty conf.toml since WithExceptStart and Setup{Cleanup,Finally,With}
stack effects already match CPython, so the TODO override entries are
stale and only cause CI hook diffs.
Regenerate opcode_metadata.rs and drop the matching SetupCleanup/
SetupFinally/SetupWith assertions on PseudoOpcode::has_arg(); their
`HAS_ARG` flag comes from pseudo definitions in bytecodes.c that the
upstream analyzer does not propagate through PseudoInstruction.properties,
so the generated has_arg() excludes them. has_target() still covers
these block-push pseudos via is_block_push().
* Drop is_block_push has_arg invariant
The CPython invariant `assert(OPCODE_HAS_ARG(op) || !IS_BLOCK_PUSH(op))`
relies on SETUP_{FINALLY,CLEANUP,WITH} carrying `HAS_ARG_FLAG` in
CPython's metadata. The autogen tool reads pseudo-opcode properties from
target instructions and does not propagate the pseudo's own
HAS_ARG flag, so PseudoOpcode::has_arg() omits these three opcodes.
Drop the debug_assert that fired inside py_freeze proc-macro expansion.
* Auto-generate has_eval_break and route AnyInstruction has_arg/has_const via macro
Add fn_has_eval_break to generate_rs_opcode_metadata.py using CPython's
Properties.eval_breaker, removing the hand-written matches! body for
Opcode::has_eval_break and PseudoOpcode::has_eval_break.
Forward has_arg/has_const from Instruction and PseudoInstruction to
their opcode, so AnyInstruction can use either_real_pseudo! like the
other has_* accessors instead of an open-coded match.
* Add `rapidhash` to list of dependencies
* Use `rapidhash::quality::RandomState` in `codegen` crate
* Use `rapidhash::quality::RandomState` in `stdlib` crate
* Use `rapidhash::quality::RandomState` in `vm` crate
* Remove `ahash` from lists of dependencies
* Align codegen CFG cleanup with CPython
* Align codegen bytecode with CPython 3.14
* Remove remove_jump_target_line_nops
The pass dropped a same-line NOP at the start of any jump-targeted block
whose body advanced to a later line, but CPython's basicblock_remove_redundant_nops
only consults the previous/next instruction inside the same block. The
extra pass deleted the else-body line trace anchor exercised by
test_nested_double_async_with.
Ignore test_conditional_break_finally_does_not_keep_break_cleanup_nop;
the break NOP lands in a separate block from the inlined finally body,
so same-line successor elision does not apply here.
* Keep except_handler blocks reachable in eliminate_unreachable_blocks
After convert_pseudo_ops lowers SETUP_FINALLY to a plain NOP, the only
remaining link from the try body to the except_handler block was the
per-instruction except_handler annotation. When earlier passes had
already removed every NOP that carried that annotation (e.g. an empty
try body with `pass`), the handler block became unreachable from the
entry block and its instructions were cleared, dropping the handler
entirely. Seed reachability with blocks already marked except_handler
so handler dispatch survives independent of the in-block annotation.
Also drop two expectedFailure markers in test_patma whose match-tracing
expectations now pass.
* Align bytecode CFG cleanup with CPython
* Add tests for Mode::BlockExpr and VirtualMachine::run_block_expr
test_block_expr_return_const() panics currently.
* Fix "tried to pop from empty stack" with BlockExpr and const
Constant expressions without side effects are optimized away unless in
an interactive mode[1]. This causes panic when last statement in a block
is a constant and Mode::BlockExpr is used as this constant value must be
returned to caller.
Example panic:
---- vm::python_run::tests::test_block_expr_return_const stdout ----
[crates/vm/src/frame.rs:9653:9] self = ExecutingFrame {
code: code: <code object <module> at ??? file "<embedded>", line 1>,
stack_len: 0,
}
thread 'vm::python_run::tests::test_block_expr_return_const' (2640752) panicked at crates/vm/src/frame.rs:9410:18:
tried to pop from empty stack
stack backtrace:
0: __rustc::rust_begin_unwind
at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/std/src/panicking.rs:689:5
1: core::panicking::panic_fmt
at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/core/src/panicking.rs:80:14
2: rustpython_vm::frame::ExecutingFrame::fatal
at ./src/frame.rs:9654:9
3: rustpython_vm::frame::ExecutingFrame::pop_stackref_opt
at ./src/frame.rs:9410:18
4: rustpython_vm::frame::ExecutingFrame::pop_stackref
at ./src/frame.rs:9420:18
5: rustpython_vm::frame::ExecutingFrame::pop_value
at ./src/frame.rs:9435:14
6: rustpython_vm::frame::ExecutingFrame::execute_instruction
at ./src/frame.rs:3391:34
7: rustpython_vm::frame::ExecutingFrame::run
at ./src/frame.rs:1623:31
8: rustpython_vm::frame::<impl rustpython_vm::object::core::Py<rustpython_vm::frame::Frame>>::run::{{closure}}
at ./src/frame.rs:1098:44
9: rustpython_vm::frame::<impl rustpython_vm::object::core::Py<rustpython_vm::frame::Frame>>::with_exec
at ./src/frame.rs:1093:9
10: rustpython_vm::frame::<impl rustpython_vm::object::core::Py<rustpython_vm::frame::Frame>>::run
at ./src/frame.rs:1098:14
11: rustpython_vm::vm::VirtualMachine::run_frame::{{closure}}
at ./src/vm/mod.rs:1201:44
12: rustpython_vm::vm::VirtualMachine::with_frame_impl::{{closure}}::{{closure}}
at ./src/vm/mod.rs:1610:60
13: rustpython_vm::vm::VirtualMachine::dispatch_traced_frame
at ./src/vm/mod.rs:1689:22
14: rustpython_vm::vm::VirtualMachine::with_frame_impl::{{closure}}
at ./src/vm/mod.rs:1610:22
15: rustpython_vm::vm::VirtualMachine::with_recursion
at ./src/vm/mod.rs:1547:9
16: rustpython_vm::vm::VirtualMachine::with_frame_impl
at ./src/vm/mod.rs:1572:14
17: rustpython_vm::vm::VirtualMachine::with_frame
at ./src/vm/mod.rs:1555:14
18: rustpython_vm::vm::VirtualMachine::run_frame
at ./src/vm/mod.rs:1201:20
19: rustpython_vm::vm::VirtualMachine::run_code_obj
at ./src/vm/mod.rs:1120:14
20: rustpython_vm::vm::python_run::<impl rustpython_vm::vm::VirtualMachine>::run_block_expr
at ./src/vm/python_run.rs:51:14
21: rustpython_vm::vm::python_run::tests::test_block_expr_return_const::{{closure}}
at ./src/vm/python_run.rs:190:47
22: rustpython_vm::vm::interpreter::Interpreter::enter::{{closure}}
at ./src/vm/interpreter.rs:366:39
23: rustpython_vm::vm::thread::set_current_vm::{{closure}}
at ./src/vm/thread.rs:107:9
24: std::thread::local::LocalKey<T>::try_with
at /home/im/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:513:12
25: std::thread::local::LocalKey<T>::with
at /home/im/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:477:20
26: rustpython_vm::vm::thread::set_current_vm
at ./src/vm/thread.rs:102:14
27: rustpython_vm::vm::thread::enter_vm
at ./src/vm/thread.rs:132:5
28: rustpython_vm::vm::interpreter::Interpreter::enter
at ./src/vm/interpreter.rs:366:9
29: rustpython_vm::vm::python_run::tests::test_block_expr_return_const
at ./src/vm/python_run.rs:188:23
30: rustpython_vm::vm::python_run::tests::test_block_expr_return_const::{{closure}}
at ./src/vm/python_run.rs:187:38
31: core::ops::function::FnOnce::call_once
at /home/im/.rustup/toolchains/stable-aarch64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
32: <fn() -> core::result::Result<(), alloc::string::String> as core::ops::function::FnOnce<()>>::call_once
at /rustc/59807616e1fa2540724bfbac14d7976d7e4a3860/library/core/src/ops/function.rs:250:5
[1] https://github.com/RustPython/RustPython/blob/883ce9d273db98b435eb3f51128925a1255ba88d/crates/codegen/src/compile.rs#L2990-L2997
* Align LOAD_FAST_BORROW analysis with CPython chain shape
Three changes that bring optimize_load_fast_borrow closer to CPython's
optimize_load_fast in flowgraph.c:
* ir.rs: split mark_cold into the CPython-style two passes. Phase 1
propagates "warm" from the entry block, phase 2 propagates "cold" from
except_handler blocks. Blocks reached by neither phase keep cold=false
and stay in their original b_next position, matching CPython's handling
of empty placeholders left by remove_unreachable (e.g. the inner_end of
a nested try/except whose incoming jumps were re-routed by optimize_cfg).
* ir.rs: in optimize_load_fast_borrow, push the fall-through successor
only when the current block has a last instruction (is_some_and).
Empty blocks now terminate fall-through propagation, matching the
`term != NULL` check in optimize_load_fast.
* compile.rs: add switch_to_new_or_reuse_empty() helper and use it in
compile_while. The helper reuses the current block when it is empty
and unlinked, mirroring USE_LABEL absorption in
cfg_builder_maybe_start_new_block. This stops a stray empty block
from appearing between e.g. a try/except end_block and the following
while loop header.
Four codegen tests that depended on the previous fall-through-through-
empty behavior are marked #[ignore] with TODO comments.
Also includes a handful of dictionary entries in .cspell.dict picked up
during the work.
* Interleave const fold passes per-block to match CPython
Mirror CPython's optimize_basic_block() (flowgraph.c) by walking each
block once in instruction order and trying tuple, list, set, unary, and
binop folding at each position before advancing. This replaces the
previous global-pass sequence where every fold_unary_constants pattern
in the whole CFG was registered before any tuple constant, leaving
negated literals like `-1` at co_consts positions earlier than CPython
produces (e.g. snippets.py: -1 at idx 280 vs CPython idx 726).
Changes:
- Extract `fold_unary_constant_at` and `fold_binop_constant_at` per-
position helpers from the existing global passes; the global passes
now call the helpers in a loop.
- Add `fold_constants_per_block` that walks each block to a fixed point,
trying all five folds at each instruction position.
- Call the new walker before the legacy global passes in
optimize_finalize so co_consts insertion order matches CPython's.
Measured on the full Lib tree: differing files 270 → 269; the only
newly matching file is `test/test_ast/snippets.py`, the case raised in
youknowone/RustPython#28.
* Inline small fast-return blocks only through unconditional jumps
`inline_small_fast_return_blocks` previously appended the target
`LOAD_FAST(_BORROW)/RETURN_VALUE` block's instructions onto any
predecessor whose fall-through eventually reached it, in addition to the
unconditional-jump case CPython handles in
`inline_small_or_no_lineno_blocks` (flowgraph.c:1210). CPython only
inlines through unconditional jumps, leaving fall-through predecessors
to reach the shared return block via the natural CFG layout. The extra
fall-through branch duplicated the return tail (e.g. `if/elif/return`
emitted two adjacent `LOAD_FAST_BORROW x; RETURN_VALUE` sequences).
Remove the fall-through inlining branch and keep only the
unconditional-jump path.
Measured on the full Lib tree: differing files 270 → 239 (-31), no new
regressions. Files newly matching include copy.py, argparse.py,
dataclasses.py, logging/__init__.py, pathlib/__init__.py, etc.
* Allow scope-exit/jump-back reorder within a shared except handler
`reorder_conditional_scope_exit_and_jump_back_blocks` previously skipped
any reorder where the conditional, scope-exit, or jump-back block had
an `except_handler` attached, even when all three shared the same
handler. CPython reorders these regardless of try/except context, as
long as the blocks stay within the same protected region. The over-
conservative guard left patterns like `try: for: if cond: return` with
the loop body's scope-exit ahead of the backedge, while CPython places
the backedge first and inverts the conditional.
Replace the `block_is_protected` triple-check with a single
`mismatched_protection` test: skip only when the three blocks do not
share the same `except_handler`. Same-handler reorders preserve the
protected range because every instruction's `except_handler` field
stays attached as `.next` pointers shift.
Measured on the full Lib tree: differing files 239 → 237; no new
regressions.
* Skip jump-over-cleanup reorder when target restarts an exception scope
reorder_jump_over_exception_cleanup_blocks was swapping a small
scope-exit target with a preceding cold cleanup chain even when the
target block began a fresh try (SETUP_FINALLY/SETUP_CLEANUP/SETUP_WITH).
The swap moved the next try's setup ahead of the prior handler's
cleanup_end/next_handler/cleanup_block, making the cleanup_body's
JUMP_FORWARD fall through directly to the cleanup_end and get elided as
redundant. The bytecode then lacked the JUMP_FORWARD that skips the
cleanup blocks and matched the prior handler's borrow tail incorrectly.
Skip the reorder when the target block contains any block-push pseudo
op so a new try's setup stays in source order. Re-enables the four
named/typed except-cleanup borrow tests that were marked #[ignore] in
commit 7481459ea.
* fix if block_idx == BlockIdx::NULL
Joshua Megnauth