* Support PEP 798 comprehension unpacking
Extract the remaining Python 3.15 comprehension unpacking implementation
from the integration branch. Use existing list/set/dict update operations
and an explicit inner iterator loop for generator unpacking. Align the
public DictComp AST representation and schema with CPython.
Migrate test_unpack_ex and test_unparse with scripts/update_lib from
CPython d220409a13b77de7a6c30abe16ea90db01528edc. Preserve existing Rust tests;
add no duplicate regression tests or snippets.
AI-assisted source extraction, canonical migration, and independent source
review with OpenAI Codex. Runtime validation is not yet complete.
Assisted-by: OpenAI Codex:model version unavailable
* Improve selected unpacking diagnostics and refresh canonical exception tests
Normalize six supported comprehension-unpacking diagnostics using existing parser context. Restrict the dictionary-spread hint to its first entry to avoid changing unrelated diagnostics.
Migrate the whole test_exceptions module from CPython d220409a13b77de7a6c30abe16ea90db01528edc with the existing updater, preserving only its two existing RustPython markers. Ruff sources and dependency pins are unchanged.
Six focused check/build/strict Clippy gates passed. A paired 34-input compile-only comparison retains six canonical diagnostic improvements, with eight of fifteen canonical cases still differing; this is not full test_exceptions or CI validation.
* Fix mapping-rest error locations and retire stale encoding xfail
* Match remaining unpacking diagnostics without changing Ruff
Use exact retained AST and error ownership to normalize the remaining CPython 3.15 unpacking diagnostics. Reparse only the owned call argument slice with the unchanged Ruff parser when its original recovery loses generator clauses.
Validated all 30 canonical compile-only cases in exec and doctest-style single modes against CPython 3.15.0, plus finite surrounding controls. Targeted compiler Clippy and source-bound builds passed. Full CI is pending. Cspell alone was omitted with the maintainer’s explicit one-commit approval because its npm bootstrap downloads returned HTTP 403; all other applicable configured commit hooks ran.
AI assistance: dot authored, reviewed, and validated the changes. The runtime does not expose its exact model version.
Assisted-by: dot: model version not exposed
---------
Co-authored-by: Codex <codex@openai.com>
* Add ExceptionGroup fields and OSError use-init
BaseExceptionGroup stores msg, excs, and excs_str; args keep
the original sequence. OSError defers parse when a subclass
overrides only __init__. characters_written can be deleted.
Exception args/traceback/cause/context reject invalid
setattr/delattr. UnicodeError.object is a deletable field.
except* remainder projection copies traceback/context/cause.
SyntaxError.__init__ clears omitted location attributes.
Assisted-by: Grok:grok-4.6
* Fix OSError payload init and UnicodeError construction
new_payload_exception now runs slot_new so BlockingIOError
parses characters_written. new_exception falls back to
invoke_exception for extra-payload types. Mixed tab/space
UnexpectedIndentation is TabError. AttributeError pickle
omits obj.
Assisted-by: Grok:grok-4.6
* address review: classify TabError from error line
Assisted-by: Grok:grok-4.6
* Add new_simple_exception; fill SyntaxError and f_lineno
new_exception constructs extra-payload types; BaseException-sized
types use new_simple_exception. Non-UTF-8 SyntaxError sets location
and text. Codegen errors load text via ProgramText. Keep Missing
parentheses capitalization. f-string adjacent atoms report a missing
comma. f_lineno may be None; lasti is not restored on unwind.
Assisted-by: Grok:grok-4.6
* address review: skip f-string continuation keywords
Assisted-by: Grok:grok-4.6
* Fix clippy map_unwrap_or in Non-UTF-8 offset
Assisted-by: Grok:grok-4.6
* Rename encode error ctor and exception setters
new_unicode_encode_error_real is new_unicode_encode_error.
set___traceback__/__cause__/__context__ are pygetset setters;
Rust uses set_traceback/set_cause/set_context.
Assisted-by: Grok:grok-4.6
* Use set_context in native pickle
Assisted-by: Grok:grok-4.6
* Collapse nested if in asyncio traceback restore
Assisted-by: Grok:grok-4.6
* Use current instruction for f_lineno addr2line
lasti is stored as the next instruction index. addr2line of
that byte offset is past RETURN_VALUE, so f_lineno was None
and stdlib_threading_current_frames hung on leftover workers.
Assisted-by: Grok:grok-4.6
* Look up format args, dir class, and frame builtins
Consume printf args before rejecting unknown conversions.
object.__dir__ uses getattr of __class__/__dict__.
_PyEval_GetBuiltin looks up iter/__import__ on the frame.
Assisted-by: Grok:grok-4.6
* Fix builtin lookups after rebase and unxfail passing tests
Assisted-by: Grok:grok-4.6
* Restore RERAISE lasti and SyntaxError locations
RERAISE with oparg restores lasti from the exception table
before unwind, so leftover f_lineno matches the original raise.
Handler lookup still uses the raising instruction.
Indentation errors outrank print/exec missing-parentheses.
BOM-with-non-utf8, bytes literals, and global+nonlocal now
report the same offsets as the parser/symtable.
object.__dir__ keeps string keys; type.__type_params__ skips
the getset wrapper stored on builtin types.
Assisted-by: Grok:grok-4.6
* Unxfail syntax/import tests and bind jump locals
IMPORT_NAME passes fromlist None instead of an empty tuple.
Jumping f_lineno binds None into unbound localsplus slots.
str lstrip/rstrip copy subclasses to exact str when unchanged.
Assisted-by: Grok:grok-4.6
* Fix import_from callers and rename frame lineno
sqlite3.iterdump now passes fromlist as a PyObject.
FrameObject::lineno is the i32 helper; f_lineno is the getset.
Assisted-by: Grok:grok-4.6
* Pass sqlite fromlist without Into conversion
from_list.into() could not infer Into<PyObjectRef> at the
import_from call. The typed tuple already converts.
Assisted-by: Grok:grok-4.6
* Honor close unraisable and deallocator messages
A failed close lookup on a yield-from target is unraisable.
Ignored GeneratorExit from finalize gets a traceback and the
closing-generator err_msg. __del__ uses the deallocator message.
Assisted-by: Grok:grok-4.6
* address review: claim, ag_running, frame steal
Keep the running claim through close() cleanup. ag_running
reads running_async. aclose ignore-yield does not mark the
generator closed. Steal the frame slot before uniqueness
and expose it with try_to_owned.
Assisted-by: Grok:grok-4.6
* Honor wrapper docs, yield assign, throw context
method-wrapper exposes the slot __doc__. Parenthesized yield
assignment uses the invalid-target messages. throw() restores
without overwriting __context__, then chains only from the
generator's own exc_info slot.
Assisted-by: Grok:grok-4.6
* Remove passing expectedFailure markers
method-wrapper docs, deallocator messages, and help pager
output now match, so drop the leftover markers.
Assisted-by: Grok:grok-4.6
* Honor wrapper signatures and assignment diagnostics
Slot docs include text signatures. method-wrapper compares and
hashes by identity. Keyword genexp and if/while assignment use
the invalid-target messages.
Assisted-by: Grok:grok-4.6
* address review: aclose, unraisable, frame publish
aclose ignore-yield does not close the generator. Format
unraisable object is None. close() leaves traceback to the
caller; finalize attaches the generator frame. Publish the
iframe before the lock-free slot.
Assisted-by: Grok:grok-4.6
* Honor member descriptors and wrapper annotations
Assisted-by: Grok:grok-4.6
* address review: getitem, bool doc, if assign
Assisted-by: Grok:grok-4.6
* Honor eval-mode assignment invalid syntax
Assisted-by: Grok:grok-4.6
* Supply getset and member docs from rustpython-doc
Assisted-by: Grok:grok-4.6
* Enable rustls SSL on WASI targets
Compile the rustls _ssl module for WASI and stub
socket waits plus the native certificate store.
Assisted-by: Grok:grok-4.6
* Add browser TLS _ssl in rustpython-wasm
Register rustls _ssl and a MemoryBIO-oriented _socket
from rustpython-wasm. Use rustls-rustcrypto because
ring cannot compile for wasm32-unknown-unknown.
Assisted-by: Grok:grok-4.6
* Define SSL protocol and cert constants as enums
Keep OpenSSL bitmask defines as integer constants.
Assisted-by: Grok:grok-4.6
* Ignore rustls deps cargo-shear cannot see
The rustpython-wasm _ssl sources are compiled through
a symlink, so shear does not count their crate uses.
Assisted-by: Grok:grok-4.6
* Replace wasm ssl symlink with a BIO module
Drop the stdlib ssl.rs link and keep a rustpython-wasm
_ssl that only covers MemoryBIO wrap_bio TLS.
Assisted-by: Grok:grok-4.6
* Guard all/any/tuple(genexpr) calls when the generator is a coroutine
`maybe_optimize_function_call()` reserves a `skip_optimization` label for
every `name(genexpr)` call and, for `all`, `any` and `tuple`, emits an
identity guard against the builtin plus an inlined loop. The port skipped
both whenever the generator expression's symbol table was a coroutine, so
`all(await x for x in xs)` and `all(x async for x in xs)` compiled to a
plain call.
An `await` or an `async for` in the generator does not disqualify the
shape: the inlined `FOR_ITER` raises the same `TypeError` that calling the
builtin on an async generator raises, which is what
test_builtin test_builtin_call_async_genexpr_no_crash asserts.
This was the last code-level difference against CPython across `Lib/`:
comparing every file's code tree by structure, opcode and constant value
(qualnames aside, which moved in 3.14.3+, and set constants compared by
value rather than by hash order) now matches on 1720 of 1720 comparable
files, up from 1719.
Assisted-by: Claude Code:claude-opus-5
* Name the ellipsis type `ellipsis`
`type(...).__name__`, its `__qualname__`, and `types.EllipsisType.__name__`
read `EllipsisType`.
Assisted-by: Claude Code:claude-opus-5
* Align compiler front end and instruction positions with CPython
Reject a bare generator expression in a class header and a format spec
nested more than two deep. Both parse here; the reference grammar has no
rule for either, since a class header only takes `arguments` and the
tokenizer runs out of nesting levels at the third format spec.
Scan the interpolations a format spec carries inside another format spec,
so `f"{x:{y:{z}}}"` reaches `z` when building the symbol table.
Give every generic function's type params scope a `.defaults` slot,
whether or not the signature has any default, so `co_varnames` and
`co_nlocals` match. Classes and type aliases keep theirs empty.
Decide an unparenthesized sole generator argument by the parser's
`parenthesized` flag instead of scanning the source for the shape of its
element. `_sum((d := x - c) * d for x in data)` opens with a parenthesized
group the rest of the expression continues, which the scan read as the
generator's own parentheses and left the call's out of its range.
Drop empty f-string fragments unconditionally. A concatenation whose
fragments are all empty now loads its empty string at the whole
concatenation instead of at whichever fragments happened to be plain
string literals.
Keep a byte order mark in source handed over as text, and report it as an
invalid non-printable character. Reading a file to freeze strips it, as
the encoded-bytes path already did.
Carry the filename into the errors `compile(..., PyCF_ONLY_AST)` raises;
they reported `<unknown>` or an empty name.
Across `Lib/`, the 1729 files that compile on both sides now agree on
opcodes, constants, flags, variable names, names, and exception tables,
and on instruction positions in all but 7 files, where a compound
statement whose body ends in a trailing `;` ends one column short.
Assisted-by: Claude Code:claude-opus-5
* Align ast module output with CPython
- Keep lone surrogates in string, f-string and t-string literal values by
reading such a literal from its source; the literal-value helpers move
out of `Compiler` into `rustpython_codegen` and `ConstantLiteral::Str`
holds `Wtf8Buf`.
- Give `{expr=}` interpolations their leading text as a Constant and
default their conversion to `repr`, in both f-strings and t-strings.
- Span folded consecutive literals from the first of them to the last.
- Take the `:` into a format spec's range only when the parser left it
out, instead of always widening by one.
- Report a node end that lands on a line start as that line, column 0.
- Give a generator expression written straight into a call's parentheses
the range of those parentheses, in the preprocess pass, replacing
codegen's source scan and `_ast`'s one-column adjustment.
- Drop the expectedFailure on test_ast_line_numbers_with_parentheses.
Assisted-by: Claude Code:claude-opus-5
* Align symbol table output with CPython
- Register an import as `DEF_IMPORT` alone, and an annotated parameter as
a plain parameter, so `import x; global x` is accepted and annotated
parameters are not reported as annotated names.
- Record a `global` declaration in the module block as well, and resolve a
name a `global` in an enclosing scope covers as an implicit global, so a
class body reading such a name emits `LOAD_NAME`.
- Resolve a name bound nowhere as an implicit global instead of leaving
its scope unknown.
- Require a binding only for a name an explicit `nonlocal` declared;
a free variable that reached the scope any other way already resolved.
- Name a generator expression's block `genexpr`; the code object keeps
`<genexpr>`.
- Locate a `def`/`class` block at its keyword rather than at the first
decorator. `decorated_definition_range()` moves from `Compiler` to
`rustpython_codegen` for that.
- Carry a comprehension symbol's flags over when inlining it into a parent
entry that only holds a free variable propagated from a child.
- Leave `__conditional_annotations__` out of an annotation block's
symbols and have the compiler cook up the free variable, which places
it after the ones the symbol table supplies.
- Check `from __future__` placement while building the symbol table, and
give `symtable.symtable()` the same docstring stripping and future
feature validation the compiler does.
- Floor `co_stacksize` at 1.
Assisted-by: Claude Code:claude-opus-5
* Locate the eval return and the interactive print like CPython
Leave `eval` mode's `RETURN_VALUE` without a location, so the exit block
is duplicated per reaching path and each copy carries the location of the
branch that jumps to it. `a and b` now ends with two returns as it does
in CPython, and `a if b else c` gives the first one the `a` it returns.
Emit an interactive `CALL_INTRINSIC_1(Print)` at the expression statement
rather than wherever the expression left the location, which differ for
`(a := 10)` and `f'{0:fz}'`.
Assisted-by: Claude Code:claude-opus-5
* Give interactive mode the conditional annotations cell
`compile_program_single()` emitted the `__conditional_annotations__` set
and its name accesses but never added the cell that holds it, so a module
compiled in `single` mode was missing the `MAKE_CELL` and the cellvar that
the same source gets in `exec` mode.
Assisted-by: Claude Code:claude-opus-5
* Give compiler-raised SyntaxErrors an end position
`CodegenError` and `SymbolTableError` now carry an `end_location`
alongside `location`, and `CompileError::python_end_location()` returns
it, so errors such as "'await' outside function" set `end_lineno` and
`end_offset` instead of leaving them None.
`SymbolTableBuilder::error_ranged()` builds both positions from a range,
replacing the inline `source_location()` calls at the error sites.
Drop the `expectedFailure` on
`test_exceptions.SyntaxErrorTests.test_file_source`, which now passes.
Assisted-by: Claude Code:claude-opus-5
* Report parser diagnostics with CPython's wording
`InvalidAnnotatedAssignmentTarget` becomes "illegal target for
annotation", `UnexpectedExpressionToken` becomes plain "invalid syntax"
rather than prefixing the parser's text, and the list-recovery
diagnostics ("Expected an expression or a '}'" and its siblings) join the
two that already mapped to "invalid syntax".
Assisted-by: Claude Code:claude-opus-5
* Punctuate the code object repr like CPython
`repr(code)` was missing the comma after the address and escaped the
filename through Rust's Debug formatting.
Drop the `expectedFailure` on the six `test_dis` tests that compare
disassembly text, which now pass.
Assisted-by: Claude Code:claude-opus-5
* Match CPython's parser error positions
A syntax error the parser reports counts its columns in characters, so
`source_location()` converts with UTF-32 rather than UTF-8; an offset
inside a character walks back to where that character starts.
Spans the parser reported differently:
- an error between two tokens covered nothing, and now covers one
character, since the narrowest token still covers one
- a "was never closed" bracket ends at column 0
- an unterminated string literal ends where it starts
- "leading zeros in decimal integer literals" spans the run of zeros
- an exponent with no digits and no sign is blamed on the digit before
the exponent letter, which the tokenizer puts back
Assisted-by: Claude Code:claude-opus-5
* codegen: seek to the finally body's own scopes before copying it
An early exit from a try block emits an extra copy of the finally body.
Nested scopes are handed out by position, and the cursors still sit inside
the try block's own run of scopes there, so the copy took those instead of
the ones the finally body opens: a generator expression got a sibling's
symbol table and raised "the symbol 'k' must be present in the symbol
table", or, where the sibling was a named scope, built a code object with
no `.0` argument that raised UnboundLocalError when it ran.
Seek the cursors to the first scope beginning on or after the finally
body's first line before compiling the copy. The existing restore then
leaves the try statement's own copies starting from the same place.
Assisted-by: Claude
* Align compiler and AST behavior with CPython
Update the compiler pipeline, AST module support, and related VM compile/eval paths toward CPython 3.14 behavior.
Use upstream Ruff parser crates directly and keep RustPython-specific syntax preflight handling in the compile path.
Refresh opcode metadata, snapshots, and targeted tests for the aligned bytecode and AST behavior.
* fix
* Fix PR 8138 CI failures
* Fix match mapping wildcard scan complexity
* fixes
* Use Ruff runtime AST fields
* Minimize residual patch diff
* refactor error handling
* Remove _ast conversion context wrappers
* Clean up _ast conversion glue and compiler warning escalation
Rename the Node conversion parameter `ctx` to `vm` throughout the _ast
module.
Remove the unused compile_program, compile_program_single,
compile_block_expression, and compile_expression forwarders; compile_top
stays as the entry point.
Stash an escalated compiler SyntaxWarning so a non-SyntaxWarning category
propagates unchanged instead of being rewritten to SyntaxError, matching
PyErr_ExceptionMatches(SyntaxWarning) in compiler_warn. Drop the now-unused
CompileWarningError::into_codegen_error.
Replace the PositionalArguments two-variant enum with a struct holding a
shared range and a PositionalArgumentsKind enum.
Assisted-by: Claude
* Represent compiler flags as a CompilerFlags bitflags
Replace the loose PY_CF_*/CO_* i32 constants in compile_mode with a
CompilerFlags bitflags covering the PyCF_* flags and __future__
CO_FUTURE_* bits. The Python-visible ast.PyCF_* attribute values are
derived from it via bits(); parser input modes stay plain consts.
builtins::compile and the compile_string flag handling test flags
through CompilerFlags::contains instead of raw bit math.
Assisted-by: Claude
* Fix PR 8138 test regressions
* Fix PR 8138 CI failures
* Avoid shell interpolation in Linux deps action
* Fix review findings in compiler/AST changes
- frame.rs: compute the MATCH_CLASS class name lazily so a successful
match no longer takes the type name lock or allocates; the
__match_args__-not-a-tuple branch reuses it instead of recomputing.
- _ast/constant.rs: add the missing leading space to the with_recursion
label ("maximum recursion depth exceeded during compilation").
- sys.set_int_max_str_digits: restore the "maxdigits must be 0 or larger
than {n}" error text.
- codegen: set CO_MAXBLOCKS to 20 in both ir.rs and compile.rs.
- _ast/exception.rs: extract the duplicated excepthandler node guard
into ensure_excepthandler_node.
- codegen: correct the validate_keywords doc comment.
- Drop redundant "CPython" qualifiers from comments added in this branch.
Assisted-by: Claude
* Update codeop from v3.14.3
* Fix traceback, syntax errors, and exception handling
- Improve unclosed bracket detection with "'(' was never closed" message
- Fix IndentationError location to point to end of line
- Implement frame.clear() with proper checks for executing/suspended frames
- Fix exception context chaining for propagated exceptions
- Add traceback.__dir__() and prevent tb_next deletion
- Fix subscript operation source range restoration in compiler
- Change "duplicate parameter" to "duplicate argument" error message
- Refactor duplicate code in asyncgenerator.rs and frame.rs
---------
Co-authored-by: CPython Developers <>
Assisted-by: Codex:gpt-6
Initially this is only meant to be a small update for `test_io.py`.
But it turns out that `test.support` need a lot of cleanup especially
for `test.support.warnings_helper`.
