* Update pstats from v3.14.5
* Update profile from v3.14.5
* Update tracemalloc from v3.14.5
* Update test_multibytecodec from v3.14.5
* Mark failing tests in test_pstats
* Mark failing test in test_profile
* Mark failing tests in test_cprofile
* Mark failing tests in test_tracemalloc
* Mark failing tests in test_multibytecodec
* Added comment marker to test_pstats.py
---------
Co-authored-by: CPython Developers <>
* Share marshal ref table between code object and its internals
read_marshal_bytes, _str, _str_vec, _name_tuple, and _const_tuple now
take a shared ref table and resolve TYPE_REF / register FLAG_REF
entries. deserialize_code is split into a public wrapper and an inner
function that receives the ref table; deserialize_value_depth opens a
fresh inner ref space when it hits Type::Code, mirroring CPython's
behaviour of putting the code object itself at ref slot 0. Nested code
objects inside const tuples reuse the surrounding code's ref space via
the new read_const_value helper.
* Align PYC magic number, FORMAT_VERSION, and header check with CPython 3.14
PYC_MAGIC_NUMBER changes from 2994 to 3627, matching CPython 3.14's
pyc_magic_number_token (0x0a0d0e2b). marshal FORMAT_VERSION drops from
5 to 4 (the encoder/marshal.version value; the decoder already accepts
both). check_pyc_magic_number_bytes now compares all four magic bytes
instead of the first two.
* Accept CPython-tagged .pyc as read-only bytecode source
SourceFileLoader.get_code now also looks for .pyc files using
_RP_FALLBACK_CACHE_TAGS (currently ('cpython-314',)) in addition to
sys.implementation.cache_tag. The matched .pyc is only used for
reading; recompilation still writes to the RustPython-tagged path, so
CPython's .pyc is never overwritten. Source-stat / hash / timestamp
validation logic is unchanged.
* Apply rustfmt to marshal helpers
* Marshal PySlice from format version 4 instead of 5
CPython's marshal supports TYPE_SLICE from format version 4 onwards
and that is the default version. Rejecting slice dumps below version
5 made marshal.dumps(slice(...)) fail with the default version and
broke test.test_marshal.SliceTestCase.test_slice.
* Revert "Accept CPython-tagged .pyc as read-only bytecode source"
Lib/importlib/_bootstrap_external.py is CPython's own code copied
verbatim; local patches here defeat compatibility tracking. The
cpython-XX cache_tag fallback needs to live on the RustPython side
(Rust code or sys.implementation.cache_tag policy), not as edits to
the imported standard library.
This reverts commit 1fc426d0fb5fcdb50d35cad13bbb43e8f6ce1c7f.
* Set marshal FORMAT_VERSION to 5 to match CPython 3.14.5
Py_MARSHAL_VERSION is 5 in CPython 3.14.5 (Include/marshal.h:16) and
TYPE_SLICE serialization rejects version < 5 (Python/marshal.c:720).
Restore the same threshold and constant so marshal.version and the
slice-marshal gate match CPython.
* Thread marshal recursion depth through nested code objects
Code objects embedded in const-tuples reset the depth budget on each
recursion, so a hostile or pathological marshal stream of code-in-tuple-
in-code can blow the stack despite MAX_MARSHAL_STACK_DEPTH. Pass the
current depth through deserialize_code_inner and read_marshal_const_tuple
and decrement at each code-object/tuple boundary.
Also route dict keys through deserialize_value_after_header so TYPE_CODE
keys decode instead of failing with BadType.
* align compiler to CPython
* Align codegen with CPython compile.c
Rename CFG helpers and accessors to the names used in CPython's
compile.c (basicblock_next_instr, basicblock_last_instr,
basicblock_append_instructions, bb_has_fallthrough, is_jump,
make_cfg_traversal_stack, mark_warm/mark_cold, etc.). Drop the unused
boolop-folding gate, mark_cpython_cfg_label_block helper, and
ComprehensionLoopControl::iter_range field.
Track an is_coroutine flag on SymbolTable, set in async def, await, and
async comprehensions, and propagate it through non-generator
comprehensions per symtable_handle_comprehension().
Mark SetupCleanup/SetupFinally/SetupWith as has_arg pseudo-ops, mark
ForIter as a terminator, and add has_arg/has_const on AnyInstruction.
Fix Instruction::stack_effect_jump to delegate to the opcode's
stack_effect_jump rather than stack_effect.
* Align codegen IR with CPython CFG structures
* Match CPython CFG annotation offset arithmetic
* Propagate CPython CFG label translation errors
* Align CPython exception target labeling flow
* Propagate CPython CFG traversal stack allocation errors
* Match CPython optimize_load_fast allocation flow
* Propagate CPython basicblock allocation errors
* Propagate CPython redundant NOP cleanup errors
* Propagate CPython unused const cleanup errors
* Propagate CPython const folding errors
* Propagate CPython swaptimize allocation errors
* Match CPython list-to-tuple fold allocation
* Skip const folding on CPython allocation failures
* Skip subscript folding on CPython allocation failures
* Propagate CPython assembler allocation errors
* Propagate CPython localsplus allocation errors
* Propagate CPython localsplus setup allocation errors
* Propagate CPython jump label map allocation errors
* Propagate CPython instruction sequence allocation errors
* Propagate CPython instruction label allocation errors
* Guard CPython codegen block allocation
* Guard CPython label shadow allocation
* Propagate CPython label shadow allocation errors
* Align CPython c-array allocation updates
* Align CPython ref stack growth
* Match CPython CFG builder debug check
* Avoid Rust-only CFG append clone
* Reuse CPython cleared block slots
* Use CPython block append in copy_basicblock
* Match CPython cfg builder creation order
* Clear label map after CPython apply pass
* Match CPython cfg builder allocation check
* Propagate CPython c-array size errors
* Drop Rust-only label uniqueness check
* Drop Rust-only label shadow debug checks
* Propagate CFG block index overflow
* Propagate CFG label oparg overflow
* Model CPython basicblock instruction storage
* Drop Rust-only recorded CFG precheck
* Model CPython instruction sequence storage
* Propagate instruction sequence offset overflow
* Model CPython instruction sequence labels
* Match CPython jump offset arithmetic
* Match CPython exception table arithmetic
* Match CPython label index arithmetic
* Match CPython instruction offset casts
* Match CPython jump offset indexing
* Match CPython oparg locals casts
* Match CPython localsplus offset arithmetic
* Match CPython cell prefix indexing
* Match CPython C array growth arithmetic
* Match CPython label map allocation arithmetic
* Match CPython label map size tracking
* Use CPython label map size in sequence passes
* Assert CPython label map clearing invariants
* Match CPython label oparg assignment
* Match CPython compiler direct arithmetic
* Match CPython load fast local casts
* Match CPython load fast depth assert
* Match CPython resume depth flagging
* Match CPython stack depth arithmetic
* Drop Rust-only stack overflow error
* Return CPython stack depth directly
* Match CPython C array growth errors
* Match CPython instruction insert asserts
* Match CPython unreachable pseudo jump
* Match CPython CFG size guard
* Match CPython superinstruction assert
* Match CPython redundant jump assert
* Match CPython stackdepth errors
* Match CPython jump offset flow
* Match CPython assembler buffer defaults
* Match CPython bytecode emit growth
* Match CPython assembler entry growth
* Match CPython assembler growth overflow check
* Match CPython remove_unreachable structure
* Match CPython static swap flow
* Inline CPython code unit preprocessing
* Match CPython C array growth checks
* Match CPython label map size guard
* Match CPython load-fast flow
* Simplify CPython CFG condition flow
* Align exception fallthrough propagation
* Match CPython pseudo target table
* Match CPython annotations CFG assert
* Match CPython inverted op assert
* Match CPython many-locals guard
* Reject deopt opcodes in CFG stack effects
* Match CPython invalid stack effect error
* Test CPython deopt stack effect guard
* Match CPython load-fast extended-arg assert
* Match CPython instruction allocation asserts
* Match CPython basicblock last-instr asserts
* Match CPython opcode range asserts
* Assert CPython fallthrough line propagation invariant
* Assert CPython CFG target offset sign
* Assert CPython exception fallthrough invariant
* Assert CPython exception stack bounds
* Assert CPython traversal stack allocation
* Match CPython label-map allocation in shadow
* Mirror CPython label-map sentinel fill
* Match CPython CFG builder allocation asserts
* Match CPython exception stack structure
* Match CPython ref stack structure
* Match CPython CFG traversal stack structure
* Mirror CPython CFG traversal stack pointer
* Use CPython fixed exception handler stack
* Mirror CPython ref stack capacity field
* Match CPython swap optimizer scratch stack
* Align static swap helpers with CPython blocks
* Align swaptimize signature with CPython
* Match CPython redundant pair pass result
* Match CPython inline pass result
* Match CPython redundant NOP pass results
* Fix bytecode metadata after upstream rebase
* Match CPython opcode stack metadata
* Add CPython identifiers to cspell dictionary
Add CNOTAB, LNOTAB, ialloc, ioffset, iused, nblocks, ncellsused,
ncellvars, nextop, noffsets, nvars, swaptimize, untargeted to
.cspell.dict/cpython.txt for the new CFG/assembler code in
crates/codegen/src/ir.rs.
* Fix CI failures from bytecode-parity work
- clippy: drop redundant `test_` prefix on three test functions and
remove an unnecessary `u32` cast in basicblock_clear_reuses_cpython_spare_slots_in_offset_order
- insta: regenerate nested_double_async_with snapshot to match the new
CFG output that drops unreferenced labels after the redundant-NOP pass
- regrtest: drop `@expectedFailure` markers from test_func_args,
test_meth_args (test_compile), test_disassemble_with,
test_disassemble_try_finally (test_dis), and test_except_star
(test_monitoring) which now pass
* Resync generated opcode metadata
Empty conf.toml since WithExceptStart and Setup{Cleanup,Finally,With}
stack effects already match CPython, so the TODO override entries are
stale and only cause CI hook diffs.
Regenerate opcode_metadata.rs and drop the matching SetupCleanup/
SetupFinally/SetupWith assertions on PseudoOpcode::has_arg(); their
`HAS_ARG` flag comes from pseudo definitions in bytecodes.c that the
upstream analyzer does not propagate through PseudoInstruction.properties,
so the generated has_arg() excludes them. has_target() still covers
these block-push pseudos via is_block_push().
* Drop is_block_push has_arg invariant
The CPython invariant `assert(OPCODE_HAS_ARG(op) || !IS_BLOCK_PUSH(op))`
relies on SETUP_{FINALLY,CLEANUP,WITH} carrying `HAS_ARG_FLAG` in
CPython's metadata. The autogen tool reads pseudo-opcode properties from
target instructions and does not propagate the pseudo's own
HAS_ARG flag, so PseudoOpcode::has_arg() omits these three opcodes.
Drop the debug_assert that fired inside py_freeze proc-macro expansion.
* Auto-generate has_eval_break and route AnyInstruction has_arg/has_const via macro
Add fn_has_eval_break to generate_rs_opcode_metadata.py using CPython's
Properties.eval_breaker, removing the hand-written matches! body for
Opcode::has_eval_break and PseudoOpcode::has_eval_break.
Forward has_arg/has_const from Instruction and PseudoInstruction to
their opcode, so AnyInstruction can use either_real_pseudo! like the
other has_* accessors instead of an open-coded match.
* sqlite3: fix Blob.__setitem__ value range validation
Previously, assigning an out-of-range integer (negative or > 255) or an
integer too large for i64 (e.g. 2**65) to a Blob index raised OverflowError
instead of ValueError.
Mirror CPython's ass_subscript_index logic:
- Convert the value via to_i64(), treating any overflow as -1
- Validate the result is in [0, 255], raising ValueError("byte must be in range(0, 256)") otherwise
- Separate deletion error messages: "item deletion" for index, "slice deletion" for slice
* sqlite3: fix Blob.__setitem__ negative-step slice write
In the step != 1 branch of Blob.ass_subscript, the loop used
i_in_temp += step as usize
where step is isize. For negative steps (e.g. step = -2),
(-2isize) as usize = 18446744073709551614
causing an out-of-bounds panic whenever slice_len >= 2.
Fix: use SaturatedSliceIter (already used by the read path) to iterate
over the correct absolute blob indices, then map each index back to a
temp buffer offset via abs_idx - range_start.
Also fix a Clippy lint: replace
val < 0 || val > 255
with the idiomatic
!(0..=255).contains(&val)
Add a regression test in extra_tests/snippets/stdlib_sqlite.py that
exercises blob[9:0:-2] (negative step, slice_len=5).
* fix: guard blob negative-step snippet from CPython 3.11 bug
* style: add blank line after import sys in stdlib_sqlite snippet (ruff)
* Update extra_tests/snippets/stdlib_sqlite.py
---------
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
* Update tests
* Add basic audit support
* Add audit for `time.sleep`
* Add some for `socket`
* Some syslog
* Some sys related audits
* some marshal
* monitoring callback
* Mark failing tests
* clippy
* Clippy
* clippy
* mark failing test
* mark more
* Update `test_sys_setprofile.py` to 3.14.5
* Mark failing tests
* Convert host_env Windows path/argv params from raw *const u16 to &WideCStr
* Migrate remaining winapi raw u16 pointer signatures to typed references
* Migrate winreg pub unsafe fn string parameters to typed references
* Add ToPyException impls for host_env error types (PyPy wrap_oserror analog)
* Add CheckLibcResult helper and apply to socket/fcntl/shm/posix_wasi
* Add Win32 BOOL/HANDLE check helpers; apply check helpers across host_env
* Apply Win32/libc check helpers to overlapped/testconsole/os.rs
* Apply Win32 check helpers to winapi.rs (partial)
* Apply Win32 check helpers across more winapi.rs functions
* Apply Win32 check helpers to nt.rs (partial)
* Add CheckWin32Sentinel helper; apply to nt.rs INVALID_HANDLE_VALUE/INVALID_FILE_ATTRIBUTES patterns
* Add OwnedHandle / HandleToOwned helper; apply to mmap create_named_mapping leak path
* Use OwnedHandle RAII in nt::pipe to eliminate manual cleanup on error path
* Use OwnedHandle in nt::chmod_follow; hoist HandleToOwned import
* Drop rustix dependency from vm crate
Remove unused IntoPyException impl for rustix::io::Errno and the
rustix entry in crates/vm/Cargo.toml. rustix is now only depended on
by host_env.
* Fix CI failures: cross-platform regressions
- winapi.rs: pass None to create_event_w; the recent Option<&WideCStr>
migration left one call site still passing a raw null pointer.
- exceptions.rs: gate ToPyException for LockfError with
cfg(any(unix, target_os = "wasi")), matching host_env::fcntl's own
cfg. The previous cfg let it compile on wasm32-unknown-unknown where
host_env::fcntl does not exist.
- io_unsupported.rs: derive Eq on FileMode alongside PartialEq to
satisfy clippy::derive_partial_eq_without_eq.
* Fix CI failures: cfg gates and unused imports
- exceptions.rs: gate ToPyException for LockfError with
cfg(all(unix, not(target_os = "redox"))) to match the type's own
cfg in host_env/src/fcntl.rs (LockfError is not built on wasi).
- signal.rs: CheckLibcResult is only used in unix-gated functions;
split import so it is not pulled in for windows.
- mmap.rs: remove CheckWin32Handle from imports; no longer used after
switching to HandleToOwned-based RAII.
- overlapped.rs: remove INVALID_HANDLE_VALUE from connect_pipe import;
the call now uses .check_valid().
* Fix CI failures: rustfmt and windows unused import
- signal.rs: reorder cfg-gated imports per rustfmt.
- socket.rs: gate ToPyException import to cfg(all(unix, not(target_os = "redox")));
it is only used inside sendmsg which has the same gate, so it was unused
on windows.
* Push remaining libc/extern callsites from vm into host_env
Add host_env wrappers and replace the corresponding vm call sites:
- host_env::errno::strerror_string for libc::strerror
- host_env::io::write_stderr_raw for libc::write(STDERR_FILENO,...)
- host_env::locale::localeconv_data reused from vm::format
- host_env::os::abort for the inline abort extern
- host_env::os::urandom wraps getrandom; getrandom moves from vm to host_env
- host_env::posix::lchmod for the macOS/BSD lchmod extern
- host_env::posix::fcopyfile for the macOS fcopyfile extern
- host_env::nt::wputenv for the Windows _wputenv extern
vm/format.rs's get_locale_info now uses host_env on both unix and windows
instead of the unix-only libc::localeconv path.
* Move time tz state and winsound FFI into host_env
- host_env::time::tz: wraps the libc tzset/timezone/daylight/tzname
globals on non-msvc, non-wasm32 targets. vm::stdlib::time now reads
these via the typed wrappers instead of declaring its own externs.
- host_env::winsound (windows): exposes PlaySoundW (via a typed
PlaySoundSource enum), Beep, and MessageBeep. vm::stdlib::winsound
drops its inline FFI block and routes through host_env.
* Migrate unsetenv to host_env::nt::wputenv; rustfmt
- vm::stdlib::os::unsetenv had a second _wputenv call site that still
referenced the removed inline extern. Route it through
host_env::nt::wputenv like putenv.
- rustfmt fixups in exceptions.rs (boolean chain layout) and the two
winsound files.
* Address PR review comments
- host_env::winapi::create_process: assert that the command_line buffer
is NUL-terminated and that the env block ends with a double-NUL,
matching the Win32 CreateProcessW contract.
- stdlib::overlapped CreateEvent: replace WideCString::from_str_truncate
with the fallible from_str(), so embedded NULs in the event name
surface as ValueError instead of being silently truncated.
- vm::exceptions::ReadlinkError::NotSymbolicLink now maps to OSError
(matches Win32 ERROR_NOT_A_REPARSE_POINT semantics) rather than
ValueError.
- winreg::ConnectRegistry: route the non-zero return through the
existing os_error_from_windows_code helper so the resulting exception
carries the real winerror/message instead of a generic OSError.
* Fix CI failures and address review follow-ups
CI failures:
- rustfmt cleanup in exceptions.rs after the ReadlinkError change.
- vm/stdlib/os.rs: drop unused ToWideString import that the wputenv
migration left behind.
- vm/stdlib/winsound.rs: replace explicit `&*buf` with `&buf` to
satisfy clippy::explicit_auto_deref.
- Lib/test/test_format.py, Lib/test/test_types.py: drop the now-stale
expectedFailureIfWindows decorators on the locale-format tests; the
Windows path now reads real `localeconv` data via host_env so these
tests pass.
Review follow-ups:
- host_env::winapi::create_process: switch the new buffer terminator
checks from `assert!` to fallible validators returning
`io::ErrorKind::InvalidInput`, so bad inputs stay recoverable at
the API boundary.
- host_env::winsound::play_sound: reject `Memory(_)` together with
`SND_ASYNC` (lifetime-unsafe) and `SND_MEMORY` without a
`Memory(_)` source. Expand `PlaySoundError` into a variant enum.
- vm::stdlib::_winapi::CreateProcess: route the Win32 path/argv strings
through `as_wtf8().to_wide_cstring()` like the rest of the Windows
API surface; `expect_str()` could panic on Python strings containing
lone surrogates.
* Align codegen CFG cleanup with CPython
* Align codegen bytecode with CPython 3.14
* Remove remove_jump_target_line_nops
The pass dropped a same-line NOP at the start of any jump-targeted block
whose body advanced to a later line, but CPython's basicblock_remove_redundant_nops
only consults the previous/next instruction inside the same block. The
extra pass deleted the else-body line trace anchor exercised by
test_nested_double_async_with.
Ignore test_conditional_break_finally_does_not_keep_break_cleanup_nop;
the break NOP lands in a separate block from the inlined finally body,
so same-line successor elision does not apply here.
* Keep except_handler blocks reachable in eliminate_unreachable_blocks
After convert_pseudo_ops lowers SETUP_FINALLY to a plain NOP, the only
remaining link from the try body to the except_handler block was the
per-instruction except_handler annotation. When earlier passes had
already removed every NOP that carried that annotation (e.g. an empty
try body with `pass`), the handler block became unreachable from the
entry block and its instructions were cleared, dropping the handler
entirely. Seed reachability with blocks already marked except_handler
so handler dispatch survives independent of the in-block annotation.
Also drop two expectedFailure markers in test_patma whose match-tracing
expectations now pass.
* Align bytecode CFG cleanup with CPython
Shahar Naveh