* Align compiler and AST behavior with CPython
Update the compiler pipeline, AST module support, and related VM compile/eval paths toward CPython 3.14 behavior.
Use upstream Ruff parser crates directly and keep RustPython-specific syntax preflight handling in the compile path.
Refresh opcode metadata, snapshots, and targeted tests for the aligned bytecode and AST behavior.
* fix
* Fix PR 8138 CI failures
* Fix match mapping wildcard scan complexity
* fixes
* Use Ruff runtime AST fields
* Minimize residual patch diff
* refactor error handling
* Remove _ast conversion context wrappers
* Clean up _ast conversion glue and compiler warning escalation
Rename the Node conversion parameter `ctx` to `vm` throughout the _ast
module.
Remove the unused compile_program, compile_program_single,
compile_block_expression, and compile_expression forwarders; compile_top
stays as the entry point.
Stash an escalated compiler SyntaxWarning so a non-SyntaxWarning category
propagates unchanged instead of being rewritten to SyntaxError, matching
PyErr_ExceptionMatches(SyntaxWarning) in compiler_warn. Drop the now-unused
CompileWarningError::into_codegen_error.
Replace the PositionalArguments two-variant enum with a struct holding a
shared range and a PositionalArgumentsKind enum.
Assisted-by: Claude
* Represent compiler flags as a CompilerFlags bitflags
Replace the loose PY_CF_*/CO_* i32 constants in compile_mode with a
CompilerFlags bitflags covering the PyCF_* flags and __future__
CO_FUTURE_* bits. The Python-visible ast.PyCF_* attribute values are
derived from it via bits(); parser input modes stay plain consts.
builtins::compile and the compile_string flag handling test flags
through CompilerFlags::contains instead of raw bit math.
Assisted-by: Claude
* Fix PR 8138 test regressions
* Fix PR 8138 CI failures
* Avoid shell interpolation in Linux deps action
* Fix review findings in compiler/AST changes
- frame.rs: compute the MATCH_CLASS class name lazily so a successful
match no longer takes the type name lock or allocates; the
__match_args__-not-a-tuple branch reuses it instead of recomputing.
- _ast/constant.rs: add the missing leading space to the with_recursion
label ("maximum recursion depth exceeded during compilation").
- sys.set_int_max_str_digits: restore the "maxdigits must be 0 or larger
than {n}" error text.
- codegen: set CO_MAXBLOCKS to 20 in both ir.rs and compile.rs.
- _ast/exception.rs: extract the duplicated excepthandler node guard
into ensure_excepthandler_node.
- codegen: correct the validate_keywords doc comment.
- Drop redundant "CPython" qualifiers from comments added in this branch.
Assisted-by: Claude
Rustix bypasses `libc` to make direct syscalls on Linux. Rustix is also
semantically cleaner than `libc` itself. For `copy_file_range`, the
offsets can't be negative but `libc`'s `off_t` type is an `i64` because
it's used elsewhere, including for functions that CAN take a negative
offset. Rustix is cleaner because the offsets are `u64`.
* Fix WASM memory allocation failure (#4989)
This commit addresses the unbounded memory allocation issue on WASM targets that caused minimal programs (like = 1) to fail with ~79MB allocation errors in constrained runtimes like wasmi.
The fixes include:
1. Capping the max memory limit at 64MB and restricting stack size to 1MB via linker flags in .cargo/config.toml.
2. Reducing the minimum chunk size of the DataStack from 16KB to 4KB on WASM targets to shrink the initial memory footprint.
3. Adding a wasm-release profile in the root Cargo.toml optimized for size with opt-level = s, LTO enabled, and symbols stripped.
* Trigger CI rerun
* Address review feedback: add comments, use cfg! for MIN_CHUNK_SIZE
* Remove duplicate MIN_CHUNK_SIZE definition
* Fix duplicate MIN_CHUNK_SIZE and format for prek lint
* Update CI to use wasm-release profile for WASM tests
* Remove leftover conflict markers
* Clean remaining conflict markers in CI workflow
* Clean remaining conflict markers in CI workflow
* Trigger CI to pick up new required check
* Remove --max-memory flag, keep stack limit only
Redo of #7800.
Bumping the random and crypto crates is not trivial. Both sets of crates
have had API changes which require manual updates.
AI use:
I wrote all of the code myself but used AI to figure out the new APIs.
AI explained SHAKE's block size and why the block_size() function
doesn't exist in the new crate. I confirmed the block size, which is
actually represented in the new types for SHAKE. `Shake128` is actually
`Shake<168>` - 168 is the block size. I cited Wikipedia as it lists the
SHAKE block size in bits.
I also used AI to figure out HMAC for SHA3.
Assisted-by: Codex:gpt-5.4
Closes: #7893
Fix 1:
`foreign-types-shared` needs to match `openssl`'s version. Bumping it is
a SemVer violation because the latest versions of the crate aren't
backwards compatible with older versions.
See: rust-openssl/rust-openssl#2461
Fix 2:
The second fix is to align the `openssl` module with the latest
`host_env` and `ssl` changes.
* Modularize rustls as work towards providers
`rustls`'s architecture is very clean and trait-driven. There are many
providers for `rustls` including the built-in `aws-lc-rs` and `ring` as
well as backends for `boringssl`, `graviola`, `openssl`, `mbedtls`, etc.
This commit removes the hard dependency on `aws-lc-rs` and adds support
for `ring`. It works towards #7059 as well.
* Clean up rustls features
* Remove ring as an explicit feature
* ssl-rustls is the default and implies aws-lc
* Support custom rustls crypto providers
The new feature, `ssl-rustls-no-provider`, enables custom rustls
providers. By default, `aws-lc-rs` is enabled which matches the old
behavior and keeps backward compatibility.
I wrote a new type that abstracts what we need from crypto providers.
CryptoExt encapsulates the ticketer as well as cipher suites and KX
groups. I wrote fallbacks to help select a reasonable default if a
provider is missing features (they all seem to support the same things
though).
I also wrote an example to show how to actually use custom providers.
* Fix duplicate VERIFY_X509 constants and unused imports in ssl module
Remove duplicate VERIFY_X509_STRICT/VERIFY_X509_PARTIAL_CHAIN definitions
from compat.rs (already defined in _ssl module with #[pyattr]).
Remove unused imports: ClientConnection, ServerConnection.
* no-provider as default
* Fix CI failures: openssl build, wasm target, cargo-shear
- Update openssl.rs to use renamed sock_wait/SockWaitKind and add vm parameter
- Add skip_ssl for wasm32-wasip2 target (aws-lc-sys cannot build for wasm)
- Remove unused workspace dependency aws-lc-rs
- Fix foreign-types-shared version to match openssl's dependency (0.1)
- Restore Cargo.lock from upstream/main
---------
Co-authored-by: Jeong, YunWon <jeong@youknowone.org>
* Add `rapidhash` to list of dependencies
* Use `rapidhash::quality::RandomState` in `codegen` crate
* Use `rapidhash::quality::RandomState` in `stdlib` crate
* Use `rapidhash::quality::RandomState` in `vm` crate
* Remove `ahash` from lists of dependencies
RustPython used an ancient, unmaintained version of the `dirs` crate.
This pulled in `winapi-rs` on Windows, which is an old crate that has
been deprecated in favor of `windows-rs`. `windows-rs` is maintained by
Microsoft. Bumping `dirs` to the latest version removes more `winapi-rs`
from Cargo.lock.
As for `uname`, `rustix` handles it safely so the additional crate isn't
needed.
The `page_size` crate is a simple libc wrapper for Unix and uses
`winapi-rs` for Windows. `windows-sys` and `windows-rs` are the modern
alternatives for the unmaintained `winapi-rs`. Both crates are
maintained by Microsoft - they're official. Getting the page size is a
simple call for both Unix and Windows.
Besides Unix and Windows, I also added the page size for wasm32 which
the `page_size` crate did not support. `wasm32`'s page size is a
constant that is defined by the spec, so I hard coded it without adding
additional dependencies.
Finally, I dropped `maplit` which is seven years old and only used in
one place. Calling `collect()` with a single item iterator is idiomatic
as well as better in this case because Rust can optimize it. `maplit`
called `HashMap::insert` which over allocates to amortize future allocs.
* PyBytes.title should be ASCII-only.
* Use icu_casemap over unicode-casing for titles
`icu_casemap` is consistently maintained, official, and tracks the
latest Unicode versions. RustPython is also using other `icu4x` crates,
so using `icu_casemap` is more consistent.
As with islower and isupper, tracking the latest Unicode version is
important because character definitions shift over time which causes
discrepancies between RustPython and CPython.
This commit fixes title().
* Use icu_casemap for capitalize()
I dropped unicode-casing because it's cleaner to use icu4x for
everything. `icu4x` will also stay up to date whereas unicode-casing
will need to be periodically updated with new Unicode tables. Dropping
unicode-casing also removes some binary bloat due to the tables.
`capitalize()` mimics CPython behavior more closely now as well.
Notably, I implemented CPython's sigma edge case handler.
* Match CPython's title() exactly
Jeong, YunWon