Dual-Root Mismatch in RustPython:
Unlike CPython, RustPython manages the standard library
in two separate locations in the workspace:
- Workspace 'Lib/': Contains the test suite ('Lib/test/...') and
standard library files.
- crates/pylib/Lib/: A helper directory used for packaging.
'crates/pylib/Lib' is a symlink pointing to the real workspace '../../Lib/'.
Runtime Directory Discrepancy:
- 'sys.path' contains 'crates/pylib/Lib' (the path to the symlink).
- Thus, when 'test.support' is imported, its '__file__' is resolved relative
to the symlink: '/opt/RustPython/crates/pylib/Lib/test/support/__init__.py'.
- Consequently, 'STDLIB_DIR' resolves to the symlink root:
'/opt/RustPython/crates/pylib/Lib'.
- However, since 'test_program.py' is executed directly from the workspace,
its 'pkg_dir' is: '/opt/RustPython/Lib/test/test_unittest/testmock'.
- When 'os.path.relpath(pkg_dir, top_dir)' is called:
---
os.path.relpath(
"/opt/RustPython/Lib/test/test_unittest/testmock",
"/opt/RustPython/crates/pylib/Lib"
)
# Result: "../../../Lib/test/test_unittest/testmock"
---
Because the relative path starts with '..', the assertion fails.
Assisted-by: Gemini
* Fix clippy warning
* Fix clippy warnings surfaced by newer clippy
- exceptions.rs: {b:#02x} -> {b:#04x} (unused_format_specs); output
unchanged for decode-error bytes (always >= 0x80)
- binascii.rs: [b'\r', b'\n'] -> b"\r\n" (byte_str_slice)
- host_env, _io.rs, ssl.rs, pyexpat.rs: expect std_instead_of_core where
the suggested core::io items (ErrorKind, Cursor) are unstable (core_io);
use expect so the suppression is flagged for removal once core::io
stabilizes. build_posix_spawn_attrs co-gates the expect with the cfg
block so it is not left unfulfilled on platforms compiling it out.
Assisted-by: Claude
* Fix more
---------
Co-authored-by: Jeong YunWon <jeong@youknowone.org>
* Align compiler and AST behavior with CPython
Update the compiler pipeline, AST module support, and related VM compile/eval paths toward CPython 3.14 behavior.
Use upstream Ruff parser crates directly and keep RustPython-specific syntax preflight handling in the compile path.
Refresh opcode metadata, snapshots, and targeted tests for the aligned bytecode and AST behavior.
* fix
* Fix PR 8138 CI failures
* Fix match mapping wildcard scan complexity
* fixes
* Use Ruff runtime AST fields
* Minimize residual patch diff
* refactor error handling
* Remove _ast conversion context wrappers
* Clean up _ast conversion glue and compiler warning escalation
Rename the Node conversion parameter `ctx` to `vm` throughout the _ast
module.
Remove the unused compile_program, compile_program_single,
compile_block_expression, and compile_expression forwarders; compile_top
stays as the entry point.
Stash an escalated compiler SyntaxWarning so a non-SyntaxWarning category
propagates unchanged instead of being rewritten to SyntaxError, matching
PyErr_ExceptionMatches(SyntaxWarning) in compiler_warn. Drop the now-unused
CompileWarningError::into_codegen_error.
Replace the PositionalArguments two-variant enum with a struct holding a
shared range and a PositionalArgumentsKind enum.
Assisted-by: Claude
* Represent compiler flags as a CompilerFlags bitflags
Replace the loose PY_CF_*/CO_* i32 constants in compile_mode with a
CompilerFlags bitflags covering the PyCF_* flags and __future__
CO_FUTURE_* bits. The Python-visible ast.PyCF_* attribute values are
derived from it via bits(); parser input modes stay plain consts.
builtins::compile and the compile_string flag handling test flags
through CompilerFlags::contains instead of raw bit math.
Assisted-by: Claude
* Fix PR 8138 test regressions
* Fix PR 8138 CI failures
* Avoid shell interpolation in Linux deps action
* Fix review findings in compiler/AST changes
- frame.rs: compute the MATCH_CLASS class name lazily so a successful
match no longer takes the type name lock or allocates; the
__match_args__-not-a-tuple branch reuses it instead of recomputing.
- _ast/constant.rs: add the missing leading space to the with_recursion
label ("maximum recursion depth exceeded during compilation").
- sys.set_int_max_str_digits: restore the "maxdigits must be 0 or larger
than {n}" error text.
- codegen: set CO_MAXBLOCKS to 20 in both ir.rs and compile.rs.
- _ast/exception.rs: extract the duplicated excepthandler node guard
into ensure_excepthandler_node.
- codegen: correct the validate_keywords doc comment.
- Drop redundant "CPython" qualifiers from comments added in this branch.
Assisted-by: Claude
* Modularize rustls as work towards providers
`rustls`'s architecture is very clean and trait-driven. There are many
providers for `rustls` including the built-in `aws-lc-rs` and `ring` as
well as backends for `boringssl`, `graviola`, `openssl`, `mbedtls`, etc.
This commit removes the hard dependency on `aws-lc-rs` and adds support
for `ring`. It works towards #7059 as well.
* Clean up rustls features
* Remove ring as an explicit feature
* ssl-rustls is the default and implies aws-lc
* Support custom rustls crypto providers
The new feature, `ssl-rustls-no-provider`, enables custom rustls
providers. By default, `aws-lc-rs` is enabled which matches the old
behavior and keeps backward compatibility.
I wrote a new type that abstracts what we need from crypto providers.
CryptoExt encapsulates the ticketer as well as cipher suites and KX
groups. I wrote fallbacks to help select a reasonable default if a
provider is missing features (they all seem to support the same things
though).
I also wrote an example to show how to actually use custom providers.
* Fix duplicate VERIFY_X509 constants and unused imports in ssl module
Remove duplicate VERIFY_X509_STRICT/VERIFY_X509_PARTIAL_CHAIN definitions
from compat.rs (already defined in _ssl module with #[pyattr]).
Remove unused imports: ClientConnection, ServerConnection.
* no-provider as default
* Fix CI failures: openssl build, wasm target, cargo-shear
- Update openssl.rs to use renamed sock_wait/SockWaitKind and add vm parameter
- Add skip_ssl for wasm32-wasip2 target (aws-lc-sys cannot build for wasm)
- Remove unused workspace dependency aws-lc-rs
- Fix foreign-types-shared version to match openssl's dependency (0.1)
- Restore Cargo.lock from upstream/main
---------
Co-authored-by: Jeong, YunWon <jeong@youknowone.org>
`version.rs` essentially consists of constants that can be baked in at
compile time. I moved most of `version.rs` to `build.rs`. The constants
are passed via rustc's environment then stored in the binary.
* Add basic capi error support
* Add missing symbols to make tests compile again
* Add `pyerrors` to dictionary
* Return exception in `Py_GetConstantBorrowed`
* Remove `allow(dead_code)`
* Fix windows
* Load stdlib when calling `Py_InitializeEx`
* Debug tests
* Revert "Load stdlib when calling `Py_InitializeEx`"
This reverts commit bccd38e981.
* Disable tests on windows for now
* Truncate `PyType_GetFlags` to be always 32 bits
* Add test for exception type checking
* Remove subclass type flags
* Use latest pyo3 to make test work on windows
* Revert "Use latest pyo3 to make test work on windows"
This reverts commit b2c2f6913f.
* `set_main_interpreter` -> `init_main_interpreter`
* Add missing _winapi functions and fix WinHandle int conversion
Add 13 functions: ReadFile, SetNamedPipeHandleState, CreateFileMapping,
OpenFileMapping, MapViewOfFile, UnmapViewOfFile, VirtualQuerySize,
CopyFile2, ResetEvent, CreateMutex, OpenEventW, LoadLibrary,
_mimetypes_read_windows_registry.
Add constants: INVALID_HANDLE_VALUE, FILE_MAP_READ/WRITE/COPY/EXECUTE.
Change WinHandle integer type from usize to isize so negative values
like INVALID_HANDLE_VALUE (-1) can be passed from Python.
* Align _winapi module with CPython
- Rename winapi.rs to _winapi.rs with #[path] attribute
- Rename CreateMutex to CreateMutexW
- Add missing constants: ERROR_ACCESS_DENIED, ERROR_PRIVILEGE_NOT_HELD,
PROCESS_ALL_ACCESS, 10 STARTF_ constants, LOCALE_NAME_SYSTEM_DEFAULT,
LOCALE_NAME_USER_DEFAULT, COPY_FILE_DIRECTORY
- Fix OpenMutexW return type and ReleaseMutex param type to use WinHandle
* Fix ReadFile/WriteFile overlapped keyword argument
Use FromArgs structs so overlapped parameter can be passed as
a keyword argument (overlapped=True), matching the CPython API.
* Remove extra constants and LoadLibrary not in CPython _winapi
Remove 19 constants (WAIT_ABANDONED, CREATE_ALWAYS, CREATE_NEW,
OPEN_ALWAYS, TRUNCATE_EXISTING, FILE_ATTRIBUTE_NORMAL, 8 FILE_FLAG_*,
3 FILE_SHARE_*, NMPWAIT_NOWAIT, NMPWAIT_USE_DEFAULT_WAIT) and
LoadLibrary function that are not present in CPython's _winapi module.
* Fix utf8_mode default to 0 and add PYTHONUTF8 env var support
Default utf8_mode was incorrectly set to 1, causing text-mode
subprocess to always decode as UTF-8 instead of locale encoding.
Changed default to 0 to match CPython 3.13 behavior on Windows.
Added PYTHONUTF8 environment variable handling with -X utf8 override.
* Fix CopyFile2 to raise proper OSError subclass
Use std::io::Error::from_raw_os_error instead of vm.new_os_error so
that winerror attribute is set and errno-to-exception mapping works
(e.g. ERROR_ACCESS_DENIED → PermissionError).
* Fix syntax_non_utf8 test to not depend on locale encoding
Use explicit encoding='latin-1' so the test works regardless of
the system locale (e.g. C/POSIX locale uses ASCII by default).
* Add host_env feature for sandbox isolation
Introduce a `host_env` feature flag that gates all host environment
access (filesystem, network, signals, processes). When disabled,
the VM operates in sandbox mode:
- _io module always available; FileIO gated by host_env
- SandboxStdio provides lightweight stdin/stdout/stderr via Rust std::io
- BytesIO/StringIO/BufferedIO/TextIOWrapper work without host_env
- open() returns UnsupportedOperation in sandbox
- stdlib modules (os, socket, signal, etc.) gated by host_env
- CI checks both host_env ON and OFF builds
* Auto-format: ruff check --select I --fix
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Refine documentation in interpreter.rs
Improve InterpreterConfig with convenience methods (with_debug, add_path, add_paths), better documentation with working examples, refactored stdlib setup into focused functions, and comprehensive unit tests while maintaining 100% backward compatibility.
* Improve error message for non-Unicode paths
* Use consistent make_module naming in all doc examples
* cut useful changes
---------
Co-authored-by: Mohamed Gharbi <galaxysea777@gmail.com>
* * Added alloc_instead_of_core, std_instead_of_alloc, and std_instead_of_core clippy rules
* Manually changed part of the code to use core/alloc
* use clippy --fix to fix issues in stdlib
* * Used clippy --fix to fix issues in vm
* Imported Range in vm/src/anystr.rs
* * Used clippy --fix to fix issues in common
* Update ruff to 0.14.1
* Fix test regression in `test_compile`
* Unmark passing test
* Update `test_syntax` from 3.13.9
---------
Co-authored-by: Noa <coolreader18@gmail.com>
There aren't any tests but being able to do nested for loops seems like
a pretty big win to me so I'm going to put up for review.
The original returned boolean clearly had **false positives** for
detecting bad errors for things like nested `if` and `for` statements.
What is less clear is if there are any **true positives** which I am no
longer catching with the updated return value.
Co-authored-by: Jack O'Connor <jack@jackoconnor.dev>
Leesoo Ahn