mirror of
https://github.com/RustPython/RustPython.git
synced 2026-10-29 03:04:38 +08:00
fc853015d1732b5b488bbd102005585f2fb3cbc2
137
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 youknowdot
|
170e772255 |
Refresh Python 3.15 HTTP, URL and documentation libraries (#9193)
* Refresh Python 3.15 HTTP, URL and documentation libraries Import the connected documentation and HTTP/URL library closure from CPython d220409a13b77de7a6c30abe16ea90db01528edc using scripts/update_lib. Preserve surviving RustPython test guards; the upstream-removed CGI test class has no remaining marker target. Assisted-by: OpenAI Codex:model version unavailable * Retire obsolete wsgiref file-wrapper test forwarder Canonical Python 3.15 removed the target method; migrate with an empty marker override rather than retain an expected-failure call to a nonexistent super method. Assisted-by: OpenAI Codex:model version unavailable * Adapt pydoc classification to the Cargo source layout Expose the existing source-directory predicate as sys._is_python_build and regenerate pinned pydoc with the explicit updater transform shared with the mega repair. Installed layouts and explicit basedir arguments retain upstream behavior. Assisted-by: OpenAI Codex:model version unavailable * Expose source-directory classification on every platform The existing platform-neutral Lib and Cargo.toml predicate is needed by sys._is_python_build on all platforms; preserve its algorithm unchanged. Assisted-by: OpenAI Codex:model version unavailable |
||
|
youknowdotandJeong YunWon
|
3f55cf4af0 |
Implement basic REPL execution and fix piped stdin handling (#9163)
* Preserve original main VM through C API execution
Run and finalize the original main VM while keeping the registered attachment template available without holding its mutex across Python callbacks.
Assisted-by: OpenAI Codex:model version unavailable
* Make unraisable reporting robust at hook boundaries
Keep the existing message API and traceback formatting while handling missing, disabled, audited and failing hooks without panicking.
Assisted-by: OpenAI Codex:model version unavailable
* Separate interactive warning collection from compilation
Retain ordinary warning behavior and typed module APIs while exposing record-local parser warnings and post-warning code generation. Add a source-only lazy future-import guard.
Assisted-by: OpenAI Codex:model version unavailable
* Evaluate native stdin with shared VM record semantics
Use one evaluator for basic REPL and root transports, preserve effective inspect snapshots and future flags, and carry owned host exit status through original-main finalization.
Assisted-by: OpenAI Codex:model version unavailable
* Refresh canonical REPL tests and retire fixed failures
Run update_lib migrate from pinned CPython 3.15.0, preserving three remaining platform/runtime markers and retiring the two traceback/startup failures verified on the native evaluator.
Assisted-by: OpenAI Codex:model version unavailable
* Keep the root compiler dependency for development targets
The native evaluator now compiles through the VM. Only root benchmarks and the parse_folder example use rustpython-compiler directly, so declare it as a dev dependency while preserving VM compiler features.
Assisted-by: OpenAI Codex:model version unavailable
* Validate explicit conditional failure marker retirement
Require exact selector and original decorator evidence for each conditional expected-failure marker retired by migrate. Preserve undeclared guards and reject missing, duplicate, modified or unconsumed declarations.
Assisted-by: OpenAI Codex:model version unavailable
* Refresh canonical stdin and exception audit tests
Migrate the pinned CPython 3.15.0 modules and their canonical helper fixtures through update_lib. Retire exactly four failure markers proven obsolete by hosted CI, preserving all other existing guards.
Assisted-by: OpenAI Codex:model version unavailable
* Synchronize runpy warning attribution and document remaining capabilities
Migrate whole canonical runpy.py to pass module names to compile. Use update_lib to annotate only reproduced missing-capability cases: absent _testcapi import-event tests, unsupported startup/import-time features, and the deferred frozen-loader module-attribution caller.
Assisted-by: OpenAI Codex:model version unavailable
* Propagate interrupted native stdin reads to signal dispatch
Preserve the shared input buffer and partial bytes while returning EINTR to the VM before retrying. Exercise the transport deterministically and verify a returning SIGINT handler while pipe stdin stays open.
Assisted-by: OpenAI Codex:model version unavailable
* Reject statements after a compound statement in single mode
single_mode_multiple_statements_error now reports "invalid syntax" at the
statement that follows a compound first statement, and interactive_input
runs it right after parsing as compile() does.
The lazy `from __future__` import check moved from lazy_future_import_error
into unsupported_grammar_error, so it applies to every compile mode.
Assisted-by: Claude Code:claude-opus-5-5
* Remove passing expected failure from test_runpy
test_run_module_filter_syntax_warnings_by_module passes.
Assisted-by: Claude Code:claude-opus-5-5
* Restructure stdin execution after Modules/main.c and pythonrun.c
- src/lib.rs follows pymain_run_python: pymain_run_stdin, pymain_repl,
pymain_run_startup, pymain_run_interactive_hook and pymain_run_pyrepl
return exit codes instead of raising SystemExit.
- VirtualMachine gains run_any_stdin (_PyRun_AnyFile),
run_interactive_loop (_PyRun_InteractiveLoop) and run_simple_stdin;
run_shell feeds rustyline lines into run_interactive_loop, and
sys._baserepl calls run_any_stdin.
- print_uncaught_exception (PyErr_Print) replaces
report_uncaught_exception, and handle_system_exit_and_keyboard_interrupt
replaces handle_reported_exception.
- An unhandled KeyboardInterrupt is recorded and the process exits
through SIGINT after finalization.
- Interpreter::run_main replaces run_with_exit_status and InterpreterExit;
rustpython_capi::run_main_interpreter takes an exit-code callback.
- config_set_inspect audits cpython.PyConfig_Set and rebuilds sys.flags
through set_flag_obj (_PySys_SetFlagObj); the inspect state is an
AtomicBool and dict replace_by_identity is removed.
- add_main_module (PyImport_AddModuleRef("__main__")) is used by the
interactive and simple-file paths.
- The interactive parser warning helpers take their state directly.
Assisted-by: Claude Code:claude-opus-5-5
* Exit through Py_Exit from PyErr_Print and run the basic REPL via stdin
- print_uncaught_exception (PyErr_Print) calls exit_process (Py_Exit) for a
SystemExit when inspect mode is off, instead of returning the exception.
run_interactive_loop no longer checks SystemExit itself and flushes after
printing an error; sys._baserepl returns None.
- exit_process finalizes through VirtualMachine::finalize_ex, which holds
the Py_FinalizeEx steps split out of Interpreter::finalize, and exits
through host_env::os::exit_with_cleanup (C exit()).
- The basic REPL on a terminal (PYTHON_BASIC_REPL, or _pyrepl missing)
runs run_any_stdin instead of the rustyline shell.
- run_shell saves the history after every line.
Assisted-by: Claude Code:claude-opus-5-5
* Display exceptions through PyErr_Display and track the main thread running __main__
- display_exception (PyErr_Display) and display_exception_to (_PyErr_Display)
try traceback._print_exception_bltin before the native printer and flush
the file. PyErr_Print's excepthook fallback, print_exception and
sys.excepthook use them. A missing sys.stderr is reported with
dump_object (PyObject_Dump); a None sys.stderr prints nothing.
- Paths gains program_name (argv[0]) from getpath. The "can't open file"
message uses it with strerror(errno) and goes to sys.stderr.
- running_main holds the ident of the thread running __main__.
pymain_run_python sets it around running the main code, Py_Exit clears it,
the fork child reinitializes it, and _interpreters.is_running reads it
instead of treating the main interpreter as always running.
- Interpreter::run_main clears the unhandled KeyboardInterrupt flag before
running and applies exit_sigint after finalizing; finalize no longer
does.
- run_interactive_loop returns a SystemExit raised outside inspect mode
again and no longer flushes after printing an error; run_shell saves the
history once after the loop.
- pymain_set_inspect reports its error through print_uncaught_exception.
Assisted-by: Claude Code:claude-opus-5-5
* Treat a parse error at the end of the source as incomplete input
A parser error that starts at the end of the source is reported as
_IncompleteInputError when incomplete input is allowed, so the
interactive loop and codeop keep reading after `try:` and its block or
after a decorator line.
Assisted-by: Claude Code:claude-opus-5-5
* Place syntax errors on tokens made up at the end of the input
A parse error on the NEWLINE, DEDENT or ENDMARKER the tokenizer makes up
at the end of the input now carries an EofToken: the number of lines
read and the last line. SyntaxError takes its lineno and text from it,
and its offset is that line's length with end_offset -1, or 0 for both
when the error is raised at the token itself (the generic syntax error
and a missing forced `:` or `(`). Exec input that does not end with a
newline is checked with one appended.
Assisted-by: Claude Code:claude-opus-5-5
* Slice the source with get when checking an indented block error at EOF
Assisted-by: Claude Code:claude-opus-5-5
* Rename the host exits after C exit() and fix wasi and Windows builds
- host_env::os::exit now calls the C exit() on every platform; on Windows
std::process::exit ends in ExitProcess, so it flushes the Rust stdout
and calls libc::exit there. The former exit, which calls _exit(), is
now exit_immediately, used by os._exit, faulthandler and
posixsubprocess.
- errno::strerror_string is available on wasi.
- PyGlobalState::reinit_running_main is built only where its fork caller
is.
Assisted-by: Claude Code:claude-opus-5-5
---------
Co-authored-by: Jeong YunWon <jeong@youknowone.org>
|
||
|
youknowdot
|
9f6f771965 |
Align lazy imports and f/t-string interpolation with Python 3.15 (#9179)
* Preserve interpolation source text and canonical metadata Source-only checkpoint; candidate validation and the existing annotation expectation reconciliation remain pending. Assisted-by: OpenAI Codex:model version unavailable * Align existing annotation expectations with CPython metadata Update only the nine existing expected strings confirmed against pinned CPython 3.15, with explicit user approval. Assisted-by: OpenAI Codex:model version unavailable * Preserve template metadata through compilation and interactive input Escape literal debug prefixes, retain None AST expression metadata internally, and continue incomplete triple-quoted t-strings in the existing shell. Assisted-by: OpenAI Codex:model version unavailable * Classify normalized unclosed brackets as REPL line continuations Assisted-by: OpenAI Codex:model version unavailable * Nest equivalent REPL parse-error alternatives for Clippy Assisted-by: OpenAI Codex:model version unavailable * Remove fulfilled annotation migration lint workaround The canonical expected strings no longer trigger literal_string_with_formatting_args, so the previous lint expectation is obsolete. Assisted-by: OpenAI Codex:model version unavailable * Complete lazy-import exception and traceback library closure Copy canonical traceback and legacy exception mappings with preserved RustPython guards. Copy explicit non-Python library assets by exact path so exception hierarchy data does not expand into the entire test package. Assisted-by: OpenAI Codex:model version unavailable * Provide Python 3.15 traceback metadata for missing modules and attributes Preserve AttributeError context for missing class attribute deletion, matching type_update_dict. Expose canonical generated missing-stdlib messages through the native sysconfig modules without importing CPython build-specific disabled-module assumptions. Assisted-by: OpenAI Codex:model version unavailable |
||
|
youknowdot
|
6e8e7e0481 |
Align startup paths and sysconfig/venv with Python 3.15 (#9174)
* Align startup home and venv paths with canonical sysconfig closure Assisted-by: OpenAI Codex:model version unavailable * Preserve selected stdlib and empty home prefix fallback Assisted-by: OpenAI Codex:model version unavailable * Collapse startup exec-prefix fallback condition Assisted-by: OpenAI Codex:model version unavailable * Preserve startup search order and freeze the ZIP bootstrap Align zipimport and its tests with CPython 3.15.0 via update_lib, preserving inherited test guards. Keep build-time fallback paths after configured paths and the default ZIP, while respecting customized embedding paths. Assisted-by: OpenAI Codex:model version unavailable * Retire proven trace coverage guards through update_lib Both canonical coverage-file tests execute successfully after preserving startup search order. Preserve the other two inherited guards and restore canonical Python 3.15 JIT optimizer decorators. Assisted-by: OpenAI Codex:model version unavailable * Align Windows symlink helper with canonical Python home layout Import the pinned CPython 3.15 PythonSymlink class through a narrow, hash-pinned update_lib mode. Preserve canonical home/VPATH behavior and explicitly supply RustPython build stdlib to the Windows child without changing runtime PYTHONHOME semantics or skipping tests. Assisted-by: OpenAI Codex:model version unavailable * Reset surviving thread completion locks after fork Reinitialize every live thread handle completion mutex and condition variable before skipping the surviving or not-yet-started handle. Preserve their completion payload and state, and retain the existing dead-thread cleanup. Assisted-by: OpenAI Codex:model version unavailable |
||
|
youknowdot
|
7d15b50cd4 |
Align weak-reference collection with Python 3.15 finalization (#9168)
* Align weak-reference collection with Python 3.15 finalization Trace and retain owned callback edges independently of weak referent links. Clear callback-bearing weakrefs before finalizers, suppress callbacks from unreachable weakrefs, and close remaining weakref ingress after resurrection checks while the world is stopped. Publish and unlink intrusive nodes safely. Refresh weakref and canonical weakref/GC/finalization tests through update_lib from pinned CPython 3.15.0. Retire four obsolete guards only after fresh body passes, retaining applicable RustPython limitations. Assisted-by: OpenAI Codex:model version unavailable * Refresh canonical IO tests for weakref finalization Migrate the complete CPython 3.15.0 test_io package at d220409 with the fail-closed legacy-map updater, preserving all ten existing guards. Retire the seven consolidated standalone suites and update io/file dependency paths with mapping regression tests. The canonical GC tests ignore ResourceWarning without retaining its source, so callback-free weakrefs correctly reflect surviving owners. This closes the stale test dependency exposed by the lifecycle change. Assisted-by: OpenAI Codex:model version unavailable * Protect weakref callbacks with the original referent stripe Follow CPython's free-threading weakref locking model instead of keeping an independent callback RwLock. Retain the original referent stripe key so callback access stays synchronized after GC clears the referent. Unlink and steal callbacks under that stripe, then release ownership outside the lock. Preserve synchronization for running-world GC paths and gc.get_referents while keeping ordinary weakref upgrades unchanged. Assisted-by: OpenAI Codex:model version unavailable * Defer weakref GC tracking until referent publication Keep freshly allocated weakrefs out of GC's object lists until they are linked and their referent is published. Release the referent stripe before tracking, since tracking can collect and invoke finalizers. Generic reuse losers remain untracked. Exercise the allocator boundary directly for reference, proxy, callable proxy and subclass payloads, and ensure completed constructors track their objects. This prevents observers from retaining a half-built dead weakref that later becomes live. Assisted-by: OpenAI Codex:model version unavailable * Retain the original weakref stripe lock like CPython Store a safe handle to the selected static weakref stripe instead of retaining and rehashing the referent address. Keep the same lock after referent clearing, matching CPython free-threaded weakrefs_lock. The nonthreaded handle is zero-sized and introduces no locking. Preserve lock selection, acquire/release ordering, fork reset, deferred GC publication, and callback-owner release outside the stripe. No new locks or unsafe code are needed. Assisted-by: OpenAI Codex:model version unavailable |
||
|
youknowdot
|
fc9f4a9541 |
Update importlib to CPython 3.15 with native bootstrap callback (#9169)
* Refresh canonical importlib and preserve native import callback Cache the original builtin __import__ callback so CPython 3.15 importlib's hierarchical locking uses the existing native parent-initialization path. Refresh the complete canonical importlib and related test/helper families from CPython d220409a13b77de7a6c30abe16ea90db01528edc through update_lib. Preserve both nondecorator importlib guards in the updater. Add explicit hash-pinned removal of reviewed obsolete canonical files, with full preflight, dry-run, idempotence and refusal tests. Preserve empty fixtures. Assisted-by: OpenAI Codex:model version unavailable * Complete canonical inspect and types closure for importlib metadata Refresh whole canonical inspect/types production and related test families through update_lib after hosted CI exposed stale __cached__ consumers. Add coroutine and async-generator state getters, with closed-state priority for reentrant frame-local finalizers, and expose the existing frame-locals proxy type with its CPython class documentation. Preserve the optional _datetime and unittest import blocks through a narrow migration adapter. Retain canonical assertions and existing guards; record the missing LazyImportType expectation before adding a conditional expected failure, and capability-skip only the two absent-_testcapi CLI fixtures. Assisted-by: OpenAI Codex:model version unavailable * Reject canonical removal junctions and read UTF-8 manifests Reject junctions alongside symlinks in the existing path-component preflight and revalidation. Decode removal manifests explicitly as UTF-8. Add regressions for whole-batch refusal, revalidation, and Unicode dry-run/apply/idempotence. Assisted-by: OpenAI Codex:model version unavailable |
||
|
youknowdot
|
19d2fe3af7 |
Align bytes and bytearray operations with Python 3.15 (#9165)
Correct quote escaping, replacement count keyword binding, signed-minimum hex grouping, and clear storage/export handling. Migrate canonical bytes coverage through fail-closed preservation of the existing RustPython customizations, and retire only two freshly verified formatting guards. Assisted-by: OpenAI Codex:model version unavailable |
||
|
youknowdot
|
1f4e9ffaf6 |
Align OS argument conversion and syscall handling with Python 3.15 (#9146)
* Align OS argument conversion and syscall handling with Python 3.15 Import canonical os tests through update_lib, retaining existing guards. Preserve raw descriptor validation, errno, audit ordering, signal retries and unlocked sendfile buffers. Validation: independent source review passed; matching release build and nine bounded canonical cases pending. Assisted-by: OpenAI Codex:model version unavailable * Import canonical root-user test support for OS package Add hash-pinned selective assignment migration within test.support and import the three Python 3.15.0 privilege helpers plus their public exports. Preserve unrelated support helpers and existing test markers. Validation: 177 host updater tests pass; byte-identical canonical helper provenance verified. Independent source review passed. Native import validation pending integration onto current main. Assisted-by: OpenAI Codex:model version unavailable |
||
|
youknowdot
|
9e18d7be6b |
Preserve explicit customizations during canonical library migration (#9142)
* Preserve explicit customizations during canonical library migration Add bounded runtime, call, and doctest preservation plus explicit remaining-marker metadata. Support mapped legacy test-package migrations without losing ordered guards, and exclude cache directories from both migration paths while retaining canonical assets. Validation: 53 bounded host-Python tooling tests passed. Tests inspect source and temporary files without executing migrated fixtures. Existing copy-library cache protection is unchanged. No Lib or interpreter changes. Assisted-by: OpenAI Codex:model version unavailable * Integrate migration preservation into existing updater modules Keep migration orchestration and guarded package moves in cmd_migrate, and marker/doctest metadata in patch_spec. Reuse AST anchor validation and existing test modules, removing all eleven newly introduced helper, test, and data files. Require caller-selected doctest metadata instead of embedding syntax-case policy. Reject guarded legacy doctests before package writes. Preserve the prior 53 test scenarios and add seven focused metadata regressions; all 60 bounded host tooling tests passed. No Lib, workflow, or native changes. Assisted-by: OpenAI Codex:model version unavailable |
||
|
youknowdot
|
498adff3a9 |
Align JSON array hooks and libraries with Python 3.15 (#9135)
Generate the JSON library and selected tests with update_lib from CPython d220409a13b77de7a6c30abe16ea90db01528edc, preserving native bootstrap workarounds and test markers. Exclude generated cache directories at the library copy boundary. Assisted-by: OpenAI Codex:model version unavailable |
||
|
youknowdot
|
92e20da822 |
Fix update_lib marker preservation for inline classes (#9131)
* Preserve inherited test markers in inline class migrations Expand canonical inline class suites before appending retained inherited overrides, and validate generated syntax. Run focused migration-tool tests in a path-filtered Python-only CI job. Assisted-by: OpenAI Codex:model version unavailable * Run migration tool tests through existing CI conditions Keep Rust build gates unchanged and run focused Python migration tests on Linux for tool-only changes. Remove the separate workflow. Assisted-by: OpenAI Codex:model version unavailable |
||
|
youknowdot
|
0f127b07e9 |
Add Python 3.15 math APIs using pymath (#9122)
* Add Python 3.15 math APIs using pymath Pin pymath to merged dd1e9a4cf26d68caf3b549c09476f52e356e16a3 and expose isnormal, issubnormal, signbit, fmax and fmin through thin wrappers. Share the existing integer functions through math.integer and _math_integer, matching CPython module identity and metadata. Migrate test_math.py and test_math_integer.py via scripts/update_lib from CPython d220409a13b77de7a6c30abe16ea90db01528edc, preserving the existing Windows remainder marker. Add bounded Python snippet coverage. AI assistance: OpenAI Codex implemented and source-reviewed this change, ran update_lib, and performed focused automated validation. No human manual testing is claimed. Assisted-by: OpenAI Codex:model version unavailable * Track integer math tests in library updates Assisted-by: OpenAI Codex:model version unavailable |
||
|
|
28393e932d |
Implement PEP 814 frozendict (#8951)
* Restore consolidated frozendict implementation on current main Preserve reverse dictionary iterator fixes and the 3.15 frame imports. Keep library refreshes separate pending the runtime dependency sequencing decision. Assisted-by: OpenAI Codex:model identifier unavailable * Support scoped canonical test updates in update_lib Index conditional test classes and methods without crossing namespaces, reject ambiguous targets, and preserve inherited override indentation. Add hash-pinned whole-test copying that verifies execution context and exact source bytes, with focused Python fixtures. Assisted-by: OpenAI Codex:model identifier unavailable * Align frozendict copy support and tests with CPython 3.15 Import exact-final copy.py with update_lib; defer Mapping ABC registration to the later library stage. Generate only proven marker corrections, retain observed debug recursion-limit expected failures, and copy the complete canonical builtin-doc discovery test using hash-pinned update_lib copy-test. Assisted-by: OpenAI Codex:model identifier unavailable * Preserve decorator rationale during canonical test copying Reject differing decorator source blocks even when their ASTs match. Cover inline and multiline marker comments, guarded async methods, and CLI no-write behavior. Assisted-by: OpenAI Codex:model identifier unavailable * Finish frozendict with borrowed PyAnyDict views Use the landed PyRef target model for a transparent dict/frozendict header view, retaining objects explicitly with PyRef<PyAnyDict>. Keep targets sealed and preserve mutable-only entry points. Borrow frame globals from the object header rather than movable reference wrappers. Unify exact-TypeError key diagnostics with final CPython semantics, including callback failure propagation and no explicit cause. Keep the Python regressions in builtin_dict and test reference layout/ownership in Rust. Preserve existing canonical library updates. Assisted-by: OpenAI Codex:model identifier unavailable * Align frozendict with CPython and use CPython 3.15 tests - Name the target type in unhashable key errors through PyDict::update_arg (dict_update_arg), replacing the separate PyFrozenDict::merge_into copy of the merge logic. - Split the | operator into dict_or (_PyDict_Or) and frozendict_or. - Return the argument from frozendict(frozendict) only when the called type is frozendict. - Call through the vectorcall slot in PyCallable::invoke when there are no keyword arguments, instead of special-casing frozendict. - Add ensure_builtins (_PyEval_EnsureBuiltins) for eval/exec, and insert __builtins__ for source strings regardless of globals type. - Read __warningregistry__ and __name__ from stored globals entries and accept only None or str for the module name in setup_context. - Remove the unused PyFrozenDict::from_object. - Migrate test_dict.py, mapping_tests.py, test_copy.py and test_builtin.py to v3.15.0 with update_lib, and add test_import/data/syntax_warnings.py. - Drop builtin_dict.py frozendict tests covered by those modules, keeping the descriptor and namespace integration tests. Assisted-by: Claude Code:claude-opus-5-5 * Update crates/vm/src/builtins/dict.rs --------- Co-authored-by: Jeong YunWon <jeong@youknowone.org> Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com> |
||
|
youknowdotandJeong, YunWon
|
06007c81fa |
Begin the Python 3.15 target and essential-library migration (#8950)
* Target Python 3.15 using the final upstream reference Update active version and comparison configuration to Python 3.15.0. Preserve the alpha implementation status, actual bytecode format, and existing libraries pending separate migration. Align generated library-upgrade workflow pins without changing permissions or actions. Assisted-by: OpenAI Codex:model identifier unavailable * Adapt essential snippet contracts to Python 3.15 final Preserve existing coverage while following verified final behavior for script metadata, builtin signatures and loading, AST fixture construction, memoryview diagnostics, mmap search bounds, and struct format errors. The unchanged final comparator exposes the earlier expectations; focused native baseline failures establish the required implementation changes. Assisted-by: OpenAI Codex:model identifier unavailable * Keep final-reference regression paths and remove expired sysconfig argument Preserve reverse multiplication syntax and native exact overflow diagnostics. Accept the two source-proven CPython 3.15 specialization messages only in affected reverse str/bytes/tuple cases. Use an actually out-of-range Windows datetime errno fixture. Remove the sysconfig check_home argument at its existing canonical 3.15 removal test boundary. Assisted-by: OpenAI Codex:model identifier unavailable * Keep README wording unchanged in Python 3.15 transition Assisted-by: OpenAI Codex:model identifier unavailable * Match Python 3.15 repetition specialization and keep universal tests Use exact sequence-type guards and preserve generic fallback diagnostics. Validate both cold and warmed expressions without implementation-specific assertions. Reject stale CPython source checkouts and cross-version library caches. Assisted-by: OpenAI Codex:model identifier unavailable * Limit in-place specialization additions to exact sequence repetition Assisted-by: OpenAI Codex:model identifier unavailable * Keep subclass regression scoped to repetition dispatch Assisted-by: OpenAI Codex:model identifier unavailable * Provision final CPython 3.15 with pinned uv builds Assisted-by: OpenAI Codex:model identifier unavailable * Enforce Python 3.15 AST constructor requirements Reject retired missing-field and arbitrary-keyword forms while preserving defaults and explicit placeholders. Sync only matching final CPython constructor test hunks. Assisted-by: OpenAI Codex:model identifier unavailable * Keep seeded Python ahead of its Windows DLL directory Assisted-by: OpenAI Codex:model identifier unavailable * Keep opcode metadata tied to the implemented bytecode format The interpreter target and reference tests use Python 3.15.0, but the current Rust opcode enum retains the 3.14 bytecode format. Generate metadata from its matching source rather than importing unsupported instruction families. Assisted-by: OpenAI Codex:model identifier unavailable * Align CSV, logarithm and decimal formatting with Python 3.15 Propagate dialect lookup errors except AttributeError, retry integer logarithms after float overflow, and update the decimal conversion diagnostic. Exercise the final reference contracts without implementation-specific branches. Assisted-by: OpenAI Codex:model identifier unavailable * Bind the logarithm index result before borrowing its integer Assisted-by: OpenAI Codex:model identifier unavailable * Align required canonical test expectations with Python 3.15 Take only demonstrated contract changes from CPython v3.15.0, d220409a13b77de7a6c30abe16ea90db01528edc: removed module __cached__, Struct format exception classes, and decimal-format diagnostic. Preserve unrelated tests and expectations. Assisted-by: OpenAI Codex:model identifier unavailable * Track pending frozendict pickle support during the 3.15 transition Keep the two existing frozendict pickle tests running as expected failures until #8951 supplies the builtin. Unexpected success will signal removal of these temporary markers. Assisted-by: OpenAI Codex:model identifier unavailable * Support final Python 3.15 Windows negative timestamps Use FILETIME for pre-epoch conversions and preserve precise Windows timestamp errors. Assisted-by: OpenAI Codex:model identifier unavailable * Derive historical Windows timezone offsets from wall time FILETIME reports historical DST as unknown, so calculate the actual offset rather than selecting standard-time bias. Assisted-by: OpenAI Codex:model identifier unavailable * Identify decimal formatting arguments in Python 3.15 errors Track consumed positional arguments, including star width and precision, and preserve mapping key representations in decimal conversion diagnostics. Assisted-by: OpenAI Codex:model identifier unavailable * Reuse mapping format keys for lookup and error context Assisted-by: OpenAI Codex:model identifier unavailable * Borrow mapping diagnostic keys as Python objects Assisted-by: OpenAI Codex:model identifier unavailable * Avoid empty components in the reference Python library path Assisted-by: OpenAI Codex:model identifier unavailable * Isolate RustPython bytecode caches during the 3.15 transition Assisted-by: OpenAI Codex:model identifier unavailable * Sync final Windows timestamp test expectations Import only the final CPython platform-limit guards and mktime roundtrip expectation required by the existing Windows CI suite. Assisted-by: OpenAI Codex:model identifier unavailable * Restore official Python setup for 3.15 final Use the existing pinned setup-python action now that its official catalog supplies CPython 3.15.0. Remove the temporary standalone provisioning action and its sparse-checkout entry while retaining interpreter outputs and requested versions. Assisted-by: OpenAI Codex:model identifier unavailable * Align merged runtime contracts with Python 3.15 final Restrict mappingproxy ordering, use final codec registry normalization, and prefer index conversion before float conversion for sleep. Retain universal Python assertions and import the matching final codec/time canonical test hunks plus the non-ASCII codec-name deprecation warning. Upstream source: CPython d220409a13b77de7a6c30abe16ea90db01528edc. The existing codec normalization implementation and bytecode contract remain unchanged. Assisted-by: OpenAI Codex:model identifier unavailable * Limit Windows launcher version check to its interpreter Exclude only the CPython py.exe version-identity case from RustPython CI. The installed launcher reports its own final version, while RustPython correctly remains alpha. Preserve the other 45 launcher cases and all RustPython version assertions without modifying Lib. Assisted-by: OpenAI Codex:model identifier unavailable --------- Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com> |
||
|
Jeong, YunWon
|
cb0f3b012b |
Add a native _zoneinfo module (#8915)
* Add a native _zoneinfo module ZoneInfo reads TZif tuples from zoneinfo._common.load_data. The stdlib module implements utcoffset, dst, tzname, fromutc, and the zone cache. whats_left compares that class. Cache tests no longer expect failure. Assisted-by: Grok:grok-4.7 * Fix ZoneInfo layout and keep its cache on the module utcoffset, dst, and tzname take the object. Zone data is boxed so the payload lines up with tzinfo on 32-bit. The strong, weak, and timedelta caches live on each interpreter's _zoneinfo module. Assisted-by: Grok:grok-4.7 |
||
|
Jeong, YunWon
|
536737f4a0 |
Stabilize whats_left signature defaults and weakref repr (#8911)
Mask sys.executable, sys.version_info, the install scheme, and set order in signatures. Compare decimal and zoneinfo with the pure-Python fallback. Print the referent type's module in weakref repr. Assisted-by: Grok:grok-4.7 |
||
|
Jeong, YunWon
|
3c654204f5 |
Match native text signatures to CPython names, kinds and defaults (#8815)
* Render CPython defaults in generated text signatures Arguments whose default rendered as `<unrepresentable>`, while the same callable has a representable default in CPython, now render that default: - Where the Rust value can be a literal, the argument becomes a plain field with `default = <literal>`. - The remaining cases keep their Rust default and add `py_default` with a comment. These are wrappers such as ArgIntoFloat or ArgIntoBool, values that need the VM, enums, and Option where None means another value. Byte string literals render as Python bytes. A function registered under several names gets an internal doc per name, so its __doc__ no longer carries the signature header. Removed expectedFailure from test_pydoc tests that now pass. Assisted-by: Grok:grok-4.7 Assisted-by: Claude:claude-opus-5-5 * Match native callable signatures to CPython names and kinds Rename parameters and adjust positional-only / keyword-only kinds and defaults of native functions and methods so their generated text signatures match CPython. Remove expectedFailure markers from tests that now pass. Assisted-by: Claude:claude-opus-5-5 Assisted-by: Grok:grok-4.7 * Fix Windows console read size and keep type.__prepare__ permissive Assisted-by: Claude:claude-opus-5-5 * Derive signature defaults from typed Rust values Param.default is now a DefaultRepr rendered by the const writer. Literals, integer and bool constants become typed defaults, and other default expressions use the type's const fn py_default. A bare `optional` renders as None, so `default = None` rewrites are reverted. Address review comments: zlib max_length defaults, get_ca_certs binary_form keyword, getpeercert der, WASM ssl certificate keywords, fstatvfs fd-only, and _sre endpos shown as sys.maxsize. Assisted-by: Claude:claude-opus-5-5 Assisted-by: Grok:grok-4.7 * Generate class constructor text signatures at compile time A #[pyclass] impl composes the type's internal doc from the Args of its Initializer or Constructor, so type.__text_signature__ and inspect.signature work for native classes. __doc__ stays the plain doc. Struct sequences get (iterable=(), /) and __reduce__ (self, /). Hand-parsed constructors are converted to FromArgs structs, and _weakref.proxy is split from ProxyType. Assisted-by: Claude:claude-opus-5-5 Assisted-by: Grok:grok-4.7 * Remove expectedFailure from test_hmac tests that now pass Assisted-by: Claude:claude-opus-5-5 * Match remaining native signatures and count missing CPython signatures as matches Give lock.__enter__ and RLock.__enter__ their own no-argument methods, add FromArgs for start_new_thread and start_joinable_thread keeping their errors, take RTLD_GLOBAL/RTLD_LOCAL from libc, and align names, kinds and <unrepresentable> defaults where CPython's signature is invalid (bytes.hex, reduce, warn, link, setlocale, add_done_callback, ...). getaddrinfo accepts keyword arguments. whats_left treats a callable CPython has no signature for as matching. Assisted-by: Claude:claude-opus-5-5 Assisted-by: Grok:grok-4.7 * Drop unfulfilled dead_code expectation on os.link dir_fd args Assisted-by: Claude:claude-opus-5-5 * Distinguish OptionalArg and Option in signature defaults An `optional` field's signature default now comes from its type: Option<T> renders None, while OptionalArg<T> (a missing argument that is not any Python value) renders <unrepresentable>. Fields where a missing argument behaves exactly like None become Option<T>; fields whose missing state differs keep OptionalArg. Redundant py_default = "<unrepresentable>" entries are removed. Assisted-by: Claude:claude-opus-5-5 Assisted-by: Grok:grok-4.7 * Remove expectedFailure from test_rlcompleter.test_attr_matches Assisted-by: Claude:claude-opus-5-5 * Use Option for optional arguments whose missing and None states match OptionalOption and OptionalArg<Option<T>> fields that treat a missing argument like None become Option<T> with #[pyarg(optional)], which renders =None in the text signature. Plain optional parameters of that kind move into small FromArgs structs. Uses where missing and None differ (reduce initial, StringIO newline, TextIOWrapper.reconfigure, set_asyncgen_hooks) keep OptionalOption. Assisted-by: Claude:claude-opus-5-5 Assisted-by: Grok:grok-4.7 * Render ::NAME defaults by name and restore named constant defaults - default = ::NAME writes NAME into the text signature and stores Into::into(NAME) as the Rust value. - A one-field tuple-struct argument pattern such as Fildes(fd) names the parameter after its binding, so fcntl and termios keep destructuring. - zlib, lzma, syslog, mmap and select defaults use their named constants again instead of literals or <unrepresentable>. Assisted-by: Claude:claude-opus-5-5 Assisted-by: Grok:grok-4.7 * Name FromArgs fields after their Python parameters Drop #[pyarg(name = ...)] where the Python name is a valid Rust field name and rename the field instead. The option stays only for keywords (fn, loop, final, type, self) and the non-snake_case memLevel. Assisted-by: Claude:claude-opus-5-5 Assisted-by: Grok:grok-4.7 * Build literal defaults from Python values and type <unrepresentable> - default = LIT on a non-primitive field converts the literal through ToPyObject and TryFromObject when the argument is missing, so object fields no longer need a vm expression plus a matching py_default. - A bare default on a primitive integer, bool or float field renders its zero value. - DefaultRepr::Unrepresentable replaces the <unrepresentable> text, and py_default = "<unrepresentable>" is a compile error. - pyexpat.ParserCreate distinguishes a missing intern from intern=None, mmap AccessMode renders its value, and sqlite blobopen names rowid. Assisted-by: Claude:claude-opus-5-5 Assisted-by: Grok:grok-4.7 * Resolve native docstrings from the doc database at compile time - rustpython-doc stores its table as a sorted const slice with const fn lookups (get, get_attr, class_attr_doc) instead of a phf map, and the update-doc-db workflow emits that form. - Class, method, getset, module function and struct sequence docs, and per-class slot wrapper docs, are chosen by const lookups in the derive output. The database text wins over Rust doc comments. - Macros reach the table through rustpython_vm::__exports, so only the vm depends on rustpython-doc. - Remove expectedFailure markers from doctest and pydoc tests that pass. Assisted-by: Claude:claude-opus-5-5 Assisted-by: Grok:grok-4.7 * Keep time.gmtime, localtime and ctime without a text signature Their [seconds] parameter has no text signature, and unittest.mock relies on inspect.signature failing for them. Take OptionalOption again so the default stays unrepresentable, and treat None like a missing argument. Assisted-by: Claude:claude-opus-5-5 |
||
|
Hanjeong Lee
|
e189ca7923 |
Compare native signatures and docstrings in whats_left.py (#8751)
* Record a module's docstring mismatches on their own
They were only kept when the module also had missing items or signature
mismatches, so annotationlib, base64, ntpath and _ctypes reported none.
Whether a module counts as implemented still depends on missing items
and signatures alone.
Assisted-by: Claude Code:claude-opus-5
* Report a builtin type method that differs as a mismatch
A method whose signature or docstring differed was listed under
"builtin items" as if it were missing, with both extra_info dicts as the
reason. That section feeds the website's builtin_items.csv. Differences
now go to mismatched_items and mismatched_doc_items, keyed by type name,
like module items. Nothing reaches this path yet because builtin
signatures are still skipped; without this change, lifting that skip
turns the one "builtin items" line into 455.
Assisted-by: Claude Code:claude-opus-5
* Compare signatures and docstrings of native callables
extra_info() skipped anything inspect treats as builtin since
|
||
|
Jeong, YunWon
|
d7e41ebff5 |
Split CI cargo run and test into build then execute (#8705)
* Run gc_collect ten times per benchmark iteration Assisted-by: Grok:4.6 * Reuse the prebuilt rustpython for CI whats_left whats_left was invoking cargo build --release without threading, so the snippets job rebuilt the interpreter after the earlier release build. Assisted-by: Grok:4.6 * Split CI cargo run and test into build then execute Assisted-by: Grok:4.6 * Compile doctests in the CI build step cargo test --no-run skips doctests, so rustdoc compilation was still happening in the run step. Assisted-by: Grok:4.6 * Drop cron-ci.yaml from this PR A pull_request path filter matches the whole PR, so every push was starting the periodic benchmark and coverage jobs. Assisted-by: Grok:4.6 * Run doctests in their own CI step cargo test --doc --no-run is rejected, so doctests cannot be compiled without also executing them. Assisted-by: Grok:4.6 * Do not run benchmark targets in rust tests --all-targets includes Criterion benches, which cargo test does not run by default. Assisted-by: Grok:4.6 |
||
|
Lee DogeonandClaude Sonnet 5
|
44dcd1de2d |
ci: add pyperformance benchmarking scripts and workflow (#8691)
* Add script to run upstream pyperformance against RustPython pyperformance can't be installed as-is on RustPython since pyperf hard-depends on psutil, a C extension RustPython cannot build or load (no CPython C-API / extension-module support). Since pyperf already disables psutil usage on interpreters reporting Py_GIL_DISABLED=1 (which RustPython does, having no GIL), a functionality-free pure-Python psutil stub is enough to satisfy pip's dependency resolution and unblock any pure-Python benchmark. scripts/pyperformance/run_all.py drives pyperformance (under a host CPython venv) with RustPython as the --python target, installs the stub via PIP_FIND_LINKS + --inherit-environ, and catalogs pass/fail/timeout per benchmark with an automatic failure classifier (C-extension build blocked vs. a genuine exception raised by the benchmark itself). Assisted-by: Claude Code:claude-sonnet-5 * pyperformance: support targeting any Python and comparing catalogs Generalize run_all.py from a RustPython-only runner to a --python/--label pair, so the same script can catalog a real CPython (with --no-psutil-stub, since it has a genuine psutil) alongside RustPython. Each label gets its own results/<label>/ subdirectory. Add compare.py, which loads two labels' catalogs and writes a per-benchmark Markdown table with both targets' status/mean and the candidate/baseline time ratio, plus the median slowdown across benchmarks both targets passed. Assisted-by: Claude Code:claude-sonnet-5 * ci: compare base, head and CPython with pyperformance on one runner A ratio between two pyperformance runs measured on different machines can move further than the change under review did, so a PR's numbers are only worth reading when base, head and CPython were measured back to back on the same runner. This adds a job that does exactly that -- two release builds kept side by side with the stdlib of their own commit, then three runs with nothing in between -- and posts the table as a PR comment. `pr_diff.py` renders the three catalogs `run_all.py` writes: per benchmark the CPython time, both RustPython times, both ratios against CPython, and the head/base change, with anything under 3% shown as `~` because a shared runner is not quiet enough to say more. The comment is handed to a `workflow_run` companion the same way `codspeed-comment.yaml` does it, since a `pull_request` run from a fork gets a read-only token however the workflow is configured. The job is opt-in behind a `run:pyperformance` label: it runs 28 benchmarks three times over and costs well over an hour, which most PRs do not need. `run_all.py` now forwards RUSTPYTHONPATH through pyperf, which a binary copied away from its checkout needs in order to find the stdlib at all; the two names go in one comma-separated `--inherit-environ`, because repeating the flag keeps only the last. Assisted-by: Claude Code:claude-fable-5-1 * ci: comment from the pyperformance run itself when the PR allows it The `workflow_run` companion only starts firing once it is on the default branch, so a pull request that introduces it -- this one -- would upload the table and never show it. A pull request opened from this repo carries a token that may write comments, so post it from the run that measured it and leave the companion to the forks that genuinely cannot. Both paths upsert on the same marker, so a fork PR that reaches the companion after this step was skipped still lands one comment. Assisted-by: Claude Code:claude-fable-5-1 * ci: address zizmor findings and fall back to head vs. CPython - Pin `actions/github-script` to the commit the rest of the repo pins it to; the hash carried over from an in-flight branch pointed elsewhere than the `v9.0.0` comment claimed (ref-version-mismatch). - Read the PR number, the CPython path and version, and the two commit shas from `env` rather than expanding them into the shell (template-injection). - Record the `workflow_run` trigger in `.github/zizmor.yml` next to the existing `pull_request_target` entry, with the same justification: the workflow reads one artifact and writes a comment, and never checks out or runs pull request code (dangerous-triggers). Building or measuring the base commit is now non-fatal, and `pr_diff.py` renders head against CPython alone when the base column has nothing in it -- a base that will not build should still leave a usable table rather than failing the job. The time and ratio columns are also named apart, since `| base | head | base | head |` gave no clue which pair was which. Assisted-by: Claude Code:claude-fable-5-1 * ci: derive pyperformance comment target from trusted workflow_run metadata pr_number.txt came from the artifact a `pull_request` run uploads, which a fork could tamper with to make this privileged `workflow_run` comment on an arbitrary PR/issue. Resolve the PR from context.payload.workflow_run instead (falling back to a commit-associated open PR lookup for workflow_dispatch runs), and escape any HTML comments already present in diff.md so they can't be mistaken for the marker. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * ci: grant actions:read to the pyperformance comment workflow download-artifact needs actions:read to fetch another run's artifact by run-id. Specifying a permissions block sets every unlisted scope to none, so without this the cross-run download silently failed for fork PRs -- the scenario this workflow exists for -- continue-on-error masked it, and the comment step found no diff.md and returned without posting. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: kill the whole process group on a benchmark timeout subprocess.run's own timeout only terminates the direct pyperformance process; pyperf re-execs the target interpreter as a child, which can survive the timeout and keep burning CPU into the next benchmark's measurement. Run each benchmark in its own session/process group instead, and on timeout SIGKILL the whole group via os.killpg. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: compute the head-vs-base cell from head_vs_base directly The cell was derived from base_vs_cpython and head_vs_cpython, both of which are None when a benchmark has no CPython time -- so it rendered "-" even though base and head both had valid times and head_vs_base was already computed. Use that precomputed ratio instead. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: invalidate the resumed catalog when the target executable changes catalog.json records results without any fingerprint of target_python, and resume skips benchmarks by name alone, so a local rebuild (or swapping the target) under the same --label could silently reuse stale measurements. Store a sha256 of the executable's contents alongside the catalog and, when it no longer matches, ignore the existing catalog and re-run everything. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * ci: fall back to github.sha for the head checkout on workflow_dispatch github.event.pull_request.head.sha is empty on a workflow_dispatch run (no pull_request payload), so `git checkout --quiet "" --` failed with "fatal: invalid reference" on this non-optional step, aborting every manual run before it could measure or render results. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: resolve --out/--cache-dir to absolute paths Every benchmark subprocess runs with cwd=work_dir (under --cache-dir), so a relative --out or --cache-dir passed on the command line resolved against that directory instead of the one run_all.py was invoked from, breaking catalog output, the pyperformance binary path, and PIP_FIND_LINKS. Both default to absolute paths already; resolve them up front so a relative override behaves the same way. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: persist catalog invalidation before the new fingerprint The fingerprint file was rewritten before the invalidated catalog was ever written back to disk. An interruption between the two (e.g. a crash during the first benchmark of the re-run) left a fingerprint matching the new executable pointing at a catalog.json still full of results measured against the old one, so the next run would trust it as fresh. Write the reset (empty) catalog first, then record the fingerprint. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: show benchmarks that fail on both base and head Rows were dropped whenever neither side had a mean, which also hid a benchmark that failed or timed out identically on base and head -- e.g. a C-extension gap unrelated to the PR under review. Only skip a benchmark never attempted on either commit; keep rows with a fail/timeout status so they still show up in the table. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: write catalog.json atomically write_catalog runs after every single benchmark, and Path.write_text truncates the file before writing the replacement. An interruption mid-write left catalog.json both missing every earlier result and invalid JSON, so the next run's resume logic failed outright at json.loads(). Write to a temporary sibling file and rename it into place instead. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: fix passed count and median population in the summary table "Benchmarks passed" counted len(base_ratios)/len(head_ratios), which require a *CPython* ratio too -- a CPython-only failure hid an otherwise-passing benchmark from both sides' counts. Count actual "ok" status per side instead. The two medians were also computed over independently-filtered populations: whichever benchmarks had a valid ratio on that side alone. A benchmark that regressed to failure on only one side dropped out of only that side's median, which could shift it well beyond what any surviving benchmark's timing actually changed by. Restrict both medians to benchmarks with a valid ratio on both sides, and label the population size. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: include RUSTPYTHONPATH's stdlib in the executable fingerprint The catalog-invalidation fingerprint hashed only the executable's bytes, but a RustPython binary copied away from its checkout finds its stdlib through RUSTPYTHONPATH -- pyperf re-execs the target interpreter into a venv and forwards that variable so it can still find Lib/. Changing that path, or editing the files under it, without rebuilding the binary left resume trusting results measured against a now-different stdlib. Fold its contents into the fingerprint too. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: require an explicit ok/fail status for pass/fail transitions only_head/only_base flagged a benchmark as newly passing or no longer passing based only on whether the other side had a mean, so a benchmark never attempted on one side (e.g. a partial --benchmarks run) was reported as a regression or fix it never was. Require the other side's status to be an actual fail/timeout, not just missing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: don't gate the head/base geometric mean on a CPython ratio The summary panel -- pass counts and the head/base geometric mean, neither of which need a CPython comparison -- only rendered when at least one benchmark also had a valid ratio against CPython on both sides. A build with a perfectly good head-vs-base comparison but no common CPython result lost the whole panel, including the one aggregate that didn't depend on CPython at all. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: normalize RUSTPYTHONPATH and extract main() into named steps Fix: a relative RUSTPYTHONPATH was fingerprinted relative to this script's invocation directory but resolved by pyperf's subprocess relative to work_dir (its cwd), so a relative value could be hashed against the wrong directory entirely, or fail to resolve at runtime. Resolve it to an absolute path up front, alongside --out and --cache-dir. Refactor: main() had accumulated argparse setup, executable resolution, and the fingerprint/resume block inline, each wrapped in an explanatory comment -- a sign the function was doing too much rather than that it needed more narration. Extracted parse_args(), resolve_target_python(), and load_resumable_catalog() so main() reads as a short sequence of named steps and each comment sits next to the one function it actually explains. No behavior change; verified with unit tests against the extracted functions (fingerprint reacting to RUSTPYTHONPATH, resume picking up a prior catalog, invalidation on an executable change). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: split pr_diff.py's main() and fix a status-transition gap Fix: only_head was still gated on have_base, which only asks whether *any* base benchmark produced a timing -- so a benchmark that flipped from failing to passing went unreported whenever every other base benchmark also happened to fail. Status transitions don't need that gate; they already require an explicit ok/fail/timeout status on both sides. Refactor: main() had grown into one long function mixing argument parsing, catalog loading, row computation, and three separate rendering concerns. Split it into parse_args(), load_base_catalog(), build_rows(), render_intro(), render_summary_panel(), render_table(), and find_transitions(), each documenting the one thing it decides. No behavior change beyond the fix above; re-verified against every scenario exercised in the last several fixes (dual-fail rows, geo mean without a CPython ratio, unattempted benchmarks, and the have_base gap) plus the existing manual smoke tests. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: don't resolve an empty RUSTPYTHONPATH to the cwd The normalization only checked "RUSTPYTHONPATH" in os.environ, so a set-but- empty value (RUSTPYTHONPATH="") resolved to the current directory via Path("").resolve(). executable_fingerprint() would then recursively hash every file under wherever run_all.py was invoked from -- including its own results directory while writing to it -- causing spurious catalog invalidation and a lot of pointless hashing. Check for a nonempty value instead, matching the truthiness check already used everywhere else this variable is read. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * docs(pyperformance): distinguish the host runner from the benchmarked target A reviewer read "pyperformance cannot be installed as-is on RustPython" and reasonably asked why RustPython would need to support pyperformance's CLI at all, since it's normally run as a command under any Python (see https://github.com/RustPython/RustPython/pull/8691#discussion_r3997957942). It doesn't: the pyperformance CLI (the host process) always runs under a real CPython regardless of --python. What actually needs psutil/pyperf importable is the *benchmark* subprocess -- pyperf re-execs the --python target and that subprocess itself does `import pyperf`. Spell out that host/worker split up front instead of leading with "cannot be installed". Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Jeong, YunWon
|
ba19382018 |
ci: hold a CodSpeed run to the machine main's baseline was measured on (#8644)
* ci: hold a CodSpeed run to the machine main's baseline was measured on CodSpeed compares a run against a base run and refuses to call the comparison meaningful when the two ran on different machines: the runner uploads a `SystemInfo` record beside the profile carrying the OS, the architecture, the CPU's vendor id, brand and flags, the physical core count and the installed memory, and a hosted pool hands out whatever machine is free. A simulated measurement is fixed against how fast the machine is, but not against what it makes the process execute -- glibc selects `memcpy` and the string routines through IFUNC on CPU flags, a new stable rustc emits different code, and these benchmarks run each program under CPython through pyo3 as well as under RustPython, so the runner's default interpreter is an input too. `scripts/codspeed-environment.py` records those fields plus the rustc, libc and CPython versions, and compares a run against a recording. The benchmark job now restores the recording main last wrote, records over it when the ref is main, and compares against it otherwise; the install, build and measure steps carry `if: steps.environment.outputs.match != 'false'`, so a run that drew a different machine finishes green having uploaded nothing. A missing or unfetchable recording is read as "nothing to disagree with" rather than as a mismatch. Assisted-by: Claude * ci: read an unreadable environment recording as not a match A cache miss, an eviction or a cache-service outage leaves the reference file absent, and the comparison then reported `match=true`: every guarded step ran and the run uploaded a measurement without having identified the machine it was taken on. An absent recording now reports `match=false`, and says so as a `::warning` rather than the `::notice` a mismatch emits, so a job that measures nothing because it has nothing to compare against is distinguishable from one that measures nothing because the machine differs. Assisted-by: Claude |
||
|
Jeong, YunWon
|
322f294a21 |
common: add VM-independent codec and text engines (#8622)
* common: add PyPy CJK codec engines
Assisted-by: Codex:GPT-5
* common: mark CJK mapping tables generated
Assisted-by: Codex:GPT-5
* common: add reusable byte and text engines
Assisted-by: Codex:GPT-5
* common: record opt-in engine dependencies
Assisted-by: Codex:GPT-5
* stdlib: add `_multibytecodec` and the six `_codecs_*` modules
Wire the CJK codec engines in `rustpython-common` up to the VM. `_multibytecodec`
holds `MultibyteCodec`, `MultibyteIncrementalEncoder`, `MultibyteIncrementalDecoder`,
`MultibyteStreamReader` and `MultibyteStreamWriter`, the encode/decode drivers and
the error-handler protocol; each `_codecs_*` module exposes `getcodec` for the
encodings of its region.
`rustpython-stdlib` now enables the `cjk-codecs` feature of `rustpython-common`.
Add `cjk::decode_reset`, which the decoder `reset()` methods need. It differs from
building a fresh decoder state: ISO 2022 keeps its G1..G3 designations and the
escape-throughout flag, and resets only G0 and the shifted flag.
Drop the markers from the CJK codec tests and from the `test_codecs`, `test_io`,
`test_email`, `test_zipfile` and `test_traceback` cases that needed a CJK codec.
`test_codecs.BasicUnicodeTest.{test_basics,test_decoder_state}` stay expected
failures for an unrelated reason: `codecs.charmap_encode` rejects a `None` mapping.
Assisted-by: Claude Code:claude-opus-5
* stdlib: use the byte and text engines from `rustpython-common`
`binascii`, `_json` and `_socket`'s address converters now call the engines in
`rustpython-common` instead of carrying their own copies, and `rustpython-stdlib`
enables the `binascii`, `inet` and `json` features for them. That leaves
`base64` and `crc32fast` with no caller in `rustpython-stdlib`, and
`crates/stdlib/src/json/machinery.rs`, the last of the json_in_type-derived code,
goes away with its last caller.
The address converters change behaviour. `inet_aton` reads the lenient forms
(`127.1`, `16909060`, `0x7f.1`, `0177.0.0.1`), `inet_pton` rejects an octet with a
redundant leading zero and a group of five hex digits, `inet_ntop` writes the
IPv4-compatible form as `::c000:201`, and an address family with no converter
raises `EAFNOSUPPORT` instead of a bare `OSError`.
Give the JSON engine the closing-quote scan and the allocation-free `\uXXXX`
escape it needs to carry `_json`'s workload. Against the code it replaces,
`json.dumps` runs 2.3x faster with `ensure_ascii=True` and 1.7x without it, and
`json.loads` 1.2x.
`binascii.a2b_base64` now reports the messages `binascii.c` does: no
`error decoding base64: ` prefix, and the data-character count in parentheses.
Assisted-by: Claude Code:claude-opus-5
* test: enable the Windows cp932 codec case
The shared CJK engine now satisfies the Windows code-page adapter, so keeping this test marked as an expected failure turns its success into a CI failure.
Assisted-by: Codex:GPT-5
|
||
|
Kevin Turcios
|
5c07fa4bd0 |
Optimize list removal lock usage (#8451)
Assisted-by: WarpForgeAgent:DeepSeek-v4-flash |
||
|
Shahar Naveh
|
9c027d4f67 |
Update test/signalinterproctester.py to 3.14.6 (#8173)
* Update `test/signalinterproctester.py` to 3.14.6 * Add `signalinterproctester.py` as a test dependency of `test_signal.py` |
||
|
Shahar Naveh
|
38d17bc12c |
scripts/update_lib migrate to preserve patches on classes (#8057)
|
||
|
Shahar Naveh
|
26dfeae512 | Autogen opcodes metadata (#7983) | ||
|
Jeong, YunWon
|
6440e64355 | Align codegen metadata with CPython (#7952) | ||
|
Jeong, YunWon
|
1d4a60a503 |
Align nested code object bytecode parity (#7942)
* Align nested code object bytecode parity * Align CPython jump-back parity * Align match guard block parity with CPython * Align CPython block borrow parity * Align CPython load-fast barrier parity * Align CPython try-end label parity * Align CPython try-except end label barriers * Align more CPython bytecode edge cases * Align bare except finally bytecode layout |
||
|
Shahar Naveh
|
4812ea819e |
Update test_xpickle.py to 3.14.5 (#7901)
|
||
|
Shahar Naveh
|
9c5a674fee |
Update pickle.py to 3.14.5 (#7895)
* Update `pickle.py` to 3.14.5 * Update pickle deps with new file * Mark failing tests |
||
|
Shahar Naveh
|
542ddd40d9 | Autogen instructions & opcodes (#7797) | ||
|
Shahar Naveh
|
a35079967a |
Fix comment confusion bug in update_lib (#7802)
* Use `unittest.expectedSuccess` to mark failing tests * Fix comment find logic * Add test for it |
||
|
Jeong, YunWon
|
12c58d242c |
Bytecode parity: align borrow optimization with CFG traversal (#7773)
* Align CFG cleanup bytecode with CPython * Bytecode parity: fblock unwind, fstring join, folding, scope - compile.rs: unwind_fblock_stack returns whether a finally ran so return-statement emission can adjust location handling; restructure try/except/finally cleanup to preserve or drop boundary NOPs based on whether the body falls through; rework f-string lowering with count/join helpers; remove the per-collection-type heuristic for AST-level folding and defer to flowgraph passes; add several folding helpers and a ComprehensionLoopControl enum. - ir.rs: re-run unary/binop folding around tuple folding, add reorder_conditional_scope_exit_and_jump_back_blocks and several block classification helpers, add MAX_STR_SIZE, change is_exit_without_lineno to take the block list. - symboltable.rs: in analyze_cells, remove names owned as cells in function-like scopes from the parent's free set; mark lambda scope type explicitly. * Refine CFG scope-exit backedge ordering |
||
|
Shahar Naveh
|
fb6142621f |
Fix libs deps check (#7732)
* Convert parsing logic to python
* Trigger job. (DROP ME BEFORE MERGE!)
* Missing EOF
* Add found modules to set
* Suggestion by @fanninpm
* Fix missing trailing slash
* Ensure no `.py` at module name
* Strip any file auffix
* Revert "Trigger job. (DROP ME BEFORE MERGE!)"
This reverts commit
|
||
|
Shahar Naveh
|
b1df45bd09 |
Update test_unicodedata.py to 3.14.4 (#7750)
* Update `test_unicodedata.py` to 3.14.4 * Add test asset * Mark failing tests * Add `expectedSuccess` docorator * Add support for nee decorator in patch_spec * Reapply patches |
||
|
Jeong, YunWon
|
9ee32a1b38 |
Bytecode parity - boolop, comprehension, CFG passes (#7631)
* Bytecode parity - boolop, comprehension, CFG passes - Flatten nested same-op BoolOp and add IfExpr to jump_if - Simplify is_name_imported to module-level only - Enable inlined comprehensions in module/class scope - Add TweakInlinedComprehensionScopes with fast_hidden tracking - Track fb_range in FBlockInfo for with-statement line info - Add constant subscript folding and unary Not folding - Add emit_return_const_no_location for implicit returns - Use JumpNoInterrupt for ternary/except jumps - Add STACK_USE_GUIDELINE threshold for collection building - Reorder CFG pipeline: inline small blocks earlier, resolve line numbers before cold block extraction - Add redirect_empty_unconditional_jump_targets, reorder_conditional_chain_and_jump_back_blocks, materialize_empty_conditional_exit_targets, duplicate_shared_jump_back_targets passes - Add borrow deoptimization for multi-handler, named-except, protected conditional tail, and protected import joins - Run folding/optimization passes twice around peephole * Bytecode parity - slice augassign, async comp inline - Augmented assignment with two-part slices uses BINARY_SLICE/STORE_SLICE - Inline async comprehensions (remove await/async guards) - Inlined comprehension cleanup jump uses JumpNoInterrupt - Class firstlineno uses first decorator line when decorated - Recurse into nested functions for static attribute collection - Fold unary positive complex constants (+0.0j) - Add deoptimize_borrow_for_folded_nonliteral_exprs pass - Add inline_single_predecessor_artificial_expr_exit_blocks pass - Skip shared artificial expr-stmt exit blocks in small-block inlining - Mark folded boolop tail as folded_from_nonliteral_expr * apply reviews |
||
|
Jeong, YunWon
|
a04189fa34 |
Align bytecode codegen structure with CPython 3.14 (#7588)
* Align bytecode codegen structure with CPython 3.14 * Bytecode parity - constant folding, annotation ordering, superinstruction alignment - Add BoolOp constant folding with short-circuit semantics in compile_expression - Add constant truthiness evaluation for assert statement optimization - Disable const collection/boolop folding in starred unpack and assignment contexts - Move annotation block generation after body with AnnotationsPlaceholder splicing - Reorder insert_superinstructions to run before push_cold_blocks (matching flowgraph.c) - Lower LOAD_CLOSURE after superinstructions to avoid false LOAD_FAST_LOAD_FAST - Add ToBool before PopJumpIf in comparisons and chained compare cleanup blocks - Unify annotation dict building to always use incremental BuildMap + StoreSubscr - Add TrueDivide constant folding for integer operands - Fold constant sets to Frozenset (not Tuple) in try_fold_constant_collection - Add PyVmBag for frozenset constant materialization in code objects - Add remove_redundant_const_pop_top_pairs pass and peephole const+branch folding - Emit Nop for skipped constant expressions and constant-true asserts - Preserve comprehension local ordering by source-order bound name collection - Simplify annotation scanning in symboltable (remove simple-name gate) * Fix CI regressions in marshal and fast-local ops * impl more * Align bytecode codegen with CPython structure * Bytecode parity - comprehension/except scope ordering, load_fast_borrow fixes - Reorder comprehension symbol-table walk so the outermost iterator registers its sub_tables in the enclosing scope before the comp scope, and rescan elt/ifs in CPython's order. Codegen peeks past the outermost iterator's nested scopes to find the comprehension table. - For plain try/except, emit handler sub_tables before the else block so codegen's linear sub_table cursor stays aligned. - Rename `collect_simple_annotations` to `collect_annotations` and evaluate non-simple annotations during __annotate__ compilation to preserve source-order side effects while keeping the simple-name index stable. - Dedupe equivalent code constants in `arg_constant` and add a structural equality check on `CodeObject`. - Disable LOAD_FAST_BORROW for the tail end block when a try has a bare `except:` clause, and have `new_block` inherit the flag from the current block. - Remove `cfg!(debug_assertions)` guard around the `optimize_load_fast_borrow` start-depth check so mismatches are handled (return instead of assert) in release builds. - Collapse nop-only blocks that precede a return epilogue and hoist the prior line number into the next real instruction so the line table matches. - Unmark now-passing `test_consts_in_conditionals`, `test_load_fast_unknown_simple`, `test_load_fast_known_because_already_loaded`, and PEP 646 f3/f4 annotation checks. * Bytecode parity - try/except line tracking, assert 0 shape - In `compile_try_except`, drop the leading Nop and set the end block's source range from the last orelse/body statement so line events after the try fall on the right line. - Recognise constant-false asserts as the direct-raise shape (no ToBool/PopJumpIfFalse) and flip the test assertion accordingly. - Extend `remove_redundant_nops_in_blocks` to also look through a trailing nop before a return-epilogue pair (LoadConst/ReturnValue or LoadSmallInt/ReturnValue) so the epilogue keeps the correct line number. - Rename `preds` to `predecessor_blocks` in the LOAD_FAST_BORROW disable pass and add a test-only `debug_late_cfg_trace` helper. - Regenerate the `nested_double_async_with` snapshot: the tail reference to `stop_exc` now emits LOAD_FAST instead of LOAD_FAST_BORROW. * Bytecode parity - iter folding, break/continue line, cold inlining - Fold a constant list iterable into a constant tuple in for-loop iterable position, matching the CPython optimizer, and strip a redundant LIST_TO_TUPLE immediately before GET_ITER in the IR peephole pass. - Emit a Nop at the break/continue source range before unwinding so line events land on the break/continue statement instead of the following instruction. - Drop `propagate_disable_load_fast_borrow`; the forward propagation was over-zealous and the per-block inheritance in `new_block` plus the bare-except marker are enough. - Relax `inline_small_or_no_lineno_blocks` so small exit blocks at the tail of a cold block are always inlined, not just return epilogues. - Add codegen tests covering the LIST_TO_TUPLE/GET_ITER peephole and the late-CFG trace helper for a for-loop list-literal iterable. |
||
|
Shahar Naveh
|
8d1000d485 |
Simplify Intruction.deopt() (#7615)
* Simplify `Instruction.deopt()` * Adjust `scripts/generate_opcode_metadata.py` |
||
|
Shahar Naveh
|
63fa928074 |
Macro for defining opcode & instruction enums (#7573)
* Macro for defining opcode & instruction enums * Convert `Instruction` as well * revert `#[repr(...)]` changes |
||
|
Jeong, YunWon
|
7abf8cc247 |
Bytecode parity - for-loop cleanup, return duplication, late jump threading fix (#7580)
- Use POP_TOP instead of POP_ITER for for-loop break/return cleanup - Expand duplicate_end_returns to clone final return for jump predecessors - Restrict late jump threading pass to unconditional jumps only - Skip exception blocks in inline/reorder passes - Simplify threaded_jump_instr NoInterrupt handling |
||
|
Jeong, YunWon
|
4e2ce58693 | Bytecode parity - direct loop backedges (#7578) | ||
|
Jeong, YunWon
|
0b7e0f007b |
Bytecode parity - exception (#7557)
* Fix exception handling: except* chaining, finally cleanup, RERAISE - Align except* bytecode chaining - Fix exception state model and finally handler cleanup - Fix RERAISE to only pop exception, preserve values below * Port IR optimization passes from flowgraph.c - BUILD_TUPLE n + UNPACK_SEQUENCE n elimination - Dead store elimination within basic blocks - apply_static_swaps for SWAP reduction * Add bytecode comparison and disassembly dump scripts - compare_bytecode.py: compare CPython vs RustPython bytecode output - dis_dump.py: extract disassembly in normalized JSON format |
||
|
Shahar Naveh
|
b1eda639f0 |
Unify lint CI job (#7505)
* use `prek` for unified linting * Fix actionlint error * Generate metadata when specific files change * `check_redundant_patches.py` to accept glob path * Test * revert defective changes * use `rustfmt` over `cargo fmt` for individual files * debug reviewdog * rustfmt * Move comment to correct location * defevtive fmt test * Fail with reviewdog * fix reviewdog perms * Try to use present token * without checks oerms * put normal perms * fmt |
||
|
Shahar Naveh
|
3069d06851 |
Align _opcode_metadata.py to 3.14.3 (#7456)
* Align `_opcode_metadata.py` to 3.14.3 * Unmark passing test * Ensure python 3.14 runs on CI * Update banner * Fix `test__opcode.py` * Adjust generate script * Fix docs |
||
|
Shahar Naveh
|
973f2f0039 |
Restructure CI with matrix approach and multi-feature support (#7376)
* `--features` flag can take multiple values * Simplify CI job * Remove bad default * Enable jit on macOS |
||
|
Shahar Naveh
|
9899deba82 |
Update libc to 0.2.182 (#7247)
* Update to libc 0.2.182, refactor update script * Fix typing |
||
|
Jeong, YunWonandgithub-actions[bot]
|
1471e7559d |
[update_lib] fast date lookup for todo (#7299)
* [update_lib] fast date lookup for todo * add deps * Auto-format: ruff format --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
||
|
Shahar Naveh
|
cf66ed14b5 | Don't return frozenset for modules (#7276) | ||
|
Shahar Naveh
|
9b29cb426a | More accurate test dependencies (#7261) | ||
|
Shahar Naveh
|
5a58e2d038 |
Reapply "ast.NodeVisitor for import tracking" (#7241)
* Reapply "`ast.NodeVisitor` for import tracking (#7229)" (#7230)
This reverts commit
|