17938 Commits
Author SHA1 Message Date
youknowdot 6e064e5842 Support import timing and cached import diagnostics (#9190)
* Implement import timing and cached import diagnostics

Support -X importtime and PYTHONPROFILEIMPORTTIME, including mode 2 cached imports. Track nested timing separately in each VM thread and share the output header per interpreter.

Assisted-by: OpenAI Codex:model version unavailable

* Enable canonical import timing coverage and align diagnostics

Retire only test_import_time expectedFailure after the independent runtime passes its canonical assertions. Preserve other cmd-line markers and align environment range errors with CPython.

Assisted-by: OpenAI Codex:model version unavailable

* Match encoded-byte padding for cached imports

Keep CPython cached-import minimum width semantics for UTF-8 and backslash-replaced surrogate names. Scope the diagnostic blank line to importtime without changing CPU-count errors.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-18 22:38:20 +09:00
youknowdot 7137540b93 Align regex, dataclasses and typing with Python 3.15 (#9189)
Expose Pattern.prefixmatch and Scanner.prefixmatch and refresh their canonical re consumers, dataclasses annotation handling, typing APIs and regression fixtures from CPython d220409a13b77de7a6c30abe16ea90db01528edc.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-18 22:37:35 +09:00
1ndahous3 7750dbcb86 Accept keyword arguments in SemLock constructors (#9188)
Assisted-by: Codex:GPT-6
2026-10-18 22:37:01 +09:00
1ndahous3 e792ec8fb3 Accept bytes-like input in pyexpat.Parse (#9187)
Assisted-by: Codex:GPT-6
2026-10-18 22:36:39 +09:00
1ndahous3 94ec89a687 Honor stored ctypes byref addresses (#9184)
Assisted-by: Codex:GPT-6
2026-10-18 22:34:59 +09:00
youknowdot 18c1358a61 Avoid duplicate AST initialization and eager import locals allocation (#9183)
Let type.__call__ own AST initialization, matching CPython generic allocation, and inspect existing frame locals without materializing them for lazy policy checks.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-18 22:34:31 +09:00
youknowdotandJeong YunWon 3f55cf4af0 Implement basic REPL execution and fix piped stdin handling (#9163)
* Preserve original main VM through C API execution

Run and finalize the original main VM while keeping the registered attachment template available without holding its mutex across Python callbacks.

Assisted-by: OpenAI Codex:model version unavailable

* Make unraisable reporting robust at hook boundaries

Keep the existing message API and traceback formatting while handling missing, disabled, audited and failing hooks without panicking.

Assisted-by: OpenAI Codex:model version unavailable

* Separate interactive warning collection from compilation

Retain ordinary warning behavior and typed module APIs while exposing record-local parser warnings and post-warning code generation. Add a source-only lazy future-import guard.

Assisted-by: OpenAI Codex:model version unavailable

* Evaluate native stdin with shared VM record semantics

Use one evaluator for basic REPL and root transports, preserve effective inspect snapshots and future flags, and carry owned host exit status through original-main finalization.

Assisted-by: OpenAI Codex:model version unavailable

* Refresh canonical REPL tests and retire fixed failures

Run update_lib migrate from pinned CPython 3.15.0, preserving three remaining platform/runtime markers and retiring the two traceback/startup failures verified on the native evaluator.

Assisted-by: OpenAI Codex:model version unavailable

* Keep the root compiler dependency for development targets

The native evaluator now compiles through the VM. Only root benchmarks and the parse_folder example use rustpython-compiler directly, so declare it as a dev dependency while preserving VM compiler features.

Assisted-by: OpenAI Codex:model version unavailable

* Validate explicit conditional failure marker retirement

Require exact selector and original decorator evidence for each conditional expected-failure marker retired by migrate. Preserve undeclared guards and reject missing, duplicate, modified or unconsumed declarations.

Assisted-by: OpenAI Codex:model version unavailable

* Refresh canonical stdin and exception audit tests

Migrate the pinned CPython 3.15.0 modules and their canonical helper fixtures through update_lib. Retire exactly four failure markers proven obsolete by hosted CI, preserving all other existing guards.

Assisted-by: OpenAI Codex:model version unavailable

* Synchronize runpy warning attribution and document remaining capabilities

Migrate whole canonical runpy.py to pass module names to compile. Use update_lib to annotate only reproduced missing-capability cases: absent _testcapi import-event tests, unsupported startup/import-time features, and the deferred frozen-loader module-attribution caller.

Assisted-by: OpenAI Codex:model version unavailable

* Propagate interrupted native stdin reads to signal dispatch

Preserve the shared input buffer and partial bytes while returning EINTR to the VM before retrying. Exercise the transport deterministically and verify a returning SIGINT handler while pipe stdin stays open.

Assisted-by: OpenAI Codex:model version unavailable

* Reject statements after a compound statement in single mode

single_mode_multiple_statements_error now reports "invalid syntax" at the
statement that follows a compound first statement, and interactive_input
runs it right after parsing as compile() does.

The lazy `from __future__` import check moved from lazy_future_import_error
into unsupported_grammar_error, so it applies to every compile mode.

Assisted-by: Claude Code:claude-opus-5-5

* Remove passing expected failure from test_runpy

test_run_module_filter_syntax_warnings_by_module passes.

Assisted-by: Claude Code:claude-opus-5-5

* Restructure stdin execution after Modules/main.c and pythonrun.c

- src/lib.rs follows pymain_run_python: pymain_run_stdin, pymain_repl,
  pymain_run_startup, pymain_run_interactive_hook and pymain_run_pyrepl
  return exit codes instead of raising SystemExit.
- VirtualMachine gains run_any_stdin (_PyRun_AnyFile),
  run_interactive_loop (_PyRun_InteractiveLoop) and run_simple_stdin;
  run_shell feeds rustyline lines into run_interactive_loop, and
  sys._baserepl calls run_any_stdin.
- print_uncaught_exception (PyErr_Print) replaces
  report_uncaught_exception, and handle_system_exit_and_keyboard_interrupt
  replaces handle_reported_exception.
- An unhandled KeyboardInterrupt is recorded and the process exits
  through SIGINT after finalization.
- Interpreter::run_main replaces run_with_exit_status and InterpreterExit;
  rustpython_capi::run_main_interpreter takes an exit-code callback.
- config_set_inspect audits cpython.PyConfig_Set and rebuilds sys.flags
  through set_flag_obj (_PySys_SetFlagObj); the inspect state is an
  AtomicBool and dict replace_by_identity is removed.
- add_main_module (PyImport_AddModuleRef("__main__")) is used by the
  interactive and simple-file paths.
- The interactive parser warning helpers take their state directly.

Assisted-by: Claude Code:claude-opus-5-5

* Exit through Py_Exit from PyErr_Print and run the basic REPL via stdin

- print_uncaught_exception (PyErr_Print) calls exit_process (Py_Exit) for a
  SystemExit when inspect mode is off, instead of returning the exception.
  run_interactive_loop no longer checks SystemExit itself and flushes after
  printing an error; sys._baserepl returns None.
- exit_process finalizes through VirtualMachine::finalize_ex, which holds
  the Py_FinalizeEx steps split out of Interpreter::finalize, and exits
  through host_env::os::exit_with_cleanup (C exit()).
- The basic REPL on a terminal (PYTHON_BASIC_REPL, or _pyrepl missing)
  runs run_any_stdin instead of the rustyline shell.
- run_shell saves the history after every line.

Assisted-by: Claude Code:claude-opus-5-5

* Display exceptions through PyErr_Display and track the main thread running __main__

- display_exception (PyErr_Display) and display_exception_to (_PyErr_Display)
  try traceback._print_exception_bltin before the native printer and flush
  the file. PyErr_Print's excepthook fallback, print_exception and
  sys.excepthook use them. A missing sys.stderr is reported with
  dump_object (PyObject_Dump); a None sys.stderr prints nothing.
- Paths gains program_name (argv[0]) from getpath. The "can't open file"
  message uses it with strerror(errno) and goes to sys.stderr.
- running_main holds the ident of the thread running __main__.
  pymain_run_python sets it around running the main code, Py_Exit clears it,
  the fork child reinitializes it, and _interpreters.is_running reads it
  instead of treating the main interpreter as always running.
- Interpreter::run_main clears the unhandled KeyboardInterrupt flag before
  running and applies exit_sigint after finalizing; finalize no longer
  does.
- run_interactive_loop returns a SystemExit raised outside inspect mode
  again and no longer flushes after printing an error; run_shell saves the
  history once after the loop.
- pymain_set_inspect reports its error through print_uncaught_exception.

Assisted-by: Claude Code:claude-opus-5-5

* Treat a parse error at the end of the source as incomplete input

A parser error that starts at the end of the source is reported as
_IncompleteInputError when incomplete input is allowed, so the
interactive loop and codeop keep reading after `try:` and its block or
after a decorator line.

Assisted-by: Claude Code:claude-opus-5-5

* Place syntax errors on tokens made up at the end of the input

A parse error on the NEWLINE, DEDENT or ENDMARKER the tokenizer makes up
at the end of the input now carries an EofToken: the number of lines
read and the last line. SyntaxError takes its lineno and text from it,
and its offset is that line's length with end_offset -1, or 0 for both
when the error is raised at the token itself (the generic syntax error
and a missing forced `:` or `(`). Exec input that does not end with a
newline is checked with one appended.

Assisted-by: Claude Code:claude-opus-5-5

* Slice the source with get when checking an indented block error at EOF

Assisted-by: Claude Code:claude-opus-5-5

* Rename the host exits after C exit() and fix wasi and Windows builds

- host_env::os::exit now calls the C exit() on every platform; on Windows
  std::process::exit ends in ExitProcess, so it flushes the Rust stdout
  and calls libc::exit there. The former exit, which calls _exit(), is
  now exit_immediately, used by os._exit, faulthandler and
  posixsubprocess.
- errno::strerror_string is available on wasi.
- PyGlobalState::reinit_running_main is built only where its fork caller
  is.

Assisted-by: Claude Code:claude-opus-5-5

---------

Co-authored-by: Jeong YunWon <jeong@youknowone.org>
2026-10-18 09:22:12 +00:00
youknowdot 9f6f771965 Align lazy imports and f/t-string interpolation with Python 3.15 (#9179)
* Preserve interpolation source text and canonical metadata

Source-only checkpoint; candidate validation and the existing annotation expectation reconciliation remain pending.

Assisted-by: OpenAI Codex:model version unavailable

* Align existing annotation expectations with CPython metadata

Update only the nine existing expected strings confirmed against pinned CPython 3.15, with explicit user approval.

Assisted-by: OpenAI Codex:model version unavailable

* Preserve template metadata through compilation and interactive input

Escape literal debug prefixes, retain None AST expression metadata internally, and continue incomplete triple-quoted t-strings in the existing shell.

Assisted-by: OpenAI Codex:model version unavailable

* Classify normalized unclosed brackets as REPL line continuations

Assisted-by: OpenAI Codex:model version unavailable

* Nest equivalent REPL parse-error alternatives for Clippy

Assisted-by: OpenAI Codex:model version unavailable

* Remove fulfilled annotation migration lint workaround

The canonical expected strings no longer trigger literal_string_with_formatting_args, so the previous lint expectation is obsolete.

Assisted-by: OpenAI Codex:model version unavailable

* Complete lazy-import exception and traceback library closure

Copy canonical traceback and legacy exception mappings with preserved RustPython guards. Copy explicit non-Python library assets by exact path so exception hierarchy data does not expand into the entire test package.

Assisted-by: OpenAI Codex:model version unavailable

* Provide Python 3.15 traceback metadata for missing modules and attributes

Preserve AttributeError context for missing class attribute deletion, matching type_update_dict. Expose canonical generated missing-stdlib messages through the native sysconfig modules without importing CPython build-specific disabled-module assumptions.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-18 17:11:26 +09:00
youknowdot 6e8e7e0481 Align startup paths and sysconfig/venv with Python 3.15 (#9174)
* Align startup home and venv paths with canonical sysconfig closure

Assisted-by: OpenAI Codex:model version unavailable

* Preserve selected stdlib and empty home prefix fallback

Assisted-by: OpenAI Codex:model version unavailable

* Collapse startup exec-prefix fallback condition

Assisted-by: OpenAI Codex:model version unavailable

* Preserve startup search order and freeze the ZIP bootstrap

Align zipimport and its tests with CPython 3.15.0 via update_lib, preserving inherited test guards. Keep build-time fallback paths after configured paths and the default ZIP, while respecting customized embedding paths.

Assisted-by: OpenAI Codex:model version unavailable

* Retire proven trace coverage guards through update_lib

Both canonical coverage-file tests execute successfully after preserving startup search order. Preserve the other two inherited guards and restore canonical Python 3.15 JIT optimizer decorators.

Assisted-by: OpenAI Codex:model version unavailable

* Align Windows symlink helper with canonical Python home layout

Import the pinned CPython 3.15 PythonSymlink class through a narrow, hash-pinned update_lib mode. Preserve canonical home/VPATH behavior and explicitly supply RustPython build stdlib to the Windows child without changing runtime PYTHONHOME semantics or skipping tests.

Assisted-by: OpenAI Codex:model version unavailable

* Reset surviving thread completion locks after fork

Reinitialize every live thread handle completion mutex and condition variable before skipping the surviving or not-yet-started handle. Preserve their completion payload and state, and retain the existing dead-thread cleanup.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-18 15:55:22 +09:00
1ndahous3 924e34200f Honor setter-only descriptors in STORE_ATTR specialization (#9181)
Assisted-by: Codex:gpt-6
2026-10-18 11:02:46 +09:00
Luan Taraschi 9fd2ba8367 Compare complex with int exactly instead of through a float (#9180)
complex.__eq__ and __ne__ converted an int operand to f64 before
comparing, so a complex whose real part is the float nearest to a large
int compared equal to that int even when the values differ:
complex(2**53 + 1) == 2**53 + 1 returned True, while
float(2**53 + 1) == 2**53 + 1 returns False and the two objects hash
differently.

When the imaginary part is zero, compare the real part against the
exact integer with float_ops::eq_int, the helper float's own comparison
uses. This matches CPython, which hands an int operand to float's rich
comparison. test_complex.test_richcompare_boundaries now passes, so drop
its expectedFailure marker.

Assisted-by: Claude Code:claude-opus-5-5
2026-10-18 11:01:54 +09:00
Joshua Megnauth cf6ca56789 Fix Unicode 3.2.0 aliases, bump unicode_names2 (#9171)
Replaces: #9166

unicode_names2 fixed a bug with its aliases. Ironically, the bug
caused a test in test_snippets.py to pass. Bumping unicode_names2 to
4.0.0 fixes the bug and thus the extra 3.2.0 test fails. I fixed the
logic for name lookup for 3.2.0. Now the test correctly passes.
2026-10-18 11:01:02 +09:00
1ndahous3 e64023020c Fix CSV field limit validation and concurrent updates (#9160)
Assisted-by: Codex:gpt-6
2026-10-18 10:59:51 +09:00
1ndahous3 99bf003f92 Avoid codec registry deadlocks during object cleanup (#9159)
Assisted-by: Codex:gpt-6
2026-10-18 10:59:15 +09:00
1ndahous3 cf3bcee3fd Release SQLite callbacks and expanded trace SQL safely (#9151)
* Release SQLite callbacks and expanded trace SQL safely

Assisted-by: Codex:gpt-6

* Drain SQLite callback cleanup before reinitialization and after blob operations

Assisted-by: Codex:gpt-6

* Drain SQLite cleanup after statement reset

Assisted-by: Codex:gpt-6

* Fix concurrent SQLite connection reinitialization

Assisted-by: Codex:gpt-6

* Restore SQLite factory defaults after callback cleanup

Assisted-by: Codex:gpt-6
2026-10-18 10:58:39 +09:00
youknowdot efa8c26929 Align code introspection and monitoring with Python 3.15 (#9178)
* Align code metadata and monitoring event semantics with Python 3.15

Preserve canonical qualified-name interning and subclass values, normalize valid debug-range tables, and correct event-specific monitoring disable and callback ownership boundaries.

Assisted-by: OpenAI Codex:model version unavailable

* Import canonical monitoring shutdown helper

Copy the missing test_monitoring dependency from pinned CPython 3.15.0 through scripts/update_lib, retaining existing suite guards.

Assisted-by: OpenAI Codex:model version unavailable

* Match observed monitoring source layers and tool release ordering

Model active source instrumentation across all subscribed tools instead of unconditional legacy edge pairing. Preserve tool names during unlocked callback finalization before releasing them.

Assisted-by: OpenAI Codex:model version unavailable

* Preserve code qualnames and monitoring event-site identity

Retain decoded WTF8 qualnames, use exact-str marshal dispatch, and model wrapped branch dispatch with physical DISABLE sites. Retire the proven monitoring shutdown guard through update_lib.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-18 10:57:23 +09:00
youknowdot 65bbacae47 Keep optional OpenSSL CI on Linux (#9177)
Remove the macOS OpenSSL build and runtime expansion from #9173 while preserving the existing Linux checks and other macOS jobs.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-18 00:42:46 +09:00
youknowdot 7d15b50cd4 Align weak-reference collection with Python 3.15 finalization (#9168)
* Align weak-reference collection with Python 3.15 finalization

Trace and retain owned callback edges independently of weak referent links.
Clear callback-bearing weakrefs before finalizers, suppress callbacks from
unreachable weakrefs, and close remaining weakref ingress after resurrection
checks while the world is stopped. Publish and unlink intrusive nodes safely.

Refresh weakref and canonical weakref/GC/finalization tests through update_lib
from pinned CPython 3.15.0. Retire four obsolete guards only after fresh body
passes, retaining applicable RustPython limitations.

Assisted-by: OpenAI Codex:model version unavailable

* Refresh canonical IO tests for weakref finalization

Migrate the complete CPython 3.15.0 test_io package at d220409 with the
fail-closed legacy-map updater, preserving all ten existing guards. Retire
the seven consolidated standalone suites and update io/file dependency
paths with mapping regression tests.

The canonical GC tests ignore ResourceWarning without retaining its
source, so callback-free weakrefs correctly reflect surviving owners.
This closes the stale test dependency exposed by the lifecycle change.

Assisted-by: OpenAI Codex:model version unavailable

* Protect weakref callbacks with the original referent stripe

Follow CPython's free-threading weakref locking model instead of keeping
an independent callback RwLock. Retain the original referent stripe key
so callback access stays synchronized after GC clears the referent.
Unlink and steal callbacks under that stripe, then release ownership
outside the lock. Preserve synchronization for running-world GC paths
and gc.get_referents while keeping ordinary weakref upgrades unchanged.

Assisted-by: OpenAI Codex:model version unavailable

* Defer weakref GC tracking until referent publication

Keep freshly allocated weakrefs out of GC's object lists until they are
linked and their referent is published. Release the referent stripe before
tracking, since tracking can collect and invoke finalizers. Generic reuse
losers remain untracked.

Exercise the allocator boundary directly for reference, proxy, callable
proxy and subclass payloads, and ensure completed constructors track their
objects. This prevents observers from retaining a half-built dead weakref
that later becomes live.

Assisted-by: OpenAI Codex:model version unavailable

* Retain the original weakref stripe lock like CPython

Store a safe handle to the selected static weakref stripe instead of
retaining and rehashing the referent address. Keep the same lock after
referent clearing, matching CPython free-threaded weakrefs_lock. The
nonthreaded handle is zero-sized and introduces no locking.

Preserve lock selection, acquire/release ordering, fork reset, deferred
GC publication, and callback-owner release outside the stripe. No new
locks or unsafe code are needed.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-17 23:53:07 +09:00
codspeed[bot]andCodSpeed Bot 008b8e1c89 Skip unit-test harnesses when building benchmarks (#9158)
cargo bench and cargo codspeed build select every target with bench = true.
That includes the library and binary targets, so the build also compiled the
library unit-test harness as an extra LTO-linked copy of the interpreter that
no benchmark runs. Mark them bench = false so only the criterion benchmarks
(and the binary they depend on) are built.

Co-authored-by: CodSpeed Bot <no-reply@codspeed.io>
2026-10-17 23:37:20 +09:00
youknowdot 51e886c224 Fix optional OpenSSL duplex I/O on Unix (#9173)
* Fix optional OpenSSL duplex I/O on Unix

Release connection locks before readiness waits while serializing native SSL
attempts and preserving owned write retries. Wake waiters on native progress
and socket identity invalidation, and construct callback/native exceptions
after unlocking.

Add regression coverage to the existing SSL snippet and run the optional
OpenSSL backend in the existing Unix CI matrix. Windows and AIX retain the
legacy transport pending a separate compatibility-safe driver.

Assisted-by: OpenAI Codex:model version unavailable

* Consolidate OpenSSL runtime checks into the existing CI step

Keep the minimal system and vendored OpenSSL build checks, and run the
Unix SSL regressions in the existing system OpenSSL step. Preserve the
runtime feature set and enable its macOS dependency in that job.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-17 23:36:27 +09:00
youknowdot 75a2417043 Align type layouts and generic descriptors with Python 3.15 (#9175)
Import canonical descriptor tests through update_lib, preserving existing guards. Support tuple member slots, distinguish base reassignment from class layout compatibility, and share object-owned generic descriptors.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-17 14:13:50 +00:00
youknowdot fc9f4a9541 Update importlib to CPython 3.15 with native bootstrap callback (#9169)
* Refresh canonical importlib and preserve native import callback

Cache the original builtin __import__ callback so CPython 3.15 importlib's
hierarchical locking uses the existing native parent-initialization path.
Refresh the complete canonical importlib and related test/helper families
from CPython d220409a13b77de7a6c30abe16ea90db01528edc through update_lib.

Preserve both nondecorator importlib guards in the updater. Add explicit
hash-pinned removal of reviewed obsolete canonical files, with full
preflight, dry-run, idempotence and refusal tests. Preserve empty fixtures.

Assisted-by: OpenAI Codex:model version unavailable

* Complete canonical inspect and types closure for importlib metadata

Refresh whole canonical inspect/types production and related test families
through update_lib after hosted CI exposed stale __cached__ consumers.
Add coroutine and async-generator state getters, with closed-state priority
for reentrant frame-local finalizers, and expose the existing frame-locals
proxy type with its CPython class documentation.

Preserve the optional _datetime and unittest import blocks through a narrow
migration adapter. Retain canonical assertions and existing guards; record
the missing LazyImportType expectation before adding a conditional expected
failure, and capability-skip only the two absent-_testcapi CLI fixtures.

Assisted-by: OpenAI Codex:model version unavailable

* Reject canonical removal junctions and read UTF-8 manifests

Reject junctions alongside symlinks in the existing path-component preflight and revalidation. Decode removal manifests explicitly as UTF-8. Add regressions for whole-batch refusal, revalidation, and Unicode dry-run/apply/idempotence.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-17 14:40:13 +09:00
Alok Pandey aac1f4cb46 Fix py_freeze! failing on Windows when Lib is a text file (#9172)
When git can't create symlinks (the default on Windows without
Developer Mode), crates/pylib/Lib is checked out as a plain text file
that holds the target path. That path is relative to the file's own
folder, but py_freeze! opened it relative to the build's working
directory, so building rustpython-pylib failed with
"Error listing dir".

Resolve the target against the folder of that file instead.
2026-10-17 13:39:04 +09:00
youknowdot 59570923c3 Align hashlib and HMAC with Python 3.15 (#9170)
Warn on the deprecated shared string keyword and refresh the complete
hashing libraries, canonical suites, and shared helper from CPython
v3.15.0 (d220409a13b77de7a6c30abe16ea90db01528edc) with update_lib.

Keep the six-file family independent and add no RustPython test guards.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-17 11:38:33 +09:00
youknowdot fb76135ec8 Align SSL APIs and transport behavior with Python 3.15 (#9167)
Refresh ssl.py and test_ssl.py through update_lib from CPython d220409a13b77de7a6c30abe16ea90db01528edc without local test changes. Implement rustls cipher-suite, group and signature policy, negotiated-group reporting, EOF and session-policy behavior.

Keep OpenSSL and wasm API imports compatible, with genuine version-gated queries and honest unsupported capabilities. Repair OpenSSL callback ownership and traversal, callback exceptions, EOF and shutdown, PHA, session destruction, and system-error decoding exposed by canonical coverage. Preserve existing I/O locking; the inherited optional-backend full-duplex hang remains a separate driver prerequisite.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-17 11:34:23 +09:00
youknowdot 19d2fe3af7 Align bytes and bytearray operations with Python 3.15 (#9165)
Correct quote escaping, replacement count keyword binding, signed-minimum hex grouping, and clear storage/export handling. Migrate canonical bytes coverage through fail-closed preservation of the existing RustPython customizations, and retire only two freshly verified formatting guards.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-17 11:33:14 +09:00
dependabot[bot] 9a16d2841d build(deps): bump sha2 (#9164)
Bumps the crypto group with 1 update in the / directory: [sha2](https://github.com/RustCrypto/hashes).


Updates `sha2` from 0.11.0 to 0.11.1
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.11.0...sha2-v0.11.1)

---
updated-dependencies:
- dependency-name: sha2
  dependency-version: 0.11.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: crypto
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-17 04:25:21 +09:00
youknowdotandCodex 8720306805 Preserve function metadata and annotation callback lifetimes (#9162)
* Fix deprecated PyO3 code input import in benchmarks

Use the canonical code module path introduced in PyO3 0.30 without suppressing deprecation warnings.

Assisted-by: OpenAI Codex:model version unavailable

* Align runtime metadata and annotation cache contracts with Python 3.15

Preserve function name identity, implicit wrapper metadata and slice generic aliases. Match annotation dictionary and reentrant publication contracts, releasing displaced Python owners outside locks in the required phases. Refresh the matching canonical annotation tests through update_lib.

Assisted-by: OpenAI Codex: gpt-6.1-sol

* Refresh canonical generic alias coverage for slice

Migrate the complete Python 3.15 test_genericalias file through update_lib. Preserve the full canonical type list and capability guards so normal test runs cover slice subscripting.

Assisted-by: OpenAI Codex: gpt-6.1-sol

---------

Co-authored-by: Codex <codex@openai.com>
2026-10-16 23:45:57 +09:00
youknowdot b2b865aa90 Fix deprecated PyO3 code input import in benchmarks (#9161)
Use the canonical code module path introduced in PyO3 0.30 without suppressing deprecation warnings.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-16 23:43:41 +09:00
youknowdot 64330b80b9 Implement Unicode 17 grapheme iteration and properties (#9145)
* Implement native Unicode 17 grapheme iteration

Add the CPython extended grapheme state machine, full-codepoint properties, and traced Segment/iterator types with positional index handling. Refresh canonical unicodedata tests through update_lib while retaining pending markers. Include a losslessly range-compressed exact-final property oracle and differential test.

Source-reviewed implementation; candidate runtime validation remains pending. Reference property generation completed against the pinned CPython Unicode extension.

Assisted-by: OpenAI Codex:model version unavailable

* Bind grapheme Segment methods to the Python object

Use the required Py<Segment> receiver for native descriptors; preserve payload ownership and representation behavior.

Assisted-by: OpenAI Codex:model version unavailable

* Use explicit object receivers for Segment descriptors

Keep the inherent impl on the local payload and receive &Py<Self>, as required for stdlib classes outside the VM crate.

Assisted-by: OpenAI Codex:model version unavailable

* Mark grapheme property lookups as must-use

Satisfy strict Clippy for the three pure property functions without changing their behavior.

Assisted-by: OpenAI Codex:model version unavailable

* Enable validated Unicode grapheme canonical tests

Remove seven stale expected-failure markers through update_lib metadata after native property parity, canonical methods, full Unicode 17 grapheme corpus, and reference-cycle GC validation.

Assisted-by: OpenAI Codex:model version unavailable

* Accept platform line endings in grapheme oracle header

Compare the complete provenance line without its line ending so the same exact Unicode 17 reference is accepted after Windows checkout conversion. Keep the complete code-point property sweep unchanged.

Assisted-by: OpenAI Codex:model version unavailable

* Keep grapheme property regression in test module

Move the existing complete Unicode 17 property regression into the existing cfg(test) module to satisfy strict Clippy. Preserve the fixture, assertions, code-point coverage, and platform-independent provenance line check unchanged.

Assisted-by: OpenAI Codex:model version unavailable

* Qualify grapheme iterator return type for generated bindings

Assisted-by: OpenAI Codex:model version unavailable

* Fix deprecated PyO3 code input import in benchmarks

Use the canonical code module path introduced in PyO3 0.30 without suppressing deprecation warnings.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-16 23:40:26 +09:00
youknowdot fd5b30eb5e Align socket and socketserver with Python 3.15 (#9156)
Import the canonical socket libraries and tests through update_lib, preserving the existing AF_ALG expected failures. Match setsockopt index/buffer conversion and error ordering, mask optlen with the final CPython deprecation warning, reject oversized Windows buffers before the host call, and preserve arbitrary-precision getaddrinfo ports and empty canonical names.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-16 10:14:55 +09:00
youknowdot 1ee9a29062 Preserve explicit AST source positions through compilation (#9138)
* Preserve explicit AST column locations through code generation

Assisted-by: OpenAI Codex:model version unavailable

* Remove obsolete location bindings after span migration

Assisted-by: OpenAI Codex:model version unavailable

* Propagate inconsistent AST span provenance without panicking

Assisted-by: OpenAI Codex:model version unavailable

* Preserve mapped positions when exporting optimized Pass nodes

Assisted-by: OpenAI Codex:model version unavailable

* Consolidate AST location regressions and satisfy strict compiler lints

Assisted-by: OpenAI Codex:model version unavailable

* Consolidate equivalent AST literal location cases

Replace the six-way literal-only optimizer matrix with one optimized-AST round-trip, preserving all assertions. Keep the distinct normalized-range identity, explicit Pass provenance, and synthetic format-spec regressions.

Validation: all four appended location cases passed using the verified final-f6a RustPython binary with unchanged native sources. Ran normal pre-commit hooks; no Cargo build or full cyclic snippet suite.

Assisted-by: OpenAI Codex:model version unavailable

* Reuse Ruff source locations in AST conversion

Replace redundant PySourceLocation, Row, and Column wrappers with the compiler-core re-export of Ruff SourceLocation. Use UTF-8 byte coordinates and retain the empty-source fallback. Share the normalized range conversion while preserving caller validation and signed semantic metadata.

Validation so far: independent source and boundary review, rustfmt, and git diff --check. Runtime validation follows after the shared build target is released.

Assisted-by: OpenAI Codex:model version unavailable

* Preserve the first AST preprocessing location error

Share first-error recording between generated-node allocation and synthetic-pass coordinate overflow. Exercise both real compiler-state failure paths in both orders using one MAX_ID location entry.

Validation: focused compiler-state regression, strict codegen all-target Clippy, unchanged full-source seals, rustfmt, and normal pre-commit hooks. No interpreter runtime or broad suite rerun for this error-only change.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-16 10:13:59 +09:00
dependabot[bot] 9a8b2f4102 build(deps): bump mac_address (#9157)
Bumps the unix group with 1 update in the / directory: [mac_address](https://github.com/repnop/mac_address).


Updates `mac_address` from 1.2.0 to 1.2.1
- [Commits](https://github.com/repnop/mac_address/commits)

---
updated-dependencies:
- dependency-name: mac_address
  dependency-version: 1.2.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: unix
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-16 10:10:56 +09:00
1ndahous3 8966195bba Provide the C++ shared-linker command in sysconfig (#9155)
Assisted-by: Codex:gpt-6
2026-10-16 10:10:24 +09:00
1ndahous3 d90aa0781f Preserve direct-base precedence in C3 method resolution (#9154)
Assisted-by: Codex:gpt-6
2026-10-16 10:08:59 +09:00
Jeong, YunWon 2dbe992438 Remove ucd_3_2_0 feature from rustpython-capi (#9153)
Drop the capi [features] table added in #9011 along with the root
ucd_3_2_0 forwarding to rustpython-capi and the default-features = false
on the workspace rustpython-capi dependency. The root ucd_3_2_0 feature
still enables rustpython-stdlib/ucd_3_2_0.

Assisted-by: Claude Code:claude-opus-5-5
2026-10-16 10:06:40 +09:00
youknowdot 1f4e9ffaf6 Align OS argument conversion and syscall handling with Python 3.15 (#9146)
* Align OS argument conversion and syscall handling with Python 3.15

Import canonical os tests through update_lib, retaining existing guards. Preserve raw descriptor validation, errno, audit ordering, signal retries and unlocked sendfile buffers.

Validation: independent source review passed; matching release build and nine bounded canonical cases pending.

Assisted-by: OpenAI Codex:model version unavailable

* Import canonical root-user test support for OS package

Add hash-pinned selective assignment migration within test.support and import the three Python 3.15.0 privilege helpers plus their public exports. Preserve unrelated support helpers and existing test markers.

Validation: 177 host updater tests pass; byte-identical canonical helper provenance verified. Independent source review passed. Native import validation pending integration onto current main.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-16 10:06:13 +09:00
Bas Schoenmaeckers 88f5189a93 Update pyo3 to 0.30 (#9152)
* Update `pyo3` to `0.30`

* Fix cfunction with no module as self
2026-10-15 17:15:18 +00:00
1ndahous3andJeong YunWon a61f37a1b7 Own generated native closures and retain their method definitions (#8962)
* Own generated native closures and retain their method definitions

Assisted-by: Codex:gpt-6

* Keep native callable GC traversal off the bind path

Revert method_descriptor and classmethod_descriptor to untracked
payloads. Their only object field, _method_def_owner, points to an
untracked HeapMethodDef.

Restore the derived Traverse for builtin_function_or_method without a
clear implementation. default_dealloc calls the clear slot on every
deallocation, so the custom clear allocated a Vec for each bound builtin.

Use the derived Traverse for builtin_method, skipping the static class.

Assisted-by: Claude Code:claude-opus-5-5

---------

Co-authored-by: Jeong YunWon <jeong@youknowone.org>
2026-10-15 16:52:54 +00:00
1ndahous3 fdd38b8193 Avoid copying unchanged bytes results (#9149)
Assisted-by: Codex:gpt-6
2026-10-15 14:31:50 +00:00
1ndahous3 dcad0988e7 Fix Unicode StringIO overwrites and cache character positions (#9150)
Assisted-by: Codex:gpt-6
2026-10-15 14:30:53 +00:00
youknowdot 8e5709e797 Align native threading and libraries with Python 3.15 (#9148)
Preserve lock ownership and elapsed timeouts during interruptible waits; reject RLock constructor arguments and use replacement encoding for thread names. Generate associated libraries and tests from CPython d220409a13b77de7a6c30abe16ea90db01528edc using update_lib, retaining five existing markers and retiring one proven removed test owner.

AI-assisted source preparation and review. Runtime validation remains pending.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 22:46:26 +09:00
Joshua Megnauth a2c11aa51d Optional Unicode 3.2.0 but enabled by default (#9011)
Useful for smaller builds, like WebAssembly. The UCD 3.2.0 path is only
directly used in a handful of places in Python.

Assisted-by: GPT-5.6-Luna
2026-10-15 22:07:08 +09:00
Luan Taraschi bf404e0b7c Keep a bytearray exported while it formats with % (#9147)
bytearray.__mod__ copied the format and ran the arguments' __repr__,
__str__, __bytes__ and __index__ without holding an export, so a callback
could clear or extend the format and formatting went on with the old
copy:

    fmt = bytearray(b"%a end")
    class S:
        def __repr__(self):
            fmt.clear()
            return "E"
    fmt % S()   # bytearray(b'E end'), fmt left empty

CPython holds a buffer on the format for the whole call, so the resize
fails with BufferError and the format is left as it was. Taking an export
for the duration of __mod__ does the same here, the way bytearray.join
already does for its separator.

This lets test_mod_concurrent_mutation in test_bytes run again.

Assisted-by: Claude Code:claude-opus-5-5
2026-10-15 22:06:16 +09:00
youknowdot 901431963b Sync ten Python format and utility modules with Python 3.15 (#9144)
* Update ten Python library modules to Python 3.15 final

Migrate dependency-ordered linecache, locale, _pydecimal, decimal, random, uuid, wave, _strptime, zipapp and poplib with their canonical tests and WAV fixtures using update_lib.

Assisted-by: OpenAI Codex:model version unavailable

* Align dependent datetime and warning tests with Python 3.15 final

Generate complete dependent tests and pure datetime through update_lib from CPython d220409a13b77de7a6c30abe16ea90db01528edc. Preserve the existing tracemalloc expected failure. Pass warning module metadata to WarningMessage, use sys context after stack exhaustion, and identify invalid category classes with the existing qualified-name helper.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 21:59:44 +09:00
Chanho Lee e01021b902 Fix module docstring global name resolution (#9095)
* Fix module docstring global name resolution

Assisted-by: Claude Code:claude-opus-5-5

* Move module docstring test into syntax_doc.py

Assisted-by: Claude Code:claude-opus-5-5
2026-10-15 20:46:54 +09:00
Luan Taraschi 8de2cbe732 Fix object.__getstate__ deadlock and panic on __slotnames__ (#8981)
object_getstate_default kept the __slotnames__ list read-locked while it
looked up each slot attribute. A getter or __getattr__ that changed the
list (as in test_descr.test_issue24097) then waited forever for the write
lock. A non-str entry in a cached __slotnames__ hit an unwrap and
panicked.

Take a reference to each name and drop the lock before the lookup, raise
TypeError for a non-str name, let errors other than AttributeError
propagate, and check the list size after each item, as CPython does.

Assisted-by: Claude Code:claude-opus-5-5
2026-10-15 20:45:42 +09:00
youknowdot 365ff68354 Align base64 encodings and binascii keywords with Python 3.15 (#9139)
* Align base64 encodings with Python 3.15

Accept hexlify keyword arguments and migrate base64 and its tests directly from CPython v3.15.0 using update_lib.

Assisted-by: OpenAI Codex:model version unavailable

* Update Hypothesis fallback for multi-argument examples

Import the CPython 3.15 final fallback helper with scripts/update_lib migrate. Keep canonical base64 round-trip examples unchanged while passing their positional arguments as one subTest message.

Validation: six canonical base64 methods, fourteen fixed examples; bounded source-only runs with the existing same-native PR binary.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 20:43:52 +09:00
youknowdot 8f12a32256 Sync ten foundational Python libraries with Python 3.15 (#9140)
* Sync ten foundational Python libraries with CPython 3.15

Migrate _py_abc, stat, genericpath, ntpath, posixpath, functools, gettext, string, optparse, and mimetypes with related canonical tests using scripts/update_lib from CPython d220409a13b77de7a6c30abe16ea90db01528edc. Preserve the existing pure LRU hash-count expected failure.

Assisted-by: OpenAI Codex:model version unavailable

* Preserve positional-only annotation order for singledispatch

Emit positional-only annotations before regular positional annotations, matching declaration order and the canonical functools regression coverage.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 20:42:50 +09:00
youknowdot 9e18d7be6b Preserve explicit customizations during canonical library migration (#9142)
* Preserve explicit customizations during canonical library migration

Add bounded runtime, call, and doctest preservation plus explicit remaining-marker metadata. Support mapped legacy test-package migrations without losing ordered guards, and exclude cache directories from both migration paths while retaining canonical assets.

Validation: 53 bounded host-Python tooling tests passed. Tests inspect source and temporary files without executing migrated fixtures. Existing copy-library cache protection is unchanged. No Lib or interpreter changes.

Assisted-by: OpenAI Codex:model version unavailable

* Integrate migration preservation into existing updater modules

Keep migration orchestration and guarded package moves in cmd_migrate, and marker/doctest metadata in patch_spec. Reuse AST anchor validation and existing test modules, removing all eleven newly introduced helper, test, and data files.

Require caller-selected doctest metadata instead of embedding syntax-case policy. Reject guarded legacy doctests before package writes. Preserve the prior 53 test scenarios and add seven focused metadata regressions; all 60 bounded host tooling tests passed. No Lib, workflow, or native changes.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 19:54:38 +09:00