Commit Graph
538 Commits
Author SHA1 Message Date
youknowdotanddependabot[bot] 8c0c242dc8 Align profiler bindings and capability checks with Python 3.15 (#9340)
* Align offline profiler argument binding with Python 3.15

Accept the canonical keyword names for binary sample writes and native
context exits. Resolve the argument layout before numeric callbacks,
then preserve float-subclass payloads while accepting numeric protocols
for required and optional profile statistics.

AI-assisted source extraction, implementation, and source review with
OpenAI Codex. Runtime validation remains pending.

Assisted-by: OpenAI Codex:model version unavailable

* Restore the offline profiler capability flag

Preserve the original mega branch's zero-valued process_vm_readv
capability export. The native live unwinder remains unsupported; this
restores the public feature query without claiming live inspection support
or changing canonical test helpers.

AI-assisted source preservation and review with OpenAI Codex.

Assisted-by: OpenAI Codex:model version unavailable

* Restore finite-EOF table bounds for offline profiles

Match CPython's EOF-wide table count and decoding windows while retaining
checked offsets, allocation limits, cursor bounds, and replay checks.
This restores the original mega's finite-file contract without changing
canonical tests or the already validated argument-binding implementation.

Assisted-by: OpenAI Codex:model version unavailable

* Preserve symtable warning exceptions and original filenames (#9282)

Keep warning-hook exceptions untouched instead of rewriting SyntaxError
subclasses. Thread the original Python filename through parser warning
emission and fresh warning-as-error SyntaxError construction, preserving
fsdecoded bytes paths and lone surrogates.

Assisted-by: OpenAI Codex:model-version-unavailable

* build(deps): align the Cranelift 0.136 dependency family (#9341)

* build(deps): bump the wasmtime group across 1 directory with 2 updates

Bumps the wasmtime group with 2 updates in the / directory: [cranelift](https://github.com/bytecodealliance/wasmtime) and [cranelift-module](https://github.com/bytecodealliance/wasmtime).


Updates `cranelift` from 0.135.2 to 0.136.3
- [Release notes](https://github.com/bytecodealliance/wasmtime/releases)
- [Changelog](https://github.com/bytecodealliance/wasmtime/blob/main/RELEASES.md)
- [Commits](https://github.com/bytecodealliance/wasmtime/commits)

Updates `cranelift-module` from 0.135.3 to 0.136.3
- [Release notes](https://github.com/bytecodealliance/wasmtime/releases)
- [Changelog](https://github.com/bytecodealliance/wasmtime/blob/main/RELEASES.md)
- [Commits](https://github.com/bytecodealliance/wasmtime/commits)

---
updated-dependencies:
- dependency-name: cranelift
  dependency-version: 0.136.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: wasmtime
- dependency-name: cranelift-module
  dependency-version: 0.136.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: wasmtime
...

Signed-off-by: dependabot[bot] <support@github.com>

* Align cranelift-jit with the 0.136 dependency family

Update cranelift-jit alongside cranelift and cranelift-module so the JIT shares their LibCall, FuncId, and Module trait identities. Regenerate only the affected dependency lock entries, preserving the original dependency PR changes.

AI assistance covered the dependency diagnosis, alignment, and static review.

Assisted-by: OpenAI Codex:model-version-unavailable

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Support selective canonical function migration

Add hash-pinned function selection to the official Lib migration tool,
preserving canonical source bytes and rejecting ambiguous dependencies.

Assisted-by: OpenAI Codex:model version unavailable

* Detect CPython runtimes for canonical inspection feature checks

Add native runtime-cookie detection and import the exact Python 3.15
inspection helpers and tests through the official Lib migration tool.
Preserve the offline profiler argument and finite-EOF corrections.

Assisted-by: OpenAI Codex:model version unavailable

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-25 15:09:49 +09:00
Jeongseop LimandJeong YunWon c271e33214 Reject surplus positional arguments before converting any (#9242)
* Count positional arguments before converting any

FuncArgs::bind_for rejected surplus positional arguments only after
FromArgs::from_args had converted every declared parameter. A
conversion can run user code (__iter__ for PyIter, __index__ for
ArgIndex), so filter(None, it, 0), hex(x, 1) and pairwise(it, 0) raised
whatever that code raised instead of the TypeError CPython reports
before it touches any argument.

Check the count against FromArgs::arity() before from_args runs. For
that bound to be safe:

- The default arity() has no upper bound, since a manual impl may take
  positional arguments itself. KwArgs, (), DirFd, OpenDirFd and csv's
  FormatOptions declare the bound they actually have.
- A derived struct adds the upper bound of each flattened field.
- A tuple sums upper bounds with saturating_add.

Assisted-by: Claude Code:claude-opus-5-5

* Note why arity upper bounds saturate

Comment that an unbounded upper bound is usize::MAX, which is why the
tuple and derived arity() sums use saturating_add, and drop the CPython
mention from the positional count check comment.

Assisted-by: Claude Code:claude-opus-5-5

---------

Co-authored-by: Jeong YunWon <jeong@youknowone.org>
2026-10-24 01:02:44 +09:00
Jeong, YunWonandyouknowdot f0b1e82652 Add tracing profiler and Python 3.15 offline profiling tools (#9271)
* Add deterministic profiler and canonical Python 3.15 tools

Use the merged operation critical sections with interpreter-bound profiler
entry checks and short payload borrows. Trace owned code references for GC.
Import the canonical tracing and offline profiling output closure via the
repository updater, retaining existing test markers.

Assisted-by: dot:GPT-6.1

* Clarify standalone profiling license cross-reference

Preserve license terms and copyright notices; point to the reproduced
Apache text below instead of CPython's original OpenSSL section above.

Assisted-by: dot:GPT-6.1

* Defer unsupported sampling package from profiler extraction

Keep the deterministic tracing profiler and its canonical tools together.
Defer the entirely new sampling production, tests, assets and their license
as one closure until native record and binary-collector support is ready.
Retain canonical pstats for deterministic empty-profile handling; its
upstream sampled-file sorting issue is not patched here.

Assisted-by: dot:GPT-6.1

* Stage profiling root as a namespace until sampling is available

Omit the whole newly added canonical package initializer rather than
advertise an absent sampling module or hand-edit canonical __all__.
Explicit profiling.tracing and cProfile imports remain available.
The namespace root intentionally lacks canonical metadata/automatic exports.

This addresses test___all__ failures in the prior Linux, macOS and Windows
CI runs. No test assertions, skips or canonical file contents are changed.

Assisted-by: dot:GPT-6.1

* Match canonical static builtin profiler labels and metadata

Normalize the raw builtin receiver instead of its call-argument accessor.
Snapshot static method module metadata at construction, matching CPython,
so explicit module mutation and deletion remain observable. Avoid a
profiler-time fallback that would obscure deleted metadata.

Correct float.__getformat__ to CPython's classmethod calling convention,
including subclass binding, rather than special-casing its profiler label.

Assisted-by: dot:GPT-6.1

* Support collision-safe frozen package containers

Generate empty frozen packages only for source namespace directories with
real frozen descendants. Preserve genuine modules and packages across
frozen-library registrations while retaining real-real replacement order.

Record the container marker in FrozenModule and its library envelope;
legacy module/package encodings remain unchanged. This is closed-tree
frozen packaging, not full PEP 420 namespace merging.

Add structural codec, mock generator, and registration-priority checks.

* Revert frozen package containers before restoring canonical profiling root

The container implementation changed filesystem namespace precedence.
Restore the existing importer and frozen envelope unchanged; the following
offline sampling closure restores the canonical regular package root.

This reverses d5a8049234.

Assisted-by: dot:GPT-6.1

* Restore canonical profiling package with offline sampling codec

Provide native records, status constants and genuine uncompressed binary
reader/writer APIs for canonical offline tools. Keep the unsupported live
process capability absent and preserve existing canonical test gates.
Restore the complete selected sampling production/test/assets closure via
update_lib, including its asyncio frame-schema dependency. The prior frozen
container extension is reverted separately; canonical packages work with
both filesystem and frozen imports without changing namespace semantics.

Assisted-by: dot:GPT-6.1

* Keep platform-dependent C long normalization explicit

Allow only the c_long-to-i32 conversion lint: Windows uses the same type,
while LP64 requires the existing truncation to match CPython normalization.
No runtime behavior or test assertions change.

Assisted-by: dot:GPT-6.1

* Include profiling notices in library and desktop packages

Bundle a byte-identical copy of the root profiling notice in rustpython-pylib
and include the root notice in configured desktop resources. Avoid a symlink
so Windows checkouts preserve the actual notice text. Terms are unchanged.

Assisted-by: dot:GPT-6.1

* Keep profiling asset license notice in pylib crate

* Serve frozen profiling assets through native resource readers

* Preserve canonical profiling asset line endings on checkout

---------

Co-authored-by: youknowdot <dot@youknowone.org>
2026-10-22 11:18:03 +09:00
youknowdot f482e0c2cf Track Unpickler payload references during cyclic GC (#9288)
Traverse input callbacks, buffer iterators, memo slots and memo proxy owners.
Clear outgoing payload edges through deferred drops and retire successful
reinitialization state outside the state lock. Retain stream and buffers
payload-cycle regressions in the existing pickle snippet.

Assisted-by: OpenAI Codex:model-version-unavailable
2026-10-22 10:17:06 +09:00
youknowdot 2be9389f2d Read native pickle file payloads incrementally (#9260)
Bound the initial file read, grow later requests geometrically, use fallible reservation for input buffers, and compare frame bounds without addition overflow.

Assisted-by: Codex:unknown (model version not exposed)
2026-10-22 01:11:26 +09:00
youknowdot a96c0570a8 Avoid epoll alias ownership aborts (#9250)
Keep Python-visible epoll descriptors in a raw host wrapper, preserving fromfd aliasing and explicit close errors without OwnedFd drop aborts. Use native epoll event layout behind a transparent host-only type and align positive timeout overflow with CPython milliseconds.

Assisted-by: OpenAI Codex:model-version-unavailable
2026-10-21 12:05:27 +09:00
1ndahous3 b0142d0464 Implement socket.recvmsg_into on Unix (#9268)
Assisted-by: Codex:GPT-6
2026-10-21 12:01:27 +09:00
1ndahous3 02f4fbbb62 Use ordinary fileno lookup in select (#9266)
* Use ordinary fileno lookup in select

Assisted-by: Codex:GPT-6

* Share select descriptor conversion

Assisted-by: Codex:GPT-6
2026-10-21 11:59:10 +09:00
1ndahous3 6f2d4a203d Share immutable tokenizer state between iterations (#9265)
Assisted-by: Codex:GPT-6
2026-10-21 11:58:48 +09:00
youknowdot 587358ba5c Support private names in native pickle globals and bound methods (#9259)
Preserve WTF-8 dotted paths in protocol 4/5 global loading so private-name round trips also support surrogate-containing names.

Assisted-by: Codex:unknown (model version not exposed)
2026-10-21 11:57:16 +09:00
1ndahous3 0c5fa209a8 Support Linux Bluetooth socket addresses (#9243)
Assisted-by: Codex:GPT-6
2026-10-20 19:52:57 +09:00
1ndahous3 c02584368a Avoid temporary big integers for small getrandbits calls (#9233)
Assisted-by: Codex:GPT-6
2026-10-20 09:25:00 +09:00
1ndahous3 aa9455082d Support Netlink socket addresses (#9206)
Assisted-by: Codex:GPT-6
2026-10-19 20:28:39 +09:00
1ndahous3 7750dbcb86 Accept keyword arguments in SemLock constructors (#9188)
Assisted-by: Codex:GPT-6
2026-10-18 22:37:01 +09:00
1ndahous3 e792ec8fb3 Accept bytes-like input in pyexpat.Parse (#9187)
Assisted-by: Codex:GPT-6
2026-10-18 22:36:39 +09:00
youknowdotandJeong YunWon 3f55cf4af0 Implement basic REPL execution and fix piped stdin handling (#9163)
* Preserve original main VM through C API execution

Run and finalize the original main VM while keeping the registered attachment template available without holding its mutex across Python callbacks.

Assisted-by: OpenAI Codex:model version unavailable

* Make unraisable reporting robust at hook boundaries

Keep the existing message API and traceback formatting while handling missing, disabled, audited and failing hooks without panicking.

Assisted-by: OpenAI Codex:model version unavailable

* Separate interactive warning collection from compilation

Retain ordinary warning behavior and typed module APIs while exposing record-local parser warnings and post-warning code generation. Add a source-only lazy future-import guard.

Assisted-by: OpenAI Codex:model version unavailable

* Evaluate native stdin with shared VM record semantics

Use one evaluator for basic REPL and root transports, preserve effective inspect snapshots and future flags, and carry owned host exit status through original-main finalization.

Assisted-by: OpenAI Codex:model version unavailable

* Refresh canonical REPL tests and retire fixed failures

Run update_lib migrate from pinned CPython 3.15.0, preserving three remaining platform/runtime markers and retiring the two traceback/startup failures verified on the native evaluator.

Assisted-by: OpenAI Codex:model version unavailable

* Keep the root compiler dependency for development targets

The native evaluator now compiles through the VM. Only root benchmarks and the parse_folder example use rustpython-compiler directly, so declare it as a dev dependency while preserving VM compiler features.

Assisted-by: OpenAI Codex:model version unavailable

* Validate explicit conditional failure marker retirement

Require exact selector and original decorator evidence for each conditional expected-failure marker retired by migrate. Preserve undeclared guards and reject missing, duplicate, modified or unconsumed declarations.

Assisted-by: OpenAI Codex:model version unavailable

* Refresh canonical stdin and exception audit tests

Migrate the pinned CPython 3.15.0 modules and their canonical helper fixtures through update_lib. Retire exactly four failure markers proven obsolete by hosted CI, preserving all other existing guards.

Assisted-by: OpenAI Codex:model version unavailable

* Synchronize runpy warning attribution and document remaining capabilities

Migrate whole canonical runpy.py to pass module names to compile. Use update_lib to annotate only reproduced missing-capability cases: absent _testcapi import-event tests, unsupported startup/import-time features, and the deferred frozen-loader module-attribution caller.

Assisted-by: OpenAI Codex:model version unavailable

* Propagate interrupted native stdin reads to signal dispatch

Preserve the shared input buffer and partial bytes while returning EINTR to the VM before retrying. Exercise the transport deterministically and verify a returning SIGINT handler while pipe stdin stays open.

Assisted-by: OpenAI Codex:model version unavailable

* Reject statements after a compound statement in single mode

single_mode_multiple_statements_error now reports "invalid syntax" at the
statement that follows a compound first statement, and interactive_input
runs it right after parsing as compile() does.

The lazy `from __future__` import check moved from lazy_future_import_error
into unsupported_grammar_error, so it applies to every compile mode.

Assisted-by: Claude Code:claude-opus-5-5

* Remove passing expected failure from test_runpy

test_run_module_filter_syntax_warnings_by_module passes.

Assisted-by: Claude Code:claude-opus-5-5

* Restructure stdin execution after Modules/main.c and pythonrun.c

- src/lib.rs follows pymain_run_python: pymain_run_stdin, pymain_repl,
  pymain_run_startup, pymain_run_interactive_hook and pymain_run_pyrepl
  return exit codes instead of raising SystemExit.
- VirtualMachine gains run_any_stdin (_PyRun_AnyFile),
  run_interactive_loop (_PyRun_InteractiveLoop) and run_simple_stdin;
  run_shell feeds rustyline lines into run_interactive_loop, and
  sys._baserepl calls run_any_stdin.
- print_uncaught_exception (PyErr_Print) replaces
  report_uncaught_exception, and handle_system_exit_and_keyboard_interrupt
  replaces handle_reported_exception.
- An unhandled KeyboardInterrupt is recorded and the process exits
  through SIGINT after finalization.
- Interpreter::run_main replaces run_with_exit_status and InterpreterExit;
  rustpython_capi::run_main_interpreter takes an exit-code callback.
- config_set_inspect audits cpython.PyConfig_Set and rebuilds sys.flags
  through set_flag_obj (_PySys_SetFlagObj); the inspect state is an
  AtomicBool and dict replace_by_identity is removed.
- add_main_module (PyImport_AddModuleRef("__main__")) is used by the
  interactive and simple-file paths.
- The interactive parser warning helpers take their state directly.

Assisted-by: Claude Code:claude-opus-5-5

* Exit through Py_Exit from PyErr_Print and run the basic REPL via stdin

- print_uncaught_exception (PyErr_Print) calls exit_process (Py_Exit) for a
  SystemExit when inspect mode is off, instead of returning the exception.
  run_interactive_loop no longer checks SystemExit itself and flushes after
  printing an error; sys._baserepl returns None.
- exit_process finalizes through VirtualMachine::finalize_ex, which holds
  the Py_FinalizeEx steps split out of Interpreter::finalize, and exits
  through host_env::os::exit_with_cleanup (C exit()).
- The basic REPL on a terminal (PYTHON_BASIC_REPL, or _pyrepl missing)
  runs run_any_stdin instead of the rustyline shell.
- run_shell saves the history after every line.

Assisted-by: Claude Code:claude-opus-5-5

* Display exceptions through PyErr_Display and track the main thread running __main__

- display_exception (PyErr_Display) and display_exception_to (_PyErr_Display)
  try traceback._print_exception_bltin before the native printer and flush
  the file. PyErr_Print's excepthook fallback, print_exception and
  sys.excepthook use them. A missing sys.stderr is reported with
  dump_object (PyObject_Dump); a None sys.stderr prints nothing.
- Paths gains program_name (argv[0]) from getpath. The "can't open file"
  message uses it with strerror(errno) and goes to sys.stderr.
- running_main holds the ident of the thread running __main__.
  pymain_run_python sets it around running the main code, Py_Exit clears it,
  the fork child reinitializes it, and _interpreters.is_running reads it
  instead of treating the main interpreter as always running.
- Interpreter::run_main clears the unhandled KeyboardInterrupt flag before
  running and applies exit_sigint after finalizing; finalize no longer
  does.
- run_interactive_loop returns a SystemExit raised outside inspect mode
  again and no longer flushes after printing an error; run_shell saves the
  history once after the loop.
- pymain_set_inspect reports its error through print_uncaught_exception.

Assisted-by: Claude Code:claude-opus-5-5

* Treat a parse error at the end of the source as incomplete input

A parser error that starts at the end of the source is reported as
_IncompleteInputError when incomplete input is allowed, so the
interactive loop and codeop keep reading after `try:` and its block or
after a decorator line.

Assisted-by: Claude Code:claude-opus-5-5

* Place syntax errors on tokens made up at the end of the input

A parse error on the NEWLINE, DEDENT or ENDMARKER the tokenizer makes up
at the end of the input now carries an EofToken: the number of lines
read and the last line. SyntaxError takes its lineno and text from it,
and its offset is that line's length with end_offset -1, or 0 for both
when the error is raised at the token itself (the generic syntax error
and a missing forced `:` or `(`). Exec input that does not end with a
newline is checked with one appended.

Assisted-by: Claude Code:claude-opus-5-5

* Slice the source with get when checking an indented block error at EOF

Assisted-by: Claude Code:claude-opus-5-5

* Rename the host exits after C exit() and fix wasi and Windows builds

- host_env::os::exit now calls the C exit() on every platform; on Windows
  std::process::exit ends in ExitProcess, so it flushes the Rust stdout
  and calls libc::exit there. The former exit, which calls _exit(), is
  now exit_immediately, used by os._exit, faulthandler and
  posixsubprocess.
- errno::strerror_string is available on wasi.
- PyGlobalState::reinit_running_main is built only where its fork caller
  is.

Assisted-by: Claude Code:claude-opus-5-5

---------

Co-authored-by: Jeong YunWon <jeong@youknowone.org>
2026-10-18 09:22:12 +00:00
youknowdot 9f6f771965 Align lazy imports and f/t-string interpolation with Python 3.15 (#9179)
* Preserve interpolation source text and canonical metadata

Source-only checkpoint; candidate validation and the existing annotation expectation reconciliation remain pending.

Assisted-by: OpenAI Codex:model version unavailable

* Align existing annotation expectations with CPython metadata

Update only the nine existing expected strings confirmed against pinned CPython 3.15, with explicit user approval.

Assisted-by: OpenAI Codex:model version unavailable

* Preserve template metadata through compilation and interactive input

Escape literal debug prefixes, retain None AST expression metadata internally, and continue incomplete triple-quoted t-strings in the existing shell.

Assisted-by: OpenAI Codex:model version unavailable

* Classify normalized unclosed brackets as REPL line continuations

Assisted-by: OpenAI Codex:model version unavailable

* Nest equivalent REPL parse-error alternatives for Clippy

Assisted-by: OpenAI Codex:model version unavailable

* Remove fulfilled annotation migration lint workaround

The canonical expected strings no longer trigger literal_string_with_formatting_args, so the previous lint expectation is obsolete.

Assisted-by: OpenAI Codex:model version unavailable

* Complete lazy-import exception and traceback library closure

Copy canonical traceback and legacy exception mappings with preserved RustPython guards. Copy explicit non-Python library assets by exact path so exception hierarchy data does not expand into the entire test package.

Assisted-by: OpenAI Codex:model version unavailable

* Provide Python 3.15 traceback metadata for missing modules and attributes

Preserve AttributeError context for missing class attribute deletion, matching type_update_dict. Expose canonical generated missing-stdlib messages through the native sysconfig modules without importing CPython build-specific disabled-module assumptions.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-18 17:11:26 +09:00
1ndahous3 e64023020c Fix CSV field limit validation and concurrent updates (#9160)
Assisted-by: Codex:gpt-6
2026-10-18 10:59:51 +09:00
1ndahous3 cf3bcee3fd Release SQLite callbacks and expanded trace SQL safely (#9151)
* Release SQLite callbacks and expanded trace SQL safely

Assisted-by: Codex:gpt-6

* Drain SQLite callback cleanup before reinitialization and after blob operations

Assisted-by: Codex:gpt-6

* Drain SQLite cleanup after statement reset

Assisted-by: Codex:gpt-6

* Fix concurrent SQLite connection reinitialization

Assisted-by: Codex:gpt-6

* Restore SQLite factory defaults after callback cleanup

Assisted-by: Codex:gpt-6
2026-10-18 10:58:39 +09:00
youknowdot 51e886c224 Fix optional OpenSSL duplex I/O on Unix (#9173)
* Fix optional OpenSSL duplex I/O on Unix

Release connection locks before readiness waits while serializing native SSL
attempts and preserving owned write retries. Wake waiters on native progress
and socket identity invalidation, and construct callback/native exceptions
after unlocking.

Add regression coverage to the existing SSL snippet and run the optional
OpenSSL backend in the existing Unix CI matrix. Windows and AIX retain the
legacy transport pending a separate compatibility-safe driver.

Assisted-by: OpenAI Codex:model version unavailable

* Consolidate OpenSSL runtime checks into the existing CI step

Keep the minimal system and vendored OpenSSL build checks, and run the
Unix SSL regressions in the existing system OpenSSL step. Preserve the
runtime feature set and enable its macOS dependency in that job.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-17 23:36:27 +09:00
youknowdot 59570923c3 Align hashlib and HMAC with Python 3.15 (#9170)
Warn on the deprecated shared string keyword and refresh the complete
hashing libraries, canonical suites, and shared helper from CPython
v3.15.0 (d220409a13b77de7a6c30abe16ea90db01528edc) with update_lib.

Keep the six-file family independent and add no RustPython test guards.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-17 11:38:33 +09:00
youknowdot fb76135ec8 Align SSL APIs and transport behavior with Python 3.15 (#9167)
Refresh ssl.py and test_ssl.py through update_lib from CPython d220409a13b77de7a6c30abe16ea90db01528edc without local test changes. Implement rustls cipher-suite, group and signature policy, negotiated-group reporting, EOF and session-policy behavior.

Keep OpenSSL and wasm API imports compatible, with genuine version-gated queries and honest unsupported capabilities. Repair OpenSSL callback ownership and traversal, callback exceptions, EOF and shutdown, PHA, session destruction, and system-error decoding exposed by canonical coverage. Preserve existing I/O locking; the inherited optional-backend full-duplex hang remains a separate driver prerequisite.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-17 11:34:23 +09:00
youknowdot 64330b80b9 Implement Unicode 17 grapheme iteration and properties (#9145)
* Implement native Unicode 17 grapheme iteration

Add the CPython extended grapheme state machine, full-codepoint properties, and traced Segment/iterator types with positional index handling. Refresh canonical unicodedata tests through update_lib while retaining pending markers. Include a losslessly range-compressed exact-final property oracle and differential test.

Source-reviewed implementation; candidate runtime validation remains pending. Reference property generation completed against the pinned CPython Unicode extension.

Assisted-by: OpenAI Codex:model version unavailable

* Bind grapheme Segment methods to the Python object

Use the required Py<Segment> receiver for native descriptors; preserve payload ownership and representation behavior.

Assisted-by: OpenAI Codex:model version unavailable

* Use explicit object receivers for Segment descriptors

Keep the inherent impl on the local payload and receive &Py<Self>, as required for stdlib classes outside the VM crate.

Assisted-by: OpenAI Codex:model version unavailable

* Mark grapheme property lookups as must-use

Satisfy strict Clippy for the three pure property functions without changing their behavior.

Assisted-by: OpenAI Codex:model version unavailable

* Enable validated Unicode grapheme canonical tests

Remove seven stale expected-failure markers through update_lib metadata after native property parity, canonical methods, full Unicode 17 grapheme corpus, and reference-cycle GC validation.

Assisted-by: OpenAI Codex:model version unavailable

* Accept platform line endings in grapheme oracle header

Compare the complete provenance line without its line ending so the same exact Unicode 17 reference is accepted after Windows checkout conversion. Keep the complete code-point property sweep unchanged.

Assisted-by: OpenAI Codex:model version unavailable

* Keep grapheme property regression in test module

Move the existing complete Unicode 17 property regression into the existing cfg(test) module to satisfy strict Clippy. Preserve the fixture, assertions, code-point coverage, and platform-independent provenance line check unchanged.

Assisted-by: OpenAI Codex:model version unavailable

* Qualify grapheme iterator return type for generated bindings

Assisted-by: OpenAI Codex:model version unavailable

* Fix deprecated PyO3 code input import in benchmarks

Use the canonical code module path introduced in PyO3 0.30 without suppressing deprecation warnings.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-16 23:40:26 +09:00
youknowdot fd5b30eb5e Align socket and socketserver with Python 3.15 (#9156)
Import the canonical socket libraries and tests through update_lib, preserving the existing AF_ALG expected failures. Match setsockopt index/buffer conversion and error ordering, mask optlen with the final CPython deprecation warning, reject oversized Windows buffers before the host call, and preserve arbitrary-precision getaddrinfo ports and empty canonical names.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-16 10:14:55 +09:00
Joshua Megnauth a2c11aa51d Optional Unicode 3.2.0 but enabled by default (#9011)
Useful for smaller builds, like WebAssembly. The UCD 3.2.0 path is only
directly used in a handful of places in Python.

Assisted-by: GPT-5.6-Luna
2026-10-15 22:07:08 +09:00
youknowdot 365ff68354 Align base64 encodings and binascii keywords with Python 3.15 (#9139)
* Align base64 encodings with Python 3.15

Accept hexlify keyword arguments and migrate base64 and its tests directly from CPython v3.15.0 using update_lib.

Assisted-by: OpenAI Codex:model version unavailable

* Update Hypothesis fallback for multi-argument examples

Import the CPython 3.15 final fallback helper with scripts/update_lib migrate. Keep canonical base64 round-trip examples unchanged while passing their positional arguments as one subTest message.

Validation: six canonical base64 methods, fourteen fixed examples; bounded source-only runs with the existing same-native PR binary.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 20:43:52 +09:00
youknowdot 0384ddcd70 Align mmap operations and capability reporting with Python 3.15 (#9141)
Separate raw descriptor inspection from owned tracking, update flush and naming behavior, and expose resize only on implemented backends. Migrate canonical mmap tests using update_lib while preserving existing RustPython guards.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 19:53:53 +09:00
youknowdot 16e333bded Align asyncio initialization and future state with Python 3.15 (#9136)
* Align asyncio initialization and future state with Python 3.15

Reject native Future and Task reinitialization, validate task contexts, and restore task registration after failed eager context entry. Generate matching asyncio and concurrent Future sources and tests with update_lib from CPython d220409a13b77de7a6c30abe16ea90db01528edc. Preserve the existing pending-task reinitialization regression.

Assisted-by: OpenAI Codex:model version unavailable

* Remove asyncio snippet duplicating canonical reinitialization coverage

Keep the exact-final Lib tests as the reinitialization coverage instead of adding a new snippet for the same contract. Native and library implementation bytes remain unchanged.

Assisted-by: OpenAI Codex:model version unavailable

* Validate both objects in asyncio awaited-by helpers

Match CPython by ignoring non-native Future/Task waiters in add and discard. Keep the existing native reentrant hashing fixture reachable using Future subclasses and use native instances for its membership checks.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 10:02:39 +00:00
youknowdot 498adff3a9 Align JSON array hooks and libraries with Python 3.15 (#9135)
Generate the JSON library and selected tests with update_lib from CPython d220409a13b77de7a6c30abe16ea90db01528edc, preserving native bootstrap workarounds and test markers. Exclude generated cache directories at the library copy boundary.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 07:32:30 +00:00
youknowdot 9c832ee26c Align native datetime conversion with Python 3.15 (#9133)
Prefer index conversion for timestamps, accept float-convertible values, validate ISO timezone components, and apply final local fold detection consistently.

Implementation, source comparison, and bounded validation were assisted by OpenAI Codex.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 15:06:38 +09:00
youknowdot 3ab5c0c2f2 Align mapping boundaries with Python 3.15 final (#9128)
Support frozen mappings in str.maketrans and native JSON encoding, preserve integer translation keys, and protect mappingproxy dictionaries from unknown reflected comparisons.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 10:31:45 +09:00
youknowdot 2ddba03c39 Support frozen ElementTree attribute mappings (#9124)
* Support frozen ElementTree attribute mappings

Preserve assigned mapping identity while copying constructor attributes, and use checked union ownership through Element operations.

Assisted-by: OpenAI Codex:gpt-6.1-sol

* Sync final ElementTree runtime and companion tests

Import ElementTree.py from CPython d220409a13b77de7a6c30abe16ea90db01528edc through update_lib migrate. Copy six complete canonical tests with update_lib copy-test, preserving all surrounding test markers. This aligns pure-Python and native frozendict diagnostic expectations and covers the runtime import companion changes.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 10:27:39 +09:00
youknowdot 13d0c83753 Align Struct construction and complex formats with Python 3.15 (#9123)
Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 10:26:51 +09:00
youknowdot 32f5ac9c05 Reject invalidated type-cache entries and retain disassembly tracebacks (#9121)
Require matching cache/type versions to be nonzero in both readers, keeping
Acquire load order and the existing sequence-lock checks. Add a deterministic
native regression for cleared versions, mismatches, and valid negative cache
entries, without executing Python or introducing concurrency stress.

Render Python tracebacks when the existing disassembly snapshot helper fails.
The intermittent Windows AttributeError that prompted this investigation has
not been reproduced locally or proven to originate from the cache defect.

AI assistance: OpenAI Codex investigated, implemented, independently
source-reviewed, and performed focused automated validation of this change.
No human manual testing is claimed.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 10:24:48 +09:00
youknowdot 0f127b07e9 Add Python 3.15 math APIs using pymath (#9122)
* Add Python 3.15 math APIs using pymath

Pin pymath to merged dd1e9a4cf26d68caf3b549c09476f52e356e16a3
and expose isnormal, issubnormal, signbit, fmax and fmin through thin
wrappers. Share the existing integer functions through math.integer
and _math_integer, matching CPython module identity and metadata.

Migrate test_math.py and test_math_integer.py via scripts/update_lib
from CPython d220409a13b77de7a6c30abe16ea90db01528edc, preserving the
existing Windows remainder marker. Add bounded Python snippet coverage.

AI assistance: OpenAI Codex implemented and source-reviewed this change,
ran update_lib, and performed focused automated validation. No human
manual testing is claimed.

Assisted-by: OpenAI Codex:model version unavailable

* Track integer math tests in library updates

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 10:02:44 +09:00
1ndahous3 05cb67524a Cache SQLite statements and synchronize connection operations (#9109)
* Cache SQLite statements per connection

Assisted-by: Codex:gpt-6

* Keep SQLite connections alive across callbacks

Assisted-by: Codex:gpt-6

* Defer SQLite cleanup and preserve operation errors

Assisted-by: Codex:gpt-6

* Synchronize SQLite results and transaction transitions

Assisted-by: Codex:gpt-6

* Use the calling thread's VM for SQLite callbacks

Assisted-by: Codex:gpt-6
2026-10-15 09:56:14 +09:00
youknowdot 1b20f6a194 Use 64-bit timestamps for Windows CRT and FILETIME conversions (#9116)
Use the explicit 64-bit CRT ABI regardless of libc time_t width, and preserve 64-bit FILETIME seconds throughout stat producers and consumers. Add bounded Windows regressions for 2040 calendar and stat conversions while retaining existing historical timestamps and error behavior.

AI assistance: OpenAI Codex implemented and independently source-reviewed this split, prepared bounded Python regressions and performed cross-target compile checks. Windows runtime and human manual testing are not claimed.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 00:57:58 +09:00
1ndahous3 db33af9c3e Validate socket receive sizes and bound truncated results (#9108)
* Validate socket receive sizes and bound truncated results

Assisted-by: Codex:gpt-6

* Fix socket receive checks across platforms

Assisted-by: Codex:gpt-6
2026-10-14 23:56:06 +09:00
Jeong, YunWonanddot 2a09ee02bd Align compiler and VM bytecode contracts with CPython 3.15 (#9105)
* Align compiler and VM with final CPython 3.15 bytecode

Reconstruct the compiler-only baseline from validated487ae47 on the current main and frozen mapping prerequisitec78d811. Preserve the full source delta while dropping obsolete prerequisite ancestry; detailed old-to-new provenance is retained in the recovery artifact.

Assisted-by: OpenAI Codex: model identifier unavailable

* Fix final script validation, exception cleanup and private cache contracts

Replay reviewed compiler CI repairs 2191a9d, 1f1fe04, a2dacaf on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable

* Align canonical compiler fixtures and generator frame state

Replay reviewed compiler CI repairs 476bb74 on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable

* Preserve code-name owners and final warning and constant-fold contracts

Replay reviewed compiler CI repairs 6482adf on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable

* Skip unavailable monitoring C API tests before tool registration

Replay reviewed compiler CI repairs 246c309 on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable

* Assert exact final constant-list iterable folding

Replay reviewed compiler CI repairs 7ecedcf on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable

* Align local monitoring events and exception lifecycle state with final 3.15

Replay reviewed compiler CI repairs ec206b1, 191fe58 on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable

* Align 3.15 code objects and codegen with CPython

- Store co_consts, co_names, co_localsplusnames, co_localspluskinds,
  co_linetable, co_exceptiontable and co_name as objects on PyCode;
  build co_code and the varnames/cellvars/freevars tuples lazily in
  _co_cached, with co_code deoptimized through deopt_code.
- Move the compiler constant cache (_PyCode_ConstantKey,
  const_cache_insert) to builtins/code/const_cache.rs.
- Write every code field with w_object in marshal and pass the read
  field objects to MarshalBag::make_code through CodeFields.
- Emit RETURN_GENERATOR/POP_TOP in enter_scope and place SETUP_CLEANUP
  after them; drop the generator part of insert_prefix_instructions.
- Add PyCode::first_traceable (_co_firsttraceable) for
  _PyFrame_IsIncomplete and sys.monitoring.
- Follow the CPython DISABLE branches in sys.monitoring, the
  check_matched signature in warnings, the 3.15 % format error
  messages for str and bytes, code_returns_only_none, and the FOR_ITER
  stack analysis.
- Compute IMPORT_NAME flags with in_exception_handler.
- Replace the linetable/exceptiontable byte tests in codegen with
  dis(show_positions=True) insta snapshots in _opcode.
- Remove the opcode metadata verification tooling, byte-exact
  scripts, private cache tests and redundant snippets.
- Restore test_code.py and test_monitoring.py to v3.15.0 plus
  RUSTPYTHON markers.

Assisted-by: Claude Code:claude-opus-5-5

* Remove generator snippet checks covered by Lib/test

Drop extra_tests/snippets/generator_iterator_prefix.py. Its loop,
delegation, async, f_back and never-awaited checks are covered by
test_generators, test_coroutines, test_asyncgen, test_yield_from and
test_monitoring. Move the remaining checks into existing files:
genexpr prefix tracing and monitoring into stdlib_sys.py, and the
caller frame seen from the eager iterator into stdlib_sys_getframe.py.

Assisted-by: Claude Code:claude-opus-5-5

* Emit the generator prefix for top-level await modules

Module code compiled with PyCF_ALLOW_TOP_LEVEL_AWAIT and marked
ste_coroutine now starts with RETURN_GENERATOR; POP_TOP. The prefix and
RESUME emission move into emit_scope_prefix, shared by enter_scope and
the module entry points, as in codegen_enter_scope. Fixes
test_builtin.test_compile_top_level_await.

Assisted-by: Claude Code:claude-opus-5-5

* Allow the c_long cast in _opcode.stack_effect

c_long is i32 on Windows, where clippy reports the cast to i32 as
unnecessary.

Assisted-by: Claude Code:claude-opus-5-5

* Update malachite crates to 0.14.0

pymath requires malachite-bigint ^0, so the example projects, which
have no lock file, resolve it to 0.14.0 while the workspace pinned
0.13.0, and fail to build.

Assisted-by: Claude Code:claude-opus-5-5

---------

Co-authored-by: dot <dot@youknowone.org>
2026-10-14 22:38:42 +09:00
Leesoo Ahn e157f645a2 Fix grp.getgrnam raising OSError for unknown groups (#9101)
Some NSS modules (e.g. sss, systemd) return ENOENT from
getgrnam_r/getgrgid_r for a missing group. Treat any lookup error
as "not found" and raise KeyError, matching CPython. This fixes
test_shutil's test_chown and test_make_archive_owner_group.

Assisted-by: Antigravity:claude-opus-5-5
2026-10-14 09:36:18 +09:00
Jeong, YunWonanddot 10cb49430a Detach during blocking Windows multiprocessing socket I/O (#9093)
Snapshot locked send buffers before waiting and preserve the host-captured Winsock error across VM reattachment.

Assisted-by: OpenAI Codex: model identifier unavailable

Co-authored-by: dot <dot@youknowone.org>
2026-10-13 20:47:12 +09:00
Jeong, YunWonanddot 8f47ced2c7 Stabilize Windows all-thread traceback reads with stop-the-world (#9092)
Use the existing normal dump_traceback stop-the-world path on Windows before traversing raw interpreter frames. Preserve registry-lock release before resuming threads. Signal and watchdog paths are unchanged.

Assisted-by: OpenAI Codex:model identifier unavailable

Co-authored-by: dot <dot@youknowone.org>
2026-10-13 14:12:22 +09:00
youknowdot 8b66b7c48f Unify PyObjectRef ownership with extensible PyRef views (#9080)
Represent PyObjectRef through PyRef<PyObject> while preserving typed
ownership, payload projection, and drop behavior. Use the sealed target
selector for borrowed set/frozenset union views and typed tuple elements.

Keep tuple storage erased behind invariant typed slice views, with checked
conversions and reference-count ownership tests. Preserve exact tuple
identity for normalized nonempty full slices. Integrate the ownership
contract into the public object module documentation and retain focused
Rust ownership/layout tests without modifying bundled Lib sources.

Spell the set/frozenset union owner directly as PyRef<PyAnySet> without
a second owned alias; retain the sealed internal view registration.

Assisted-by: OpenAI Codex:model identifier unavailable
2026-10-13 14:11:37 +09:00
Jeong, YunWon c3e1276229 Release unused capacity after more short reads (#9089)
Call shrink_to_fit on the result buffer in os.pread, os.getrandom,
WindowsConsoleIO.read, _winapi.PeekNamedPipe, _winapi.ReadFile,
_multiprocessing.recv and the openssl MemoryBIO.read.

Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
2026-10-13 10:33:55 +09:00
1ndahous3 85d5956dbb Release unused capacity after short reads (#9028)
Assisted-by: Codex:gpt-6
2026-10-12 22:29:24 +09:00
youknowdot 5f3daa5386 Implement Python 3.15 base64 APIs (#9085)
* Implement Python 3.15 base64 encoding and decoding

Follow CPython 3.15.0 final d220409a13b77de7a6c30abe16ea90db01528edc for base64 options, alphabets, padding and canonical decoding. Keep reverse-table caches in traced module-native state and preserve module-bound callable semantics. Add bounded Python regressions without editing Lib.

Assisted-by: OpenAI Codex:model identifier unavailable

* Complete binascii codecs for canonical Python 3.15 tests

Migrate Lib/test/test_binascii.py unchanged from CPython 3.15.0 final using scripts/update_lib and replace the superseded custom base64 snippet. Implement the base32, Base85, Ascii85 and hex ignorechars APIs required by the canonical suite. Share the existing module-bound original-key alphabet cache across base32/base64/base85 and preserve argument and buffer cleanup semantics. Keep the existing base64 kernels and VM state integration unchanged.

Assisted-by: OpenAI Codex:model identifier unavailable
2026-10-12 16:12:24 +09:00
youknowdot 924ef2f434 Match hashlib constructor name type diagnostics (#9066)
Use native type names, None spelling, and UTF-8 byte precision for invalid name arguments without changing accepted strings or hashing behavior.

Assisted-by: OpenAI Codex:model identifier unavailable
2026-10-12 02:06:41 +09:00
youknowdot 4638b6a5f3 Raise IndexError for oversized SQLite blob indices (#9048)
Use index-sized conversion, preserve __index__ exceptions, and bound native type names to the UTF-8-safe diagnostic limit. Include in-memory blob read and write regressions without changing callback ownership.

Assisted-by: OpenAI Codex:model identifier unavailable
2026-10-12 01:10:08 +09:00
1ndahous3 a10744303f Release SQLite aggregate instances outside locks (#9014)
Assisted-by: Codex:gpt-6
2026-10-11 23:50:12 +09:00
youknowdot 2ab8af332f Preserve Expat qualified names without namespace expansion (#9070)
Retain raw element and attribute prefixes when namespace_separator is None. Keep expanded names and parser configuration unchanged, with regression coverage in the pyexpat snippet.

Assisted-by: OpenAI Codex:model identifier unavailable
2026-10-11 23:48:08 +09:00