Commit Graph
17299 Commits
Author SHA1 Message Date
Jeong, YunWon 7c6510334c Keep import module names as PyUtf8Str (#8690)
Add PyStr::as_utf8() for a non-erroring UTF-8 view. Thread
PyUtf8Str through import resolution, module attribute errors,
and IMPORT_FROM instead of converting names to owned String.

Assisted-by: Grok:4.6
2026-09-13 09:41:15 +09:00
Jeong, YunWon ce45fc24c7 Honor coroutine reuse, origin, and unawaited warnings (#8695)
Raise RuntimeError on send/throw of a closed coroutine, track
cr_origin, emit never-awaited warnings through the warnings helper,
and add cr_suspended. Stop treating coroutines as iterable. Warn on
unawaited asyncgen asend/athrow/aclose, match async for TypeError
messages, and attach coroutine method and name getset docs.

Assisted-by: Grok:grok-4.6
2026-09-13 08:17:17 +09:00
Jiseok CHOI 407ba437cd sqlite3: fix row_factory deletion error (#8693)
Match CPython 3.14 by distinguishing row_factory assignment from deletion and raising AttributeError with the compatible message. Unmark the passing row_factory deletion test.

Assisted-by: GitHub Copilot:GPT-5.6 Luna (high)
2026-09-12 15:51:06 +00:00
Jeong, YunWon a2e2bd299e asyncio: honor pending-task, unawaited, and get_referrers (#8689)
* asyncio: honor pending-task, unawaited, and get_referrers

Report Task destructor as 'Task was destroyed but it is pending!'.
Warn when a never-started coroutine is collected. Preserve signal()
errno. Attribute generator-owned frame locals to the owner in
gc.get_referrers. Remove the expectedFailure markers that now pass.

Assisted-by: Grok:grok-4.6

* tests: drop expectedFailure after unawaited warning

The never-started coroutine RuntimeWarning unblocks
test_bpo_45813_1 and the AsyncMock assertNeverAwaited cases.

Assisted-by: Grok:grok-4.6
2026-09-13 00:01:26 +09:00
Jeong, YunWon 66a3d9daa7 common: implement remaining builtin text codecs (#8687)
* common: implement remaining builtin text codecs

Move utf-16/32/7, unicode-escape, raw-unicode-escape and the
bytes escape transform into rustpython-common and call them from
_codecs instead of _pycodecs.

Assisted-by: Claude:grok-4.6

* host_env: share locale grouping, uuid and pipe state

Add localeconv_numeric so grouping keeps a CHAR_MAX terminator,
UuidCreateSequential for _uuid, and GetNamedPipeHandleStateW.

Assisted-by: Claude:grok-4.6

* Fix utf-7 clippy and the utf-16-be snippet length

Assisted-by: Claude:grok-4.6

* codecs: treat escape decode as final and accept str

unicode_escape_decode and raw_unicode_escape_decode default final to
true and accept a str as UTF-8. Fix clippy on wide_order, escape
encode, and encode closures. Remove expectedFailure markers that now
pass.

Assisted-by: Grok:grok-4.6
2026-09-12 15:51:28 +09:00
Jeong, YunWon 43ed0a80b6 Warn on invalid octal escapes and parse-fail strings (#8686)
* Warn on invalid octal escapes and parse-fail strings

Treat three-digit octal values above 0o377 as invalid escapes and
scan quoted literals even when parse fails, so SyntaxWarning is
emitted before a later SyntaxError. Keep warning and SyntaxError
text on InvalidEscape.

Enable the remaining test_string_literals invalid-escape cases.

Assisted-by: Grok:4.6

* Remove expectedFailure from passing test_codeop cases

test_invalid_warning and test_raw_raises_error now emit
SyntaxWarning on parse-fail invalid escapes.

Assisted-by: Grok:4.6

* Scan unparsed strings with token-boundary prefixes

Require a complete prefix at an identifier boundary, and walk
f-string interpolations separately after a parse failure.

Assisted-by: Grok:4.6
2026-09-12 13:14:01 +09:00
Lee Dogeon a582b938bd Exclude C-API tests from workspace coverage (#8619)
Assisted-by: Codex:gpt-5.6-sol
2026-09-12 09:28:09 +09:00
Jeong, YunWon cfee8a7ffa Fix -m argv[0] and file SyntaxError display (#8684)
* Fix -m argv[0] and file SyntaxError display

Set sys.argv[0] to -m for module execution. Convert an escalated
invalid-escape SyntaxWarning into a SyntaxError that spans just
the `\X` pair. Report null bytes in source files with filename,
lineno, and the truncated line.

Enable the four corresponding test_cmd_line_script cases.

Assisted-by: Grok:4.6

* Pass SyntaxError text when escalating escape warnings

Keep the warning and SyntaxError messages as separate strings at
the invalid-escape site. CompileWarningError carries an optional
replacement instead of rewriting the warning text later.

Assisted-by: Grok:4.6

* Remove expectedFailure from bytes invalid-escape test

The invalid-escape SyntaxError path now satisfies
test_eval_bytes_invalid_escape.

Assisted-by: Grok:4.6
2026-09-11 09:54:59 +09:00
Roshan Ramani 1345a82443 Report bytes out-of-range elements as CPython does (#8679)
* Report `bytes` out-of-range elements as CPython does

`collect_bytes` is shared by the `bytes` and `bytearray` entry points and
hardcoded one element error, so `bytes([256])` said "byte must be in
range(0, 256)" where CPython says "bytes". Every `bytearray` path does say
"byte", so only the one caller was wrong.

The element error is now a parameter, alongside the type error this
function already varies per caller. Verified against CPython 3.14 for all
seven paths.

Assisted-by: Claude Code:claude-opus-5

* Keep the bytearray wording in PyByteArray_FromObject

It called `bytes_from_object`, so the previous commit made it report
"bytes must be in range(0, 256)" where CPython's `PyByteArray_FromObject`
says "byte". It is the C-level bytearray constructor, so it now uses
`bytearray_from_object`, which carries that wording and the constructor's
unsized iteration.

Assisted-by: Claude Code:claude-opus-5
2026-09-11 09:54:27 +09:00
cui fliter 521b75ec4a cli: avoid panic when -X int_max_str_digits has no value (#8677)
Signed-off-by: cuishuang <imcusg@gmail.com>
2026-09-10 22:38:19 +09:00
Jeong, YunWon fb88ea834f ssl: honor hostname_checks_common_name and tls-unique (#8683)
Expose HOSTFLAG_NEVER_CHECK_SUBJECT and _host_flags so ssl.py can
disable Common Name fallback. HAS_TLS_UNIQUE matches the TLS 1.2
channel binding we already implement.

Assisted-by: Grok:grok-4.6
2026-09-10 22:37:46 +09:00
Hanjeong Lee 616cc259a8 Stop generating a __text_signature__ that inspect cannot parse (#8681)
* Leave Callee out of a generated __text_signature__

The interpreter supplies `callee` the way it supplies `vm`, but only `vm`
was filtered, so the argument surfaced as a Python parameter and left a
string inspect cannot parse:

    >>> object.__getstate__.__text_signature__
    '(*args, **kwargs, callee)'
    >>> inspect.signature(object.__getstate__)
    ValueError: builtin has invalid signature

Filter it in `func_sig`, and in `infer_native_call_flags`, which counted
it as a fixed positional argument. `object.__getstate__` is the only
#[pymethod] taking one today, inherited by 95 types.

Assisted-by: Claude Code:claude-opus-5

* Rename maketrans's first parameter to frm

`from` is a Python keyword, so the generated signature could not be
parsed:

    >>> bytes.maketrans.__text_signature__
    '(from, to, /)'
    >>> inspect.signature(bytes.maketrans)
    ValueError: builtin has invalid signature

CPython's Argument Clinic spells it `frm` for the same reason. Both
parameters are positional-only, so the name is documentation only.
`bytes.maketrans` and `bytearray.maketrans` now report what CPython
reports, `(frm, to, /)`.

Assisted-by: Claude Code:claude-opus-5
2026-09-10 15:45:54 +09:00
Jeong, YunWon 5afe140abd Store ContextVar.name as the original str object (#8682)
* Store ContextVar.name as the original str object

Keep the constructor argument instead of converting it to a Rust String
so lone surrogates and str subclasses are preserved. Build __repr__
from the name and default reprs.

Assisted-by: Grok:4.6

* Compare Context objects by their variable maps

Implement Context ==/!= from the mapping contents and mark Context
unhashable. Snapshot the map before value equality so re-entrant
ContextVar.set during __eq__ cannot deadlock.

Enable test_context.ContextTest.test_context_run_4.

Assisted-by: Grok:4.6
2026-09-10 13:46:20 +09:00
Jeong, YunWon 2fc2479f72 openssl: use host_env for CA paths and file checks (#8680)
Replace openssl-probe and local std::fs checks with host_env
native_certs and fs. Vendored OpenSSL loads system CA paths
from host_env; OpenSSL's own defaults stay for a system lib.

Assisted-by: Grok:grok-4.6
2026-09-09 20:18:00 +09:00
Jeong, YunWon 3fe3f37dac unicodedata: resolve name aliases and named sequences (#8676)
* unicodedata: resolve name aliases and named sequences

Vendor Unicode 17 NameAliases.txt and NamedSequences.txt into
rustpython-unicode so unicodedata.lookup matches CPython: formal
aliases and named sequences on the modern view, none on ucd_3_2_0.
Tangut derived-name lookup is case-insensitive. \N{} still rejects
named sequences because they are not a single character.

Assisted-by: Claude

* test_ucn: drop expectedFailure on test_aliases

The name-alias tables make the test pass; leaving the marker
fails CI as an unexpected success.

Assisted-by: Claude

* unicode: reject whitespace in Tangut derived names

parse_hex_code does not skip spaces around the hex digits.

Assisted-by: Claude
2026-09-09 11:47:16 +09:00
Jeong, YunWon 979576e2e4 ssl: move TlsConnection and session store to host_env (#8678)
TlsConnection, RecordCursor, rustls error mapping, hostname/ALPN/version
helpers, and the capturing client session store now live in host_env.
stdlib _ssl keeps socket/MemoryBIO I/O and Python objects.

Remove unused stdlib sha2 and leftover workspace TLS crates.

Assisted-by: Grok:grok-4.6
2026-09-09 04:04:16 +09:00
Roshan Ramani b701e5cbe9 Report which format string mistake was made (#8659)
* fix(vm): report which format string mistake was made

`ToPyException for FormatParseError` mapped two variants and sent the rest to
"Unexpected error parsing format string", so most format-string mistakes gave
no hint about the mistake. Against CPython 3.14.0:

| expression | before | CPython |
| --- | --- | --- |
| `"a}b".format()` | Unexpected error… | Single `}` encountered in format string |
| `"{0.}".format(1)` | Unexpected error… | Empty attribute in format string |
| `"{0[]}".format([1])` | Unexpected error… | Empty attribute in format string |
| `"{0[1}".format([1,2])` | Unexpected error… | expected `}` before end of string |
| `"{0[0]x}".format([1])` | Unexpected error… | Only `.` or `[` may follow `]` in format field specifier |
| `"{:{{}}".format(1)` | Unexpected error… | unmatched `{` in format spec |

The match is now exhaustive, so a new variant cannot silently inherit the
generic message.

Two cases are deliberately left generic, with the reason in a comment beside
each. `UnknownConversion` is raised both for a multi-character conversion
(`{0!xy}`, where CPython says "expected ':' after conversion specifier") and
for an empty one (`{0!}`, "unmatched '{' in format spec"), so one message
cannot serve both without the parser saying which it hit. And a trailing lone
`{` reaches `UnmatchedBracket` rather than
`UnescapedStartBracketInLiteral`, so `"{".format()` still differs from CPython;
that needs a parser change, not a message.

Expectations in the new snippet are CPython 3.14.0's output and pass under both
interpreters. Reverting only format.rs fails the snippet's first assertion.

Assisted-by: Claude Code:claude-opus-5

* fix(common): separate a stray `{` from an unclosed field

`parse_literal` already reports `UnescapedStartBracketInLiteral` for a brace
that was not doubled, but `from_str` discards it with `or_else` and retries as a
field, so `"{".format()` surfaced `UnmatchedBracket` and read as
"expected '}' before end of string". CPython separates the two by whether
anything follows the brace:

    "{"   "a{"    Single '{' encountered in format string
    "{s"  "a{b"   expected '}' before end of string
    "{0"          expected '}' before end of string

`parse_spec` now returns the literal error when the text is the brace alone,
which is also what makes `UnescapedStartBracketInLiteral` reachable — the
previous commit could map it but not trigger it.

`format_parse_lone_start_bracket` pins both directions at the parser level, and
the snippet covers the four messages under CPython 3.14.0 as well. The existing
`format_parse_fail` assertion for `"{s"` is unchanged.

Assisted-by: Claude Code:claude-opus-5
2026-09-08 17:52:57 +09:00
Jeong, YunWon bfd52eda61 ssl: move remaining rustls engine into host_env (#8675)
* ssl: move remaining rustls engine into host_env

Cert loading, rustls verifiers, ClientConfig/ServerConfig construction,
key logging, acceptor helpers, and X509 error mapping now live next to
the trust store. stdlib _ssl keeps the Python object and socket adapters.

Assisted-by: Claude

* Remove unused workspace SSL dependencies

Drop oid-registry, rustls-platform-verifier, and x509-cert from
workspace.dependencies.

Assisted-by: Grok 4.6
2026-09-08 13:39:44 +09:00
dependabot[bot] 63e0ff7ebf build(deps): bump is-macro from 0.3.7 to 0.3.8 (#8674)
Bumps [is-macro](https://github.com/dudykr/ddbase) from 0.3.7 to 0.3.8.
- [Commits](https://github.com/dudykr/ddbase/commits)

---
updated-dependencies:
- dependency-name: is-macro
  dependency-version: 0.3.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 01:35:56 +09:00
dependabot[bot] 860658b470 build(deps): bump github/gh-aw/actions/setup from 0.87.3 to 0.87.10 (#8673)
Bumps [github/gh-aw/actions/setup](https://github.com/github/gh-aw) from 0.87.3 to 0.87.10.
- [Release notes](https://github.com/github/gh-aw/releases)
- [Changelog](https://github.com/github/gh-aw/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/gh-aw/compare/466b8ad27fcfaaa861da3f16ba7c01272df25f46...ff62cdbec36230acbae869ddb28806e8eca01ea1)

---
updated-dependencies:
- dependency-name: github/gh-aw/actions/setup
  dependency-version: 0.87.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 01:35:48 +09:00
dependabot[bot] 5f0bcf6ca8 build(deps): bump lz4_flex from 0.13.1 to 0.14.0 (#8672)
Bumps [lz4_flex](https://github.com/pseitz/lz4_flex) from 0.13.1 to 0.14.0.
- [Release notes](https://github.com/pseitz/lz4_flex/releases)
- [Changelog](https://github.com/PSeitz/lz4_flex/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pseitz/lz4_flex/compare/0.13.1...0.14.0)

---
updated-dependencies:
- dependency-name: lz4_flex
  dependency-version: 0.14.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 01:35:40 +09:00
dependabot[bot] 79dad2b2c9 build(deps): bump thin-vec from 0.2.18 to 0.2.19 (#8671)
Bumps [thin-vec](https://github.com/mozilla/thin-vec) from 0.2.18 to 0.2.19.
- [Changelog](https://github.com/mozilla/thin-vec/blob/main/RELEASES.md)
- [Commits](https://github.com/mozilla/thin-vec/compare/v0.2.18...v0.2.19)

---
updated-dependencies:
- dependency-name: thin-vec
  dependency-version: 0.2.19
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 01:35:32 +09:00
dependabot[bot] c9f501e48d build(deps): bump writeable from 0.6.3 to 0.6.4 (#8670)
Bumps [writeable](https://github.com/unicode-org/icu4x) from 0.6.3 to 0.6.4.
- [Release notes](https://github.com/unicode-org/icu4x/releases)
- [Changelog](https://github.com/unicode-org/icu4x/blob/main/CHANGELOG.md)
- [Commits](https://github.com/unicode-org/icu4x/commits)

---
updated-dependencies:
- dependency-name: writeable
  dependency-version: 0.6.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 01:35:25 +09:00
dependabot[bot] ad7649ff44 build(deps): bump https://github.com/streetsidesoftware/cspell-cli (#8668)
Bumps [https://github.com/streetsidesoftware/cspell-cli](https://github.com/streetsidesoftware/cspell-cli) from v10.0.1 to 10.1.0.
- [Release notes](https://github.com/streetsidesoftware/cspell-cli/releases)
- [Changelog](https://github.com/streetsidesoftware/cspell-cli/blob/main/CHANGELOG.md)
- [Commits](https://github.com/streetsidesoftware/cspell-cli/compare/v10.0.1...v10.1.0)

---
updated-dependencies:
- dependency-name: https://github.com/streetsidesoftware/cspell-cli
  dependency-version: 10.1.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 01:34:40 +09:00
dependabot[bot] cf3594eb15 build(deps): bump taiki-e/install-action from 2.87.1 to 2.87.2 (#8667)
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.87.1 to 2.87.2.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/install-action/compare/742a3317eac7bd62f91cd888b4eead5e784ba833...1ed6d7be6168f6c9046541087ff549b6bc581fdf)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.87.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 01:34:30 +09:00
dependabot[bot] 4f72567d08 build(deps): bump zizmorcore/zizmor-action from 0.6.2 to 0.6.3 (#8666)
Bumps [zizmorcore/zizmor-action](https://github.com/zizmorcore/zizmor-action) from 0.6.2 to 0.6.3.
- [Release notes](https://github.com/zizmorcore/zizmor-action/releases)
- [Commits](https://github.com/zizmorcore/zizmor-action/compare/3dc1ecc9bcb9e94e9b2c709687979e1298497054...70fb788f84895a7701f5643d103d587e460b5c99)

---
updated-dependencies:
- dependency-name: zizmorcore/zizmor-action
  dependency-version: 0.6.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-08 01:34:21 +09:00
Jeong, YunWon 3a8a196996 ssl: share rustls engine in host_env and fire msg callbacks (#8665)
* ssl: fire msg callbacks and compute tls-unique

Observe TLS records to invoke _msg_callback, and derive TLS 1.2
tls-unique from the KeyLog master secret plus rustls PRF.
Restore CHANNEL_BINDING_TYPES and drop the remaining test_ssl
RUSTPYTHON skips.

Assisted-by: Claude

* lzma: build the xz-core engine on wasm32

Expose `_lzma` on wasm32 and Android by compiling the xz-core
engine on those targets instead of gating it out.

Assisted-by: Claude

* ssl: move the rustls engine into rustpython-common

Extract the VM-independent rustls helpers (MemoryBio, certificate
decode, cipher strings, OID table, chain builder, providers, and
TLS record observation) so stdlib _ssl wraps a shared engine.

Assisted-by: Claude

* ssl: keep the rustls engine in host_env

TLS helpers share the host trust store and default verify paths, so
they belong next to native_certs rather than in rustpython-common.

Assisted-by: Claude

* stdlib: drop unused jiff after moving cert decode

Assisted-by: Claude
2026-09-08 00:25:16 +09:00
Jeong, YunWon ae7bc5b619 lzma: build the xz-core engine on wasm32 (#8664)
Expose `_lzma` on wasm32 and Android by compiling the xz-core
engine on those targets instead of gating it out.

Assisted-by: Claude
2026-09-07 10:19:03 +09:00
Jeong, YunWon ac5fcb9be3 ssl: drop rustls lock before socket I/O (#8662)
Hold the rustls connection mutex only around rustls operations
in handshake, read, write, and shutdown. Socket and MemoryBIO
waits run without that lock so a blocking recv cannot deadlock
a concurrent send.

Remove the hang skip on test_thread_recv_while_main_thread_sends.

Assisted-by: Claude
2026-09-07 08:40:01 +09:00
Jeong, YunWon eee9ad6420 Align contextvars, CLI startup, and stat validation with CPython (#8657)
* Align contextvars, CLI startup, and stat validation with CPython

Validate Context and ContextVar arguments and mapping keys, hash names through the Python protocol, and preserve string contents without invoking __str__. Set the main module loader for command and stdin execution, report missing scripts with exit status 2, and reject file descriptors combined with follow_symlinks=False.

Enable the nine corresponding stdlib regression tests.

Assisted-by: Codex:gpt-6

* Move set_main_builtin_importer off the compiler feature

Define the method in vm_new.rs so builds without rustpython-compiler,
such as example_projects/barebone, can compile. Also set BuiltinImporter
on an existing __main__ when its loader is missing or None.

Assisted-by: Grok:4.6
2026-09-06 23:54:30 +09:00
Jeong, YunWon def52d06b1 ssl: cap rustls unsent output at 4 KiB (#8660)
Keep BIO writes from filling asyncio's socket transport past its
high-water mark when SO_SNDBUF is small. Drop the Unix skip on
test_sendfile_ssl_pre_and_post_data.

Assisted-by: Claude
2026-09-06 23:11:02 +09:00
Jeong, YunWon 96925f49f8 Update Ruff to 0.16.5 (#8617)
Pin the RustPython Ruff fork to the patch commit based on Ruff 0.16.5 and adapt compiler and AST integration to the updated APIs.

Assisted-by: Codex:GPT-5
2026-09-06 21:11:05 +09:00
Jeong, YunWon 531cf2b378 ssl: unify handshake, transport and shutdown state (#8656)
* ssl: accept ClientHello before selecting the SNI context

Replace the provisional server connection and replay flags with explicit handshake states. Retain fatal alerts across nonblocking retries and preserve fragmented input on timeout. Enable the three SNI alert regressions.

Assisted-by: Codex:GPT-5

* ssl: share socket and MemoryBIO transport progress

Queue drained TLS records before fallible output and use one handshake record loop for both transports.

Assisted-by: Codex:GPT-5

* ssl: unify shutdown and fatal alert lifecycle

Track close-notify progress in TlsState, preserve pending alerts across retries, and share record-boundary shutdown for sockets and MemoryBIO. Retain rustls errors until Python exception conversion.

Assisted-by: Codex:GPT-5

* ssl: emit the full peer certificate dict

Use the same certificate dict helper for getpeercert() and
Certificate.get_info() so OCSP, caIssuers, and CRL URLs are present.
Clear CHANNEL_BINDING_TYPES on the rustls backend, and add a MemoryBIO
leftover-plaintext snippet.

Assisted-by: Claude
2026-09-06 07:54:59 +09:00
Jeong, YunWon 7f9fa32d6e ssl: add key logging with a shared host append sink (#8655)
* ssl: add key logging with a shared host append sink

Port the context-owned keylog sink design from Pyre, retaining CPython 3.14 path and context-switch semantics. Keep host file lifecycle in host_env and TLS formatting in stdlib.

Assisted-by: Codex:GPT-5

* ssl: use core and alloc imports for key logging

Assisted-by: Codex:GPT-5
2026-09-05 21:21:47 +09:00
Jeong, YunWon e3bf84d1e7 Fix resource index conversion and REPL warning (#8654)
Accept __index__ values when converting resource limits and only report PyREPL fallback reasons for terminal-backed interactive sessions.

Assisted-by: Codex:gpt-5
2026-09-05 19:49:56 +09:00
Jeong, YunWon ad4532c0e0 ssl: retain WebPKI-selected certificate chains (#8653)
Port the connection-owned chain builder from Pyre. Preserve verification inputs with TLS configurations and sessions, report the WebPKI-selected path, and publish only its selected capath anchor. Match CPython chain reporting for CERT_NONE and resumed sessions.

Assisted-by: Codex:GPT-6
2026-09-05 19:35:23 +09:00
Roshan Ramani 4f55bde7c8 fix(vm): keep str's concat error when __radd__ declines (#8652)
`PyStr::__add__` delegated to the right operand's `__radd__` whenever it had
one, and returned whatever came back. For every numeric type that is a
`NotImplemented`, which then reached the generic binary-op handler:

    >>> "a" + 1
    TypeError: unsupported operand type(s) for +: 'str' and 'int'

CPython falls back to `str`'s `sq_concat` once the reflected call declines, so
it names the concatenation instead:

    TypeError: can only concatenate str (not "int") to str

The specific message was already here, but only reachable for operands with no
`__radd__` at all, so `list`, `object`, `bytes` and `None` were correct while
`int`, `float`, `bool` and any class with a declining `__radd__` were not.

A `__radd__` that returns a value still wins, unchanged.

Verified against CPython 3.14.0: the new snippet asserts the message for
`int`, `float`, `bool`, `list`, `None`, `bytes` and a declining `__radd__`,
and passes under both interpreters. Reverting only the `str.rs` change fails
its first assertion.

Assisted-by: Claude Code:claude-opus-5
2026-09-05 18:52:07 +09:00
Jeong, YunWon 25b5dfe5ca Fix /dev/fd, rlimit, spawn scheduler, and context cache (#8651)
* Fix /dev/fd, rlimit, spawn scheduler, and context cache

Honor PYTHONEXECUTABLE when computing sys.executable.
Rewind the file offset after a pyc magic probe.
Stop masking rlimit values with RLIM_INFINITY.
Print the pyrepl-unavailable warning before the rustyline shell.
Parse posix_spawn scheduler and apply it where the host supports it.
Update ContextVar cache fields under one lock.
Remove expectedFailure and skip markers for tests that now pass.

Assisted-by: Grok:4.6

* address review: check rlim_t overflow on convert

Replace the u64-to-rlim_t as-cast in py2rlim_limit with
a checked try_from, matching py2rlim.

Assisted-by: Grok:4.6
2026-09-05 17:57:29 +09:00
강동안 7376e96665 Raise MemoryError without allocating via a MemoryError freelist (#8645)
* Add no_memory_error for allocation failures

Assisted-by: Claude Code:claude-fable-5-1

* Use no_memory_error at allocation failure sites

Assisted-by: Claude Code:claude-fable-5-1

* Raise MemoryError instead of aborting on sequence repeat allocation failure

Vec::with_capacity and Vec::reserve in SequenceExt::mul and
SequenceMutExt::imul call handle_alloc_error on failure, which aborts
the process under panic = "abort". `[1, 2, 3] * 10**17` passes the
MAX_MEMORY_SIZE guard (2.4e18 bytes < isize::MAX) and aborted at the
allocation instead of raising MemoryError.

Use try_reserve_exact and map failures to vm.no_memory_error(). This
also removes the n != 2 special case in imul: with the reserve done
unconditionally, the copy loop runs zero times for n == 2 and the
final append uses the reserved capacity.

Assisted-by: Claude Code:claude-fable-5-1

* Recycle MemoryError instances through a dedicated payload freelist

no_memory_error allocated the exception object on every raise, so
under real memory exhaustion the raise itself could abort. Keep dead
MemoryError husks in a process-wide reserve and revive them on the
next raise without touching the allocator.

The freelist is scoped to MemoryError at the type level. PyMemoryError
already exists as a transparent wrapper payload, but instances never
used it: the default exception slot_new delegates to the base class,
so everything was constructed as PyBaseException. Route
no_memory_error through a PyMemoryError-typed construction instead;
the freelist hooks live on PyMemoryError's own PyPayload impl, the
dealloc glue is monomorphized per payload type, and the existing
exact-type push gate admits exactly MemoryError husks.

The #[pyexception] macro always generates the PyPayload impl, which
left no way to attach freelist configuration; a payload = "manual"
attribute now skips that generation so a type can write its own impl,
mirroring the generated PAYLOAD_TYPE_ID chain and class-based
downcast validation that keep the payload flavors interchangeable.

Exceptions are the first HAS_DICT freelist type, so their husks carry
an ObjExt prefix that PyInner::dealloc would normally drop; clear its
instance dict and member slots before pushing so no stale attributes
survive into a revived instance.

The reserve is a process-wide Mutex rather than a thread_local so a
single prewarm at interpreter init covers every thread; MemoryError
is a cold path, so the lock is uncontended. Python-level MemoryError()
still constructs a PyBaseException payload through the delegated
slot_new and simply does not participate, matching CPython, where
only the PyErr_NoMemory path uses the preallocated reserve.

Assisted-by: Claude Code:claude-fable-5-1

* Clear MemoryError husks before they enter the freelist

Assisted-by: Claude Code:claude-fable-5-1

* Use no_memory_error for the bz2 allocation failure

The shared bz2 engine maps BZ_MEM_ERROR to Bz2Error::Mem; report it
through no_memory_error like the other allocation-failure sites, matching
PyErr_NoMemory in CPython's _bz2module.

Assisted-by: Claude Code:claude-fable-5-1

* Change MemoryError husks Vec to a fixed-size array

Assisted-by: Claude Code:claude-fable-5-1

* Increase MemoryError freelist size from 4 to 16

* Address review: keep repeat and husk clearing allocation-safe

Assisted-by: Claude Code:claude-fable-5-1
2026-09-05 11:43:10 +09:00
Jeong, YunWon 699913d8cd Align codec, call, exception, and Unix module behavior with CPython (#8647)
* Collect consecutive charmap encoding failures

Pass one contiguous run of unmappable characters to the codec error handler, and use the full run when a text replacement is itself unmappable.

Assisted-by: Codex:gpt-5

* Leave conversions out of standalone format specs

A conversion belongs to a replacement field rather than the format specification consumed by format(), so stop accepting and storing it in FormatSpec.

Assisted-by: Codex:gpt-5

* Decode empty byte strings without looking up a codec

Match PyUnicode_FromEncodedObject by returning the shared empty string before resolving the requested encoding for empty bytes and bytearray inputs.

Assisted-by: Codex:gpt-5

* Validate call shape before converting builtin arguments

Reject unknown keywords and excess arguments before conversions can raise a different error. Align Python-function positional-only handling, use declared list arguments in heapq, keep delegated codecs positional-only, and preserve CPython parameter names and validation order.

Assisted-by: Codex:gpt-5

* Align attribute and exception initialization errors

Give __dict__ and type __qualname__ their CPython diagnostics, preserve OSError filename fields correctly, accept seven-item SyntaxError details, and report Unicode error constructor arity before converting values.

Assisted-by: Codex:gpt-5

* Expose missing Unix and Darwin module APIs

Add the platform constants exported by fcntl, mmap, select, socket, syslog, termios, stat, errno, time, and _sre. Implement posix.killpg and resource.getpagesize, and expose resource.error as OSError.

Assisted-by: Codex:gpt-5

* Honor the configured Python CPU count

Parse -X cpu_count and PYTHON_CPU_COUNT, expose the stored value through sys._get_cpu_count_config, and let os.cpu_count prefer it over the host count. Also publish getsizeof with its CPython parameter name.

Assisted-by: Codex:gpt-5

* Fix CPU count and WASI CI coverage

Remove the obsolete expected-failure marker now that CPU count overrides pass, and avoid exporting FASYNC where WASI libc has no O_ASYNC constant.

Assisted-by: Codex:gpt-5

* self review
2026-09-05 11:34:43 +09:00
Joshua Megnauth 9b5e550f51 Bump deps & fix deprecated Atomic::fetch_update (#8650)
AI disclosure: I linted all of the JIT changes with AI since I wanted to
be sure I followed cranelift's API properly.

Assisted-by: GPT Luna-5.6
2026-09-05 11:33:52 +09:00
Jeong, YunWonandIvan Mironov 4a267b2616 ssl: discover default certificate paths in host_env (#8649)
Keep host certificate path probing with the native certificate provider, select the first available OpenSSL-style file and directory, and leave environment overrides to ssl.py. This also gives Android string defaults instead of the previous None values.

Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>

Assisted-by: OpenAI Codex: GPT-5
2026-09-05 02:52:03 +09:00
Jeong, YunWonandIvan Mironov d67526cb08 ssl: load hashed capath certificates lazily (#8648)
Retain CA directories on SSLContext, materialize their OpenSSL-style hashed entries when building rustls configurations, and cache parsed DER until directory mtimes change. Publish only the trust anchor used by a completed handshake.

Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>

Assisted-by: OpenAI Codex: GPT-5
2026-09-05 02:49:32 +09:00
Joshua Megnauth 3442228b59 unicodedata: Fix name, unmask test_name (#8599)
This commit fixes most of UCD's names. The two main fixes are:

* Handling chars missing in 3.2.0 by checking for membership #8548
* Handling names not present in unicodenames

AI disclosure: I used AI for the initial research of this patch to
determine why the Tangut characters are missing. AI also assisted with a
bug fix where I neglected to parse the Tangut u32 as hex. I wrote all of
the code.

Assisted-by: GPT 5.6 Luna
2026-09-05 01:25:48 +09:00
Jeong, YunWonandIvan Mironov de6053099a ssl: take the OID table and the OpenSSL cipher-string parser from #8007 (#8646)
* ssl: read the OID table out of OpenSSL's object database

`txt2obj` and `nid2obj` have to answer with OpenSSL's own numbers, and the
table behind them was 39 entries transcribed by hand, so every object
outside that set was reported unknown.

Read OpenSSL's own files instead: `obj_mac.num` carries each object's NID
and `objects.txt` carries its OID, short name and description, and the
identifier `objects.pl` derives per object -- the `!Cname` override, else
the description, else the short name, each qualified by the enclosing
`!module` with `-` spelled `_` -- joins the two.  `OidEntry`, `OidTable`
and the `find_by_*` functions keep their shape, so `ssl.rs` only changes
where an object has no OID to report.

Three details the object database settles that a curated table did not
have to:

- an object written with one name carries it as both, so
  `nid2obj(48)` is `('emailAddress', 'emailAddress', ...)`;
- an object need not have an OID at all -- `DES-EDE3` and 139 other
  algorithm names have a NID and nothing to encode -- so the fourth
  field of the tuple is None for those, as it is on OpenSSL;
- one arc does not encode, since DER packs the first two into a byte,
  so the bare roots `iso` and `itu-t` are OID-less too, while two arcs
  such as X500's `2.5` are a real object.

Checked against `_ssl.nid2obj` on CPython built with OpenSSL, over every
NID it answers: 1387 of its 1390 present, with no disagreement on any of
them, up from 39.  The three absent are OpenSSL's unnamed placeholder
slots, which it reports as ('NULL', 'NULL', None).

The data files come from openssl/openssl `crypto/objects/`, via the
rustls integration rewrite in #8007.

Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
Assisted-by: Claude

* ssl: parse OpenSSL cipher strings in set_ciphers

Replaces the pattern matcher `set_ciphers` used with the grammar
`man openssl-ciphers` documents, in a new `ssl/cipher.rs` taken from
#8007: `!`, `-`, `+`, `@STRENGTH`, `@SECLEVEL=n`, `+`-joined
conjunctions, the DEFAULT/ALL/COMPLEMENTOF* groups, protocol versions,
SUITEB*, and the aXXX/kXXX/eXXX prefixes.

`set_ciphers` raises "No cipher can be selected." for a string that
names nothing the provider offers, which is the one failure
SSL_CTX_set_cipher_list reports, and keeps the TLS 1.3 suites selected
in provider order because that setter does not reach them. A SUITEB*
string pins its own key exchange groups, which `prepare_kx_groups`
now reads.

Against CPython's OpenSSL over 78 cipher strings, the selections agree
except under `@SECLEVEL=n`, which OpenSSL applies at handshake time and
this applies to the selection.

`CryptoExt` gains `default_cipher_suites` so that `DEFAULT` and `ALL`
are distinguishable.

Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
Assisted-by: Claude

* ssl: finish cipher reporting and provider handling

Use the provider-ordered cipher metadata table as the single source for parsing, reporting, OpenSSL names and descriptions. This makes get_ciphers reflect the context selection and validates curves and Suite B against the active provider.

Keep set_ciphers from changing TLS 1.3 suites, preserve the provider's order, fix empty conjunctions, and keep FIPS initialization within the provider's filtered cipher and key-exchange defaults.

Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>

Assisted-by: Claude

Assisted-by: CodeRabbit

Assisted-by: OpenAI Codex: GPT-5

* ssl: preserve OpenSSL OID lookup rules

Keep object-name lookup case-sensitive and canonicalize numeric OID arcs before consulting the generated OpenSSL object table.

Assisted-by: OpenAI Codex: GPT-5

* ssl: preserve cleartext after MemoryBIO shutdown

Read at most one TLS record while waiting for close_notify in BIO mode, leaving any post-TLS cleartext queued for the BIO owner.

Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>

Assisted-by: OpenAI Codex: GPT-5

* ssl: reject empty cipher conjunction operands

Keep empty operands visible to the parser so malformed OpenSSL cipher strings such as AES+ and ALL++AES are rejected instead of normalized.

Assisted-by: CodeRabbit

Assisted-by: OpenAI Codex: GPT-5

* ssl: make Suite B cipher ids unambiguous

Spell out the u16 conversion so Windows-only iterator implementations cannot make collection type inference ambiguous.

Assisted-by: OpenAI Codex: GPT-5

* ssl: move native certificate loading behind host_env

Keep platform trust-store loading and filesystem predicates on the host boundary while exposing backend-neutral certificate bytes to the rustls integration.

Assisted-by: OpenAI Codex: GPT-5

* ssl: invalidate server config after cipher changes

Rebuild cached server configuration after set_ciphers updates the selected suites or Suite B key-exchange groups.

Assisted-by: CodeRabbit

Assisted-by: OpenAI Codex: GPT-5

* ssl: load default certificate sources independently

Honor SSL_CERT_FILE and SSL_CERT_DIR independently. Keep directory certificates as lazy capath candidates, and fall back to native roots only when no environment certificate file was loaded.

Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>

Assisted-by: OpenAI Codex: GPT-5

---------

Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
2026-09-04 22:49:06 +09:00
Jeong, YunWon ba19382018 ci: hold a CodSpeed run to the machine main's baseline was measured on (#8644)
* ci: hold a CodSpeed run to the machine main's baseline was measured on

CodSpeed compares a run against a base run and refuses to call the
comparison meaningful when the two ran on different machines: the runner
uploads a `SystemInfo` record beside the profile carrying the OS, the
architecture, the CPU's vendor id, brand and flags, the physical core
count and the installed memory, and a hosted pool hands out whatever
machine is free. A simulated measurement is fixed against how fast the
machine is, but not against what it makes the process execute -- glibc
selects `memcpy` and the string routines through IFUNC on CPU flags, a
new stable rustc emits different code, and these benchmarks run each
program under CPython through pyo3 as well as under RustPython, so the
runner's default interpreter is an input too.

`scripts/codspeed-environment.py` records those fields plus the rustc,
libc and CPython versions, and compares a run against a recording. The
benchmark job now restores the recording main last wrote, records over it
when the ref is main, and compares against it otherwise; the install,
build and measure steps carry `if: steps.environment.outputs.match !=
'false'`, so a run that drew a different machine finishes green having
uploaded nothing. A missing or unfetchable recording is read as "nothing
to disagree with" rather than as a mismatch.

Assisted-by: Claude

* ci: read an unreadable environment recording as not a match

A cache miss, an eviction or a cache-service outage leaves the reference file
absent, and the comparison then reported `match=true`: every guarded step ran
and the run uploaded a measurement without having identified the machine it was
taken on.

An absent recording now reports `match=false`, and says so as a `::warning`
rather than the `::notice` a mismatch emits, so a job that measures nothing
because it has nothing to compare against is distinguishable from one that
measures nothing because the machine differs.

Assisted-by: Claude
2026-09-04 01:28:25 +09:00
Jeong, YunWon 6e6792ee5c common: share the hashlib engine, and publish blake2b/blake2s as types (#8643)
The digest, HMAC, PBKDF2 and scrypt state moves out of the `_hashlib`
module into a feature-gated `rustpython_common::hashlib` that owns each
state behind a `Mutex` and reports plain Rust results; `_hashlib` and
`_blake2` keep the object glue.

The engine writes its state through a raw pointer into a caller-owned
buffer, so the buffer is boxed and only the pointer travels with the
Python object, and it carries the alignment the engine asserts.  The
wrapper is built only after a successful init, so a rejected digest name
leaves nothing to drop over storage the engine never wrote.

blake2 moves from `blake2` to `blake2b_simd` / `blake2s_simd`, whose
`Params` carry the whole parameter block.  `_blake2.blake2b` and
`blake2s` become immutable types with `SALT_SIZE`, `PERSON_SIZE`,
`MAX_KEY_SIZE` and `MAX_DIGEST_SIZE` as class attributes, and their
instances are instances of those types.  They accept `key`, `salt`,
`person`, `fanout`, `depth`, `leaf_size`, `node_offset`, `node_depth`,
`inner_size` and `last_node`.  `_hashlib.scrypt` is published on the
same engine.

Six `test_hashlib` expectedFailure markers drop with it.  `blake2`,
`digest`, `dyn-clone`, `hmac` and `pbkdf2` lose their last consumer.

Assisted-by: Grok
Assisted-by: Claude
2026-09-03 22:50:41 +09:00
dependabot[bot] b3d348c711 build(deps-dev): bump fast-uri from 3.1.5 to 3.1.7 in /wasm/demo (#8641)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.5 to 3.1.7.
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.5...v3.1.7)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.7
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-03 22:27:28 +09:00
dependabot[bot] 29ebe5fb51 build(deps-dev): bump browserslist from 4.28.1 to 4.28.8 in /wasm/demo (#8640)
Bumps [browserslist](https://github.com/browserslist/browserslist) from 4.28.1 to 4.28.8.
- [Release notes](https://github.com/browserslist/browserslist/releases)
- [Changelog](https://github.com/browserslist/browserslist/blob/main/CHANGELOG.md)
- [Commits](https://github.com/browserslist/browserslist/compare/4.28.1...4.28.8)

---
updated-dependencies:
- dependency-name: browserslist
  dependency-version: 4.28.8
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-03 22:27:16 +09:00
dependabot[bot] d7e68aa766 build(deps-dev): bump postcss-selector-parser in /wasm/demo (#8636)
Bumps [postcss-selector-parser](https://github.com/postcss/postcss-selector-parser) from 7.1.0 to 7.1.5.
- [Release notes](https://github.com/postcss/postcss-selector-parser/releases)
- [Changelog](https://github.com/postcss/postcss-selector-parser/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss-selector-parser/compare/v7.1.0...7.1.5)

---
updated-dependencies:
- dependency-name: postcss-selector-parser
  dependency-version: 7.1.5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-03 22:27:13 +09:00