mirror of
https://github.com/RustPython/RustPython.git
synced 2026-10-28 18:59:04 +08:00
eee9ad642006f582d20ae805c2f84dfb328fdf04
17270
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jeong, YunWon
|
eee9ad6420 |
Align contextvars, CLI startup, and stat validation with CPython (#8657)
* Align contextvars, CLI startup, and stat validation with CPython Validate Context and ContextVar arguments and mapping keys, hash names through the Python protocol, and preserve string contents without invoking __str__. Set the main module loader for command and stdin execution, report missing scripts with exit status 2, and reject file descriptors combined with follow_symlinks=False. Enable the nine corresponding stdlib regression tests. Assisted-by: Codex:gpt-6 * Move set_main_builtin_importer off the compiler feature Define the method in vm_new.rs so builds without rustpython-compiler, such as example_projects/barebone, can compile. Also set BuiltinImporter on an existing __main__ when its loader is missing or None. Assisted-by: Grok:4.6 |
||
|
Jeong, YunWon
|
def52d06b1 |
ssl: cap rustls unsent output at 4 KiB (#8660)
Keep BIO writes from filling asyncio's socket transport past its high-water mark when SO_SNDBUF is small. Drop the Unix skip on test_sendfile_ssl_pre_and_post_data. Assisted-by: Claude |
||
|
Jeong, YunWon
|
96925f49f8 |
Update Ruff to 0.16.5 (#8617)
Pin the RustPython Ruff fork to the patch commit based on Ruff 0.16.5 and adapt compiler and AST integration to the updated APIs. Assisted-by: Codex:GPT-5 |
||
|
Jeong, YunWon
|
531cf2b378 |
ssl: unify handshake, transport and shutdown state (#8656)
* ssl: accept ClientHello before selecting the SNI context Replace the provisional server connection and replay flags with explicit handshake states. Retain fatal alerts across nonblocking retries and preserve fragmented input on timeout. Enable the three SNI alert regressions. Assisted-by: Codex:GPT-5 * ssl: share socket and MemoryBIO transport progress Queue drained TLS records before fallible output and use one handshake record loop for both transports. Assisted-by: Codex:GPT-5 * ssl: unify shutdown and fatal alert lifecycle Track close-notify progress in TlsState, preserve pending alerts across retries, and share record-boundary shutdown for sockets and MemoryBIO. Retain rustls errors until Python exception conversion. Assisted-by: Codex:GPT-5 * ssl: emit the full peer certificate dict Use the same certificate dict helper for getpeercert() and Certificate.get_info() so OCSP, caIssuers, and CRL URLs are present. Clear CHANNEL_BINDING_TYPES on the rustls backend, and add a MemoryBIO leftover-plaintext snippet. Assisted-by: Claude |
||
|
Jeong, YunWon
|
7f9fa32d6e |
ssl: add key logging with a shared host append sink (#8655)
* ssl: add key logging with a shared host append sink Port the context-owned keylog sink design from Pyre, retaining CPython 3.14 path and context-switch semantics. Keep host file lifecycle in host_env and TLS formatting in stdlib. Assisted-by: Codex:GPT-5 * ssl: use core and alloc imports for key logging Assisted-by: Codex:GPT-5 |
||
|
Jeong, YunWon
|
e3bf84d1e7 |
Fix resource index conversion and REPL warning (#8654)
Accept __index__ values when converting resource limits and only report PyREPL fallback reasons for terminal-backed interactive sessions. Assisted-by: Codex:gpt-5 |
||
|
Jeong, YunWon
|
ad4532c0e0 |
ssl: retain WebPKI-selected certificate chains (#8653)
Port the connection-owned chain builder from Pyre. Preserve verification inputs with TLS configurations and sessions, report the WebPKI-selected path, and publish only its selected capath anchor. Match CPython chain reporting for CERT_NONE and resumed sessions. Assisted-by: Codex:GPT-6 |
||
|
Roshan Ramani
|
4f55bde7c8 |
fix(vm): keep str's concat error when __radd__ declines (#8652)
`PyStr::__add__` delegated to the right operand's `__radd__` whenever it had
one, and returned whatever came back. For every numeric type that is a
`NotImplemented`, which then reached the generic binary-op handler:
>>> "a" + 1
TypeError: unsupported operand type(s) for +: 'str' and 'int'
CPython falls back to `str`'s `sq_concat` once the reflected call declines, so
it names the concatenation instead:
TypeError: can only concatenate str (not "int") to str
The specific message was already here, but only reachable for operands with no
`__radd__` at all, so `list`, `object`, `bytes` and `None` were correct while
`int`, `float`, `bool` and any class with a declining `__radd__` were not.
A `__radd__` that returns a value still wins, unchanged.
Verified against CPython 3.14.0: the new snippet asserts the message for
`int`, `float`, `bool`, `list`, `None`, `bytes` and a declining `__radd__`,
and passes under both interpreters. Reverting only the `str.rs` change fails
its first assertion.
Assisted-by: Claude Code:claude-opus-5
|
||
|
Jeong, YunWon
|
25b5dfe5ca |
Fix /dev/fd, rlimit, spawn scheduler, and context cache (#8651)
* Fix /dev/fd, rlimit, spawn scheduler, and context cache Honor PYTHONEXECUTABLE when computing sys.executable. Rewind the file offset after a pyc magic probe. Stop masking rlimit values with RLIM_INFINITY. Print the pyrepl-unavailable warning before the rustyline shell. Parse posix_spawn scheduler and apply it where the host supports it. Update ContextVar cache fields under one lock. Remove expectedFailure and skip markers for tests that now pass. Assisted-by: Grok:4.6 * address review: check rlim_t overflow on convert Replace the u64-to-rlim_t as-cast in py2rlim_limit with a checked try_from, matching py2rlim. Assisted-by: Grok:4.6 |
||
|
강동안
|
7376e96665 |
Raise MemoryError without allocating via a MemoryError freelist (#8645)
* Add no_memory_error for allocation failures Assisted-by: Claude Code:claude-fable-5-1 * Use no_memory_error at allocation failure sites Assisted-by: Claude Code:claude-fable-5-1 * Raise MemoryError instead of aborting on sequence repeat allocation failure Vec::with_capacity and Vec::reserve in SequenceExt::mul and SequenceMutExt::imul call handle_alloc_error on failure, which aborts the process under panic = "abort". `[1, 2, 3] * 10**17` passes the MAX_MEMORY_SIZE guard (2.4e18 bytes < isize::MAX) and aborted at the allocation instead of raising MemoryError. Use try_reserve_exact and map failures to vm.no_memory_error(). This also removes the n != 2 special case in imul: with the reserve done unconditionally, the copy loop runs zero times for n == 2 and the final append uses the reserved capacity. Assisted-by: Claude Code:claude-fable-5-1 * Recycle MemoryError instances through a dedicated payload freelist no_memory_error allocated the exception object on every raise, so under real memory exhaustion the raise itself could abort. Keep dead MemoryError husks in a process-wide reserve and revive them on the next raise without touching the allocator. The freelist is scoped to MemoryError at the type level. PyMemoryError already exists as a transparent wrapper payload, but instances never used it: the default exception slot_new delegates to the base class, so everything was constructed as PyBaseException. Route no_memory_error through a PyMemoryError-typed construction instead; the freelist hooks live on PyMemoryError's own PyPayload impl, the dealloc glue is monomorphized per payload type, and the existing exact-type push gate admits exactly MemoryError husks. The #[pyexception] macro always generates the PyPayload impl, which left no way to attach freelist configuration; a payload = "manual" attribute now skips that generation so a type can write its own impl, mirroring the generated PAYLOAD_TYPE_ID chain and class-based downcast validation that keep the payload flavors interchangeable. Exceptions are the first HAS_DICT freelist type, so their husks carry an ObjExt prefix that PyInner::dealloc would normally drop; clear its instance dict and member slots before pushing so no stale attributes survive into a revived instance. The reserve is a process-wide Mutex rather than a thread_local so a single prewarm at interpreter init covers every thread; MemoryError is a cold path, so the lock is uncontended. Python-level MemoryError() still constructs a PyBaseException payload through the delegated slot_new and simply does not participate, matching CPython, where only the PyErr_NoMemory path uses the preallocated reserve. Assisted-by: Claude Code:claude-fable-5-1 * Clear MemoryError husks before they enter the freelist Assisted-by: Claude Code:claude-fable-5-1 * Use no_memory_error for the bz2 allocation failure The shared bz2 engine maps BZ_MEM_ERROR to Bz2Error::Mem; report it through no_memory_error like the other allocation-failure sites, matching PyErr_NoMemory in CPython's _bz2module. Assisted-by: Claude Code:claude-fable-5-1 * Change MemoryError husks Vec to a fixed-size array Assisted-by: Claude Code:claude-fable-5-1 * Increase MemoryError freelist size from 4 to 16 * Address review: keep repeat and husk clearing allocation-safe Assisted-by: Claude Code:claude-fable-5-1 |
||
|
Jeong, YunWon
|
699913d8cd |
Align codec, call, exception, and Unix module behavior with CPython (#8647)
* Collect consecutive charmap encoding failures Pass one contiguous run of unmappable characters to the codec error handler, and use the full run when a text replacement is itself unmappable. Assisted-by: Codex:gpt-5 * Leave conversions out of standalone format specs A conversion belongs to a replacement field rather than the format specification consumed by format(), so stop accepting and storing it in FormatSpec. Assisted-by: Codex:gpt-5 * Decode empty byte strings without looking up a codec Match PyUnicode_FromEncodedObject by returning the shared empty string before resolving the requested encoding for empty bytes and bytearray inputs. Assisted-by: Codex:gpt-5 * Validate call shape before converting builtin arguments Reject unknown keywords and excess arguments before conversions can raise a different error. Align Python-function positional-only handling, use declared list arguments in heapq, keep delegated codecs positional-only, and preserve CPython parameter names and validation order. Assisted-by: Codex:gpt-5 * Align attribute and exception initialization errors Give __dict__ and type __qualname__ their CPython diagnostics, preserve OSError filename fields correctly, accept seven-item SyntaxError details, and report Unicode error constructor arity before converting values. Assisted-by: Codex:gpt-5 * Expose missing Unix and Darwin module APIs Add the platform constants exported by fcntl, mmap, select, socket, syslog, termios, stat, errno, time, and _sre. Implement posix.killpg and resource.getpagesize, and expose resource.error as OSError. Assisted-by: Codex:gpt-5 * Honor the configured Python CPU count Parse -X cpu_count and PYTHON_CPU_COUNT, expose the stored value through sys._get_cpu_count_config, and let os.cpu_count prefer it over the host count. Also publish getsizeof with its CPython parameter name. Assisted-by: Codex:gpt-5 * Fix CPU count and WASI CI coverage Remove the obsolete expected-failure marker now that CPU count overrides pass, and avoid exporting FASYNC where WASI libc has no O_ASYNC constant. Assisted-by: Codex:gpt-5 * self review |
||
|
Joshua Megnauth
|
9b5e550f51 |
Bump deps & fix deprecated Atomic::fetch_update (#8650)
AI disclosure: I linted all of the JIT changes with AI since I wanted to be sure I followed cranelift's API properly. Assisted-by: GPT Luna-5.6 |
||
|
Jeong, YunWonandIvan Mironov
|
4a267b2616 |
ssl: discover default certificate paths in host_env (#8649)
Keep host certificate path probing with the native certificate provider, select the first available OpenSSL-style file and directory, and leave environment overrides to ssl.py. This also gives Android string defaults instead of the previous None values. Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com> Assisted-by: OpenAI Codex: GPT-5 |
||
|
Jeong, YunWonandIvan Mironov
|
d67526cb08 |
ssl: load hashed capath certificates lazily (#8648)
Retain CA directories on SSLContext, materialize their OpenSSL-style hashed entries when building rustls configurations, and cache parsed DER until directory mtimes change. Publish only the trust anchor used by a completed handshake. Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com> Assisted-by: OpenAI Codex: GPT-5 |
||
|
Joshua Megnauth
|
3442228b59 |
unicodedata: Fix name, unmask test_name (#8599)
This commit fixes most of UCD's names. The two main fixes are: * Handling chars missing in 3.2.0 by checking for membership #8548 * Handling names not present in unicodenames AI disclosure: I used AI for the initial research of this patch to determine why the Tangut characters are missing. AI also assisted with a bug fix where I neglected to parse the Tangut u32 as hex. I wrote all of the code. Assisted-by: GPT 5.6 Luna |
||
|
Jeong, YunWonandIvan Mironov
|
de6053099a |
ssl: take the OID table and the OpenSSL cipher-string parser from #8007 (#8646)
* ssl: read the OID table out of OpenSSL's object database
`txt2obj` and `nid2obj` have to answer with OpenSSL's own numbers, and the
table behind them was 39 entries transcribed by hand, so every object
outside that set was reported unknown.
Read OpenSSL's own files instead: `obj_mac.num` carries each object's NID
and `objects.txt` carries its OID, short name and description, and the
identifier `objects.pl` derives per object -- the `!Cname` override, else
the description, else the short name, each qualified by the enclosing
`!module` with `-` spelled `_` -- joins the two. `OidEntry`, `OidTable`
and the `find_by_*` functions keep their shape, so `ssl.rs` only changes
where an object has no OID to report.
Three details the object database settles that a curated table did not
have to:
- an object written with one name carries it as both, so
`nid2obj(48)` is `('emailAddress', 'emailAddress', ...)`;
- an object need not have an OID at all -- `DES-EDE3` and 139 other
algorithm names have a NID and nothing to encode -- so the fourth
field of the tuple is None for those, as it is on OpenSSL;
- one arc does not encode, since DER packs the first two into a byte,
so the bare roots `iso` and `itu-t` are OID-less too, while two arcs
such as X500's `2.5` are a real object.
Checked against `_ssl.nid2obj` on CPython built with OpenSSL, over every
NID it answers: 1387 of its 1390 present, with no disagreement on any of
them, up from 39. The three absent are OpenSSL's unnamed placeholder
slots, which it reports as ('NULL', 'NULL', None).
The data files come from openssl/openssl `crypto/objects/`, via the
rustls integration rewrite in #8007.
Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
Assisted-by: Claude
* ssl: parse OpenSSL cipher strings in set_ciphers
Replaces the pattern matcher `set_ciphers` used with the grammar
`man openssl-ciphers` documents, in a new `ssl/cipher.rs` taken from
#8007: `!`, `-`, `+`, `@STRENGTH`, `@SECLEVEL=n`, `+`-joined
conjunctions, the DEFAULT/ALL/COMPLEMENTOF* groups, protocol versions,
SUITEB*, and the aXXX/kXXX/eXXX prefixes.
`set_ciphers` raises "No cipher can be selected." for a string that
names nothing the provider offers, which is the one failure
SSL_CTX_set_cipher_list reports, and keeps the TLS 1.3 suites selected
in provider order because that setter does not reach them. A SUITEB*
string pins its own key exchange groups, which `prepare_kx_groups`
now reads.
Against CPython's OpenSSL over 78 cipher strings, the selections agree
except under `@SECLEVEL=n`, which OpenSSL applies at handshake time and
this applies to the selection.
`CryptoExt` gains `default_cipher_suites` so that `DEFAULT` and `ALL`
are distinguishable.
Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
Assisted-by: Claude
* ssl: finish cipher reporting and provider handling
Use the provider-ordered cipher metadata table as the single source for parsing, reporting, OpenSSL names and descriptions. This makes get_ciphers reflect the context selection and validates curves and Suite B against the active provider.
Keep set_ciphers from changing TLS 1.3 suites, preserve the provider's order, fix empty conjunctions, and keep FIPS initialization within the provider's filtered cipher and key-exchange defaults.
Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
Assisted-by: Claude
Assisted-by: CodeRabbit
Assisted-by: OpenAI Codex: GPT-5
* ssl: preserve OpenSSL OID lookup rules
Keep object-name lookup case-sensitive and canonicalize numeric OID arcs before consulting the generated OpenSSL object table.
Assisted-by: OpenAI Codex: GPT-5
* ssl: preserve cleartext after MemoryBIO shutdown
Read at most one TLS record while waiting for close_notify in BIO mode, leaving any post-TLS cleartext queued for the BIO owner.
Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
Assisted-by: OpenAI Codex: GPT-5
* ssl: reject empty cipher conjunction operands
Keep empty operands visible to the parser so malformed OpenSSL cipher strings such as AES+ and ALL++AES are rejected instead of normalized.
Assisted-by: CodeRabbit
Assisted-by: OpenAI Codex: GPT-5
* ssl: make Suite B cipher ids unambiguous
Spell out the u16 conversion so Windows-only iterator implementations cannot make collection type inference ambiguous.
Assisted-by: OpenAI Codex: GPT-5
* ssl: move native certificate loading behind host_env
Keep platform trust-store loading and filesystem predicates on the host boundary while exposing backend-neutral certificate bytes to the rustls integration.
Assisted-by: OpenAI Codex: GPT-5
* ssl: invalidate server config after cipher changes
Rebuild cached server configuration after set_ciphers updates the selected suites or Suite B key-exchange groups.
Assisted-by: CodeRabbit
Assisted-by: OpenAI Codex: GPT-5
* ssl: load default certificate sources independently
Honor SSL_CERT_FILE and SSL_CERT_DIR independently. Keep directory certificates as lazy capath candidates, and fall back to native roots only when no environment certificate file was loaded.
Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
Assisted-by: OpenAI Codex: GPT-5
---------
Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
|
||
|
Jeong, YunWon
|
ba19382018 |
ci: hold a CodSpeed run to the machine main's baseline was measured on (#8644)
* ci: hold a CodSpeed run to the machine main's baseline was measured on CodSpeed compares a run against a base run and refuses to call the comparison meaningful when the two ran on different machines: the runner uploads a `SystemInfo` record beside the profile carrying the OS, the architecture, the CPU's vendor id, brand and flags, the physical core count and the installed memory, and a hosted pool hands out whatever machine is free. A simulated measurement is fixed against how fast the machine is, but not against what it makes the process execute -- glibc selects `memcpy` and the string routines through IFUNC on CPU flags, a new stable rustc emits different code, and these benchmarks run each program under CPython through pyo3 as well as under RustPython, so the runner's default interpreter is an input too. `scripts/codspeed-environment.py` records those fields plus the rustc, libc and CPython versions, and compares a run against a recording. The benchmark job now restores the recording main last wrote, records over it when the ref is main, and compares against it otherwise; the install, build and measure steps carry `if: steps.environment.outputs.match != 'false'`, so a run that drew a different machine finishes green having uploaded nothing. A missing or unfetchable recording is read as "nothing to disagree with" rather than as a mismatch. Assisted-by: Claude * ci: read an unreadable environment recording as not a match A cache miss, an eviction or a cache-service outage leaves the reference file absent, and the comparison then reported `match=true`: every guarded step ran and the run uploaded a measurement without having identified the machine it was taken on. An absent recording now reports `match=false`, and says so as a `::warning` rather than the `::notice` a mismatch emits, so a job that measures nothing because it has nothing to compare against is distinguishable from one that measures nothing because the machine differs. Assisted-by: Claude |
||
|
Jeong, YunWon
|
6e6792ee5c |
common: share the hashlib engine, and publish blake2b/blake2s as types (#8643)
The digest, HMAC, PBKDF2 and scrypt state moves out of the `_hashlib` module into a feature-gated `rustpython_common::hashlib` that owns each state behind a `Mutex` and reports plain Rust results; `_hashlib` and `_blake2` keep the object glue. The engine writes its state through a raw pointer into a caller-owned buffer, so the buffer is boxed and only the pointer travels with the Python object, and it carries the alignment the engine asserts. The wrapper is built only after a successful init, so a rejected digest name leaves nothing to drop over storage the engine never wrote. blake2 moves from `blake2` to `blake2b_simd` / `blake2s_simd`, whose `Params` carry the whole parameter block. `_blake2.blake2b` and `blake2s` become immutable types with `SALT_SIZE`, `PERSON_SIZE`, `MAX_KEY_SIZE` and `MAX_DIGEST_SIZE` as class attributes, and their instances are instances of those types. They accept `key`, `salt`, `person`, `fanout`, `depth`, `leaf_size`, `node_offset`, `node_depth`, `inner_size` and `last_node`. `_hashlib.scrypt` is published on the same engine. Six `test_hashlib` expectedFailure markers drop with it. `blake2`, `digest`, `dyn-clone`, `hmac` and `pbkdf2` lose their last consumer. Assisted-by: Grok Assisted-by: Claude |
||
|
dependabot[bot]
|
b3d348c711 |
build(deps-dev): bump fast-uri from 3.1.5 to 3.1.7 in /wasm/demo (#8641)
Bumps [fast-uri](https://github.com/fastify/fast-uri) from 3.1.5 to 3.1.7. - [Release notes](https://github.com/fastify/fast-uri/releases) - [Commits](https://github.com/fastify/fast-uri/compare/v3.1.5...v3.1.7) --- updated-dependencies: - dependency-name: fast-uri dependency-version: 3.1.7 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
29ebe5fb51 |
build(deps-dev): bump browserslist from 4.28.1 to 4.28.8 in /wasm/demo (#8640)
Bumps [browserslist](https://github.com/browserslist/browserslist) from 4.28.1 to 4.28.8. - [Release notes](https://github.com/browserslist/browserslist/releases) - [Changelog](https://github.com/browserslist/browserslist/blob/main/CHANGELOG.md) - [Commits](https://github.com/browserslist/browserslist/compare/4.28.1...4.28.8) --- updated-dependencies: - dependency-name: browserslist dependency-version: 4.28.8 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
d7e68aa766 |
build(deps-dev): bump postcss-selector-parser in /wasm/demo (#8636)
Bumps [postcss-selector-parser](https://github.com/postcss/postcss-selector-parser) from 7.1.0 to 7.1.5. - [Release notes](https://github.com/postcss/postcss-selector-parser/releases) - [Changelog](https://github.com/postcss/postcss-selector-parser/blob/main/CHANGELOG.md) - [Commits](https://github.com/postcss/postcss-selector-parser/compare/v7.1.0...7.1.5) --- updated-dependencies: - dependency-name: postcss-selector-parser dependency-version: 7.1.5 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Jeong, YunWon
|
aa59f1ee53 |
common: share the bzip2 stream engine (#8638)
Move the VM-independent bzip2 stream owner into a `bz2`-gated `rustpython-common` module beside the zlib and lzma engines, and keep `rustpython-stdlib` as the Python object and exception adapter. The adapter no longer names the `bzip2` crate; the `bz2` feature carries it. `_bz2` was the last user of the generic `DecompressState` / `DecompressStatus` / `Decompressor` machinery in `stdlib/src/compression.rs`, so that goes with it; `DecompressArgs` stays for zlib and lzma. The decompressor feeds the stream through the existing `Chunker`, so buffered and freshly supplied input are no longer joined into a new allocation first. `test_decompress_after_data_error` passes on the ported engine, so its `expectedFailure` marker goes. Assisted-by: Grok |
||
|
OkJa
|
93f151691c |
interpolated strings: align f-string and t-string diagnostics with CPython (#8601)
* builtins: name the other operand with tp_name in str and Template __add__
`str.__add__` and `Template.__add__` built their TypeError with
`PyType::name()`, which drops the module, where CPython formats `tp_name`:
>>> t"a" + "b"
TypeError: can only concatenate Template (not 'str') to Template
CPython 3.14 reports the qualified name and quotes the operand with double
quotes, as `Objects/unicodeobject.c` and `Objects/templateobject.c` do:
TypeError: can only concatenate string.templatelib.Template
(not "str") to string.templatelib.Template
`Template.__add__` also spelled its own name literally rather than taking it
from `PyClassDef::TP_NAME`, so the two halves of the message could drift apart.
Known gap: `slot_name()` still does not qualify a class defined in a module, so
`"a" + collections.OrderedDict()` names `OrderedDict` where CPython names
`collections.OrderedDict`. This affects both methods.
Unblocks test_template_concatenation in test_tstring.
Assisted-by: Claude Code:claude-opus-5
* interpolated strings: align f-string and t-string diagnostics with CPython
RustPython re-derives CPython's syntax diagnostics by scanning the source rather
than by translating ruff's parse errors, and the scanner shared by f-strings and
t-strings misread several replacement field states:
- A field that ran off the end of the literal (`f'{'`) was reported as an
expression that failed to start, where CPython asks for the closing brace.
- A field whose expression cannot start (`f'{;'`) and one whose expression runs
into a stray character (`f'{a;'`) both fell through to the missing brace
message instead of CPython's two distinct ones.
- A conversion or format spec following `=` was never validated, and a format
spec following a valid conversion was skipped entirely.
- Format specs were scanned with the `{{` escape rule that only applies to
literal text, so nested replacement fields inside them went unchecked.
- Brackets opened inside a field were never matched, so `f'{a[4)}'` and
`f'{3)+(4}'` fell back to a bare "invalid syntax".
- Comments inside a field were not recognised, so a `#` that swallows the
closing brace was reported as an unterminated literal, and a comment in a
multi-line field hid the expression behind it.
- Unterminated literals were named "string" whatever their prefix, and an
interpolated literal left with a replacement field open reported the missing
quote rather than the brace.
A field is only "never closed" while the tokenizer is still reading its
expression. Past the field's own `:` it is emitting literal text again, so
`f'{a:>5` runs out of input as an ordinary unterminated literal while `f'{a`
reports the brace. The field's own `:` is also not the one in a slice, a display
or a lambda, and an unclosed bracket inside the expression is what CPython names
rather than the field around it, so this walk tracks the whole delimiter stack.
Two messages were worded from the wrong branch of CPython's tokenizer. The hint
"perhaps you escaped the end quote?" is raised only where lexer.c handles a
literal without an interpolation prefix; its `%c-string` branch has just the
triple-quoted and plain forms, so pairing the hint with a prefix produced
`unterminated f-string literal (...); perhaps you escaped the end quote?`, which
CPython never emits. And a bracket mismatch inside a field dropped the
` on line %d` clause that lexer.c adds whenever `parenlinenostack[level]` differs
from the current `lineno`; the general bracket scanner in this file already
computed that suffix, so `f"""{a[\n4)}"""` now names line 1 as CPython does.
The rules these now follow are the ones CPython spells out in
Grammar/python.gram (`invalid_fstring_replacement_field` and its t-string twin)
and in Parser/lexer/lexer.c, which likewise parameterises the literal's prefix
rather than duplicating the messages.
Only a field's expression is code. A format spec is text, so a `(` there opens
nothing and a `#` selects the alternate form instead of starting a comment;
inside the expression a `#` does start one. Scanning the whole field for either
therefore reported diagnostics against valid literals, and because these
scanners only run once the source has already failed to parse, that let a good
literal take the blame for an error further down the file. Expression-level
checks now stop at the field's top-level separator and skip comments, and the
literal is walked field by field rather than byte by byte so that a `{` inside a
comment no longer opens one. A pre-existing instance of the same bug in the
line-continuation check is fixed along with them.
Ruff reports a mixed literal concatenation only as a bytes/non-bytes mix, so
`t"x" b"y"` arrived as a bytes error where CPython names the t-string. CPython's
`invalid_string_tstring_concat` is an `invalid_` rule, reached only on the error
pass, and `strings` tries `(fstring|string)+` ahead of it: that alternative
consumes the concatenation's leading run of non-t-string literals, so a mix among
those raises from `_PyPegen_concatenate_strings` on the first pass and sets
`error_indicator`, which suppresses the t-string rule. The t-string message
therefore wins for `t"x" b"y"` and the bytes message keeps precedence for
`"a" b"b" t"c"`. Both are now settled here rather than left to whichever scanner
runs next, which had been answering the second case with an unrelated
"Is this intended to be part of the string?".
A leading doubled `=` or `!` was read as a marker with an empty expression before it,
so `f"{==a}"` reported `valid expression required before '='` where CPython has no
expression to report at all. The same lookahead now guards that branch, and a
doubled `=` or `!` counts as an expression that cannot start.
A top-level `=` or `!` was read as a debug or conversion marker without checking
whether it belonged to a longer operator, so `f"{a==b}"` and `f"{a!=b}"` were also
read as fields ending early. CPython tokenises `==`, `!=`, `<=` and the rest as
single tokens before it ever considers the debug marker, so the separator scan now
skips a `=` that follows one of `= ! < > + - * / % & | ^ @ :` or precedes another
`=`, and a `!` that precedes one.
An expression cannot end on an operator that still wants an operand, and CPython
points at that operator rather than at the brace. `f"{a==}"`, `f"{a and}"` and
`f"{a.b.}"` fell through to a plain `invalid syntax`; they now carry the same
message and column CPython gives, reusing the scan that already handled `;` and
`$` for exactly this shape. `is not` and `not in` are single operators, so the
first word is what gets pointed at, and `...` is consumed in whole triples so
that `f"{....}"` blames the fourth dot rather than the first. The region has to
be walked forwards: a comment's terminating newline is whitespace, so trimming
backwards from the end would step into the comment body and read a triple-quoted
field whose comment ends in `+` as an expression ending in `+`.
That check runs only once the field is known to close, which is where a stray
character and a dangling operator part ways. A stray character is a finished
token, so the parser rejects it on the lookahead in
`annotated_rhs !('='|'!'|':'|'}')` and `f"{a;"` gets the separator message even
with no closing brace. A dangling operator instead makes the parser ask for one
more token, and producing it runs into the literal's closing quote, where
lexer.c answers from its `INSIDE_FSTRING(tok)` branch with
`%c-string: expecting '}'` before any `invalid_` rule is reached. So `f"{a and"`
is a missing brace while `f"{a and}"` names the operator.
A leading `.` was read as a character that cannot start an expression, so the
Ellipsis literal `f"{...}"`, the float `f"{.5}"` and its signed form `f"{-.5}"`
were reported as broken whenever the file failed to parse somewhere else.
`pegen` joins the cpython spelling dictionary, for the `_PyPegen_*` names these
comments cite.
Known gaps, none covered by either test file. Needing the longest valid expression
prefix, which a character scanner cannot compute: a starred tuple (`f"{*a,}"`) and
a dict-unpacking display are read as unable to start an expression, and
`f"{a===b}"`, `f"{a b}"` and `f"{a,,}"` fall through to a bare `invalid syntax`.
The caret still differs from CPython's on `expecting a valid expression after '{'`,
`expecting '}'`, `expecting '}', or format specs` and `unterminated ... literal`,
and on the messages CPython points at a whole token for, such as
`f"{lambda x: x}"` and `f"{x! r}"`. A quote inside a format spec is still treated
as a string delimiter.
Two more are reachable only through a t-string concatenation that ruff does not
report as a bytes mix, so `mixed_tstring_literal_error` never runs: three or more
literals with no bytes literal among them (`t"a" t"b" "c"`, `"a" "b" t"c"`) answer
with `invalid syntax. Is this intended to be part of the string?`, and so does a
tokenizer-level nesting overflow where CPython has
`too many nested f-strings or t-strings`. Two-literal mixes are correct because
ruff's own error carries the wording.
Two pre-existing bugs outside this change are worth naming, since it touches
their neighbourhood. `unterminated triple-quoted ... literal` reports
`detected at line 2` for a one-line source where CPython reports line 1, for every
prefix including none, so it is in the shared line arithmetic rather than the
interpolated path. And `str.__add__`'s message is unreachable for an operand that
defines `__radd__`: `"a" + 1` falls through to the generic
`unsupported operand type(s)` from the binop dispatch, where CPython has
`can only concatenate str (not "int") to str`.
Unblocks test_syntax_errors and test_literal_concatenation in test_tstring, and
test_comments,
test_conversions, test_invalid_syntax_error_message, test_mismatched_braces,
test_mismatched_parens, test_parens_in_expressions and
test_syntax_error_after_debug in test_fstring.
Assisted-by: Claude Code:claude-opus-5
* diagnostics: give the source scanners a diagnostic type
Every scanner in `cpython_parse_diagnostic_override` returned a bare
`Option<(String, usize, usize)>` — 69 signatures and 114 construction sites of
message, start offset, end offset — which the consumer then reassembled into a
message and a range one call later. They return a `CpythonDiagnostic` now, and
the two things that consume one, `NormalizedParseDiagnostic::other` and
`CompileError::from_source_error`, take it whole.
The type earns its name. These scanners run *after* ruff's parse has failed, so
what they produce is not a parse error but a reconstruction of what CPython
would have said about the same source; nothing here ever hands one back to ruff.
Reusing `ruff_python_parser::ParseError` would have fit the shape — it is
`{ ParseErrorType, TextRange }` — but every one of the 114 sites would have
wrapped its message in the single `OtherError` variant of a ninety-variant enum,
widened `other` and `from_source_error` to accept variants no caller
constructs, and, being a foreign type, ruled out the constructor that now
carries the `u32` cast and its justification. The `OtherError` wrapping happens
once per consumer instead, at the boundary where this does cross into ruff's
vocabulary.
Two nameless tuples that carried more than a diagnostic get names, following
`CallArgFrame` and `AssignmentContext` in this file:
- `bracket_syntax_error` returned the message alongside a bare `bool` for
whether the bracket was left open, which the caller needs apart from the
message because ruff reports that case as an EOF error. It returns a
`BracketError` now.
- `unclosed_replacement_field_error` walked its delimiter stack as
`Vec<(usize, u8, bool)>`, where the third element decided whether the field
had reached its format spec — `.2 = true` and `Some((_, b'{', false))` at the
use sites. `OpenDelimiter` names all three.
The `source_error!` macro is untouched at the 34 call sites it already had and
drops to five lines. Two copies of its body had been written out by hand
(`invalid_number_literal_error`, `unterminated_string_error`) and fold into it,
bringing it to 36.
One thing does change beyond the packaging. `TextRange::new` asserts that its
start does not exceed its end, and it asserts unconditionally. Building the
range at the scanner rather than at the consumer therefore puts that assert in
front of the hundred-odd scanners that reach `NormalizedParseDiagnostic::other`,
which never built a range before — they handed their two offsets straight to
`source_locations`. A scanner that emitted a reversed span used to produce a
garbled location; now it aborts. Every site was read for this and each clamp is
anchored to a bound at or after its start, and neither two hundred thousand
generated sources nor eleven thousand mutations of stdlib files reached it, so
this is a latent invariant made loud rather than a new failure — but it is not
packaging, so it is written down here.
Otherwise a pure refactor: no diagnostic changes message, column or line.
Verified by building the parent commit and diffing its output against this one
over a malformed-source corpus and over every `Lib/**/*.py` that parses clean
(1728 files, each recompiled with a trailing `$` so a scanner that misreads
valid code shows up as a moved line) — byte-identical, and the count of files
where the reported line moves is 1149 on both. test_fstring, test_tstring,
test_string_literals, test_syntax, test_exceptions, test_grammar and
test_compile all pass.
Addresses the review on #8601.
Assisted-by: Claude Code:claude-opus-5
* diagnostics: name the operator-character set behind the debug marker
`is_replacement_field_marker` carried its list of operator characters inline, a
sixteen-line `matches!` wrapped around three lines of logic, and it looked very
much like `is_dangling_operator_byte` a few functions down — enough that a
reviewer asked whether they were the same set.
They are not, and neither contains the other. Written in the same order the
difference is three characters:
precedes_equals_in_one_operator ! % & * + - / : < = > @ ^ |
is_dangling_operator_byte ! % & * + - . / < = > @ ^ | ~
The two are derived from different things. One asks which bytes pair with a
following `=` to make a single token, so that the `=` is part of that operator
rather than the start of a debug specifier — read off Python's token table, and
`.=` and `~=` are not on it, while `:=` is. The other asks which bytes make up
an operator that still wants an operand, which is why `~a` and `a.b.` put `.`
and `~` there and why the separator `:` stays out.
So the list moves next to the one it resembles, under a name, both sorted the
same way, and each says what the other has that it does not. Neither is defined
in terms of the other: the thirteen characters they share are two answers that
happen to coincide, not a set with a meaning of its own, and a common list would
tie each to the other's reasons to change.
No behavior change: the extracted list is character for character what was
inline. Verified against a build of the parent over a malformed-source corpus
and every `Lib/**/*.py` that parses clean — byte-identical — and with
test_fstring, test_tstring, test_string_literals, test_syntax, test_exceptions,
test_grammar and test_compile.
Addresses the review on #8601.
Assisted-by: Claude Code:claude-opus-5
* diagnostics: give the unterminated-literal scan one exit
A single-quoted literal that meets a newline and one that runs off the end of
the source are the same failure, and CPython's lexer says so in one condition:
if (c == EOF || (quote_size == 1 && c == '\n')) {
Everything after that — the check for a replacement field left open, and the
choice of message — is written once there. This scan had grown a second exit
for the newline case with its own copy of both, so the guard added for
interpolated literals had to be added twice and the message assembled twice.
The newline case now breaks out of the scan and falls into the exit that was
already handling end of source. `line` cannot have moved from `start_line` on
that path, and `quote_size` is 1, so the surviving exit computes the same
`detected_line` and the same `triple` the deleted one did.
No behavior change: verified against a build of the parent over a
malformed-source corpus and every `Lib/**/*.py` that parses clean —
byte-identical — and with test_fstring, test_tstring, test_string_literals,
test_syntax, test_exceptions, test_grammar and test_compile.
Assisted-by: Claude Code:claude-opus-5
* diagnostics: classify an unterminated triple-quoted t-string as incomplete input
`analyze_compile_error` pairs the f-string and t-string error variants
everywhere it names a message, but the match that decides whether a failure is
a `SyntaxError` or an `IncompleteInputError` listed only the f-string one. So a
`t'''` typed at the prompt would not have asked for another line where an
`f'''` would.
Would not have, rather than does not: the arm is unreachable today. The
compiler's `unterminated_string_error` scanner claims these first and hands
back an `OtherError`, so `f'''` and a bare `'''` get a `SyntaxError` at the
prompt too, on `main` as much as here — which is why
`test_codeop.test_incomplete` carries an `expectedFailure`. Whoever untangles
that precedence should not then find that f-strings work and t-strings
silently do not.
CPython does not separate the two either: `lexer.c` emits one message
parameterised with `%c` for the prefix and sets `E_EOFS` the same way for both,
and `_is_end_of_source` (`Parser/pegen.c`) looks only at that code, never at
which kind of literal produced it.
No behavior change, and none is testable while the arm cannot be reached.
Assisted-by: Claude Code:claude-opus-5
|
||
|
Joshua Megnauth
|
7ceada0048 |
Dedupe Unix & WASI stdlib (#8635)
Follow up to #8591 because I figured out how to deduplicate the code. This is nicer for future patches that improve WASI support because Unix-likes and WASI can often share the same dispatch functions. |
||
|
Jeong, YunWon
|
39682562b3 |
Raise the shared argument-binding errors from the slots that count for themselves (#8634)
* Raise the shared argument-binding errors from the slots that count for themselves
Sites all over the tree checked their own argument counts and keywords and
wrote the message by hand, so they drifted: `attrgetter expected 1 argument,
got 0.` carried a full stop, `TypeVar() got unexpected keyword argument(s):
bogus` named neither the keyword the way its counterpart does nor the
function the way `typevar()` is named, and `_csv`'s dialect parser handed a
whole formatted sentence to `InvalidKeywordArgument`, which then wrapped it
in another one.
`Callee::arity_error` and `Callee::unexpected_keyword` are now public, and
these sites raise through them, so there is one place the wording lives.
`FrameLocalsProxy` counts its arguments before reading its keywords, and
`min`/`max` do too, which is the order `framelocalsproxy_new` and `min_max`
check in. `weakref.ref`, `GenericAlias`, `frozenset`, `attrgetter`,
`itemgetter`, `islice`, `start_new_thread`, `TextIOWrapper`, `AttributeError`,
`NameError`, `TypeVar`, `ParamSpec`, `TypeVarTuple` and `TypeAliasType` all
follow.
Of 25 calls measured against CPython 3.14.6, 21 now produce the identical
string. The four left over need what `Callee` cannot carry: `weakref.proxy`
is a function of its own there rather than a type, `frozenset` names the
subclass being constructed, and `TextIOWrapper` names its first parameter
and counts before converting.
Assisted-by: Claude
* Give the argument-binding errors the constructors the rest of them have
Raising one of these meant writing `Callee::of::<Self>(vm).arity_error(1..=3,
0, vm)` — the vm twice, and a type to know about before you could say what
you meant. Every other error in the tree is a `vm.new_*_error`.
So the message texts move out of `Callee` into functions the vm can call, and
the vm gets `new_arity_type_error` and `new_unexpected_keyword_type_error`
alongside `new_unsupported_bin_op_error` and the rest. A slot names itself
with the `NAME` its class already carries:
return Err(vm.new_arity_type_error(Self::NAME, 1..=3, 0));
return Err(vm.new_unexpected_keyword_type_error(Some("typevar"), &key));
The keyword one takes the name as an `Option` because the parser sometimes
has none of its own, which is what `_PyArg_Parser.fname` being NULL means and
what `_csv`'s dialect parser wants.
`bind_for` and `check_kwargs_empty_for` take anything a `Callee` converts
from, so the slots pass `Self::NAME` there too. `Callee` itself is left to the
binding machinery, which is the only place that needs to carry a name around
rather than use one.
No message changes: 22 of 26 calls still match CPython 3.14.6 exactly, and the
43-call set still stands at 23.
Assisted-by: Claude
|
||
|
dependabot[bot]
|
cfdbffae75 |
build(deps): bump libsqlite3-sys from 0.38.1 to 0.38.2 (#8546)
Bumps [libsqlite3-sys](https://github.com/rusqlite/rusqlite) from 0.38.1 to 0.38.2. - [Release notes](https://github.com/rusqlite/rusqlite/releases) - [Changelog](https://github.com/rusqlite/rusqlite/blob/master/Changelog.md) - [Commits](https://github.com/rusqlite/rusqlite/commits) --- updated-dependencies: - dependency-name: libsqlite3-sys dependency-version: 0.38.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
00ef37e346 |
build(deps): bump base64 from 0.22.1 to 0.23.1 (#8545)
Bumps [base64](https://github.com/marshallpierce/rust-base64) from 0.22.1 to 0.23.1. - [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md) - [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.22.1...v0.23.1) --- updated-dependencies: - dependency-name: base64 dependency-version: 0.23.1 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Jeong, YunWon
|
c716a7d070 |
common: share the zlib stream engine (#8631)
* common: share the zlib stream engine Move the VM-independent zlib-rs stream owner and state machine into a feature-gated rustpython-common module. Make rustpython-stdlib adapt that engine instead of carrying its own flate2 implementation, including native stream copy/deepcopy, Z_BLOCK flushes, and wbits=0 decompression. Assisted-by: Codex:GPT-5 * common: share the lzma stream engine Move the VM-independent liblzma stream owner, filter properties, and compression state into a target-gated rustpython-common module. Keep rustpython-stdlib as the Python object and exception adapter, and remove its direct xz dependencies and lzma-only generic compressor machinery. Assisted-by: Codex:GPT-5 * common: address compression engine reviews |
||
|
Lee Dogeon
|
29c622ca47 |
Document separate C-API Clippy checks (#8615)
Assisted-by: Codex:gpt-5.6-sol |
||
|
Lee Dogeon
|
99a63f92aa |
benchmarks: add pyperformance workloads (#8633)
Add dependency-free CPU-bound workloads adapted from pyperformance 1.14.0 to cover interpreter, standard-library serialization, regex, numeric, and garbage-collection paths under the existing Criterion and CodSpeed harnesses. Keep inputs deterministic and scale expensive cases for simulation while checking that benchmark execution does not leak interpreter state. Assisted-by: Codex:gpt-5.6-sol |
||
|
Joshua Megnauth
|
7ec2682153 |
ucd: Rewrite parser to be faster & more flexible (#8611)
Related: #8599 My original parser was monolithic and inflexible. It worked as intended for the derived data files, but anything more complicated required hacky code. For example, the original parser always expected to build a vector of values, yet sometimes we required other data structures such as BTreeMaps. I split up the parser into helper functions that are both cleaner and more flexible. The actual parser should still yield the same results - this doesn't "fix" anything yet. However, this is preliminary work for fixing more of UCD since we will need a more flexible parser for fixes like the linked PR. |
||
|
Jeong, YunWon
|
6aac4653e6 |
Fix post-merge subinterpreter lifecycle issues (#8632)
Preserve logical memoryview descriptors across interpreter transfers, make channel-end release idempotent, and scope runtime-owned interpreter cleanup to the creating top-level runtime. Re-read owned interpreters during shutdown so finalizers cannot leave newly-created children behind. Add regression coverage for sliced and cast memoryviews, repeated channel release, and multiple embedded runtimes. Assisted-by: Codex:GPT-5 |
||
|
Jeong, YunWon
|
516a116f9a |
Name the function a failed argument binding was binding for (#8630)
* Name the function a failed argument binding was binding for A built-in call that binds badly reported neither the function nor, for a method, the right numbers: `self` was prepended before binding, so it was counted as both an expected parameter and a given argument. `[].append()` read `expected at least 2 arguments, got 1`. `Callee` carries what a message says about the function: the name, and whether the leading argument fills the instance parameter. `PyNativeFn` takes one, `PyNativeFunction` and `PyMethodDescriptor` supply it, and the four method-descriptor call specializations in `frame.rs` do the same. `bind_for` subtracts the instance from both numbers. The messages now read as their counterparts do: - `_PyArg_CheckPositional`: `find expected at least 1 argument, got 0`, with the singular when the count is one. - `_PyArg_UnpackKeywords`: `split() got an unexpected keyword argument 'bogus'`, and `cast() missing required argument 'format' (pos 1)` for a parameter a call may pass by name. A positional-only parameter is only ever counted, so it keeps the first form. A binding that happens where the name isn't known leaves it off, the way `_PyArg_Parser.fname` is NULL. `test_find_etc_raise_correct_error_messages` now passes in both `test_bytes` and `string_tests`. Assisted-by: Claude * Name the type whose constructor or initializer bound the arguments `Constructor::slot_new` and `Initializer::slot_init` bound their arguments without a name, and so did the slots that override them, so a constructor that was called wrongly said only `expected 1 argument, got 0`. Both defaults and every override now name the type the slot was written for, not the subclass being constructed and not the module `tp_name` carries: `float expected at most 1 argument, got 2`, and `deque expected at most 2 arguments, got 3` for a `deque` subclass. `slice()` and `range()` counted their own arguments and raised messages of their own; they now raise the one `_PyArg_CheckPositional` raises, which makes `test_range_constructor_error_messages` pass. Assisted-by: Claude |
||
|
dependabot[bot]
|
d9d3b6bc7a |
build(deps): bump https://github.com/astral-sh/ruff-pre-commit (#8625)
Bumps [https://github.com/astral-sh/ruff-pre-commit](https://github.com/astral-sh/ruff-pre-commit) from v0.16.3 to 0.16.4. - [Release notes](https://github.com/astral-sh/ruff-pre-commit/releases) - [Commits](https://github.com/astral-sh/ruff-pre-commit/compare/v0.16.3...v0.16.4) --- updated-dependencies: - dependency-name: https://github.com/astral-sh/ruff-pre-commit dependency-version: 0.16.4 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
afb19d6239 |
build(deps): bump github/gh-aw/actions/setup from 0.86.2 to 0.87.3 (#8626)
Bumps [github/gh-aw/actions/setup](https://github.com/github/gh-aw) from 0.86.2 to 0.87.3. - [Release notes](https://github.com/github/gh-aw/releases) - [Changelog](https://github.com/github/gh-aw/blob/main/CHANGELOG.md) - [Commits](https://github.com/github/gh-aw/compare/48e5fa3ff52294d91d97715017a9f8693a48387f...466b8ad27fcfaaa861da3f16ba7c01272df25f46) --- updated-dependencies: - dependency-name: github/gh-aw/actions/setup dependency-version: 0.87.3 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
7841d5a5a1 |
build(deps): bump system-configuration (#8627)
Bumps the system-configuration group with 1 update in the / directory: [system-configuration](https://github.com/mullvad/system-configuration-rs). Updates `system-configuration` from 0.7.0 to 0.8.0 - [Changelog](https://github.com/mullvad/system-configuration-rs/blob/main/CHANGELOG.md) - [Commits](https://github.com/mullvad/system-configuration-rs/compare/v0.7.0...v0.8.0) --- updated-dependencies: - dependency-name: system-configuration dependency-version: 0.8.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: system-configuration ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Jeong, YunWon
|
306f104847 |
codecs: address post-merge review findings (#8623)
Assisted-by: Codex:GPT-5 |
||
|
Jeong, YunWon
|
322f294a21 |
common: add VM-independent codec and text engines (#8622)
* common: add PyPy CJK codec engines
Assisted-by: Codex:GPT-5
* common: mark CJK mapping tables generated
Assisted-by: Codex:GPT-5
* common: add reusable byte and text engines
Assisted-by: Codex:GPT-5
* common: record opt-in engine dependencies
Assisted-by: Codex:GPT-5
* stdlib: add `_multibytecodec` and the six `_codecs_*` modules
Wire the CJK codec engines in `rustpython-common` up to the VM. `_multibytecodec`
holds `MultibyteCodec`, `MultibyteIncrementalEncoder`, `MultibyteIncrementalDecoder`,
`MultibyteStreamReader` and `MultibyteStreamWriter`, the encode/decode drivers and
the error-handler protocol; each `_codecs_*` module exposes `getcodec` for the
encodings of its region.
`rustpython-stdlib` now enables the `cjk-codecs` feature of `rustpython-common`.
Add `cjk::decode_reset`, which the decoder `reset()` methods need. It differs from
building a fresh decoder state: ISO 2022 keeps its G1..G3 designations and the
escape-throughout flag, and resets only G0 and the shifted flag.
Drop the markers from the CJK codec tests and from the `test_codecs`, `test_io`,
`test_email`, `test_zipfile` and `test_traceback` cases that needed a CJK codec.
`test_codecs.BasicUnicodeTest.{test_basics,test_decoder_state}` stay expected
failures for an unrelated reason: `codecs.charmap_encode` rejects a `None` mapping.
Assisted-by: Claude Code:claude-opus-5
* stdlib: use the byte and text engines from `rustpython-common`
`binascii`, `_json` and `_socket`'s address converters now call the engines in
`rustpython-common` instead of carrying their own copies, and `rustpython-stdlib`
enables the `binascii`, `inet` and `json` features for them. That leaves
`base64` and `crc32fast` with no caller in `rustpython-stdlib`, and
`crates/stdlib/src/json/machinery.rs`, the last of the json_in_type-derived code,
goes away with its last caller.
The address converters change behaviour. `inet_aton` reads the lenient forms
(`127.1`, `16909060`, `0x7f.1`, `0177.0.0.1`), `inet_pton` rejects an octet with a
redundant leading zero and a group of five hex digits, `inet_ntop` writes the
IPv4-compatible form as `::c000:201`, and an address family with no converter
raises `EAFNOSUPPORT` instead of a bare `OSError`.
Give the JSON engine the closing-quote scan and the allocation-free `\uXXXX`
escape it needs to carry `_json`'s workload. Against the code it replaces,
`json.dumps` runs 2.3x faster with `ensure_ascii=True` and 1.7x without it, and
`json.loads` 1.2x.
`binascii.a2b_base64` now reports the messages `binascii.c` does: no
`error decoding base64: ` prefix, and the data-character count in parentheses.
Assisted-by: Claude Code:claude-opus-5
* test: enable the Windows cp932 codec case
The shared CJK engine now satisfies the Windows code-page adapter, so keeping this test marked as an expected failure turns its success into a CI failure.
Assisted-by: Codex:GPT-5
|
||
|
Hanjeong Lee
|
d1bbf4bc1a |
Report every generated __text_signature__, and keep it out of __doc__ (#8614)
* Strip the signature prefix from builtin __doc__
A function's signature and its documentation share one string, separated
by `\n--\n\n`, as in CPython. __text_signature__ read the front of it,
but __doc__ returned the whole thing:
>>> len.__doc__
'len(obj, /)\n--\n\nReturn the number of items in a container.' # was
'Return the number of items in a container.' # now
44 of the 46 builtin functions leaked the prefix.
get_doc_from_internal_doc already stripped it but was reachable only from
PyType.__doc__. Split it the way CPython does, into doc_without_signature
(_PyType_DocWithoutSignature) and a wrapper mapping an empty result to
None (_PyType_GetDocFromInternalDoc), and call it from PyNativeFunction
and PyMethodDescriptor as well.
Assisted-by: Claude Code:claude-opus-5
* Terminate a generated __text_signature__ that carries no documentation
A signature and its documentation share one string, separated by
`\n--\n\n`, and readers find the signature by searching for that marker.
A #[pyfunction] with no doc comment stored the bare signature, which no
reader could parse:
>>> time.clock_getres.__doc__
'clock_getres(clk_id, /)'
>>> time.clock_getres.__text_signature__
None
Emit the marker in that case too, as Argument Clinic does for an
undocumented function. 136 of the 915 functions reachable from the
importable modules were affected.
__doc__ now reports None for them, matching CPython, because nothing
follows the marker.
Assisted-by: Claude Code:claude-opus-5
* Report a __text_signature__ for every method
A #[pymethod] built its doc only when the method carried a doc comment,
so one without a comment dropped the generated signature too:
>>> list.append.__text_signature__
None
Build it from either part, as #[pyfunction] does. All 1101 method
descriptors reachable from the builtin types now report a signature, up
from 1, and 713 of the 1030 that CPython also describes match it exactly.
A method that takes its receiver as an ordinary first argument instead of
`&self` reported that argument by name, so binding the method could not
drop it:
>>> inspect.signature([].__dir__)
(obj, /) # was
() # now
Mark it $self, or $type for a classmethod, as CPython does. 310 methods
were affected.
test_unbound_builtin_method_noargs and test_bound_builtin_method_noargs
pass now.
Assisted-by: Claude Code:claude-opus-5
* Drop the expectedFailure on test_simple_completion
rlcompleter closes the parenthesis for a callable whose signature takes
no parameters, and leaves it open when the signature cannot be read:
def _callable_postfix(self, val, word):
if callable(val):
word += "("
try:
if not inspect.signature(val).parameters:
word += ")"
except ValueError:
pass
os.getpid reports `()` since the generated signature carries the `--`
terminator, so completing `os.getpid` yields `os.getpid()` and the test
passes.
Assisted-by: Claude Code:claude-opus-5
* Name every #[pymethod] receiver zelf
A method that needs an owned handle cannot take `&self`, so it takes the
receiver as its first argument and names it `zelf`, `self` being a Rust
keyword. 24 of the 163 such methods named it something else: `instance`
(11), `obj` (4), `exc` (3), `_self` (3), `_instance` (2).
Rename them. The generated signature is unaffected either way, since the
macro marks the receiver by position rather than by name.
Assisted-by: Claude Code:claude-opus-5
* Drop the unreachable zelf name check in func_sig
The receiver is now marked by position, so the name it carries no longer
matters, and no path reaches this check with an argument named zelf:
- a #[pymethod] or #[pyclassmethod] without a `&self` receiver has its
first argument replaced by the marker before the name is read
- a `&self` receiver is handled as syn::FnArg::Receiver
- a #[pymethod(raw)] is passed to static_raw_func, whose PyNativeFn bound
fixes its signature to (&VirtualMachine, FuncArgs)
- no #[pystaticmethod] or #[pyfunction] names an argument zelf
- #[pymember], #[pygetset] and #[pyslot] never reach func_sig
Every generated signature is byte for byte identical without it.
Assisted-by: Claude Code:claude-opus-5
* Keep the argument bundle when marking the receiver
A #[pyclassmethod] that takes FuncArgs receives the class inside the
bundle along with every other argument, but the marker was spent on the
bundle and the arguments went unreported:
>>> object.__subclasshook__.__text_signature__
'($type, /)' # was, claims it takes nothing
'($type, *args, **kwargs)' # now
Report both, the shape CPython uses for __new__. Two methods take their
arguments this way, __subclasshook__ on object and on type.
Assisted-by: Claude Code:claude-opus-5
|
||
|
Jeong, YunWon
|
5b1a1c08d7 |
Leave a byte that starts no UTF-8 sequence at the end of a console write (#8621)
* Leave a byte that starts no UTF-8 sequence at the end of a console write `find_last_utf8_boundary` read every byte at or above 0xc0 as the start of a sequence, giving 0xf8..=0xff an expected length of 4 and cutting the tail back to before it. `_find_last_utf8_boundary` stops the four-byte range at 0xf8 and returns the full length for anything above it, leaving the byte for MultiByteToWideChar to answer with U+FFFD. `write_console_utf8` therefore reported fewer bytes written than it was given whenever one of those bytes fell in the last three, and for a buffer that began with one it cut the length to 0, converted nothing and returned Ok(0) -- a write that never advances. The scan also ran a fourth iteration the C loop does not; the arms happen to answer the same there, but the bound is now the same 3. * Drop the expectedFailure on test_winconsoleio's test_write The marker went stale with the boundary fix in the parent commit. The test writes `b'\xff'*10` and `data + b'\xff'` to CONOUT$ and asserts the returned count, which is the case `find_last_utf8_boundary` was answering wrong, so regrtest now reports it as an unexpected success. |
||
|
Lee Dogeon
|
578217e1f6 |
Fix tuple subclass preservation in type bases (#8420)
* Fix tuple subclass preservation in type bases Assisted-by: Codex:gpt-5.6-sol * Refine type bases representation Store validated base classes in the original typed tuple, while retaining a bootstrap vector only until a Python tuple can be materialized. Assisted-by: gpt-5.6-sol Assisted-by: Codex:gpt-5.6-sol * Bootstrap tuple type with type hierarchy Initialize tuple and the canonical empty tuple alongside type and object so every type stores an actual typed bases tuple from construction. Remove the bootstrap bases representation and keep raw initialization Miri-safe. Assisted-by: Codex:gpt-5.6-sol |
||
|
Lee Dogeon
|
07c0fda72c |
ci: bump actions and opt into Node.js 24 (#8616)
Assisted-by: Codex:gpt-5.6-sol |
||
|
Lee Dogeon
|
1c9783ae0b |
Fix CodSpeed cargo install cache (#8620)
Assisted-by: Codex:gpt-5.6-sol |
||
|
김은빈
|
8b65a5455f | feat: reuse weak callable proxy (#8618) | ||
|
Jeong, YunWon
|
2c8d3ce50b |
Implement PEP 667 frame locals semantics (#8602)
* Implement PEP 667 frame locals semantics Assisted-by: Codex:GPT-5 * Fix C API test CString import Assisted-by: Codex:GPT-5 * Address PEP 667 review findings Assisted-by: Codex:GPT-5 |
||
|
Lee Dogeon
|
a21377a97b |
Match uuid.getnode() native node detection to CPython (#8612)
* uuid: expose stable hardware node Cache native MAC address detection and advertise an extractable node only when a hardware address is available. This lets uuid.getnode() use the native UUID path while preserving its existing fallbacks. Assisted-by: Codex:gpt-5.6-sol * Prefer universally administered MAC addresses Match uuid.getnode()'s documented interface-selection rule by scanning available addresses before falling back to a locally administered one. Ignore zero addresses and cover the selection behavior with Rust unit tests. Assisted-by: Codex:gpt-5.6-sol |
||
|
codspeed-hq[bot]andCodSpeed Bot
|
062bd72c60 |
Add CodSpeed continuous benchmarking (#8613)
Alias criterion to codspeed-criterion-compat so the existing benchmark suites can be measured by CodSpeed with CPU simulation on every pull request, and add the workflow running them. Co-authored-by: CodSpeed Bot <no-reply@codspeed.io> |
||
|
Jeong, YunWon
|
7c6c78f955 |
Implement CPython 3.14 subinterpreter APIs (#8605)
* Add _interpreters and _interpchannels modules
Apply the patch baseline for the low-level subinterpreter modules:
- crates/vm/src/stdlib/_interpreters.rs: create/destroy/list_all/get_main/
get_current/is_running/exec/call/run_string/run_func/set___main___attrs/
is_shareable/whence and the Interpreter*Error exceptions
- crates/vm/src/stdlib/_interpchannels.rs: channel create/destroy/send/recv/
list_all/list_interpreters/release/close and the Channel*Error exceptions
- crates/vm/src/vm/crossinterp.rs: cross-interpreter data protocol
- crates/vm/src/vm/{interpreter,mod,runtime,vm_new}.rs: interpreter registry
and lifecycle plumbing the modules need
- Lib/concurrent/interpreters/: high-level PEP 734 package
Assisted-by: Claude:claude-opus-5
* Correct _interpreters and _interpchannels against CPython 3.14
Argument matching:
- Add `function::ArgSpec`, a `PyArg_ParseTupleAndKeywords` equivalent that
applies the `|`, `$` and `:` markers of a format string to a `kwlist`, and
route every module function of both modules through it. This enforces
keyword-only parameters, rejects unexpected keywords, accepts keyword forms
that were previously positional-only (`whence(id=)`, `get_config(id=)`,
`set___main___attrs(id=, updates=)`, `capture_exception(exc=)`,
`_register_end_types(send=, recv=)`), and reproduces the arity messages.
- `O!` slots (`shared`, `updates`, `call`'s `args`/`kwargs`) now reject None,
and `_PyArg_BadArgument` renders None as "None" rather than "NoneType".
- `new_config` takes at most one positional `str`; `list_all` of
`_interpchannels` is argument-less.
- Convert arguments in `kwlist` order, so an argument's own error is raised
ahead of the checks that follow it.
_interpreters:
- Add the `CrossInterpreterBufferView` type and build the received memoryview
on it, keeping the sending interpreter's exporter out of the destination.
- Build `excinfo.errdisplay` from `traceback.TracebackException`.
Feature flags:
- `os.fork` checks finalization before `allow_fork`, matching `os_fork_impl`.
- `_thread.start_new_thread` and `start_joinable_thread` reject an interpreter
without `allow_threads`.
Lib/concurrent/interpreters/_crossinterp.py: restore the stripped docstrings
and comments.
Assisted-by: Claude:claude-opus-5
* Correct the stateless checks and error shapes of _interpreters against CPython 3.14
verify_stateless_function follows _PyFunction_VerifyStateless: it rejects
non-dict builtins, a non-empty __defaults__, __kwdefaults__ or __closure__,
and code whose LOAD_GLOBAL names are held by the function's globals or
absent from its builtins.
code_returns_only_none follows _PyCode_ReturnsOnlyNone: generator,
coroutine and async-generator code is rejected up front, the instruction
walk skips inline caches and maps specialized and instrumented opcodes
back, and the LOAD_CONST preceding each RETURN_VALUE is compared against
the index of None in co_consts.
ExcInfo::capture keeps an empty exception message instead of dropping it,
and builds `formatted` as `module.qualname: msg`, leaving out the builtins
and __main__ modules.
_interpreters.call packs the callable through _PyFunction_GetXIData before
pickle and re-raises the stateless-check failure when both fail. A func,
args or kwargs that cannot be rebuilt in the target, and a result without
cross-interpreter data, now raise NotShareableError with the snapshot as
the cause rather than being returned as excinfo.
_interpreters.capture_exception() without an argument returns None.
pickle_loads and _PyFunction_GetXIData attach the underlying failure as
__cause__.
channel_send converts the object to cross-interpreter data after the
channel's `closing` check.
ChannelID rich comparison returns the result of comparing its id with the
other number instead of coercing that result to bool.
PyMemoryView_FromObjectAndFlags raises "memoryview: a bytes-like object is
required, not 'X'" for an object that is not a buffer.
Assisted-by: Claude:claude-opus-5
* Correct cross-interpreter data conversion and channel argument parsing
Bound shareable ints by isize, raising OverflowError("try sending as
bytes") outside that range and keeping it as the NotShareableError cause.
Report an unshareable object by its repr, and attach a failing conversion
as __cause__ of the NotShareableError raised for the object it was called
for, so each level of a nested tuple appears in the chain.
Guard each tuple item conversion with with_recursion("while sharing a
tuple").
Gate the memoryview getdata function on `_interpreters` having been
imported, and route channel_send_buffer through a memoryview so it uses
that function and the resolved fallback.
Create the cross-interpreter exception types from the module_exec of
either module that raises them, instead of only `_interpreters`.
parse_cid raises OverflowError("int too big to convert"); int_arg
reproduces the `i` converter's three overflow messages and is also used
while parsing channel_send arguments.
Add BASETYPE to _queue.Empty, and gate nt.execv/nt.execve on allow_exec.
Assisted-by: Claude:claude-opus-5
* Correct interpreter ID parsing and the channel_send defaults lookup
_PyInterpreterState_ObjectToID accepts any object with __index__, raises
OverflowError("int too big to convert") outside the int64 range, and
reports a negative ID with the repr of the original object.
channelsmod_send reads the channel's default unboundop and fallback only
when one of the two arguments is negative, so an explicit pair is
validated before the channel is looked up.
Assisted-by: Claude:claude-opus-5
* Add the _interpqueues module and concurrent.interpreters._queues
_interpqueues holds a process-wide queue table and exposes create,
destroy, list_all, put, get, bind, release, get_maxsize,
get_queue_defaults, is_full, get_count and _register_heap_types, raising
QueueError and QueueNotFoundError. QueueEmpty and QueueFull, which
subclass queue.Empty and queue.Full, are registered per interpreter by
concurrent.interpreters._queues.
Cross-interpreter data carries a queue as a refcounted queue ID that
binds when the data is captured and releases when it is dropped, and
is_shareable reports a registered Queue as shareable.
Queue and channel items owned by an interpreter are now cleared from
Interpreter::finalize, after module finalization, rather than from
destroy_owned_interpreter, so values sent by an atexit callback also
become unbound.
is_shareable gates memoryview on the same registration flag as the
getdata lookup.
Assisted-by: Claude:claude-opus-5
* Add the high-level concurrent.interpreters API
Interpreter, ExecutionFailed and the queue aliases, copied verbatim from
CPython 3.14.
Assisted-by: Claude:claude-opus-5
* Destroy leftover subinterpreters when the main interpreter finalizes
Interpreter::finalize now runs finalize_subinterpreters between the atexit
handlers and the finalizing flag: when the main interpreter still owns
subinterpreters it emits the RuntimeWarning "remaining subinterpreters;
close them with Interpreter.close()" and then finalizes each of them.
runtime::owned_interpreter_ids lists the runtime-owned interpreters.
Assisted-by: Claude:claude-opus-5
* Absolutize the script path in run_file
_Py_abspath and _PyPathConfig_ComputeSysPath0 arrive as abs_path and
script_sys_path0. run_file hands the absolutized path to get_importer
and to the runner, so __file__ and co_filename no longer stay relative,
and inserts the symlink-resolved script directory as sys.path[0]
instead of the argument's parent.
test_cmd_line_script.test_script_abspath no longer expects a failure.
Assisted-by: Claude:claude-opus-5
* Name types in error messages the way tp_name does
The object, number, sequence and mapping protocols, the
unsupported-operand helpers, and the type-specific messages fed from
them now read PyType::slot_name() where they read PyType::name(),
so a type declared with a module keeps it.
Message texts corrected along the way:
- PySequence_Size, PyMapping_Size, PySequence_GetItem and
PySequence_SetItem/DelItem split their merged wording in two, and
PyObject_SetItem/DelItem say "object".
- PyObject_GenericSetAttr reports a read-only attribute and the
missing __dict__.
- check_class drops the type name it appended.
- __int__ and __index__ drop the class prefix; __float__ keeps it.
- float(), complex(), round(), next(), str subscripting and list
subscripting take their own wording.
- str.join and bytes.join number the item they turn down and name what
they wanted, and report a non-iterable as "can only join an
iterable".
test_class.testObjectAttributeAccessErrorMessages,
string_tests.test_subscript and the test_descrtut doctest no longer
expect a failure.
Assisted-by: Claude:claude-opus-5
* Gate script_code's source compilation on the compiler feature
crossinterp::script_code called vm.compile unconditionally, which only
exists with the rustpython-compiler feature. Builds without it, such as
example_projects/barebone, now raise a TypeError instead of failing to
compile.
Assisted-by: Claude:claude-opus-5
* Skip test_pyinit_function_raises_exception
The test imports _testsinglephase in its body without a decorator
guarding it, and now that _interpreters exists requires_subinterpreters
no longer skips it.
Assisted-by: Claude:claude-opus-5
|
||
|
Jeong, YunWon
|
c47e4170f5 |
posixsubprocess: call libc setgroups directly in the child (#8609)
`setgroups_if_needed` was compiled out on macOS, so `extra_groups` was accepted and then silently ignored there. nix gates `unistd::setgroups` off `apple_targets`, `redox` and `haiku`; calling `libc::setgroups` and checking the return with `nix::Error::result` is the shape the adjacent `setregid_if_needed` and `setreuid_if_needed` already use, and libc declares the function everywhere except redox. `u32` and `libc::gid_t` have the same layout, so the caller's slice is passed as-is instead of being collected into a fresh `Vec` after fork and before exec. `ngroups` is `c_int` on the BSDs and `size_t` on linux, hence `as _`. Assisted-by: Claude |
||
|
Shahar Naveh
|
bf3b263a3e |
CI: run cargo with --locked (#8607)
|