* common: add PyPy CJK codec engines
Assisted-by: Codex:GPT-5
* common: mark CJK mapping tables generated
Assisted-by: Codex:GPT-5
* common: add reusable byte and text engines
Assisted-by: Codex:GPT-5
* common: record opt-in engine dependencies
Assisted-by: Codex:GPT-5
* stdlib: add `_multibytecodec` and the six `_codecs_*` modules
Wire the CJK codec engines in `rustpython-common` up to the VM. `_multibytecodec`
holds `MultibyteCodec`, `MultibyteIncrementalEncoder`, `MultibyteIncrementalDecoder`,
`MultibyteStreamReader` and `MultibyteStreamWriter`, the encode/decode drivers and
the error-handler protocol; each `_codecs_*` module exposes `getcodec` for the
encodings of its region.
`rustpython-stdlib` now enables the `cjk-codecs` feature of `rustpython-common`.
Add `cjk::decode_reset`, which the decoder `reset()` methods need. It differs from
building a fresh decoder state: ISO 2022 keeps its G1..G3 designations and the
escape-throughout flag, and resets only G0 and the shifted flag.
Drop the markers from the CJK codec tests and from the `test_codecs`, `test_io`,
`test_email`, `test_zipfile` and `test_traceback` cases that needed a CJK codec.
`test_codecs.BasicUnicodeTest.{test_basics,test_decoder_state}` stay expected
failures for an unrelated reason: `codecs.charmap_encode` rejects a `None` mapping.
Assisted-by: Claude Code:claude-opus-5
* stdlib: use the byte and text engines from `rustpython-common`
`binascii`, `_json` and `_socket`'s address converters now call the engines in
`rustpython-common` instead of carrying their own copies, and `rustpython-stdlib`
enables the `binascii`, `inet` and `json` features for them. That leaves
`base64` and `crc32fast` with no caller in `rustpython-stdlib`, and
`crates/stdlib/src/json/machinery.rs`, the last of the json_in_type-derived code,
goes away with its last caller.
The address converters change behaviour. `inet_aton` reads the lenient forms
(`127.1`, `16909060`, `0x7f.1`, `0177.0.0.1`), `inet_pton` rejects an octet with a
redundant leading zero and a group of five hex digits, `inet_ntop` writes the
IPv4-compatible form as `::c000:201`, and an address family with no converter
raises `EAFNOSUPPORT` instead of a bare `OSError`.
Give the JSON engine the closing-quote scan and the allocation-free `\uXXXX`
escape it needs to carry `_json`'s workload. Against the code it replaces,
`json.dumps` runs 2.3x faster with `ensure_ascii=True` and 1.7x without it, and
`json.loads` 1.2x.
`binascii.a2b_base64` now reports the messages `binascii.c` does: no
`error decoding base64: ` prefix, and the data-character count in parentheses.
Assisted-by: Claude Code:claude-opus-5
* test: enable the Windows cp932 codec case
The shared CJK engine now satisfies the Windows code-page adapter, so keeping this test marked as an expected failure turns its success into a CI failure.
Assisted-by: Codex:GPT-5
* Strip the signature prefix from builtin __doc__
A function's signature and its documentation share one string, separated
by `\n--\n\n`, as in CPython. __text_signature__ read the front of it,
but __doc__ returned the whole thing:
>>> len.__doc__
'len(obj, /)\n--\n\nReturn the number of items in a container.' # was
'Return the number of items in a container.' # now
44 of the 46 builtin functions leaked the prefix.
get_doc_from_internal_doc already stripped it but was reachable only from
PyType.__doc__. Split it the way CPython does, into doc_without_signature
(_PyType_DocWithoutSignature) and a wrapper mapping an empty result to
None (_PyType_GetDocFromInternalDoc), and call it from PyNativeFunction
and PyMethodDescriptor as well.
Assisted-by: Claude Code:claude-opus-5
* Terminate a generated __text_signature__ that carries no documentation
A signature and its documentation share one string, separated by
`\n--\n\n`, and readers find the signature by searching for that marker.
A #[pyfunction] with no doc comment stored the bare signature, which no
reader could parse:
>>> time.clock_getres.__doc__
'clock_getres(clk_id, /)'
>>> time.clock_getres.__text_signature__
None
Emit the marker in that case too, as Argument Clinic does for an
undocumented function. 136 of the 915 functions reachable from the
importable modules were affected.
__doc__ now reports None for them, matching CPython, because nothing
follows the marker.
Assisted-by: Claude Code:claude-opus-5
* Report a __text_signature__ for every method
A #[pymethod] built its doc only when the method carried a doc comment,
so one without a comment dropped the generated signature too:
>>> list.append.__text_signature__
None
Build it from either part, as #[pyfunction] does. All 1101 method
descriptors reachable from the builtin types now report a signature, up
from 1, and 713 of the 1030 that CPython also describes match it exactly.
A method that takes its receiver as an ordinary first argument instead of
`&self` reported that argument by name, so binding the method could not
drop it:
>>> inspect.signature([].__dir__)
(obj, /) # was
() # now
Mark it $self, or $type for a classmethod, as CPython does. 310 methods
were affected.
test_unbound_builtin_method_noargs and test_bound_builtin_method_noargs
pass now.
Assisted-by: Claude Code:claude-opus-5
* Drop the expectedFailure on test_simple_completion
rlcompleter closes the parenthesis for a callable whose signature takes
no parameters, and leaves it open when the signature cannot be read:
def _callable_postfix(self, val, word):
if callable(val):
word += "("
try:
if not inspect.signature(val).parameters:
word += ")"
except ValueError:
pass
os.getpid reports `()` since the generated signature carries the `--`
terminator, so completing `os.getpid` yields `os.getpid()` and the test
passes.
Assisted-by: Claude Code:claude-opus-5
* Name every #[pymethod] receiver zelf
A method that needs an owned handle cannot take `&self`, so it takes the
receiver as its first argument and names it `zelf`, `self` being a Rust
keyword. 24 of the 163 such methods named it something else: `instance`
(11), `obj` (4), `exc` (3), `_self` (3), `_instance` (2).
Rename them. The generated signature is unaffected either way, since the
macro marks the receiver by position rather than by name.
Assisted-by: Claude Code:claude-opus-5
* Drop the unreachable zelf name check in func_sig
The receiver is now marked by position, so the name it carries no longer
matters, and no path reaches this check with an argument named zelf:
- a #[pymethod] or #[pyclassmethod] without a `&self` receiver has its
first argument replaced by the marker before the name is read
- a `&self` receiver is handled as syn::FnArg::Receiver
- a #[pymethod(raw)] is passed to static_raw_func, whose PyNativeFn bound
fixes its signature to (&VirtualMachine, FuncArgs)
- no #[pystaticmethod] or #[pyfunction] names an argument zelf
- #[pymember], #[pygetset] and #[pyslot] never reach func_sig
Every generated signature is byte for byte identical without it.
Assisted-by: Claude Code:claude-opus-5
* Keep the argument bundle when marking the receiver
A #[pyclassmethod] that takes FuncArgs receives the class inside the
bundle along with every other argument, but the marker was spent on the
bundle and the arguments went unreported:
>>> object.__subclasshook__.__text_signature__
'($type, /)' # was, claims it takes nothing
'($type, *args, **kwargs)' # now
Report both, the shape CPython uses for __new__. Two methods take their
arguments this way, __subclasshook__ on object and on type.
Assisted-by: Claude Code:claude-opus-5
* Leave a byte that starts no UTF-8 sequence at the end of a console write
`find_last_utf8_boundary` read every byte at or above 0xc0 as the start of a
sequence, giving 0xf8..=0xff an expected length of 4 and cutting the tail back
to before it. `_find_last_utf8_boundary` stops the four-byte range at 0xf8 and
returns the full length for anything above it, leaving the byte for
MultiByteToWideChar to answer with U+FFFD.
`write_console_utf8` therefore reported fewer bytes written than it was given
whenever one of those bytes fell in the last three, and for a buffer that began
with one it cut the length to 0, converted nothing and returned Ok(0) -- a
write that never advances.
The scan also ran a fourth iteration the C loop does not; the arms happen to
answer the same there, but the bound is now the same 3.
* Drop the expectedFailure on test_winconsoleio's test_write
The marker went stale with the boundary fix in the parent commit. The test
writes `b'\xff'*10` and `data + b'\xff'` to CONOUT$ and asserts the returned
count, which is the case `find_last_utf8_boundary` was answering wrong, so
regrtest now reports it as an unexpected success.
* Fix tuple subclass preservation in type bases
Assisted-by: Codex:gpt-5.6-sol
* Refine type bases representation
Store validated base classes in the original typed tuple, while retaining a bootstrap vector only until a Python tuple can be materialized.
Assisted-by: gpt-5.6-sol
Assisted-by: Codex:gpt-5.6-sol
* Bootstrap tuple type with type hierarchy
Initialize tuple and the canonical empty tuple alongside type and object so every type stores an actual typed bases tuple from construction. Remove the bootstrap bases representation and keep raw initialization Miri-safe.
Assisted-by: Codex:gpt-5.6-sol
* uuid: expose stable hardware node
Cache native MAC address detection and advertise an extractable node only when a hardware address is available. This lets uuid.getnode() use the native UUID path while preserving its existing fallbacks.
Assisted-by: Codex:gpt-5.6-sol
* Prefer universally administered MAC addresses
Match uuid.getnode()'s documented interface-selection rule by scanning available addresses before falling back to a locally administered one. Ignore zero addresses and cover the selection behavior with Rust unit tests.
Assisted-by: Codex:gpt-5.6-sol
Alias criterion to codspeed-criterion-compat so the existing benchmark suites can be measured by CodSpeed with CPU simulation on every pull request, and add the workflow running them.
Co-authored-by: CodSpeed Bot <no-reply@codspeed.io>
* Add _interpreters and _interpchannels modules
Apply the patch baseline for the low-level subinterpreter modules:
- crates/vm/src/stdlib/_interpreters.rs: create/destroy/list_all/get_main/
get_current/is_running/exec/call/run_string/run_func/set___main___attrs/
is_shareable/whence and the Interpreter*Error exceptions
- crates/vm/src/stdlib/_interpchannels.rs: channel create/destroy/send/recv/
list_all/list_interpreters/release/close and the Channel*Error exceptions
- crates/vm/src/vm/crossinterp.rs: cross-interpreter data protocol
- crates/vm/src/vm/{interpreter,mod,runtime,vm_new}.rs: interpreter registry
and lifecycle plumbing the modules need
- Lib/concurrent/interpreters/: high-level PEP 734 package
Assisted-by: Claude:claude-opus-5
* Correct _interpreters and _interpchannels against CPython 3.14
Argument matching:
- Add `function::ArgSpec`, a `PyArg_ParseTupleAndKeywords` equivalent that
applies the `|`, `$` and `:` markers of a format string to a `kwlist`, and
route every module function of both modules through it. This enforces
keyword-only parameters, rejects unexpected keywords, accepts keyword forms
that were previously positional-only (`whence(id=)`, `get_config(id=)`,
`set___main___attrs(id=, updates=)`, `capture_exception(exc=)`,
`_register_end_types(send=, recv=)`), and reproduces the arity messages.
- `O!` slots (`shared`, `updates`, `call`'s `args`/`kwargs`) now reject None,
and `_PyArg_BadArgument` renders None as "None" rather than "NoneType".
- `new_config` takes at most one positional `str`; `list_all` of
`_interpchannels` is argument-less.
- Convert arguments in `kwlist` order, so an argument's own error is raised
ahead of the checks that follow it.
_interpreters:
- Add the `CrossInterpreterBufferView` type and build the received memoryview
on it, keeping the sending interpreter's exporter out of the destination.
- Build `excinfo.errdisplay` from `traceback.TracebackException`.
Feature flags:
- `os.fork` checks finalization before `allow_fork`, matching `os_fork_impl`.
- `_thread.start_new_thread` and `start_joinable_thread` reject an interpreter
without `allow_threads`.
Lib/concurrent/interpreters/_crossinterp.py: restore the stripped docstrings
and comments.
Assisted-by: Claude:claude-opus-5
* Correct the stateless checks and error shapes of _interpreters against CPython 3.14
verify_stateless_function follows _PyFunction_VerifyStateless: it rejects
non-dict builtins, a non-empty __defaults__, __kwdefaults__ or __closure__,
and code whose LOAD_GLOBAL names are held by the function's globals or
absent from its builtins.
code_returns_only_none follows _PyCode_ReturnsOnlyNone: generator,
coroutine and async-generator code is rejected up front, the instruction
walk skips inline caches and maps specialized and instrumented opcodes
back, and the LOAD_CONST preceding each RETURN_VALUE is compared against
the index of None in co_consts.
ExcInfo::capture keeps an empty exception message instead of dropping it,
and builds `formatted` as `module.qualname: msg`, leaving out the builtins
and __main__ modules.
_interpreters.call packs the callable through _PyFunction_GetXIData before
pickle and re-raises the stateless-check failure when both fail. A func,
args or kwargs that cannot be rebuilt in the target, and a result without
cross-interpreter data, now raise NotShareableError with the snapshot as
the cause rather than being returned as excinfo.
_interpreters.capture_exception() without an argument returns None.
pickle_loads and _PyFunction_GetXIData attach the underlying failure as
__cause__.
channel_send converts the object to cross-interpreter data after the
channel's `closing` check.
ChannelID rich comparison returns the result of comparing its id with the
other number instead of coercing that result to bool.
PyMemoryView_FromObjectAndFlags raises "memoryview: a bytes-like object is
required, not 'X'" for an object that is not a buffer.
Assisted-by: Claude:claude-opus-5
* Correct cross-interpreter data conversion and channel argument parsing
Bound shareable ints by isize, raising OverflowError("try sending as
bytes") outside that range and keeping it as the NotShareableError cause.
Report an unshareable object by its repr, and attach a failing conversion
as __cause__ of the NotShareableError raised for the object it was called
for, so each level of a nested tuple appears in the chain.
Guard each tuple item conversion with with_recursion("while sharing a
tuple").
Gate the memoryview getdata function on `_interpreters` having been
imported, and route channel_send_buffer through a memoryview so it uses
that function and the resolved fallback.
Create the cross-interpreter exception types from the module_exec of
either module that raises them, instead of only `_interpreters`.
parse_cid raises OverflowError("int too big to convert"); int_arg
reproduces the `i` converter's three overflow messages and is also used
while parsing channel_send arguments.
Add BASETYPE to _queue.Empty, and gate nt.execv/nt.execve on allow_exec.
Assisted-by: Claude:claude-opus-5
* Correct interpreter ID parsing and the channel_send defaults lookup
_PyInterpreterState_ObjectToID accepts any object with __index__, raises
OverflowError("int too big to convert") outside the int64 range, and
reports a negative ID with the repr of the original object.
channelsmod_send reads the channel's default unboundop and fallback only
when one of the two arguments is negative, so an explicit pair is
validated before the channel is looked up.
Assisted-by: Claude:claude-opus-5
* Add the _interpqueues module and concurrent.interpreters._queues
_interpqueues holds a process-wide queue table and exposes create,
destroy, list_all, put, get, bind, release, get_maxsize,
get_queue_defaults, is_full, get_count and _register_heap_types, raising
QueueError and QueueNotFoundError. QueueEmpty and QueueFull, which
subclass queue.Empty and queue.Full, are registered per interpreter by
concurrent.interpreters._queues.
Cross-interpreter data carries a queue as a refcounted queue ID that
binds when the data is captured and releases when it is dropped, and
is_shareable reports a registered Queue as shareable.
Queue and channel items owned by an interpreter are now cleared from
Interpreter::finalize, after module finalization, rather than from
destroy_owned_interpreter, so values sent by an atexit callback also
become unbound.
is_shareable gates memoryview on the same registration flag as the
getdata lookup.
Assisted-by: Claude:claude-opus-5
* Add the high-level concurrent.interpreters API
Interpreter, ExecutionFailed and the queue aliases, copied verbatim from
CPython 3.14.
Assisted-by: Claude:claude-opus-5
* Destroy leftover subinterpreters when the main interpreter finalizes
Interpreter::finalize now runs finalize_subinterpreters between the atexit
handlers and the finalizing flag: when the main interpreter still owns
subinterpreters it emits the RuntimeWarning "remaining subinterpreters;
close them with Interpreter.close()" and then finalizes each of them.
runtime::owned_interpreter_ids lists the runtime-owned interpreters.
Assisted-by: Claude:claude-opus-5
* Absolutize the script path in run_file
_Py_abspath and _PyPathConfig_ComputeSysPath0 arrive as abs_path and
script_sys_path0. run_file hands the absolutized path to get_importer
and to the runner, so __file__ and co_filename no longer stay relative,
and inserts the symlink-resolved script directory as sys.path[0]
instead of the argument's parent.
test_cmd_line_script.test_script_abspath no longer expects a failure.
Assisted-by: Claude:claude-opus-5
* Name types in error messages the way tp_name does
The object, number, sequence and mapping protocols, the
unsupported-operand helpers, and the type-specific messages fed from
them now read PyType::slot_name() where they read PyType::name(),
so a type declared with a module keeps it.
Message texts corrected along the way:
- PySequence_Size, PyMapping_Size, PySequence_GetItem and
PySequence_SetItem/DelItem split their merged wording in two, and
PyObject_SetItem/DelItem say "object".
- PyObject_GenericSetAttr reports a read-only attribute and the
missing __dict__.
- check_class drops the type name it appended.
- __int__ and __index__ drop the class prefix; __float__ keeps it.
- float(), complex(), round(), next(), str subscripting and list
subscripting take their own wording.
- str.join and bytes.join number the item they turn down and name what
they wanted, and report a non-iterable as "can only join an
iterable".
test_class.testObjectAttributeAccessErrorMessages,
string_tests.test_subscript and the test_descrtut doctest no longer
expect a failure.
Assisted-by: Claude:claude-opus-5
* Gate script_code's source compilation on the compiler feature
crossinterp::script_code called vm.compile unconditionally, which only
exists with the rustpython-compiler feature. Builds without it, such as
example_projects/barebone, now raise a TypeError instead of failing to
compile.
Assisted-by: Claude:claude-opus-5
* Skip test_pyinit_function_raises_exception
The test imports _testsinglephase in its body without a decorator
guarding it, and now that _interpreters exists requires_subinterpreters
no longer skips it.
Assisted-by: Claude:claude-opus-5
`setgroups_if_needed` was compiled out on macOS, so `extra_groups` was
accepted and then silently ignored there. nix gates `unistd::setgroups`
off `apple_targets`, `redox` and `haiku`; calling `libc::setgroups` and
checking the return with `nix::Error::result` is the shape the adjacent
`setregid_if_needed` and `setreuid_if_needed` already use, and libc
declares the function everywhere except redox.
`u32` and `libc::gid_t` have the same layout, so the caller's slice is
passed as-is instead of being collected into a fresh `Vec` after fork
and before exec. `ngroups` is `c_int` on the BSDs and `size_t` on
linux, hence `as _`.
Assisted-by: Claude
* memoryview: walk the view and the buffer wrapper for the collector
Neither `memoryview` nor the `_buffer_wrapper` a `__buffer__` produces was
traversed, so the object a view looks at was never reported and a cycle
running through one was classified as reachable:
class Src:
def __buffer__(self, flags): return memoryview(b'abcdef')
o = Src(); o.keep = memoryview(o) # o is never freed
Both now report what they hold, as `memory_traverse` and `mbuf_traverse` do.
Assisted-by: Claude
* Name the type in an unhashable-type message the way `tp_name` does
The message took the short name, so a type defined in a module lost the
module: `hash(array.array('B', b'a'))` said `unhashable type: 'array'`
where the name is `array.array`, and `collections.deque` said `'deque'`.
`slot_name()` is what `tp_name` holds, and it leaves a class defined in
Python reported by its own name as before.
Assisted-by: Claude
* Fix %-formatting of bytes precision, width and %c errors
`format_bytes` subtracted the byte length from the field width without
saturating, so `b'%1s' % (b'abcde',)` aborted with a capacity overflow.
A precision written as a bare dot now truncates to nothing, and `%r`/`%a`
on bytes now honour the precision and width like the other conversions.
`%c` with a bytes, bytearray or str argument that is not one unit long
now names the argument by its length instead of by its type.
Assisted-by: Claude Code:claude-opus-5
* Fix format spec handling of grouping, bool, 'c' and complex
A grouping option next to an unrecognised presentation type now reports
"Cannot specify ',' with 'a'." instead of falling through to the type
check. `format(True, spec)` now validates the spec the way an int does.
The 'c' type now pads and aligns its character, and an integer outside
the C long range raises OverflowError rather than reporting a code point
out of range.
A complex formatted with the 'n'/'g' types or with a precision and no
type now goes through the general format without forcing a fraction.
Assisted-by: Claude Code:claude-opus-5
* Round floats at a negative ndigits from the exact decimal digits
Dividing by a power of ten rounded twice: once when the quotient was
stored and again at the tie, so e.g. `round(2.5e2, -2)` came out as 300.0.
The digits now come from the exact decimal expansion of the value.
Assisted-by: Claude Code:claude-opus-5
* Report the requested base in int literal errors and reject a bare sign
An invalid literal parsed with base 0 named the base that the prefix
turned out to mean rather than 0. A string of nothing but a sign was
taken for the old octal form and returned 0 instead of raising.
Assisted-by: Claude Code:claude-opus-5
* Split, strip and expand tabs over the Py_UNICODE_ISSPACE set
`str.split`, `rsplit`, `strip`, `lstrip` and `rstrip` with no separator
used Rust's whitespace predicate, which differs from the 29 code points
`Py_UNICODE_ISSPACE` names: U+001C-U+001F and U+0085 were skipped and
U+200B was treated as a separator.
`expandtabs` went through `str`, so it dropped the lone surrogates of a
WTF-8 string; it now works on `Wtf8` throughout.
Indexing a str out of range now says "string index out of range".
Assisted-by: Claude Code:claude-opus-5
* Read Unicode numeric values as exact rationals
`DerivedNumericValues.txt` field 1 carries a rounded decimal, so
`unicodedata.numeric('⅐')` and its neighbours came back short of the
value CPython reports. The generator now reads the rational in field 3,
and takes the 3.2 values from field 8 of `UnicodeData-3.2.0.txt`.
Assisted-by: Claude Code:claude-opus-5
* Return a mappingproxy from Pattern.groupindex
`pattern_groupindex` wraps the group index in a dict proxy, and hands
back a plain dict only when the pattern has no named group.
Assisted-by: Claude Code:claude-opus-5
* Fix the utf-16/32/7 codecs' error handling
The utf-32 decoder dropped a code point above U+10FFFF instead of calling
the error handler, and both the utf-16 and utf-32 handlers were told the
encoding was "utf-16"/"utf-32" whichever byte order the data was in.
The encoders now take a bytes replacement from the handler as it stands,
rejecting one that does not fill whole code units, rather than iterating
over it as text; a str replacement from the utf-16 encoder now splits an
astral code point into a surrogate pair.
The utf-7 decoder names its encoding "utf7", as its codec does.
Assisted-by: Claude Code:claude-opus-5
* Rework complex multiplication, division and power
Division now divides by whichever part of the divisor is larger, the way
`_Py_c_quot` does, and both it and multiplication recover the infinities
that the plain formulas turn into nan, following C11 Annex G. An operand
that is a real number is now multiplied and divided component-wise, which
keeps the sign of a zero and the other part of an infinity; a real
numerator has its own quotient so that its zero keeps a sign too.
An integer exponent no greater than 100 in magnitude is now reached by
repeated multiplication rather than through the polar form, so e.g.
`(1+2j) ** 2` is exact. A zero exponent gives 1, and an exponent with no
imaginary part no longer reaches for a logarithm that turns an infinite
magnitude into a nan. A result that came out infinite raises
OverflowError.
`complex() arg is a malformed string` is replaced by `could not convert
string to complex: ...` when the string carries a misplaced underscore,
`complex division by zero` by `division by zero`, and the message for
zero raised to a negative or complex power no longer names the base.
Assisted-by: Claude Code:claude-opus-5
* Intern the string constants that read as names
A code object's string constants were interned by length, so a short
string that is not a name was interned and a long name was not. They are
now picked out the way `all_name_chars` does. The one-character latin-1
strings are interned as well, so that every route to one of them lands on
the same object.
Assisted-by: Claude Code:claude-opus-5
* Honour the marshal version and separate its EOF errors
Versions below 2 now write a float or complex as its digits under
TYPE_FLOAT/TYPE_COMPLEX, versions from 4 on write a tuple of fewer than
256 items under TYPE_SMALL_TUPLE and a long ascii string under
TYPE_ASCII, and a string is marked interned only when it is.
`marshal.loads` now tells apart the three ways its data can run out:
"EOF read where object expected" where an object was to start, "EOF read
where not expected" where a single byte was, and "marshal data too short"
where a run of bytes was cut short.
Assisted-by: Claude Code:claude-opus-5
Replace `PyType::attributes` (`PyRwLock<PyAttributes>`) with a `TypeNamespace`
enum. A type created while an interpreter is running holds a `PyDict`; the
types `Context::genesis` and `PyType::new_static` build keep the interned-key
`IndexMap`, since hashing a string needs a VM.
`type.__new__` binds `__classdictcell__` to the type's own namespace rather
than the namespace dict passed to it, so an annotation scope reads attribute
changes made after the class body ran. Removes the `expectedFailure` on
`test_type_params.TypeParamsClassScopeTest.test_modified_later`.
`PyGetSet` holds its class as a `PyRef<PyType>` instead of a non-owning
`PointerSlot`, and traverses it, because a namespace can outlive the type it
belongs to. `PointerSlot` and the `unsafe` on `Context::new_getset` are gone.
`subs_parameters` raises `TypeError` when `__typing_subst__` returns a
non-tuple in an unpack position, using the new
`PyType::fully_qualified_name` for the message. Removes the
`expectedFailure` on
`test_typing.GenericTests.test_return_non_tuple_while_unpacking`.
Assisted-by: Claude
`malachite-bigint` 0.11.0 released while the workspace pinned 0.10.0. `pymath`
requires `malachite-bigint = "0"`, so a fresh resolve picks 0.11 for it and
leaves the workspace on 0.10, putting two incompatible copies of `BigInt` in
one graph. `crates/stdlib/src/math.rs` then fails to compile, which is what the
example projects do -- they carry no lockfile and resolve fresh on every run.
`malachite-q` and `malachite-base` move with it; bumping `malachite-bigint`
alone splits `malachite-nz` the same way. No source change is needed.
Assisted-by: Claude
* lock: add an opt-in rwlock that detaches before it blocks
A thread blocked acquiring a lock reaches no safepoint, so stop-the-world
cannot stop it, and the lock it waits for is routinely one a thread the
requester already suspended is holding. `RawDetachingRwLock` wraps the raw
rwlock and hands the wait for a contended acquire to a hook that leaves the
interpreter first; an acquire that takes the lock on its first try does not
reach the hook. The vm installs the hook during interpreter init and
implements it with `allow_threads`.
The wait ends with the lock acquired while detached, so re-attaching can park
the thread holding it. That is only safe where nothing reachable from a
stop-the-world section takes the same lock, so it is opt-in per lock:
`PyDetachingRwLock` is a separate type from `PyRwLock`, and `Traverse` is not
implemented for it, so a payload holding one cannot derive `Traverse`.
`PyByteArray::inner` takes it. `BorrowedValue`/`BorrowedValueMut` gain the
matching mapped-guard variants.
`a_thread_blocked_on_a_lock_does_not_stall_stop_the_world` blocks an
interpreter thread on a `PyDetachingRwLock` and asserts stop-the-world still
completes, running the stop on its own thread with a timeout so a stop that
never completes fails rather than hangs. Without the hook installed it fails
on the 10 s timeout; with it, it passes in 0.07 s.
Assisted-by: Claude
* lock: stop detaching where the lock may already be held
`upgrade` runs with the upgradable lock held, and `lock_shared_recursive` may
be the re-entrant take of a lock the calling thread holds; detaching there
parks a thread holding the lock, which is what this type documents it must not
do. They forward to the wrapped lock instead. `lock_upgradable` starts from
holding nothing, but nothing takes an upgradable read of one of these, so it
forwards too. `lock_shared` and `lock_exclusive` still detach.
Also narrow two claims the comments overstated. Not implementing `Traverse`
enforces the opt-in rule only against collections, not against the other things
that stop the world. And the requester exemption the hook relies on is wider
than `_PyEval_StopTheWorld` gives, so it is a local invariant.
Assisted-by: Claude
* os.readinto: read aside when the fd can wait
`readinto` held the destination's write lock for the whole call, including the
`read(2)` inside `allow_threads`. On a pipe, socket or terminal that read
returns only when the other end writes, so a thread reaching the same object
waited on a lock for an unbounded time, reaching no safepoint while it did.
Take the fd that answers without waiting directly, as before, and otherwise
read into scratch and take the lock only for the copy. This is what
`FileIO.readinto`, `socket.recv_into` and `socket.recvfrom_into` already do;
`os.readinto` was the site left over.
The EINTR retry moves to `read_into_slice`, unchanged.
Assisted-by: Claude
* fcntl: detach around the calls that wait
Every call in this module ran with the thread attached, so a thread inside one
reached no safepoint until it returned. `flock(LOCK_EX)` and `lockf(F_LOCK)`
return when whoever holds the lock gives it up, which may be never, and an
ioctl on a terminal or socket answers when the device is ready to; the world
could not be stopped for that long. `fcntl_fcntl_impl`, `fcntl_ioctl_impl`,
`fcntl_flock_impl` and `fcntl_lockf_impl` all release around the call.
`ioctl` with `mutate_flag` additionally held the target's write lock for the
whole call, so a thread reaching the same object waited on a lock for as long
as the device took. Its bytes now go in and come back through a buffer of our
own, as `fcntl_ioctl_impl` copies through one of its own for anything up to
IOCTL_BUFSZ. The export the argument holds is what keeps the length from
changing in between.
test_fcntl and test_ioctl pass.
Assisted-by: Claude
* openssl: read aside instead of holding the caller's buffer locked
`SSLSocket.read` wrote straight into the destination buffer, holding the lock
that reaching its bytes takes for the whole call. That read returns when the
peer writes, which may be never, so a thread touching the same object waited on
that lock for as long as the peer took, reaching no safepoint while it did.
Read into a buffer of our own and take the destination's lock only for the
copy. The rustls backend already reads this way, and `_ssl__SSLSocket_read_impl`
works from a `Py_buffer` whose critical section ended before the read.
`test_ssl` on this backend fails the same 16 tests before and after.
Assisted-by: Claude
* vm: add the inverse of allow_threads for callbacks
A call that detaches hands the thread to stop-the-world, which counts it as
parked. A callback that reaches Python from inside such a call would then run
on a thread the requester believes is stopped, and nothing in the vm could
stop it: `attach_thread` and `detach_thread` are private to `thread.rs`, and
`allow_threads` only goes the one way.
`attach_for_callback` attaches for the duration of the closure and returns the
thread to where it was, the way `PyGILState_Ensure` and `PyGILState_Release`
bracket `_servername_callback`. It tests for "not ATTACHED" rather than for
DETACHED, so a thread a stop-the-world has already moved to SUSPENDED routes
through `attach_thread` and parks there until the world starts again.
`a_callback_inside_a_detached_call_waits_for_the_world` stops the world with a
thread detached, then turns that thread loose at a callback and asserts it does
not run until the world starts. With the transition disabled it fails.
Assisted-by: Claude
* openssl: attach around the callbacks that run Python
`_servername_callback` and `_msg_callback` reach the interpreter from inside an
SSL call -- they take a reference to the Python callback, build arguments and
call it. That call is about to detach, and running Python from a detached
thread runs it on a thread a stop-the-world requester counts as parked.
`_servername_callback` opens with `PyGILState_Ensure()` for the same reason.
Both now rejoin the interpreter for the duration of the callback and give the
thread back afterwards. The reference to the callback moves inside that
section, since taking it is itself an interpreter operation; the check for
whether a callback is set at all stays outside, so a socket with none set never
reaches the interpreter.
No behavior change yet: nothing detaches around these calls, so
`attach_for_callback` finds the thread already attached and just runs.
Assisted-by: Claude
* openssl: detach around the SSL calls that wait
On a socket with a timeout the wait lands in `select` -> `sock_wait`, which
already detaches. On a blocking socket there is no such return: `SSL_read`
blocks in `recv(2)` through `impl Read for &PySocket`, with the thread attached
and the connection's write lock held, so the world could not be stopped for as
long as the peer stayed silent. `SSL_do_handshake`, `SSL_read_ex`,
`SSL_write_ex` and `SSL_shutdown` all run between `Py_BEGIN_ALLOW_THREADS` and
`Py_END_ALLOW_THREADS`; these now do too, in both socket and BIO mode as there.
The connection lock stays held across the call and so becomes a detaching lock:
a thread reaching the same socket gives up its interpreter rather than wait for
it attached. `connection` is `#[pytraverse(skip)]`, so a collection does not
walk into it and never takes that lock -- which is the rule for opting in,
though the skip is what supplies it here rather than the missing `Traverse`.
A server that completes a handshake and then says nothing used to deadlock the
whole process: the collector suspended the main thread at a safepoint and then
waited forever for the reader, so even the test's own timeout could not fire.
It now collects in 3 ms. Verified separately that a Python `_msg_callback`
still runs from inside the handshake -- 920 invocations across 40 handshakes
with a collector looping, five runs clean.
test_ssl on this backend fails the same 8 tests before and after, by name, and
openssl.rs draws no clippy warning it did not draw before.
Assisted-by: Claude
* lock: assert the opt-in rule against every stop-the-world section
Not implementing `Traverse` for `PyDetachingRwLock` states the rule to a
collection: a payload holding one cannot derive `Traverse`, so a collection
cannot walk into it. It says nothing to the other sections that stop the world
-- fork, traceback dumps, frame enumeration -- and `#[pytraverse(skip)]` steps
around it besides, which is how `_SSLSocket.connection` holds one. For those
the rule was a comment.
`set_world_stopped` records, on the one thread still running inside a stopped
world, that it is that thread; `lock_shared` and `lock_exclusive` assert it is
not set. A section that took one of these could block on a lock a parked thread
holds and only that section can release, which is the deadlock the rule exists
to prevent. Debug builds only; release builds track nothing.
Nothing in the tree trips it: 180 rounds of collect, `sys._current_frames`,
`faulthandler.dump_traceback` and 18 forks with four threads churning
bytearrays, plus test_gc/test_bytes/test_threading/test_memoryview/test_buffer
on a debug build, all clean.
`taking_one_while_stopping_the_world_is_caught` takes one with the flag set and
asserts the panic, so the guard is not dead code.
Assisted-by: Claude
* clippy: drop std_instead_of_core suppressions
1.98 no longer reports `std::io` items for the lint, so the six `expect`
attributes for it are unfulfilled. Also drops `from_iter_instead_of_collect`
from the workspace lint table, which 1.98 removed.
Assisted-by: Claude
* lock: spell out why the detaching rwlock exists
States the motivation as the three-thread cycle it avoids: how a stop reaches
a DETACHED thread but not an ATTACHED one, why a thread blocked on a lock
reaches no safepoint, and why the waiter rather than the holder gives way.
Puts it on `RawDetachingRwLock`, which is public and so rendered; the module
doc is private and keeps only the hook description.
Assisted-by: Claude
* fcntl: raise the error the call returned
The six detached calls discarded the `io::Error` and read `errno` again after
`allow_threads`, which re-attaches in between and can park on the way. `lockf`
already converts the returned error; these now do too.
Assisted-by: Claude
* openssl: allocate the read buffer through the vm
Without a `buffer` argument `read_len` is whatever non-negative size the caller
passed, so `vec![0u8; read_len]` aborts on allocation failure.
`new_zeroed_bytes` raises `MemoryError` instead, as the two other reads in this
file already do.
Assisted-by: Claude
* lock: leave no attached blocking acquire on a detaching lock
`lock_upgradable` starts from holding nothing, so it detaches like
`lock_shared`. A recursive read cannot: it may be the re-entrant take of a lock
this thread holds, so detaching there parks a thread holding it, and staying
attached stalls stop-the-world. `RawRwLockRecursive` is no longer implemented,
which removes `read_recursive` from these locks; nothing took one.
The assertion test no longer replaces the panic hook, which is process-wide and
was suppressing panic output from whatever else ran beside it.
Assisted-by: Claude
* vm: wait for the actual block in the stop-the-world test
The worker signalled before `read()` and the test slept 50ms, so a stop could
complete with nothing blocked on the lock and the test would pass having
checked nothing. It now publishes its thread id from inside the interpreter and
the test waits for that slot to reach DETACHED, bounded so an acquire that
never detaches fails rather than hangs.
Assisted-by: Claude
`missing_comma_expression_error` returned its span as `(index, next + 1)`,
where `next` is the byte index at which the second atom starts. The `+ 1`
is a fixed one-byte width, so the reported `end_offset` matched CPython
only when that atom was a single ASCII character:
(a b) end 5, CPython 5
(a bb) end 5, CPython 6
(a bbb) end 5, CPython 7
(a β) end 4, CPython 5
`adjacent_atom_end` already measures an atom -- identifiers including
non-ASCII ones, string literals and numbers -- and is what the first atom
is measured with three lines above. Use it for the second one too.
Assisted-by: Claude
Like the other patches, this greatly simplifies our unlinkat code and is
overall nicer because it uses the syscall directly on Linux.
As an added benefit, this commit adds WASI support and fixes a clippy
deny. The old code caused clippy to complain that the WASI branch
bypassed host_env for remove/unlink.
* Warn instead of aborting when a class namespace holds a non-string key
type() took the whole process down when the namespace it was handed had
a key that was not an exact str:
>>> type("MyClass", (), {1: 2})
thread 'main' panicked at crates/vm/src/builtins/dict.rs:781:65:
dict has non-string keys: [PyObject PyInt { value: 1 }]
to_attributes called expect() on downcast_exact, so a str subclass hit
the same line, and that one names an attribute as well as a str does.
PyAttributes is keyed by interned strings and can hold nothing else, so
a str subclass is now interned like any other name, and a key that is no
kind of string is skipped, with the caller deciding what the skip means.
type() warns once per class, the way CPython does, and
_thread.start_new_thread raises the TypeError an ordinary call already
raises for the same mapping.
The key itself still does not land in the class dict, which is what
test_type_lookup_mro_reference needs, so that one stays skipped with the
reason rewritten.
Assisted-by: Claude Code:claude-opus-5
* Mark test_type_lookup_mro_reference as an expected failure
It aborted the runner before, which is why it was skipped. Now that it
only fails, the run can carry it: five runs in a row report it as an
expected failure, and it cannot turn into a pass while the class dict
drops the key its lookup needs.
Assisted-by: Claude Code:claude-opus-5
We can save space by calculating the diffs between modern Unicode and
3.2.0 and storing membership information where valid. This avoids
storing entire tables for 3.2.0, and is also more correct in the long
run since it handles absence from 3.2.0 correctly. I switched over Bidi
to this new method which partially fixed the test. Unfortunately, our
Unicode data is more up to date than Python 3.14 so the test fails for
modern Unicode. I updated the test to 3.15 as per review, and now our
bidi passes the test.
* Guard all/any/tuple(genexpr) calls when the generator is a coroutine
`maybe_optimize_function_call()` reserves a `skip_optimization` label for
every `name(genexpr)` call and, for `all`, `any` and `tuple`, emits an
identity guard against the builtin plus an inlined loop. The port skipped
both whenever the generator expression's symbol table was a coroutine, so
`all(await x for x in xs)` and `all(x async for x in xs)` compiled to a
plain call.
An `await` or an `async for` in the generator does not disqualify the
shape: the inlined `FOR_ITER` raises the same `TypeError` that calling the
builtin on an async generator raises, which is what
test_builtin test_builtin_call_async_genexpr_no_crash asserts.
This was the last code-level difference against CPython across `Lib/`:
comparing every file's code tree by structure, opcode and constant value
(qualnames aside, which moved in 3.14.3+, and set constants compared by
value rather than by hash order) now matches on 1720 of 1720 comparable
files, up from 1719.
Assisted-by: Claude Code:claude-opus-5
* Name the ellipsis type `ellipsis`
`type(...).__name__`, its `__qualname__`, and `types.EllipsisType.__name__`
read `EllipsisType`.
Assisted-by: Claude Code:claude-opus-5
* Align compiler front end and instruction positions with CPython
Reject a bare generator expression in a class header and a format spec
nested more than two deep. Both parse here; the reference grammar has no
rule for either, since a class header only takes `arguments` and the
tokenizer runs out of nesting levels at the third format spec.
Scan the interpolations a format spec carries inside another format spec,
so `f"{x:{y:{z}}}"` reaches `z` when building the symbol table.
Give every generic function's type params scope a `.defaults` slot,
whether or not the signature has any default, so `co_varnames` and
`co_nlocals` match. Classes and type aliases keep theirs empty.
Decide an unparenthesized sole generator argument by the parser's
`parenthesized` flag instead of scanning the source for the shape of its
element. `_sum((d := x - c) * d for x in data)` opens with a parenthesized
group the rest of the expression continues, which the scan read as the
generator's own parentheses and left the call's out of its range.
Drop empty f-string fragments unconditionally. A concatenation whose
fragments are all empty now loads its empty string at the whole
concatenation instead of at whichever fragments happened to be plain
string literals.
Keep a byte order mark in source handed over as text, and report it as an
invalid non-printable character. Reading a file to freeze strips it, as
the encoded-bytes path already did.
Carry the filename into the errors `compile(..., PyCF_ONLY_AST)` raises;
they reported `<unknown>` or an empty name.
Across `Lib/`, the 1729 files that compile on both sides now agree on
opcodes, constants, flags, variable names, names, and exception tables,
and on instruction positions in all but 7 files, where a compound
statement whose body ends in a trailing `;` ends one column short.
Assisted-by: Claude Code:claude-opus-5
* Align ast module output with CPython
- Keep lone surrogates in string, f-string and t-string literal values by
reading such a literal from its source; the literal-value helpers move
out of `Compiler` into `rustpython_codegen` and `ConstantLiteral::Str`
holds `Wtf8Buf`.
- Give `{expr=}` interpolations their leading text as a Constant and
default their conversion to `repr`, in both f-strings and t-strings.
- Span folded consecutive literals from the first of them to the last.
- Take the `:` into a format spec's range only when the parser left it
out, instead of always widening by one.
- Report a node end that lands on a line start as that line, column 0.
- Give a generator expression written straight into a call's parentheses
the range of those parentheses, in the preprocess pass, replacing
codegen's source scan and `_ast`'s one-column adjustment.
- Drop the expectedFailure on test_ast_line_numbers_with_parentheses.
Assisted-by: Claude Code:claude-opus-5
* Align symbol table output with CPython
- Register an import as `DEF_IMPORT` alone, and an annotated parameter as
a plain parameter, so `import x; global x` is accepted and annotated
parameters are not reported as annotated names.
- Record a `global` declaration in the module block as well, and resolve a
name a `global` in an enclosing scope covers as an implicit global, so a
class body reading such a name emits `LOAD_NAME`.
- Resolve a name bound nowhere as an implicit global instead of leaving
its scope unknown.
- Require a binding only for a name an explicit `nonlocal` declared;
a free variable that reached the scope any other way already resolved.
- Name a generator expression's block `genexpr`; the code object keeps
`<genexpr>`.
- Locate a `def`/`class` block at its keyword rather than at the first
decorator. `decorated_definition_range()` moves from `Compiler` to
`rustpython_codegen` for that.
- Carry a comprehension symbol's flags over when inlining it into a parent
entry that only holds a free variable propagated from a child.
- Leave `__conditional_annotations__` out of an annotation block's
symbols and have the compiler cook up the free variable, which places
it after the ones the symbol table supplies.
- Check `from __future__` placement while building the symbol table, and
give `symtable.symtable()` the same docstring stripping and future
feature validation the compiler does.
- Floor `co_stacksize` at 1.
Assisted-by: Claude Code:claude-opus-5
* Locate the eval return and the interactive print like CPython
Leave `eval` mode's `RETURN_VALUE` without a location, so the exit block
is duplicated per reaching path and each copy carries the location of the
branch that jumps to it. `a and b` now ends with two returns as it does
in CPython, and `a if b else c` gives the first one the `a` it returns.
Emit an interactive `CALL_INTRINSIC_1(Print)` at the expression statement
rather than wherever the expression left the location, which differ for
`(a := 10)` and `f'{0:fz}'`.
Assisted-by: Claude Code:claude-opus-5
* Give interactive mode the conditional annotations cell
`compile_program_single()` emitted the `__conditional_annotations__` set
and its name accesses but never added the cell that holds it, so a module
compiled in `single` mode was missing the `MAKE_CELL` and the cellvar that
the same source gets in `exec` mode.
Assisted-by: Claude Code:claude-opus-5
* Give compiler-raised SyntaxErrors an end position
`CodegenError` and `SymbolTableError` now carry an `end_location`
alongside `location`, and `CompileError::python_end_location()` returns
it, so errors such as "'await' outside function" set `end_lineno` and
`end_offset` instead of leaving them None.
`SymbolTableBuilder::error_ranged()` builds both positions from a range,
replacing the inline `source_location()` calls at the error sites.
Drop the `expectedFailure` on
`test_exceptions.SyntaxErrorTests.test_file_source`, which now passes.
Assisted-by: Claude Code:claude-opus-5
* Report parser diagnostics with CPython's wording
`InvalidAnnotatedAssignmentTarget` becomes "illegal target for
annotation", `UnexpectedExpressionToken` becomes plain "invalid syntax"
rather than prefixing the parser's text, and the list-recovery
diagnostics ("Expected an expression or a '}'" and its siblings) join the
two that already mapped to "invalid syntax".
Assisted-by: Claude Code:claude-opus-5
* Punctuate the code object repr like CPython
`repr(code)` was missing the comma after the address and escaped the
filename through Rust's Debug formatting.
Drop the `expectedFailure` on the six `test_dis` tests that compare
disassembly text, which now pass.
Assisted-by: Claude Code:claude-opus-5
* Match CPython's parser error positions
A syntax error the parser reports counts its columns in characters, so
`source_location()` converts with UTF-32 rather than UTF-8; an offset
inside a character walks back to where that character starts.
Spans the parser reported differently:
- an error between two tokens covered nothing, and now covers one
character, since the narrowest token still covers one
- a "was never closed" bracket ends at column 0
- an unterminated string literal ends where it starts
- "leading zeros in decimal integer literals" spans the run of zeros
- an exponent with no digits and no sign is blamed on the digit before
the exponent letter, which the tokenizer puts back
Assisted-by: Claude Code:claude-opus-5
* codegen: seek to the finally body's own scopes before copying it
An early exit from a try block emits an extra copy of the finally body.
Nested scopes are handed out by position, and the cursors still sit inside
the try block's own run of scopes there, so the copy took those instead of
the ones the finally body opens: a generator expression got a sibling's
symbol table and raised "the symbol 'k' must be present in the symbol
table", or, where the sibling was a named scope, built a code object with
no `.0` argument that raised UnboundLocalError when it ran.
Seek the cursors to the first scope beginning on or after the finally
body's first line before compiling the copy. The existing restore then
leaves the try statement's own copies starting from the same place.
Assisted-by: Claude
Both walk nested groups by calling themselves back through
vm.call_method, and neither pushes a Python frame, so the frame limit
never sees the nesting and the native stack runs out first. A group
nested past the recursion limit segfaults:
e = TypeError(1)
for _ in range(exceeds_recursion_limit()):
e = ExceptionGroup("eg", [e])
e.split(TypeError) # dumps core
vm.with_recursion is the guard for exactly this, the same one comparison
and repr already use, and it turns the crash into the RecursionError
CPython raises. test_deep_split and test_deep_subgroup were skipped on
the segfault and now pass.
Assisted-by: Claude Code:claude-opus-5
Three skips in test_array say "Hangs", one in test_marshal carries no
reason at all, and two in test_thread say "panic" on Linux. None of the
six does any of that now. Each ran three to five times in a row and
passed, and the array trio was also checked case by case against CPython
3.14, including the mutation-during-index-conversion shapes the tests
were written for.
The neighbouring markers that say "Flaky", "Flakey on CI" and "flaky
EOFError" are left alone. One green run says nothing about those.
Assisted-by: Claude Code:claude-opus-5
trigger_signals borrowed the handler table for the whole dispatch loop
and the borrow was still alive while the Python handler ran. A handler
that disarms itself, which is the ordinary way to write one, reached for
the same cell:
>>> import signal
>>> def h(signum, frame): signal.signal(signal.SIGUSR1, signal.SIG_IGN)
...
>>> signal.signal(signal.SIGUSR1, h)
>>> signal.raise_signal(signal.SIGUSR1)
thread 'main' panicked at crates/vm/src/stdlib/_signal.rs:255:43:
RefCell already borrowed
The handler now comes out of the table one signal at a time, and the
borrow ends before it runs. Reading inside the loop rather than once up
front also keeps the table current, so a handler that arms or disarms
another signal is obeyed for the rest of the pass.
Assisted-by: Claude Code:claude-opus-5
* vm: use non-raising lookup for default attribute access
get_attribute_opt() backs hasattr() and getattr() with a default. It used
get_attr_inner() for every object, then converted AttributeError from a
missing attribute into None. The usual user-object miss therefore built and
dropped an exception just to answer an optional lookup.
Route objects using PyBaseObject::getattro through generic_getattr_opt(),
which returns the missing case without constructing AttributeError. Objects
with a custom getattro slot keep the raising path: their __getattribute__ and
__getattr__ hooks can run Python code, so their exceptions must preserve the
existing behavior. AttributeError from either route, including a descriptor,
is still converted to None; non-AttributeError exceptions propagate.
The new snippet covers ordinary hits and misses, custom __getattribute__ /
__getattr__, descriptor exceptions, and dynamic getattro-slot updates.
Refs youknowone#48
Assisted-by: Codex:GPT-5.6-terra
* vm: use non-raising lookup for optional attributes
* Give the strong count the bits an unused weak field held
RefCount packs its state into one usize and splits the non-flag bits evenly
between a strong and a weak count, so a 32-bit target left the strong count
15 bits. An object reaching 32 767 references then called refcount_overflow,
which aborts the process; on wasm that surfaces as an unreachable trap with
no Python-level error. Two ordinary module imports pass that total.
The weak half is never read and never written. WEAK_COUNT appears once
outside its own definition, in RefCount::new, and nothing in the repository
observes those bits: weak references live in the object's WeakRefList, which
Py::weak_count walks. The strong count now takes every bit the three flags
leave, which is 29 on a 32-bit target and 61 on a 64-bit one.
The tests run on wasm32-wasip2, where the old layout traps and the new one
passes. A 64-bit host passes either way, since 31 bits already covered them.
Fixes#8469
Assisted-by: Claude Code:claude-opus-5
* Return the strong count as usize so a 64-bit target cannot truncate it
The count is 61 bits wide on a 64-bit target once it takes the bits the weak
field held, so a u32 return truncates above 4 294 967 295 references: inc
would read a wrapped 0 and re-increment, and dec would read a wrapped 1 and
free a live object. A 32-bit target is unaffected, where 29 bits fit u32.
Assisted-by: Claude Code:claude-opus-5
* Cover the refcount ceiling through inc and dec
inc_by writes its own overflow check, so a test built on it leaves inc and
dec unexercised. The reported aborts came through inc, one reference at a
time.
Assisted-by: Claude Code:claude-opus-5
---------
Co-authored-by: Jean-Marc Le Roux <jeanmarc@lx.industries>
A sign, a space or an alternate form parsed fine on a string spec and
then had no effect:
>>> format('ab', '+')
'ab'
>>> f"{'ab':#5}"
'ab '
CPython raises ValueError for all three. format_string already refused z
and an explicit '=' alignment with the same family of messages, so the
three checks go next to those, in the order CPython reports them: sign,
then z, then the alternate form, then the alignment.
Assisted-by: Claude Code:claude-opus-5
With strict_mode=True the decoder accepted padding CPython rejects:
>>> binascii.a2b_base64(b'YWJj=', strict_mode=True)
b'abc'
It also dropped data in the default mode, because it returned at the
first pad that completed a quad:
>>> binascii.a2b_base64(b'abc=a')
b'i\xb7' # CPython: b'i\xb7\x1a'
The loop now counts pads and decides at the end, the way CPython does,
so a stray pad is ignored outside strict mode and named as leading,
excess or discontinuous inside it. DecodeError::InvalidPadding was dead
in this file and now carries the excess padding message.
test_base64_strict_mode and test_base64_excess_data were marked as
expected failures and pass now.
Assisted-by: Claude Code:claude-opus-5
Jeong, YunWon