`malachite-bigint` 0.11.0 released while the workspace pinned 0.10.0. `pymath`
requires `malachite-bigint = "0"`, so a fresh resolve picks 0.11 for it and
leaves the workspace on 0.10, putting two incompatible copies of `BigInt` in
one graph. `crates/stdlib/src/math.rs` then fails to compile, which is what the
example projects do -- they carry no lockfile and resolve fresh on every run.
`malachite-q` and `malachite-base` move with it; bumping `malachite-bigint`
alone splits `malachite-nz` the same way. No source change is needed.
Assisted-by: Claude
* lock: add an opt-in rwlock that detaches before it blocks
A thread blocked acquiring a lock reaches no safepoint, so stop-the-world
cannot stop it, and the lock it waits for is routinely one a thread the
requester already suspended is holding. `RawDetachingRwLock` wraps the raw
rwlock and hands the wait for a contended acquire to a hook that leaves the
interpreter first; an acquire that takes the lock on its first try does not
reach the hook. The vm installs the hook during interpreter init and
implements it with `allow_threads`.
The wait ends with the lock acquired while detached, so re-attaching can park
the thread holding it. That is only safe where nothing reachable from a
stop-the-world section takes the same lock, so it is opt-in per lock:
`PyDetachingRwLock` is a separate type from `PyRwLock`, and `Traverse` is not
implemented for it, so a payload holding one cannot derive `Traverse`.
`PyByteArray::inner` takes it. `BorrowedValue`/`BorrowedValueMut` gain the
matching mapped-guard variants.
`a_thread_blocked_on_a_lock_does_not_stall_stop_the_world` blocks an
interpreter thread on a `PyDetachingRwLock` and asserts stop-the-world still
completes, running the stop on its own thread with a timeout so a stop that
never completes fails rather than hangs. Without the hook installed it fails
on the 10 s timeout; with it, it passes in 0.07 s.
Assisted-by: Claude
* lock: stop detaching where the lock may already be held
`upgrade` runs with the upgradable lock held, and `lock_shared_recursive` may
be the re-entrant take of a lock the calling thread holds; detaching there
parks a thread holding the lock, which is what this type documents it must not
do. They forward to the wrapped lock instead. `lock_upgradable` starts from
holding nothing, but nothing takes an upgradable read of one of these, so it
forwards too. `lock_shared` and `lock_exclusive` still detach.
Also narrow two claims the comments overstated. Not implementing `Traverse`
enforces the opt-in rule only against collections, not against the other things
that stop the world. And the requester exemption the hook relies on is wider
than `_PyEval_StopTheWorld` gives, so it is a local invariant.
Assisted-by: Claude
* os.readinto: read aside when the fd can wait
`readinto` held the destination's write lock for the whole call, including the
`read(2)` inside `allow_threads`. On a pipe, socket or terminal that read
returns only when the other end writes, so a thread reaching the same object
waited on a lock for an unbounded time, reaching no safepoint while it did.
Take the fd that answers without waiting directly, as before, and otherwise
read into scratch and take the lock only for the copy. This is what
`FileIO.readinto`, `socket.recv_into` and `socket.recvfrom_into` already do;
`os.readinto` was the site left over.
The EINTR retry moves to `read_into_slice`, unchanged.
Assisted-by: Claude
* fcntl: detach around the calls that wait
Every call in this module ran with the thread attached, so a thread inside one
reached no safepoint until it returned. `flock(LOCK_EX)` and `lockf(F_LOCK)`
return when whoever holds the lock gives it up, which may be never, and an
ioctl on a terminal or socket answers when the device is ready to; the world
could not be stopped for that long. `fcntl_fcntl_impl`, `fcntl_ioctl_impl`,
`fcntl_flock_impl` and `fcntl_lockf_impl` all release around the call.
`ioctl` with `mutate_flag` additionally held the target's write lock for the
whole call, so a thread reaching the same object waited on a lock for as long
as the device took. Its bytes now go in and come back through a buffer of our
own, as `fcntl_ioctl_impl` copies through one of its own for anything up to
IOCTL_BUFSZ. The export the argument holds is what keeps the length from
changing in between.
test_fcntl and test_ioctl pass.
Assisted-by: Claude
* openssl: read aside instead of holding the caller's buffer locked
`SSLSocket.read` wrote straight into the destination buffer, holding the lock
that reaching its bytes takes for the whole call. That read returns when the
peer writes, which may be never, so a thread touching the same object waited on
that lock for as long as the peer took, reaching no safepoint while it did.
Read into a buffer of our own and take the destination's lock only for the
copy. The rustls backend already reads this way, and `_ssl__SSLSocket_read_impl`
works from a `Py_buffer` whose critical section ended before the read.
`test_ssl` on this backend fails the same 16 tests before and after.
Assisted-by: Claude
* vm: add the inverse of allow_threads for callbacks
A call that detaches hands the thread to stop-the-world, which counts it as
parked. A callback that reaches Python from inside such a call would then run
on a thread the requester believes is stopped, and nothing in the vm could
stop it: `attach_thread` and `detach_thread` are private to `thread.rs`, and
`allow_threads` only goes the one way.
`attach_for_callback` attaches for the duration of the closure and returns the
thread to where it was, the way `PyGILState_Ensure` and `PyGILState_Release`
bracket `_servername_callback`. It tests for "not ATTACHED" rather than for
DETACHED, so a thread a stop-the-world has already moved to SUSPENDED routes
through `attach_thread` and parks there until the world starts again.
`a_callback_inside_a_detached_call_waits_for_the_world` stops the world with a
thread detached, then turns that thread loose at a callback and asserts it does
not run until the world starts. With the transition disabled it fails.
Assisted-by: Claude
* openssl: attach around the callbacks that run Python
`_servername_callback` and `_msg_callback` reach the interpreter from inside an
SSL call -- they take a reference to the Python callback, build arguments and
call it. That call is about to detach, and running Python from a detached
thread runs it on a thread a stop-the-world requester counts as parked.
`_servername_callback` opens with `PyGILState_Ensure()` for the same reason.
Both now rejoin the interpreter for the duration of the callback and give the
thread back afterwards. The reference to the callback moves inside that
section, since taking it is itself an interpreter operation; the check for
whether a callback is set at all stays outside, so a socket with none set never
reaches the interpreter.
No behavior change yet: nothing detaches around these calls, so
`attach_for_callback` finds the thread already attached and just runs.
Assisted-by: Claude
* openssl: detach around the SSL calls that wait
On a socket with a timeout the wait lands in `select` -> `sock_wait`, which
already detaches. On a blocking socket there is no such return: `SSL_read`
blocks in `recv(2)` through `impl Read for &PySocket`, with the thread attached
and the connection's write lock held, so the world could not be stopped for as
long as the peer stayed silent. `SSL_do_handshake`, `SSL_read_ex`,
`SSL_write_ex` and `SSL_shutdown` all run between `Py_BEGIN_ALLOW_THREADS` and
`Py_END_ALLOW_THREADS`; these now do too, in both socket and BIO mode as there.
The connection lock stays held across the call and so becomes a detaching lock:
a thread reaching the same socket gives up its interpreter rather than wait for
it attached. `connection` is `#[pytraverse(skip)]`, so a collection does not
walk into it and never takes that lock -- which is the rule for opting in,
though the skip is what supplies it here rather than the missing `Traverse`.
A server that completes a handshake and then says nothing used to deadlock the
whole process: the collector suspended the main thread at a safepoint and then
waited forever for the reader, so even the test's own timeout could not fire.
It now collects in 3 ms. Verified separately that a Python `_msg_callback`
still runs from inside the handshake -- 920 invocations across 40 handshakes
with a collector looping, five runs clean.
test_ssl on this backend fails the same 8 tests before and after, by name, and
openssl.rs draws no clippy warning it did not draw before.
Assisted-by: Claude
* lock: assert the opt-in rule against every stop-the-world section
Not implementing `Traverse` for `PyDetachingRwLock` states the rule to a
collection: a payload holding one cannot derive `Traverse`, so a collection
cannot walk into it. It says nothing to the other sections that stop the world
-- fork, traceback dumps, frame enumeration -- and `#[pytraverse(skip)]` steps
around it besides, which is how `_SSLSocket.connection` holds one. For those
the rule was a comment.
`set_world_stopped` records, on the one thread still running inside a stopped
world, that it is that thread; `lock_shared` and `lock_exclusive` assert it is
not set. A section that took one of these could block on a lock a parked thread
holds and only that section can release, which is the deadlock the rule exists
to prevent. Debug builds only; release builds track nothing.
Nothing in the tree trips it: 180 rounds of collect, `sys._current_frames`,
`faulthandler.dump_traceback` and 18 forks with four threads churning
bytearrays, plus test_gc/test_bytes/test_threading/test_memoryview/test_buffer
on a debug build, all clean.
`taking_one_while_stopping_the_world_is_caught` takes one with the flag set and
asserts the panic, so the guard is not dead code.
Assisted-by: Claude
* clippy: drop std_instead_of_core suppressions
1.98 no longer reports `std::io` items for the lint, so the six `expect`
attributes for it are unfulfilled. Also drops `from_iter_instead_of_collect`
from the workspace lint table, which 1.98 removed.
Assisted-by: Claude
* lock: spell out why the detaching rwlock exists
States the motivation as the three-thread cycle it avoids: how a stop reaches
a DETACHED thread but not an ATTACHED one, why a thread blocked on a lock
reaches no safepoint, and why the waiter rather than the holder gives way.
Puts it on `RawDetachingRwLock`, which is public and so rendered; the module
doc is private and keeps only the hook description.
Assisted-by: Claude
* fcntl: raise the error the call returned
The six detached calls discarded the `io::Error` and read `errno` again after
`allow_threads`, which re-attaches in between and can park on the way. `lockf`
already converts the returned error; these now do too.
Assisted-by: Claude
* openssl: allocate the read buffer through the vm
Without a `buffer` argument `read_len` is whatever non-negative size the caller
passed, so `vec![0u8; read_len]` aborts on allocation failure.
`new_zeroed_bytes` raises `MemoryError` instead, as the two other reads in this
file already do.
Assisted-by: Claude
* lock: leave no attached blocking acquire on a detaching lock
`lock_upgradable` starts from holding nothing, so it detaches like
`lock_shared`. A recursive read cannot: it may be the re-entrant take of a lock
this thread holds, so detaching there parks a thread holding it, and staying
attached stalls stop-the-world. `RawRwLockRecursive` is no longer implemented,
which removes `read_recursive` from these locks; nothing took one.
The assertion test no longer replaces the panic hook, which is process-wide and
was suppressing panic output from whatever else ran beside it.
Assisted-by: Claude
* vm: wait for the actual block in the stop-the-world test
The worker signalled before `read()` and the test slept 50ms, so a stop could
complete with nothing blocked on the lock and the test would pass having
checked nothing. It now publishes its thread id from inside the interpreter and
the test waits for that slot to reach DETACHED, bounded so an acquire that
never detaches fails rather than hangs.
Assisted-by: Claude
`missing_comma_expression_error` returned its span as `(index, next + 1)`,
where `next` is the byte index at which the second atom starts. The `+ 1`
is a fixed one-byte width, so the reported `end_offset` matched CPython
only when that atom was a single ASCII character:
(a b) end 5, CPython 5
(a bb) end 5, CPython 6
(a bbb) end 5, CPython 7
(a β) end 4, CPython 5
`adjacent_atom_end` already measures an atom -- identifiers including
non-ASCII ones, string literals and numbers -- and is what the first atom
is measured with three lines above. Use it for the second one too.
Assisted-by: Claude
Like the other patches, this greatly simplifies our unlinkat code and is
overall nicer because it uses the syscall directly on Linux.
As an added benefit, this commit adds WASI support and fixes a clippy
deny. The old code caused clippy to complain that the WASI branch
bypassed host_env for remove/unlink.
* Warn instead of aborting when a class namespace holds a non-string key
type() took the whole process down when the namespace it was handed had
a key that was not an exact str:
>>> type("MyClass", (), {1: 2})
thread 'main' panicked at crates/vm/src/builtins/dict.rs:781:65:
dict has non-string keys: [PyObject PyInt { value: 1 }]
to_attributes called expect() on downcast_exact, so a str subclass hit
the same line, and that one names an attribute as well as a str does.
PyAttributes is keyed by interned strings and can hold nothing else, so
a str subclass is now interned like any other name, and a key that is no
kind of string is skipped, with the caller deciding what the skip means.
type() warns once per class, the way CPython does, and
_thread.start_new_thread raises the TypeError an ordinary call already
raises for the same mapping.
The key itself still does not land in the class dict, which is what
test_type_lookup_mro_reference needs, so that one stays skipped with the
reason rewritten.
Assisted-by: Claude Code:claude-opus-5
* Mark test_type_lookup_mro_reference as an expected failure
It aborted the runner before, which is why it was skipped. Now that it
only fails, the run can carry it: five runs in a row report it as an
expected failure, and it cannot turn into a pass while the class dict
drops the key its lookup needs.
Assisted-by: Claude Code:claude-opus-5
We can save space by calculating the diffs between modern Unicode and
3.2.0 and storing membership information where valid. This avoids
storing entire tables for 3.2.0, and is also more correct in the long
run since it handles absence from 3.2.0 correctly. I switched over Bidi
to this new method which partially fixed the test. Unfortunately, our
Unicode data is more up to date than Python 3.14 so the test fails for
modern Unicode. I updated the test to 3.15 as per review, and now our
bidi passes the test.
* Guard all/any/tuple(genexpr) calls when the generator is a coroutine
`maybe_optimize_function_call()` reserves a `skip_optimization` label for
every `name(genexpr)` call and, for `all`, `any` and `tuple`, emits an
identity guard against the builtin plus an inlined loop. The port skipped
both whenever the generator expression's symbol table was a coroutine, so
`all(await x for x in xs)` and `all(x async for x in xs)` compiled to a
plain call.
An `await` or an `async for` in the generator does not disqualify the
shape: the inlined `FOR_ITER` raises the same `TypeError` that calling the
builtin on an async generator raises, which is what
test_builtin test_builtin_call_async_genexpr_no_crash asserts.
This was the last code-level difference against CPython across `Lib/`:
comparing every file's code tree by structure, opcode and constant value
(qualnames aside, which moved in 3.14.3+, and set constants compared by
value rather than by hash order) now matches on 1720 of 1720 comparable
files, up from 1719.
Assisted-by: Claude Code:claude-opus-5
* Name the ellipsis type `ellipsis`
`type(...).__name__`, its `__qualname__`, and `types.EllipsisType.__name__`
read `EllipsisType`.
Assisted-by: Claude Code:claude-opus-5
* Align compiler front end and instruction positions with CPython
Reject a bare generator expression in a class header and a format spec
nested more than two deep. Both parse here; the reference grammar has no
rule for either, since a class header only takes `arguments` and the
tokenizer runs out of nesting levels at the third format spec.
Scan the interpolations a format spec carries inside another format spec,
so `f"{x:{y:{z}}}"` reaches `z` when building the symbol table.
Give every generic function's type params scope a `.defaults` slot,
whether or not the signature has any default, so `co_varnames` and
`co_nlocals` match. Classes and type aliases keep theirs empty.
Decide an unparenthesized sole generator argument by the parser's
`parenthesized` flag instead of scanning the source for the shape of its
element. `_sum((d := x - c) * d for x in data)` opens with a parenthesized
group the rest of the expression continues, which the scan read as the
generator's own parentheses and left the call's out of its range.
Drop empty f-string fragments unconditionally. A concatenation whose
fragments are all empty now loads its empty string at the whole
concatenation instead of at whichever fragments happened to be plain
string literals.
Keep a byte order mark in source handed over as text, and report it as an
invalid non-printable character. Reading a file to freeze strips it, as
the encoded-bytes path already did.
Carry the filename into the errors `compile(..., PyCF_ONLY_AST)` raises;
they reported `<unknown>` or an empty name.
Across `Lib/`, the 1729 files that compile on both sides now agree on
opcodes, constants, flags, variable names, names, and exception tables,
and on instruction positions in all but 7 files, where a compound
statement whose body ends in a trailing `;` ends one column short.
Assisted-by: Claude Code:claude-opus-5
* Align ast module output with CPython
- Keep lone surrogates in string, f-string and t-string literal values by
reading such a literal from its source; the literal-value helpers move
out of `Compiler` into `rustpython_codegen` and `ConstantLiteral::Str`
holds `Wtf8Buf`.
- Give `{expr=}` interpolations their leading text as a Constant and
default their conversion to `repr`, in both f-strings and t-strings.
- Span folded consecutive literals from the first of them to the last.
- Take the `:` into a format spec's range only when the parser left it
out, instead of always widening by one.
- Report a node end that lands on a line start as that line, column 0.
- Give a generator expression written straight into a call's parentheses
the range of those parentheses, in the preprocess pass, replacing
codegen's source scan and `_ast`'s one-column adjustment.
- Drop the expectedFailure on test_ast_line_numbers_with_parentheses.
Assisted-by: Claude Code:claude-opus-5
* Align symbol table output with CPython
- Register an import as `DEF_IMPORT` alone, and an annotated parameter as
a plain parameter, so `import x; global x` is accepted and annotated
parameters are not reported as annotated names.
- Record a `global` declaration in the module block as well, and resolve a
name a `global` in an enclosing scope covers as an implicit global, so a
class body reading such a name emits `LOAD_NAME`.
- Resolve a name bound nowhere as an implicit global instead of leaving
its scope unknown.
- Require a binding only for a name an explicit `nonlocal` declared;
a free variable that reached the scope any other way already resolved.
- Name a generator expression's block `genexpr`; the code object keeps
`<genexpr>`.
- Locate a `def`/`class` block at its keyword rather than at the first
decorator. `decorated_definition_range()` moves from `Compiler` to
`rustpython_codegen` for that.
- Carry a comprehension symbol's flags over when inlining it into a parent
entry that only holds a free variable propagated from a child.
- Leave `__conditional_annotations__` out of an annotation block's
symbols and have the compiler cook up the free variable, which places
it after the ones the symbol table supplies.
- Check `from __future__` placement while building the symbol table, and
give `symtable.symtable()` the same docstring stripping and future
feature validation the compiler does.
- Floor `co_stacksize` at 1.
Assisted-by: Claude Code:claude-opus-5
* Locate the eval return and the interactive print like CPython
Leave `eval` mode's `RETURN_VALUE` without a location, so the exit block
is duplicated per reaching path and each copy carries the location of the
branch that jumps to it. `a and b` now ends with two returns as it does
in CPython, and `a if b else c` gives the first one the `a` it returns.
Emit an interactive `CALL_INTRINSIC_1(Print)` at the expression statement
rather than wherever the expression left the location, which differ for
`(a := 10)` and `f'{0:fz}'`.
Assisted-by: Claude Code:claude-opus-5
* Give interactive mode the conditional annotations cell
`compile_program_single()` emitted the `__conditional_annotations__` set
and its name accesses but never added the cell that holds it, so a module
compiled in `single` mode was missing the `MAKE_CELL` and the cellvar that
the same source gets in `exec` mode.
Assisted-by: Claude Code:claude-opus-5
* Give compiler-raised SyntaxErrors an end position
`CodegenError` and `SymbolTableError` now carry an `end_location`
alongside `location`, and `CompileError::python_end_location()` returns
it, so errors such as "'await' outside function" set `end_lineno` and
`end_offset` instead of leaving them None.
`SymbolTableBuilder::error_ranged()` builds both positions from a range,
replacing the inline `source_location()` calls at the error sites.
Drop the `expectedFailure` on
`test_exceptions.SyntaxErrorTests.test_file_source`, which now passes.
Assisted-by: Claude Code:claude-opus-5
* Report parser diagnostics with CPython's wording
`InvalidAnnotatedAssignmentTarget` becomes "illegal target for
annotation", `UnexpectedExpressionToken` becomes plain "invalid syntax"
rather than prefixing the parser's text, and the list-recovery
diagnostics ("Expected an expression or a '}'" and its siblings) join the
two that already mapped to "invalid syntax".
Assisted-by: Claude Code:claude-opus-5
* Punctuate the code object repr like CPython
`repr(code)` was missing the comma after the address and escaped the
filename through Rust's Debug formatting.
Drop the `expectedFailure` on the six `test_dis` tests that compare
disassembly text, which now pass.
Assisted-by: Claude Code:claude-opus-5
* Match CPython's parser error positions
A syntax error the parser reports counts its columns in characters, so
`source_location()` converts with UTF-32 rather than UTF-8; an offset
inside a character walks back to where that character starts.
Spans the parser reported differently:
- an error between two tokens covered nothing, and now covers one
character, since the narrowest token still covers one
- a "was never closed" bracket ends at column 0
- an unterminated string literal ends where it starts
- "leading zeros in decimal integer literals" spans the run of zeros
- an exponent with no digits and no sign is blamed on the digit before
the exponent letter, which the tokenizer puts back
Assisted-by: Claude Code:claude-opus-5
* codegen: seek to the finally body's own scopes before copying it
An early exit from a try block emits an extra copy of the finally body.
Nested scopes are handed out by position, and the cursors still sit inside
the try block's own run of scopes there, so the copy took those instead of
the ones the finally body opens: a generator expression got a sibling's
symbol table and raised "the symbol 'k' must be present in the symbol
table", or, where the sibling was a named scope, built a code object with
no `.0` argument that raised UnboundLocalError when it ran.
Seek the cursors to the first scope beginning on or after the finally
body's first line before compiling the copy. The existing restore then
leaves the try statement's own copies starting from the same place.
Assisted-by: Claude
Both walk nested groups by calling themselves back through
vm.call_method, and neither pushes a Python frame, so the frame limit
never sees the nesting and the native stack runs out first. A group
nested past the recursion limit segfaults:
e = TypeError(1)
for _ in range(exceeds_recursion_limit()):
e = ExceptionGroup("eg", [e])
e.split(TypeError) # dumps core
vm.with_recursion is the guard for exactly this, the same one comparison
and repr already use, and it turns the crash into the RecursionError
CPython raises. test_deep_split and test_deep_subgroup were skipped on
the segfault and now pass.
Assisted-by: Claude Code:claude-opus-5
Three skips in test_array say "Hangs", one in test_marshal carries no
reason at all, and two in test_thread say "panic" on Linux. None of the
six does any of that now. Each ran three to five times in a row and
passed, and the array trio was also checked case by case against CPython
3.14, including the mutation-during-index-conversion shapes the tests
were written for.
The neighbouring markers that say "Flaky", "Flakey on CI" and "flaky
EOFError" are left alone. One green run says nothing about those.
Assisted-by: Claude Code:claude-opus-5
trigger_signals borrowed the handler table for the whole dispatch loop
and the borrow was still alive while the Python handler ran. A handler
that disarms itself, which is the ordinary way to write one, reached for
the same cell:
>>> import signal
>>> def h(signum, frame): signal.signal(signal.SIGUSR1, signal.SIG_IGN)
...
>>> signal.signal(signal.SIGUSR1, h)
>>> signal.raise_signal(signal.SIGUSR1)
thread 'main' panicked at crates/vm/src/stdlib/_signal.rs:255:43:
RefCell already borrowed
The handler now comes out of the table one signal at a time, and the
borrow ends before it runs. Reading inside the loop rather than once up
front also keeps the table current, so a handler that arms or disarms
another signal is obeyed for the rest of the pass.
Assisted-by: Claude Code:claude-opus-5
* vm: use non-raising lookup for default attribute access
get_attribute_opt() backs hasattr() and getattr() with a default. It used
get_attr_inner() for every object, then converted AttributeError from a
missing attribute into None. The usual user-object miss therefore built and
dropped an exception just to answer an optional lookup.
Route objects using PyBaseObject::getattro through generic_getattr_opt(),
which returns the missing case without constructing AttributeError. Objects
with a custom getattro slot keep the raising path: their __getattribute__ and
__getattr__ hooks can run Python code, so their exceptions must preserve the
existing behavior. AttributeError from either route, including a descriptor,
is still converted to None; non-AttributeError exceptions propagate.
The new snippet covers ordinary hits and misses, custom __getattribute__ /
__getattr__, descriptor exceptions, and dynamic getattro-slot updates.
Refs youknowone#48
Assisted-by: Codex:GPT-5.6-terra
* vm: use non-raising lookup for optional attributes
* Give the strong count the bits an unused weak field held
RefCount packs its state into one usize and splits the non-flag bits evenly
between a strong and a weak count, so a 32-bit target left the strong count
15 bits. An object reaching 32 767 references then called refcount_overflow,
which aborts the process; on wasm that surfaces as an unreachable trap with
no Python-level error. Two ordinary module imports pass that total.
The weak half is never read and never written. WEAK_COUNT appears once
outside its own definition, in RefCount::new, and nothing in the repository
observes those bits: weak references live in the object's WeakRefList, which
Py::weak_count walks. The strong count now takes every bit the three flags
leave, which is 29 on a 32-bit target and 61 on a 64-bit one.
The tests run on wasm32-wasip2, where the old layout traps and the new one
passes. A 64-bit host passes either way, since 31 bits already covered them.
Fixes#8469
Assisted-by: Claude Code:claude-opus-5
* Return the strong count as usize so a 64-bit target cannot truncate it
The count is 61 bits wide on a 64-bit target once it takes the bits the weak
field held, so a u32 return truncates above 4 294 967 295 references: inc
would read a wrapped 0 and re-increment, and dec would read a wrapped 1 and
free a live object. A 32-bit target is unaffected, where 29 bits fit u32.
Assisted-by: Claude Code:claude-opus-5
* Cover the refcount ceiling through inc and dec
inc_by writes its own overflow check, so a test built on it leaves inc and
dec unexercised. The reported aborts came through inc, one reference at a
time.
Assisted-by: Claude Code:claude-opus-5
---------
Co-authored-by: Jean-Marc Le Roux <jeanmarc@lx.industries>
A sign, a space or an alternate form parsed fine on a string spec and
then had no effect:
>>> format('ab', '+')
'ab'
>>> f"{'ab':#5}"
'ab '
CPython raises ValueError for all three. format_string already refused z
and an explicit '=' alignment with the same family of messages, so the
three checks go next to those, in the order CPython reports them: sign,
then z, then the alternate form, then the alignment.
Assisted-by: Claude Code:claude-opus-5
With strict_mode=True the decoder accepted padding CPython rejects:
>>> binascii.a2b_base64(b'YWJj=', strict_mode=True)
b'abc'
It also dropped data in the default mode, because it returned at the
first pad that completed a quad:
>>> binascii.a2b_base64(b'abc=a')
b'i\xb7' # CPython: b'i\xb7\x1a'
The loop now counts pads and decides at the end, the way CPython does,
so a stray pad is ignored outside strict mode and named as leading,
excess or discontinuous inside it. DecodeError::InvalidPadding was dead
in this file and now carries the excess padding message.
test_base64_strict_mode and test_base64_excess_data were marked as
expected failures and pass now.
Assisted-by: Claude Code:claude-opus-5
append and appendleft trimmed before pushing, and the test they used,
maxlen == len, holds for an empty deque whose bound is zero. The pop then
had nothing to remove and the item stayed:
>>> d = deque(maxlen=0)
>>> d.append(1)
>>> list(d)
[1]
Both now push first and trim after, which is the order CPython uses, so a
bound of zero drops what just arrived. extend, extendleft, insert, rotate,
the operators and the constructor were already right.
Assisted-by: Claude Code:claude-opus-5
format_bool answered "True" or "False" for every spec that carries no
presentation type, so width, fill, alignment and sign were all discarded:
>>> f"{True:>5}"
'True'
CPython has no bool.__format__ of its own. It uses int's, where an empty
spec on a subclass gives str(self) and everything else formats the integer.
The empty spec keeps the spelled out answer, the rest now goes to
format_int, which also brings back the errors an integer spec raises.
Assisted-by: Claude Code:claude-opus-5
insort_left and insort_right rebound `x` to `key(x)` and then handed that
same value to both the search and `a.insert`, so the object the caller
passed in never reached the list:
>>> words = ["a", "ccc"]
>>> bisect.insort(words, "bb", key=len)
>>> words
['a', 2, 'ccc']
The key now feeds the search only, and the insert keeps the original.
Assisted-by: Claude Code:claude-opus-5
unicodeescape_string is a port of CPython 2.7's
PyUnicode_EncodeUnicodeEscape, where the branch mapping a character at or
above 0x10000 to \U00xxxxxx sits under #ifdef Py_UNICODE_WIDE and the
branch folding a surrogate pair sits under the #else. The port kept the
"# ifdef Py_UNICODE_WIDE" and "# endif" comments around the first one but
dropped the #else, so both branches run.
The pairing branch reads the next character without checking that there
is one, so a high surrogate at the end of the string walks off it:
>>> "\ud800".encode("unicode_escape")
IndexError: index out of range # CPython: b'\ud800'
And when there is a next character it folds the two, which no wide build
does:
>>> "𐀀".encode("unicode_escape")
b'\U00010000' # CPython: b'\ud800\udc00'
With the branch gone a high surrogate falls through to the 16-bit case
and comes out as \udXXX, which is where a low surrogate already went.
test_incremental_surrogatepass in UnicodeEscapeTest carried an
expectedFailure for the IndexError. It passes now, so the override goes
with it.
Assisted-by: Claude Code:claude-opus-5
`"a\tb".expandtabs(0)` panicked with a capacity overflow. CPython returns
`'ab'`: with no width to advance to, the tabs come out and nothing else
moves. `expandtabs(-1)` is the same call, since `ExpandTabsArgs::tabsize`
sends every negative value to 0.
`expandtabs` keeps the tab stop in `tab_size` and the current column in
`col_count`, and on a tab it does `tab_size - col_count`. With a tab size of
zero both start at 0, the first character makes `col_count` 1 while
`tab_size` stays 0, and the subtraction underflows. The run of spaces asked
for next is `usize::MAX`, and the allocation aborts the process. A tab has to
follow something on the line to reach it: `"\ta".expandtabs(0)` subtracts 0
from 0 and comes out right by accident.
`BytesInner::expandtabs` already returns early for this and filters the tabs
out. The string version now does the same.
Assisted-by: Claude Code:claude-opus-5
* csv: validate dialect options
Resolve each dialect once and validate the merged options before
constructing readers and writers. Handle Unicode character parsing
consistently and enable the corresponding CPython CSV tests.
Assisted-by: Tau:gpt-5.6-luna
* cargo fmt
* Validate csv dialect conflicts with line terminators
- Reject dialect characters that overlap any lineterminator character
- Validate Dialect construction consistently and allow non-ASCII terminators
- Unskip csv invalid character coverage now that validation matches behavior
* Remove stale CSV rejection snippet
Removes a RustPython-only assertion that contradicts the
CPython-compatible dialect validation added by this branch.
Scope: test-only
AI-Assisted-By: Codex
* Revert "Remove stale CSV rejection snippet"
This reverts commit 3a145dbba4.
* Reapply "Remove stale CSV rejection snippet"
This reverts commit 0754731a2e.
* fix(csv): complete dialect merge resolution
Restore dialect validation and remove stale quoting logic.
Assisted-by: Codex: GPT-5
* Rebase range iterator __reduce__ to match CPython
range_iterator.__reduce__ (and longrange_iterator) returned the original
range plus the current index as pickle state; CPython returns the range
rebased to the current position with a None state. Rebase start by
index * step (clamped to the length) and emit None. __setstate__ is kept
so pickles carrying an integer state still load.
Assisted-by: Claude Code:claude-opus-4-8
* Add snippet test for range iterator __reduce__
Assert the rebased range and None state added in the previous commit. The
snippet runner executes it under both CPython and RustPython, so it also
cross-checks the representation against CPython.
Assisted-by: Claude Code:claude-opus-4-8
* Update extra_tests/snippets/builtin_range.py
---------
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
findall returns the matched text rather than a match object, so a group that
took no part in the match is reported as an empty value. With one group it
was reported as `None`:
>>> re.findall(r"(a)?b", "b ab")
[None, 'a'] # CPython: ['', 'a']
The branch for two or more groups was already right, since it passes `""` to
`Match.groups` as the default, so the two halves of the same function
disagreed with each other.
That default is built as a `str` whatever the pattern is, so a `bytes`
pattern came back with `str` mixed into it:
>>> re.findall(rb"(a)|(b)", b"ab")
[(b'a', ''), ('', b'b')] # CPython: [(b'a', b''), (b'', b'b')]
The empty value is now built once from `isbytes` and both branches use it.
`Match.groups` still reports `None`, which is what CPython does.
Assisted-by: Claude Code:claude-opus-5
`clippy::std_instead_of_core` stopped firing at the four places that carry
an `#[expect]` for it. All four were added for the same false positive, on
items whose `core` counterpart is still unstable.
`unfulfilled_lint_expectations` is denied through `-D warnings`, so the
three clippy jobs and the WASM check fail on current stable for every pull
request, whatever it touches.
Assisted-by: Claude Code:claude-opus-5
Wtf8::replace looks for the pattern with a byte search. An empty pattern
matches at every byte, so the replacement was inserted between the bytes of
a multi-byte character and the result was no longer WTF-8:
>>> "á".replace("", "-")
'-ím' # CPython: '-á-'
from_bytes_unchecked then took that as valid without looking, so every later
read of the string returned something else. An empty pattern now walks code
points instead. A pattern that is not empty stays on the byte search, which
is safe because a WTF-8 sequence never starts inside another one.
Assisted-by: Claude Code:claude-opus-5
* Align compiler futures and annotations with CPython
Handle Barry parsing consistently across AST, type-comment, REPL, and WASM paths. Preserve deferred annotation source and scope metadata, and make _symtable conversion cached and linear-time.
Assisted-by: Codex:gpt-5.4
* Fix f-string debug ranges and update Barry/annotate test expectations
Compute the debug f-string source range from the raw leading and
trailing text instead of the comment-stripped text, so a comment inside
a multi-line replacement field no longer shifts the emitted LOAD_CONST
location.
Restrict the obsolete-operator location shift to `<>` by also requiring
a `>` at the reported offset; without it any `ExpectedExpression` that
followed a `<` (`2 <;`) was reported one column to the left.
Add `CodeFlags::FUTURE_MASK` and use it in builtins, the shell, and the
WASM VM instead of repeating the eight-flag union, and collapse the
duplicated Barry diagnostic sequences into `BarrySource::diagnostic`.
Update the `__annotate__` disassembly snapshot to LOAD_FAST_BORROW and
replace the RustPython-only `barry_as_FLUFL` co_flags assertion in
extra_tests with the CPython-matching one.
Assisted-by: Claude Code:claude-opus-5
* Restore the annotated function in __annotate__ qualnames
Signature annotation scopes are compiled in the scope enclosing the
annotated function, so the function name has to be folded back into the
qualname: `f.__annotate__`, `C.m.__annotate__`,
`outer.<locals>.inner.__annotate__`. test_type_annotations
test_annotate_qualname covers this.
test_pyrepl test_future_barry_as_flufl now passes, so drop its
expectedFailure marker.
Assisted-by: Claude Code:claude-opus-5
* Pass safepoints in the attached-thread interpreter tests
`main_and_subinterpreter_run_sections_overlap` parked its workers on a
condvar while attached, and `busy_main_interpreter_does_not_block_subinterpreter`
looped over protocol calls that never reach a safepoint. An attached
thread that never reaches a safepoint cannot be suspended, so a
concurrent process-wide stop-the-world (from the gc test running in
parallel) never completes and the already-stopped sibling interpreter
stays stopped; the second worker then waits in `wait_while_suspended`
until the 30s deadline.
Both loops now call `vm.check_signals()` each iteration, and the condvar
wait uses `wait_timeout` so the lock is released between safepoints.
Reproduced on Linux (4 CPUs): the two tests with `--test-threads=2`
failed 2/40 before and 0/40 after; the full `rustpython-vm` lib binary
failed 1/30 before and 0/30 after.
Assisted-by: Claude Code:claude-opus-5
* Report the obsolete `<>` operator from one place
`<>` was located twice: `BarrySource` recorded the token ranges it
rewrote to `!=`, and `cpython_parse_diagnostic_override` separately
peeked at the bytes around an `ExpectedExpression` location. Both
produced "invalid syntax" spanning the whole operator.
`prepare_barry_as_flufl_source` now records the operator ranges in the
non-Barry case too -- by plain text search, since nothing is rewritten
there -- so `BarrySource::invalid_legacy_operator` covers both, and the
byte-peeking helper is gone. It also defers to an unclosed bracket
earlier in the source, which the diagnostic chain it used to sit in did
for it.
A `<>` that starts a statement now spans both characters: the parser
stops at the `<` rather than one character in, which the byte peek did
not handle.
Assisted-by: Claude Code:claude-opus-5
* Drop a py object's payload before its class
`PyInner` declares `typ` before `payload`, so `drop_in_place` dropped the
class first and `PyAtomicRef::drop` left it null. A `PyWeak` payload is
still linked into the weakref list of the object it points at until its
own `Drop` unlinks it under the stripe lock, which happens after that.
`WeakRefList::add` walks the same list looking for a proxy to reuse and
reads `node.class()` off every node it passes, so it could dereference
the null class of a node on its way out.
`PyInner::dealloc` now runs the two destructors in the other order at all
three of its exit paths, with a compile-time check that no other field
has one.
Reproduced by the new test in about 0.1s, as the panic CI hit in the
c-api `weakrefobject::tests::new_proxy` run:
panicked at crates/vm/src/object/ext.rs:312:
unsafe precondition(s) violated: hint::unreachable_unchecked must
never be reached
Assisted-by: Claude Code:claude-opus-5
`Constants<C>` exposed `Deref` and `Index<ConstIdx>` but neither mutable
counterpart, so an owner of a `CodeObject` could read a constant but not
replace one in place and had to rebuild the whole boxed slice.
`[T]` already carries both halves of the `VarNum` pair in this file.
Assisted-by: Claude
* Correct what the non-unix frame stack is for
The comment said non-unix threading builds have no stop-the-world.
CollectStopTheWorld is gated on `feature = "threading"` alone, and
sys._current_frames stops the world on both paths; the field is the
fallback a reader uses when there is no `top_iframe` to materialize from.
Assisted-by: Claude
* Seek by the whole file position on Windows
`SetFilePointer` answers with the low half of the new position and signals
failure with INVALID_SET_FILE_POINTER, which is also that half of a
position four gigabytes in; telling them apart takes the error code, which
this did not read, so such a seek was reported as an error. Deciding
seekability from it also called such a file unseekable.
`SetFilePointerEx` returns the whole position and a success flag of its
own, which also removes the transmute of the position into halves.
Assisted-by: Claude
* memoryview: tell a value of the wrong kind from one that does not fit
Assigning an item reported every packing failure as a TypeError, so
m[0] = 300 on a 'B' view said the value was the wrong type rather than out
of range. Packing now says which of the two it was, and whether the value's
own code raised, in which case that error is the answer as it is:
m[0] = 300 ValueError: invalid value for format 'B'
m[0] = "x" TypeError: invalid type for format 'B'
m[0] = <__index__ that raises> the raised error
`struct` reports both as `struct.error` and is unchanged; the kind travels
beside the exception for the caller that tells them apart.
Also: None was read as a deletion, so m[0] = None answered "cannot delete
memory" instead of packing it; and deleting through the mapping protocol
never reached the read-only check, which comes first.
Assisted-by: Claude
* Raise MemoryError for allocations sized by Python input
os.read, _RawIOBase.read, int.to_bytes, struct.pack, ctypes array
creation and the _ssl RAND functions sized a Vec from a Python-supplied
length with vec![], which calls handle_alloc_error and, under
panic = "abort", ends the process. They now allocate through
vm.new_zeroed_bytes and raise MemoryError.
itertools.product built its pools without checking that
len(iterables) * repeat is representable; it now raises OverflowError
"repeat argument too large" and reserves the pool and index vectors
fallibly. repeat is read as isize, so a negative one raises ValueError
"repeat argument cannot be negative" instead of the conversion's message.
_ssl.RAND_bytes and RAND_pseudo_bytes read n as i32, matching the int
they are declared with.
Assisted-by: Claude
* Collect with the count in the object header
A collection kept the count it was working with in a table keyed by the
object's address, and the objects it had proved reachable in a second one.
Between them they were hashed once per candidate and twice per edge in the
heap, which is where most of a collection over a live heap went.
The count now lives in `PyInner::gc_refs`, with `GcBits::COLLECTING` saying
it is meaningful, and reachability is `gc_refs == GC_REACHABLE` rather than
membership in a set. Step 5 splits the candidates and clears the bit in one
pass. gcbench, five interleaved pairs, median: a live heap of 423k objects
goes from 0.101s to 0.055s and a dead one from 0.402s to 0.383s.
The bits, generation, owner and count take eight bytes between them. A
64-bit header had those eight as the padding its alignment forces, so it is
unchanged at 48 bytes; a 32-bit header grows from 24 to 28.
Assisted-by: Claude
* Take an explicit thread stack size as a floor in debug builds
A debug build already started Python threads on 8 MiB rather than Rust's
2 MiB default, but an explicit threading.stack_size(N) went through
verbatim. test_threading asks for 256 KiB, and starting a thread on that
walked off the end of the stack: the guard page fault landed in the
prologue of ExecutingFrame::run.
Unoptimized, that prologue reserves 80,848 bytes where the optimized one
reserves 656 -- execute_instruction is #[inline(always)] and LLVM only
colors stack slots from opt-level 1, so the frame is the sum of all 200
instruction arms' temporaries rather than the largest. A Python call costs
88,672 bytes of native stack there, and threading's bootstrap is six
frames deep, so 256 KiB holds less than half of what starting a thread
takes.
The floor reaches thread::Builder only; threading.stack_size() still
answers with what was asked for, and release builds are unchanged.
Assisted-by: Claude
* Check the native stack on every frame entry
The C-stack guard ran on one frame entry in eight. That asks the margin to
cover eight frames rather than one, and it does not: an unoptimized frame
entered through native code takes 88,672 bytes against a debug margin of
262,144. A recursion whose steps re-enter that way -- `__add__` calling
itself, a sort key that sorts -- ran off the end of the stack instead of
raising RecursionError. On a debug build `class Add: __add__ = lambda s, o:
s + o; Add() + 1` segfaulted on the main thread; it now raises, as it does
under CPython and in release builds.
enter_iframe checked and then called enter_iframe_unchecked, which checked
again; it now leaves the check to the one call.
Measured on a call-dominated benchmark, five interleaved pairs: instructions
retired go up 0.17%, about four per call, which is the stack pointer read
and the compare.
Assisted-by: Claude
* Take an iterable's length hint when a list is filled from it
`map_py_iter` read `__length_hint__` only to pass it to `PyIterIter`, and
returned an empty vector when the hint was `isize::MAX` or more. Collecting
through `PyResult` dropped the iterator's lower bound, so nothing reserved
the room the hint asked for.
`list()`, `list.extend()` and `list.__iadd__()` now reserve it and report a
hint they cannot honour as `MemoryError`; a hint that leaves no room for the
elements the list already holds is passed over, as `list_extend()` does.
`tuple()` and the other callers keep filling up without reserving.
`length_hint_opt` errors other than the `TypeError` it already turns into
`None` now reach the caller instead of being dropped.
`__iadd__` and `inplace_concat` went through `extract_cloned`, which reads
`__len__` and not `__length_hint__`; both call `PyList::extend` now, the way
`list_inplace_concat()` calls `list_extend()`.
The tuple, list and dict fast paths of `extract_elements_inner` reserve their
known length, which the `collect()` they used dropped.
Assisted-by: Claude
* Drop map.__length_hint__
`map_methods` has no `__length_hint__`, so `operator.length_hint()` on a map
answers 0, not the length of what it draws from.
The method walked into the length hint of every iterator it holds, and a
chain of maps 10000 long overflowed the native stack answering for the
outermost one. It also took the longest of its iterators, where a map stops
at the shortest.
Assisted-by: Claude
* Ask for a length hint where each caller asks for it
`map_py_iter` asked the iterable it was handed, for every caller, and reported
what asking raised. Only some callers ask it: `list_extend()` and
`_PyBytes_FromIterator()` ask the iterable, `PySequence_Tuple()` asks the
iterator, and the bytearray constructor asks nothing. `tuple()`, `min()`,
`max()`, `collections.deque()` and `f(*x)` raised for an iterable whose
`__len__` or `__length_hint__` does, where they answer.
Which object is asked is now the caller's to say. `sorted()` asks the iterable,
being `PySequence_List()`.
`bytes_from_object()` stood in for `PyBytes_FromObject()`, for
`bytearray_extend()` and for the bytearray constructor, which do not agree on
this: the first two ask, the last does not. It is split, and assigning to a
bytearray slice takes the constructor's side with `PyByteArray_FromObject()`.
`list.extend()` counted what it held before the iterable had been asked, where
`list_extend()` reads `Py_SIZE(self)` after. A `__length_hint__` that adds to
the list made the overflow guard read a count too small and raise `MemoryError`
where nothing is wrong; one that empties it made the guard skip a reservation
that cannot be served.
Assisted-by: Claude
* Settle product's pool count before it reads its arguments
`product_new()` checks `repeat` and works out `npools` before it calls
`PySequence_Tuple()` on any argument, and fills the pools `npools` times.
The pools were filled by repeating the arguments `repeat` times instead, which
walks that many steps even with no arguments to repeat: `product(repeat=2**62)`
counted up to it rather than answering `[()]`. The count was also worked out
after the arguments had been read, so a repeat too large to serve ran their
code first.
Assisted-by: Claude
* Let the bool format answer with the error its value raised
`pack_single()` leaves `'?'` to `PyObject_IsTrue()` and returns what that
raised. Packing classified the error instead, so a `ValueError` from a
`__bool__` came back as "memoryview: invalid value for format '?'".
Assisted-by: Claude
* Release a cell's old value after the lock
`PyCell::set` dropped what it replaced while still holding the mutex guarding
the cell contents. A `__del__` running from that drop and reading the same cell
waited on a lock its own caller held, so `del it` on a closure variable whose
value has such a `__del__` deadlocked.
The replaced value is now released once the guard is gone, as `Py_XSETREF`
stores before it decrefs.
Assisted-by: Claude
* Have a set iterator hold the set it iterates
The iterator kept only a reference to the inner hash table, so in
`it = iter(A(*args))` the `A()` temporary was the last owner and died as the
call returned, before `it` was bound. A `__del__` reading `it` there saw an
unbound name and its error was printed and ignored.
The iterator now holds the set object, as `si_set` does, and releases it once
exhausted. That release happens after the lock is dropped, where
`setiter_iternext()` puts its `Py_DECREF(so)` past `Py_END_CRITICAL_SECTION()`,
so a `__del__` that iterates again does not wait on a lock the call holds.
Assisted-by: Claude
* Release an exhausted iterator's container after the lock
`PositionIterInternal::_next` overwrote its `IterStatus::Active` while the
caller still held the mutex around it. Dropping the container there ran any
`__del__` under that lock, and a `__del__` that iterated the same object again
blocked on it.
`exhaust()` now hands the container back instead of dropping it, and
`locked_step()` releases it after the guard. list, list_reverseiterator, tuple,
str, dict and its views and reverse views, bytes, bytearray, memoryview,
array, deque, and the enumerate and sequence iterators all go through it.
Assisted-by: Claude
* Unskip test_free_after_iterating
Assisted-by: Claude
* Stop listing test_set as an environment polluter
`check_free_after_iterating` no longer leaves an ignored exception behind, and
the job that reruns the listed tests ten times fails once one of them stops
polluting.
Assisted-by: Claude
* Drop winsound imports left unused
`TryFromBorrowedObject`, `exceptions`, and `ToWideString` have no reference in
the module, which fails the Windows clippy line under `-Dwarnings`.
Assisted-by: Claude
* Ask nothing where the caller takes no room
`map_py_iter` asked the iterator for a length hint on behalf of every caller
that does not reserve, and reported what asking raised. Those callers ask
nothing at all: `tuple()`, `f(*x)`, `bytearray(x)`, `min()`, `max()` and
`collections.deque()` answer for an iterator whose `__length_hint__` raises,
where they had been raising it.
The answer was also never spent. It reached `PyIterIter` for a `size_hint()`
the push loop does not read, so the lookup and any call it made were work
thrown away: `tuple()` over a generator drops 22% of its instructions, and 17%
over an iterator with a `__length_hint__` written in Python.
Assisted-by: Claude
* Stop asking an iterator how long it is to walk it
`PyIter::iter` and `PyIter::into_iter` asked for a length hint and reported
what asking raised. Nothing spent the answer: it reached `PyIterIter` for a
`size_hint()` that every caller either loops past or drops, since collecting
into a `Result` reports no lower bound.
23 operations answered for an iterator whose `__length_hint__` raises, where
they had been raising it: `set`, `frozenset` and the nine `set` methods that
take an iterable, `dict.fromkeys` and the dict view operators, `array` and
`array.extend`, `all`, `any`, `sum`, `io.writelines`, `math.fsum`,
`math.prod`, and `csv.writerow` and `writerows`.
Over a generator, `set()` drops 16% of its instructions and `all()` 23%.
`str.join` and `bytes.join` do ask, reaching their elements through
`PySequence_Fast()`, which fills a list from the iterator. They take
`iter_sized()`, which is now the only way to ask. `iter_without_hint` is gone,
its callers being what `iter` already does.
Assisted-by: Claude
* Name the matrix jobs rather than let the matrix name them
A generated job name lists every value in the matrix entry, so `cargo check`
carried the booleans its dependencies and `skip_ssl` keys expand to, and the
snippets job carried its test arguments and timeout. Adding or removing a key
renames the check, which drops it from the required list until that list is
edited to match. Emptying `env_polluting_tests` renamed three checks this way.
`Run rust tests` and `clippy` keep the names they had. `cargo check` drops the
booleans from six of its nine, and the snippets job drops its arguments and
timeout from all three.
Assisted-by: Claude
* Keep a sequence iterator active when an element raises
`PositionIterInternal::_next` exhausted the iterator for any non-`Return`
result, so an error from `__getitem__` ended the walk. `iter_iternext()`
lets go of its sequence for `IndexError` and `StopIteration` alone, and
`PyIterReturn::from_getitem_result` has already turned the first of those
into the second, so only `StopIteration` exhausts now.
Both deque iterators keep exhausting on their own mutation guard, which
`deque_iternext()` does by zeroing the counter before it raises; they
share one step function for it, and the message it raises is lowercased
to match the three other sites in the module.
Assisted-by: Claude
* Name the caller in the bytes conversion TypeError
An object with no iteration protocol reached `PyObject_GetIter`'s "not
iterable" message, and `bytearray.extend()` reported bytes. Each entry
point now checks for the protocol first and names itself:
bytes(object()) cannot convert 'object' object to bytes
bytearray(object()) cannot convert 'object' object to bytearray
bytearray().extend(object()) can't extend bytearray with object
Assisted-by: Claude
* Take a strong count too large to hold as reachable
`start_gc_refs` clipped a count at `GC_REACHABLE - 1`, a number the
per-reference subtraction could still walk down to zero and collect a
live object. It now stores `GC_REACHABLE`, and `subtract_gc_ref` leaves
that value alone.
Assisted-by: Claude
* Raise AssertionError where a snippet asserts False
`assert False` is removed under `-O`, which the snippet suite may run.
Assisted-by: Claude
* Keep raising for a collection that moved under its iterator
A deque, set or dict iterator raised once and then read as spent. The
guard is sticky: `deque_iternext()` looks at the deque's state before the
count it keeps, and `dictiter_iternextkey()` and `setiter_iternext()`
write a size no collection can have, so every later call finds the same
thing and raises again. `dequereviter_next()` is the one exception,
looking at its count first, so it runs out after the first raise.
Both deque iterators now carry `dequeiterobject.counter` rather than
reading a length back from the deque, and the dict and set iterators
compare the size they captured against the collection's own every time
they are asked how much is left, which is what makes that answer nothing
from the moment the collection changes rather than only once the
iterator has raised.
The set's message is capitalized to match `setiter_iternext()`.
Measured against 3.14.6, for each of deque, reversed deque, set, dict
and a reversed dict view: the hint after the change, the error, the hint
after the error, and what a later call answers.
Assisted-by: Claude
* Give apt-get update a deadline to fail on
The step waits on `apt-get update`, which has no deadline of its own, so
a source that takes the connection and then stops answering holds the
job until the workflow's own timeout. The retry that disables the
Microsoft and azure-cli sources runs only when the update exits
non-zero, which a held connection never does; three jobs on this branch
sat on this step for 20 minutes to five and a half hours.
Each attempt is now bounded, and the transports are given a timeout, so
a source that stops answering reaches the retry.
Assisted-by: Claude
Shahar Naveh