* gc: add CollectResult, stats fields, get_referrers, and fix count reset
- Add CollectResult struct with collected/uncollectable/candidates/duration
- Add candidates and duration fields to GcStats and gc.get_stats()
- Pass CollectResult to gc.callbacks info dict
- Reset generation counts for all collected generations (0..=N)
- Return 0 for third value in gc.get_threshold() (3.13+)
- Implement gc.get_referrers() by scanning all tracked objects
- Add DEBUG_COLLECTABLE output for collectable objects
- Update test_gc.py to expect candidates/duration in stats
* Update test_gc from v3.14.3
* Update test_gc.py from CPython v3.15.0a5
Taken from v3.15 (not v3.14.3) because get_stats() candidates/duration
fields were added in 3.13+ and the corresponding test assertions only
exist in 3.15.
* Fix gc_state build on wasm32: skip Instant timing
* Add candidates/duration to gc callback info, mark v3.15 test failures
* Fix gc.get_referrers to exclude executing frames, fix Future cancelled exc leak
- get_referrers: skip frame objects on the execution stack, since
they are not GC-tracked in CPython (_PyInterpreterFrame)
- _asyncio Future/Task make_cancelled_error_impl: clear the stored
cancelled exception after returning it, matching the Python
_make_cancelled_error behavior
* Fix gc.get_threshold to return actual gen2 threshold value
* Fix inconsistent GC count reset in early-return paths
Use the same reset_end formula in unreachable-empty early returns
as in the main collection path and collecting-empty path.
* Accept keyword arguments in socket.__init__
Use a FromArgs struct instead of a positional-only tuple so that
family, type, proto, and fileno can be passed as keyword arguments.
* Disable comp_inlined in symbol table to match compiler
The compiler does not yet implement PEP 709 inlined comprehensions
(is_inlined_comprehension_context always returns false), but the
symbol table was marking comprehensions as inlined. This mismatch
could cause comprehension-local symbols to be merged into the parent
scope while the compiler still looks them up in a separate scope.
---------
Co-authored-by: CPython Developers <>
* Fix atexit unraisable exception message format
Match PyErr_FormatUnraisable behavior: use
"Exception ignored in atexit callback {func!r}" as err_msg
and pass None as object instead of the callback function.
* Fix atexit unregister deadlock with reentrant __eq__
Release the lock during equality comparison in unregister so
that __eq__ can safely call atexit.unregister or atexit._clear.
Store callbacks in LIFO order (insert at front) and use
identity-based search after comparison to handle list mutations,
matching atexitmodule.c behavior.
Also pass None as err_msg when func.repr() fails, matching
CPython's PyErr_FormatUnraisable fallback.
* Use patched parking_lot_core with fork-safe HASHTABLE reset
parking_lot_core's global HASHTABLE retains stale ThreadData after
fork(), causing segfaults when contended locks enter park(). Use the
patched version from youknowone/parking_lot (rustpython branch) which
registers a pthread_atfork handler to reset the hash table.
Unskip test_asyncio TestFork. Add Manager+fork integration test.
* Unskip fork-related flaky tests after parking_lot fix
With parking_lot_core's HASHTABLE now properly reset via
pthread_atfork, fork-related segfaults and connection errors
in multiprocessing tests should be resolved.
Remove skip/expectedFailure markers from:
- test_concurrent_futures/test_wait.py (6 tests)
- test_concurrent_futures/test_process_pool.py (1 test)
- test_multiprocessing_fork/test_manager.py (all WithManagerTest*)
- test_multiprocessing_fork/test_misc.py (5 tests)
- test_multiprocessing_fork/test_threads.py (2 tests)
- _test_multiprocessing.py (2 shared_memory tests)
Keep test_repr_rlock skipped (flaky thread start latency,
not fork-related).
* Base implementation of _tokenize module
Port from PR #6240 by ShaharNaveh, adapted to current codebase.
Uses ruff_python_parser for tokenization via TokenizerIter.
* Update tokenize from v3.14.3
* Rewrite _tokenize with 2-phase model
Replace per-line reparsing with single-pass tokenization:
- Read all lines via readline, parse once, yield tokens
- Fix token type values (COMMENT=65, NL=66, OP=55)
- Fix NEWLINE/NL end positions and implicit newline handling
- Fix DEDENT positions via look-ahead to next non-DEDENT token
- Handle FSTRING_MIDDLE brace unescaping ({{ → {, }} → })
- Emit implicit NL before ENDMARKER when source lacks trailing newline
- Raise IndentationError from lexer errors
- Remove 13 expectedFailure marks for now-passing tests
---------
Co-authored-by: ShaharNaveh <shaharnaveh@users.noreply.github.com>
Co-authored-by: CPython Developers <>
* Add per-size tuple freelist (20 buckets × 2000 each)
Implement PyTuple freelist matching tuples[PyTuple_MAXSAVESIZE]:
- TupleFreeList with 20 per-size buckets (sizes 1..=20, 2000 capacity each)
- freelist_push uses pre-clear size hint for correct bucket selection
- freelist_pop takes &Self payload to select bucket by size
- Type guard in new_ref handles structseq types sharing PyTuple vtable
- Add pyinner_layout<T>() helper for custom freelist Drop impls
- Update freelist_pop/push signatures across all freelist types
* freelist: exact type check at pop call-site
Move exact-type filtering from freelist_pop implementations to the
single call-site in new_ref. This prevents structseq and other subtypes
from popping tuple freelist entries entirely, rather than popping and
then deallocating on type mismatch.
Add Context::try_genesis() that returns None during bootstrap to avoid
deadlock when genesis() is called during Context initialization.
* Move exact type check from freelist_push to call-site in default_dealloc
Remove typ parameter from freelist_push trait signature. The exact type
check is now done once at the call-site alongside the heaptype check,
simplifying all freelist_push implementations.
* Remove freelist_hint; call freelist_push before tp_clear
By calling freelist_push before tp_clear, the payload is still intact
and can be read directly (e.g. tuple element count for bucket selection).
This eliminates freelist_hint and the hint parameter entirely.
* Introduce TimeoutSeconds utility type for timeout parameters
Follow-up refactoring from #7237.
Python timeout parameters typically accept both float and int. Several
places in the codebase used Either<f64, i64> for this, each repeating
the same match-and-convert boilerplate. This extracts that into a
TimeoutSeconds type in vm::function::number.
Refactored sites:
- _sqlite3::ConnectArgs.timeout
- _thread::AcquireArgs.timeout
- _thread::ThreadHandle::join timeout
Either<f64, i64> in time.rs (6 sites) left unchanged: those are
timestamp values with per-branch logic (floor, range checks, etc).
Either<f64, isize> in select.rs also left unchanged (different type).
* Validate timeout in ThreadHandle::join to prevent panic
* refactor: move TimeoutSeconds from number to time module
- Remove weak_list from ObjExt, allocate WeakRefList as its own
prefix slot before PyInner
- Add MANAGED_WEAKREF flag (1 << 3) to PyTypeFlags
- Normalize MANAGED_WEAKREF from HAS_WEAKREF after flag assembly
- Use Layout::extend offsets in bootstrap allocator
The method cache (TYPE_CACHE) was storing strong references (Arc
clones) to cached attribute values, which inflated sys.getrefcount().
This caused intermittent test_memoryview failures where refcount
assertions would fail depending on GC collection timing.
Store borrowed raw pointers instead. Safety is guaranteed because:
- type_cache_clear() nullifies all entries during GC collection,
before the collector breaks cycles
- type_cache_clear_version() nullifies entries when a type is
modified, before the source dict entry is removed
- Readers use try_to_owned_from_ptr (safe_inc) to atomically
validate and increment the refcount on cache hit
* apply more allow_threads
* Simplify STW thread state transitions
- Fix park_detached_threads: successful CAS no longer sets
all_suspended=false, avoiding unnecessary polling rounds
- Replace park_timeout(50µs) with park() in wait_while_suspended
- Remove redundant self-suspension in attach_thread and detach_thread;
the STW controller handles DETACHED→SUSPENDED via park_detached_threads
- Add double-check under mutex before condvar wait to prevent lost wakes
- Remove dead stats_detach_wait_yields field and add_detach_wait_yields
* Representable for ThreadHandle
* Set ThreadHandle state to Running in parent thread after spawn
Like CPython's ThreadHandle_start, set RUNNING state in the parent
thread immediately after spawn() succeeds, rather than in the child.
This eliminates a race where join() could see Starting state if called
before the child thread executes.
Also reverts the macOS skip for test_start_new_thread_failed since the
root cause is fixed.
* Set ThreadHandle state to Running in parent thread after spawn
* Add debug_assert for thread state in start_the_world
* Unskip now-passing test_get_event_loop_thread and test_start_new_thread_at_finalization
* Wrap IO locks and file ops in allow_threads
Add lock_wrapped to ThreadMutex for detaching thread state
while waiting on contended locks. Use it for buffered and
text IO locks. Wrap FileIO read/write in allow_threads via
crt_fd to prevent STW hangs on blocking file operations.
* Use std::sync for thread start/ready events
Replace parking_lot Mutex/Condvar with std::sync (pthread-based)
for started_event and handle_ready_event. This prevents hangs
in forked children where parking_lot's global HASHTABLE may be
corrupted.
On Windows, SimpleQueue skips write locking because pipe
writes are assumed atomic. Without GIL, PipeConnection.
_send_bytes races on _send_ov when multiple threads call
send_bytes concurrently (e.g. _terminate_pool vs workers).
Wait for pending overlapped writes inside WriteFile before
returning to Python, so ERROR_IO_PENDING is never exposed
and the _send_ov assignment window is negligible.
* Suspend Python threads before fork()
Add stop-the-world thread suspension around fork() to prevent
deadlocks from locks held by dead parent threads in the child.
- Thread states: DETACHED / ATTACHED / SUSPENDED with atomic CAS
transitions matching _PyThreadState_{Attach,Detach,Suspend}
- stop_the_world / start_the_world: park all non-requester threads
before fork, resume after (parent) or reset (child)
- allow_threads (Py_BEGIN/END_ALLOW_THREADS): detach around blocking
syscalls (os.read/write, waitpid, Lock.acquire, time.sleep) so
stop_the_world can force-park via CAS
- Acquire/release import lock around fork lifecycle
- zero_reinit_after_fork: generic lock reset for parking_lot types
- gc_clear_raw: detach dict instead of clearing entries
- Lock-free double-check for descriptor cache reads (no read-side
seqlock); write-side seqlock retained for writer serialization
- fork() returns PyResult, checks PythonFinalizationError, calls
sys.audit
* Object header slimming: prefix allocation for ObjExt
Extract dict, weak_list, and slots fields from PyInner<T> into a
separate ObjExt struct allocated as a prefix before PyInner using
Layout::extend(). Objects that don't need these fields (int, str,
float, list, tuple, dict, etc.) skip the prefix entirely.
- Add HAS_WEAKREF flag to PyTypeFlags for per-type weakref control
- Add HAS_EXT bit to GcBits indicating prefix presence
- Define ObjExt struct with dict, weak_list, slots
- Shrink PyInner header from ~80-88 bytes to ~32 bytes for lightweight objects
- Update all accessor methods to go through ext_ref()
- Update bootstrap type hierarchy to use prefix allocation
- Add __weakref__ getset descriptor for heap types
- Set HAS_WEAKREF on builtin types that support weak references
- Remove test_weak_keyed_bad_delitem expectedFailure (now passes)
* Add HAS_WEAKREF to _asyncio Future/Task, rename weakref helpers
- Add HAS_WEAKREF flag to PyFuture and PyTask (matches CPython)
- Rename subtype_getweakref/setweakref to subtype_get_weakref/set_weakref
to fix cspell unknown word lint
* Add HAS_WEAKREF to array, deque, _grouper; remove expectedFailure markers
- Add HAS_WEAKREF to PyArray and PyDeque (matches CPython)
- Add HAS_WEAKREF to PyItertoolsGrouper (internal use by groupby)
- Remove 6 expectedFailure markers from test_dataclasses for weakref/slots
tests that now pass
* Add HAS_WEAKREF to code, union, partial, lock, IO, mmap, sre, sqlite3, typevar types
Add HAS_WEAKREF flag to built-in types that support weakref:
- PyCode, PyUnion, PyPartial, Lock, RLock
- All IO base/concrete classes (_IOBase, _RawIOBase, _BufferedIOBase,
_TextIOBase, BufferedReader, BufferedWriter, BufferedRandom,
BufferedRWPair, TextIOWrapper, StringIO, BytesIO, FileIO,
WindowsConsoleIO)
- PyMmap, sre Pattern, sqlite3 Connection/Cursor
- TypeVar, ParamSpec, ParamSpecArgs, ParamSpecKwargs, TypeVarTuple
Remove 3 expectedFailure markers from test_descr for now-passing tests.
* Add HAS_DICT to type flags and handle non-METHOD/CLASS in descr_get
- Add HAS_DICT flag to PyType (type metaclass) alongside existing
HAS_WEAKREF. All type objects are instances of type and need dict
support, matching CPython's PyType_Type.
- Replace unimplemented!() in PyMethodDescriptor::descr_get with
fallback to bind obj directly, matching CPython's method_get which
uses PyCFunction_NewEx for non-METH_METHOD methods.
* Fix ext detection, HeapMethodDef ownership, WASM error
- Remove HAS_EXT gc_bits flag; detect ext from type flags
using raw pointer reads to avoid Stacked Borrows violations
- Store HeapMethodDef owner in payload instead of dict hack
- Clear dict entries in gc_clear_raw to break cycles
- Add WASM error fallback when exception serialization fails
* Implement locale-aware 'n' format specifier for int, float, complex
Add LocaleInfo struct and locale-aware formatting methods to FormatSpec.
The 'n' format type now reads thousands_sep, decimal_point, and grouping
from C localeconv() and applies proper locale-based number grouping.
Remove @unittest.skip from test_format.test_locale.
* Fix complex 'n' format and remove locale expectedFailure markers
Rewrite format_complex_locale to reuse format_complex_re_im, matching
formatter_unicode.c: add_parens=0 and skip_re=0 for 'n' type.
Remove @expectedFailure from test_float__format__locale and
test_int__format__locale in test_types.py.
* Auto-format: cargo fmt --all
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
ShaharNaveh