* gc: add CollectResult, stats fields, get_referrers, and fix count reset
- Add CollectResult struct with collected/uncollectable/candidates/duration
- Add candidates and duration fields to GcStats and gc.get_stats()
- Pass CollectResult to gc.callbacks info dict
- Reset generation counts for all collected generations (0..=N)
- Return 0 for third value in gc.get_threshold() (3.13+)
- Implement gc.get_referrers() by scanning all tracked objects
- Add DEBUG_COLLECTABLE output for collectable objects
- Update test_gc.py to expect candidates/duration in stats
* Update test_gc from v3.14.3
* Update test_gc.py from CPython v3.15.0a5
Taken from v3.15 (not v3.14.3) because get_stats() candidates/duration
fields were added in 3.13+ and the corresponding test assertions only
exist in 3.15.
* Fix gc_state build on wasm32: skip Instant timing
* Add candidates/duration to gc callback info, mark v3.15 test failures
* Fix gc.get_referrers to exclude executing frames, fix Future cancelled exc leak
- get_referrers: skip frame objects on the execution stack, since
they are not GC-tracked in CPython (_PyInterpreterFrame)
- _asyncio Future/Task make_cancelled_error_impl: clear the stored
cancelled exception after returning it, matching the Python
_make_cancelled_error behavior
* Fix gc.get_threshold to return actual gen2 threshold value
* Fix inconsistent GC count reset in early-return paths
Use the same reset_end formula in unreachable-empty early returns
as in the main collection path and collecting-empty path.
* Accept keyword arguments in socket.__init__
Use a FromArgs struct instead of a positional-only tuple so that
family, type, proto, and fileno can be passed as keyword arguments.
* Disable comp_inlined in symbol table to match compiler
The compiler does not yet implement PEP 709 inlined comprehensions
(is_inlined_comprehension_context always returns false), but the
symbol table was marking comprehensions as inlined. This mismatch
could cause comprehension-local symbols to be merged into the parent
scope while the compiler still looks them up in a separate scope.
---------
Co-authored-by: CPython Developers <>
* Fix atexit unraisable exception message format
Match PyErr_FormatUnraisable behavior: use
"Exception ignored in atexit callback {func!r}" as err_msg
and pass None as object instead of the callback function.
* Fix atexit unregister deadlock with reentrant __eq__
Release the lock during equality comparison in unregister so
that __eq__ can safely call atexit.unregister or atexit._clear.
Store callbacks in LIFO order (insert at front) and use
identity-based search after comparison to handle list mutations,
matching atexitmodule.c behavior.
Also pass None as err_msg when func.repr() fails, matching
CPython's PyErr_FormatUnraisable fallback.
* Use patched parking_lot_core with fork-safe HASHTABLE reset
parking_lot_core's global HASHTABLE retains stale ThreadData after
fork(), causing segfaults when contended locks enter park(). Use the
patched version from youknowone/parking_lot (rustpython branch) which
registers a pthread_atfork handler to reset the hash table.
Unskip test_asyncio TestFork. Add Manager+fork integration test.
* Unskip fork-related flaky tests after parking_lot fix
With parking_lot_core's HASHTABLE now properly reset via
pthread_atfork, fork-related segfaults and connection errors
in multiprocessing tests should be resolved.
Remove skip/expectedFailure markers from:
- test_concurrent_futures/test_wait.py (6 tests)
- test_concurrent_futures/test_process_pool.py (1 test)
- test_multiprocessing_fork/test_manager.py (all WithManagerTest*)
- test_multiprocessing_fork/test_misc.py (5 tests)
- test_multiprocessing_fork/test_threads.py (2 tests)
- _test_multiprocessing.py (2 shared_memory tests)
Keep test_repr_rlock skipped (flaky thread start latency,
not fork-related).
* apply more allow_threads
* Simplify STW thread state transitions
- Fix park_detached_threads: successful CAS no longer sets
all_suspended=false, avoiding unnecessary polling rounds
- Replace park_timeout(50µs) with park() in wait_while_suspended
- Remove redundant self-suspension in attach_thread and detach_thread;
the STW controller handles DETACHED→SUSPENDED via park_detached_threads
- Add double-check under mutex before condvar wait to prevent lost wakes
- Remove dead stats_detach_wait_yields field and add_detach_wait_yields
* Representable for ThreadHandle
* Set ThreadHandle state to Running in parent thread after spawn
Like CPython's ThreadHandle_start, set RUNNING state in the parent
thread immediately after spawn() succeeds, rather than in the child.
This eliminates a race where join() could see Starting state if called
before the child thread executes.
Also reverts the macOS skip for test_start_new_thread_failed since the
root cause is fixed.
* Set ThreadHandle state to Running in parent thread after spawn
* Add debug_assert for thread state in start_the_world
* Unskip now-passing test_get_event_loop_thread and test_start_new_thread_at_finalization
* Wrap IO locks and file ops in allow_threads
Add lock_wrapped to ThreadMutex for detaching thread state
while waiting on contended locks. Use it for buffered and
text IO locks. Wrap FileIO read/write in allow_threads via
crt_fd to prevent STW hangs on blocking file operations.
* Use std::sync for thread start/ready events
Replace parking_lot Mutex/Condvar with std::sync (pthread-based)
for started_event and handle_ready_event. This prevents hangs
in forked children where parking_lot's global HASHTABLE may be
corrupted.
* Suspend Python threads before fork()
Add stop-the-world thread suspension around fork() to prevent
deadlocks from locks held by dead parent threads in the child.
- Thread states: DETACHED / ATTACHED / SUSPENDED with atomic CAS
transitions matching _PyThreadState_{Attach,Detach,Suspend}
- stop_the_world / start_the_world: park all non-requester threads
before fork, resume after (parent) or reset (child)
- allow_threads (Py_BEGIN/END_ALLOW_THREADS): detach around blocking
syscalls (os.read/write, waitpid, Lock.acquire, time.sleep) so
stop_the_world can force-park via CAS
- Acquire/release import lock around fork lifecycle
- zero_reinit_after_fork: generic lock reset for parking_lot types
- gc_clear_raw: detach dict instead of clearing entries
- Lock-free double-check for descriptor cache reads (no read-side
seqlock); write-side seqlock retained for writer serialization
- fork() returns PyResult, checks PythonFinalizationError, calls
sys.audit
* Object header slimming: prefix allocation for ObjExt
Extract dict, weak_list, and slots fields from PyInner<T> into a
separate ObjExt struct allocated as a prefix before PyInner using
Layout::extend(). Objects that don't need these fields (int, str,
float, list, tuple, dict, etc.) skip the prefix entirely.
- Add HAS_WEAKREF flag to PyTypeFlags for per-type weakref control
- Add HAS_EXT bit to GcBits indicating prefix presence
- Define ObjExt struct with dict, weak_list, slots
- Shrink PyInner header from ~80-88 bytes to ~32 bytes for lightweight objects
- Update all accessor methods to go through ext_ref()
- Update bootstrap type hierarchy to use prefix allocation
- Add __weakref__ getset descriptor for heap types
- Set HAS_WEAKREF on builtin types that support weak references
- Remove test_weak_keyed_bad_delitem expectedFailure (now passes)
* Add HAS_WEAKREF to _asyncio Future/Task, rename weakref helpers
- Add HAS_WEAKREF flag to PyFuture and PyTask (matches CPython)
- Rename subtype_getweakref/setweakref to subtype_get_weakref/set_weakref
to fix cspell unknown word lint
* Add HAS_WEAKREF to array, deque, _grouper; remove expectedFailure markers
- Add HAS_WEAKREF to PyArray and PyDeque (matches CPython)
- Add HAS_WEAKREF to PyItertoolsGrouper (internal use by groupby)
- Remove 6 expectedFailure markers from test_dataclasses for weakref/slots
tests that now pass
* Add HAS_WEAKREF to code, union, partial, lock, IO, mmap, sre, sqlite3, typevar types
Add HAS_WEAKREF flag to built-in types that support weakref:
- PyCode, PyUnion, PyPartial, Lock, RLock
- All IO base/concrete classes (_IOBase, _RawIOBase, _BufferedIOBase,
_TextIOBase, BufferedReader, BufferedWriter, BufferedRandom,
BufferedRWPair, TextIOWrapper, StringIO, BytesIO, FileIO,
WindowsConsoleIO)
- PyMmap, sre Pattern, sqlite3 Connection/Cursor
- TypeVar, ParamSpec, ParamSpecArgs, ParamSpecKwargs, TypeVarTuple
Remove 3 expectedFailure markers from test_descr for now-passing tests.
* Add HAS_DICT to type flags and handle non-METHOD/CLASS in descr_get
- Add HAS_DICT flag to PyType (type metaclass) alongside existing
HAS_WEAKREF. All type objects are instances of type and need dict
support, matching CPython's PyType_Type.
- Replace unimplemented!() in PyMethodDescriptor::descr_get with
fallback to bind obj directly, matching CPython's method_get which
uses PyCFunction_NewEx for non-METH_METHOD methods.
* Fix ext detection, HeapMethodDef ownership, WASM error
- Remove HAS_EXT gc_bits flag; detect ext from type flags
using raw pointer reads to avoid Stacked Borrows violations
- Store HeapMethodDef owner in payload instead of dict hack
- Clear dict entries in gc_clear_raw to break cycles
- Add WASM error fallback when exception serialization fails
* Implement locale-aware 'n' format specifier for int, float, complex
Add LocaleInfo struct and locale-aware formatting methods to FormatSpec.
The 'n' format type now reads thousands_sep, decimal_point, and grouping
from C localeconv() and applies proper locale-based number grouping.
Remove @unittest.skip from test_format.test_locale.
* Fix complex 'n' format and remove locale expectedFailure markers
Rewrite format_complex_locale to reuse format_complex_re_im, matching
formatter_unicode.c: add_parens=0 and skip_re=0 for 'n' type.
Remove @expectedFailure from test_float__format__locale and
test_int__format__locale in test_types.py.
* Auto-format: cargo fmt --all
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Fix thread-safety in GC, type cache, and instruction cache
GC / refcount:
- Add safe_inc() check for strong()==0 in RefCount
- Add try_to_owned() to PyObject for atomic refcount acquire
- Replace strong_count()+to_owned() with try_to_owned() in GC
collection and weakref callback paths to prevent TOCTOU races
Type cache:
- Add proper SeqLock (sequence counter) to TypeCacheEntry
- Readers spin-wait on odd sequence, validate after read
- Writers bracket updates with begin_write/end_write
- Use try_to_owned + pointer revalidation on read path
- Call modified() BEFORE attribute modification in set_attr
Instruction cache:
- Add pointer_cache (AtomicUsize array) to CodeUnits for
single atomic pointer load/store (prevents torn reads)
- Add try_read_cached_descriptor with try_to_owned + pointer
and version revalidation after increment
- Add write_cached_descriptor with version-bracketed writes
RLock:
- Fix release() to check is_owned_by_current_thread
- Add _release_save/_acquire_restore methods
* Fix RLock _acquire_restore tuple handling and unxfail threading test
* Align type cache seqlock writer protocol with CPython
* RLock: use single parking_lot level, track recursion manually
Instead of calling lock()/unlock() N times for recursion depth N,
keep parking_lot at 1 level and manage the count ourselves.
This makes acquire/release O(1) and matches CPython's
_PyRecursiveMutex approach (lock once + set level directly).
* Add try_to_owned_from_ptr to avoid &PyObject on stale ptrs
Use addr_of! to access ref_count directly from a raw pointer
without forming &PyObject first. Applied in type cache and
instruction cache hit paths where the pointer may be stale.
* Fix CI: spelling typo and xfail flaky test_thread_safety
- Fix "minimising" -> "minimizing" for cspell
- xfail test_thread_safety: dict iteration races with
concurrent GC mutations in _finalizer_registry
* Reduce per-instruction overhead in interpreter loop
- Defer prev_line update to when tracing is active
- Inline skip_caches_if_fallthrough, compute cache_entries once per instruction
* Unmark test_pdb_set_trace EXPECTED_FAILURE
prev_line deferral fix corrects pdb line tracking, making
these doctests pass.
* Fix symbol table sub_table desync for non-simple annotation targets
Non-simple annotations (subscript/attribute/parenthesized targets like
`a[0]: expr`) were scanned in the annotation scope during symbol table
analysis, creating sub_tables for any comprehensions. But codegen only
compiles simple name annotations into __annotate__, so those sub_tables
were never consumed. This caused subsequent simple annotations'
comprehension sub_tables to get the wrong index, resulting in
"the symbol 'X' must be present in the symbol table" errors.
Fix: skip entering annotation scope for non-simple annotations since
they are never compiled into __annotate__.
* Validate forbidden expressions in non-simple annotation targets
Fix cspell "desynchronize" warning and validate yield/await/named/async
comprehension expressions in non-simple annotations without creating
annotation scopes.
* Restore in_annotation flag before propagating error
* Emit CACHE code units in bytecode to match CPython 3.14
- Add cache_entries() method to Instruction enum
- Emit CACHE code units after opcodes in finalize_code
- Handle NO_LOCATION (line=-1) in linetable for CACHE entries
- Account for CACHE entries in exception table generation
- Skip CACHE entries in VM execution loop (with jump detection)
- Handle CACHE in InstrumentedLine/InstrumentedInstruction/InstrumentedForIter/InstrumentedNotTaken
- Skip CACHE in monitoring instrumentation phases
- Update co_branches() for cache-adjusted offsets
- Restore _cache_format in Lib/opcode.py
- Remove expectedFailure from test_c_call, test_start_offset
* Use relative jump offsets and fix bytecode layout
- Convert jump arguments from absolute to relative offsets
in frame.rs, monitoring.rs, and stack_analysis
- Add jump_relative_forward/backward helpers to ExecutingFrame
- Resolve pseudo jump instructions before offset fixpoint loop
- Emit NOP for break, continue, pass to match line-tracing
- Fix async for: emit EndAsyncFor with correct target, add NotTaken
- Fix comprehension if-cleanup to use separate block
- Fix super() source range for multi-line calls
- Fix NOP removal to preserve line-marker NOPs
- Fix InstrumentedLine cache skipping after re-dispatch
- Match InstrumentedResume/YieldValue in yield_from_target
- Remove CALL_FUNCTION_EX cache entry from opcode.py
- Remove resolved expectedFailure markers
* Align CPython 3.14 LOAD_GLOBAL null-bit and RERAISE semantics
* Remove redundant CPython-referencing comments
Clean up comments that unnecessarily mention CPython per project
convention. Replace with concise descriptions of the behavior itself.
- Add TraceEvent::Exception and Opcode variants with profile filtering
- Extract dispatch_traced_frame helper for Call/Return trace events
- Fire exception trace on new raises, SEND StopIteration, FOR_ITER StopIteration
- Fire opcode trace events gated by f_trace_opcodes
- Move prev_line to FrameState for persistence across generator suspend/resume
- Reset prev_line in gen_throw for correct LINE monitoring after yield
- Add per-code event mask (events_for_code) to prevent unrelated code instrumentation
- Remove expectedFailure markers from test_bdb (5) and test_sys_setprofile (14)
Add non-ASCII string check to _hashlib.compare_digest, matching the
behavior of _operator._compare_digest. When both arguments are strings,
non-ASCII characters now correctly raise TypeError.
Also replace the non-constant-time == comparison with constant_time_eq
for proper timing-attack resistance, and return PyResult<bool> instead
of PyResult<PyObjectRef>.
Add #[pygetset] getter/setter for Cursor.row_factory so that Python-level
attribute access reads/writes the Rust struct field instead of the
instance dict.
Fix Connection.cursor() to only propagate the connection's row_factory
to the cursor when the connection's row_factory is not None, matching
CPython behavior. Previously it unconditionally overwrote the cursor's
row_factory, discarding any factory set by a cursor subclass __init__.
When a custom `closed` property on the underlying buffer calls
`detach()` during the `file_closed()` check in `close()`, the
wrapper's internal buffer becomes None. Subsequent flush/close
operations then fail with AttributeError on NoneType.
Add a guard after the `file_closed()` check to detect if the buffer
was detached reentrantly, and return early in that case (detach has
already flushed the stream).
This mirrors the fix applied in CPython
(https://github.com/python/cpython/issues/142594\).
