In free-threaded builds, concurrent calls to PyDict_AddWatcher, PyDict_ClearWatcher, PyDict_Watch, and PyDict_Unwatch can race on the shared callback array and the per-dict watcher tags. This change adds a mutex to serialize watcher registration and removal, atomic operations for tag updates, and atomic acquire/release synchronization for callback dispatch in _PyDict_SendEvent.
(cherry picked from commit 8a4895985f)
Co-authored-by: Alper <alperyoney@fb.com>
Fixes a sequential consistency bug whereby two threads that are deleting a struct member may observe both their deletions to be successful.
(cherry picked from commit 1bdfc0f253)
Co-authored-by: Daniele Parmeggiani <8658291+dpdani@users.noreply.github.com>
gh-149122: Fix segfault in compiler when certain builtin functions are passed a coroutine as arg (GH-149138)
(cherry picked from commit 16f292ef4e)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
Improve `hash()` builtin docstring with caveats.
Mention its return type and that the value can be expected to change between
processes (hash randomization).
Why? The `hash` builtin gets reached for and used by a lot of people whether it
is the right tool or not. IDEs surface docstrings and people use pydoc and
`help(hash)`.
(cherry picked from commit 665b7dfcfa)
gh-146455: Fix O(N²) in add_const() after constant folding moved to CFG (GH-146456)
The add_const() function in flowgraph.c uses a linear search over the
consts list to find the index of a constant. After gh-126835 moved
constant folding from the AST optimizer to the CFG optimizer, this
function is now called N times for N inner tuple elements during
fold_tuple_of_constants(), resulting in O(N²) total time.
Fix by maintaining an auxiliary _Py_hashtable_t that maps object
pointers to their indices in the consts list, providing O(1) lookup.
For a file with 100,000 constant 2-tuples:
- Before: 10.38s (add_const occupies 83.76% of CPU time)
- After: 1.48s
(cherry picked from commit 5d416324c5)
Co-authored-by: zSirius <107359899+zSirius@users.noreply.github.com>
_PyRawMutex_UnlockSlow CAS-removes the waiter from the list and then
calls _PySemaphore_Wakeup, with no handshake. If _PySemaphore_Wait
returns Py_PARK_INTR, the waiter can destroy its stack-allocated
semaphore before the unlocker's Wakeup runs, causing a fatal error from
ReleaseSemaphore / sem_post.
Loop in _PyRawMutex_LockSlow until _PySemaphore_Wait returns Py_PARK_OK,
which is only signalled when a matching Wakeup has been observed.
Also include GetLastError() and the handle in the Windows fatal messages
in _PySemaphore_Init, _PySemaphore_Wait, and _PySemaphore_Wakeup to make
similar races easier to diagnose in the future.
(cherry picked from commit ad3c5b7958)
Forbid marshalling recursive code and slice objects which
cannot be correctly unmarshalled.
Add multiple tests for recursive data structures.
(cherry picked from commit 2e37d83641)
gh-137814: [3.14] Fix __qualname__ of __annotate__ functions in the interpreter
I'd still like to do #137842 on 3.15+, but that requires changing bytecode and we can't
really afford to do that in 3.14. So to fix this in 3.14, let's patch things up in the
ceval loop instead.
This is safe because the compiler only sets __annotate__ to just-created dedicated
annotate functions.
Fixes data races between dict mutation and watch/unwatch on the same dict.
(cherry picked from commit 3ab94d6842)
Co-authored-by: Sam Gross <colesbury@gmail.com>
Align the QSBR thread state array to a 64-byte cache line boundary
and add padding at the end of _PyThreadStateImpl. Depending on heap
layout, the QSBR array could end up sharing a cache line with a
thread's tlbc_index, causing QSBR quiescent state updates to contend
with reads of tlbc_index in RESUME_CHECK. This is sensitive to
earlier allocations during interpreter init and can appear or
disappear with seemingly unrelated changes.
Either change alone is sufficient to fix the specific issue, but both
are worthwhile to avoid similar problems in the future.
(cherry picked from commit 6577d870b0)
gh-146480: Override the exception in _PyErr_SetKeyError() (GH-146486)
If _PyErr_SetKeyError() is called with an exception set, it now
replaces the current exception with KeyError (as expected), instead
of setting a SystemError or failing with a fatal error (in debug
mode).
(cherry picked from commit d4153a9f76)
Co-authored-by: Victor Stinner <vstinner@python.org>
Add special cases for classmethod and staticmethod descriptors in
_PyObject_GetMethodStackRef() to avoid calling tp_descr_get, which
avoids reference count contention on the bound method and underlying
callable. This improves scaling when calling classmethods and
staticmethods from multiple threads.
Also refactor method_vectorcall in classobject.c into a new
_PyObject_VectorcallPrepend() helper so that it can be used by
PyObject_VectorcallMethod as well.
(cherry picked from commit e0f7c1097e)
Fix error handling in _PyFloat_FromDouble_ConsumeInputs() used by
_BINARY_OP_ADD_FLOAT, _BINARY_OP_SUBTRACT_FLOAT and
_BINARY_OP_MULTIPLY_FLOAT opcodes. PyStackRef_FromPyObjectSteal()
must not be called with a NULL pointer.
Fix also _BINARY_OP_INPLACE_ADD_UNICODE opcode.
gh-146092: Handle _PyFrame_GetFrameObject() failures properly (#146124)
* Fix _PyFrame_GetLocals() and _PyFrame_GetLocals() error handling.
* _PyEval_ExceptionGroupMatch() now fails on _PyFrame_GetLocals()
error.
(cherry picked from commit e1e4852133)
Cache one datachunk per tstate to prevent alloc/dealloc thrashing when repeatedly hitting the same call depth at exactly the wrong boundary.
Move new _ts member to the end to not mess up remote debuggers' ideas of the
struct's layout. (The struct is only created by the runtime, and the new
field only used by the runtime, so it should be safe.)
(cherry picked from commit 706fd4ec08)
Co-authored-by: T. Wouters <thomas@python.org>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
This undoes a change made as a part of PR 137470, for compatibility with EMSDK
4.0.19. It adds `emscripten_trampoline` field in `pycore_runtime_structs.h`
and initializes it from JS initialization code with the wasm-gc based trampoline
if possible. Otherwise we fall back to the JS trampoline.
(cherry picked from commit 43fdb7037e)
Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com>
gh-145187: Fix crash on invalid type parameter bound expression in conditional block (GH-145188)
Fix parsing crash found by oss-fuzz
(cherry picked from commit 5e61a16c10)
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
gh-144763: Fix race conditions in tracemalloc (#144779)
Avoid PyUnstable_InterpreterFrame_GetLine() since it uses a critical
section which can lead to a deadlock.
_PyTraceMalloc_Stop() now also calls PyRefTracer_SetTracer() without
holding TABLES_LOCK() to prevent another deadlock.
(cherry picked from commit 83f4fffe3d)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
gh-144601: Avoid sharing exception objects raised in a `PyInit` function across multiple interpreters (GH-144602)
(cherry picked from commit fd6b639a49)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
gh-143650: Fix importlib race condition on import failure (GH-143651)
Fix a race condition where a thread could receive a partially-initialized
module when another thread's import fails. The race occurs when:
1. Thread 1 starts importing, adds module to sys.modules
2. Thread 2 sees the module in sys.modules via the fast path
3. Thread 1's import fails, removes module from sys.modules
4. Thread 2 returns a stale module reference not in sys.modules
The fix adds verification after the "skip lock" optimization in both Python
and C code paths to check if the module is still in sys.modules. If the
module was removed (due to import failure), we retry the import so the
caller receives the actual exception from the import failure rather than
a stale module reference.
(cherry picked from commit ac8b5b6890)
Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
