gh-159652: Fix import lock leak when the C stack is nearly exhausted (GH-159660)
When an import failed with a stack overflow RecursionError, releasing the import locks on the error path could itself raise RecursionError, because the cleanup runs in a new C frame. This left the global import lock or a module lock held forever, and later imports in other threads blocked indefinitely.
_imp.acquire_lock() now raises RecursionError when there is not enough stack left to release the lock. importlib calls it before taking any import lock, so this covers the per-module locks too. The module lock weakref callback skips cleanup in that case instead of
reporting an unraisable exception.
(cherry picked from commit 926b0bf311)
Co-authored-by: Cary Hawkins <hawkinscary23@gmail.com>
Concurrent calls to codecs.unregister() can shrink the search path between its length check and indexed lookup in a free-threaded build. The lookup then returns NULL and an unconditional decref crashes the interpreter; hold the registry mutex across the scan and removal. Also reported in gh-157078. Backport to 3.15 and 3.14.
(cherry picked from commit 784b305df8)
CPython copies the trampoline templates into executable memory. Keep the
ELF source in .rodata so BOLT preserves its bytes and boundaries.
(cherry picked from commit 1ff83ad7ad)
Co-authored-by: Zanie Blue <contact@zanie.dev>
Adapt the reference-flag propagation from GH-156819 to fix gh-160299 without backporting the comprehensive changes to comprehension scope representation and compilation.
(cherry picked from commit cc3a0c9d3a)
Resetting a ContextVar can finalize the replaced value, whose finalizer may set the variable again. Updating the lookup cache after releasing the old context root overwrites that newer cached value, so get() can return a stale value or use a freed object. Update the cache before the old root can be released. Backport to 3.15 and 3.14.
(cherry picked from commit f4f8b933e3)
Co-authored-by: Jelle Zijlstra <jelle@openai.com>
A context watcher callback can clear a later watcher while a context-switch notification is being dispatched. The dispatcher still uses its earlier snapshot of active watchers, so it calls a NULL callback and crashes; check that each callback is still registered before invoking it. Backport to 3.15 and 3.14.
(cherry picked from commit 0a5762621f)
Co-authored-by: Jelle Zijlstra <jelle@openai.com>
Propagate the error from PyUnicode_FromOrdinal() before using the
result.
(cherry picked from commit 5b0ab21364)
Co-authored-by: Jelle Zijlstra <jelle@openai.com>
* [3.14] gh-158121: Preserve instruction events when line monitoring is disabled (GH-158456)
(cherry picked from commit f0f76de0c8)
Co-authored-by: limadog9 <jakewallin42@gmail.com>
Fix typo in free-threaded GC prefetch check for 32-bit x86 MSVC.
(cherry picked from commit 1b015e66f0)
Co-authored-by: Pengyu Lee <lipengyu@kylinos.cn>
Use free() instead of PyMem_RawFree() in pythread_wrapper() to avoid
a data race if another thread calls PyMem_SetAllocator() in parallel.
(cherry picked from commit d1878ccb44)
Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>
gh-159320: Enable stack size detection on Android (GH-159321)
Extend the use of pthread_getattr_np in stack size detection from any glibc linux to any glibc *or* android linux. Update tests for android's new recursion limit.
(cherry picked from commit 8ff41c2ae7)
Co-authored-by: T. Wouters <thomas@python.org>
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Set elements are written in the order of their dumps. They were sorted
as (dump, element) pairs, so elements with the same dump were compared.
That failed for a set of several complex NaNs, which the compiler can
produce by constant folding, making the module impossible to marshal.
On this branch the failure was reported as MemoryError.
Sort (dump, index, element) tuples, in which the indexes are all
different. Elements with the same dump stay in the order of the set,
as float NaNs did before.
(cherry picked from commit 1967a87ad3)
Marshalling a set element can run Python code (__buffer__, an audit
hook) which changes the set. The number of elements was read before
the loop, so a set which grew was written past the end of the list and
one which shrank left NULL items in the list which was then sorted.
Raise RuntimeError("set changed size during iteration") instead.
(cherry picked from commit cd71799c47)
[3.15] gh-158893: Make os.strerror() thread-safe (#159120)
* gh-157695: Parse also _Py_HAVE_xxx variables in sysconfig (#158926)
Add parse_config_h() tests to test_sysconfig.
(cherry picked from commit 978a8befa5)
* gh-158893: Make os.strerror() thread-safe (#158927)
Make os.strerror() thread-safe: use the reentrant strerror_r()
function if available.
* The configure script now checks if strerror_r() is supported.
* Add a stress test to test_free_threading.test_os (new module).
* Add a comment on decode_current_locale() assertion which fails if
the input string is mutated.
* Add an assertion to _Py_DecodeLocale() to detect if the input
string was mutated during the function call.
* gh-158893: Add internal _Py_strerror() function (#158982)
Add a new internal _Py_strerror() function to Python/fileutils.c. It
uses strerror_r() if available, or use strerror() otherwise.
Replace all strerror(code) calls with _Py_strerror(code).
(cherry picked from commit 15dd7356f3)
(cherry picked from commit c774d1526f)
gh-152075: Avoid lock contention in _Py_Specialize_LoadGlobal under free threading (gh-153720)
Under high thread concurrency in free-threaded builds, `_Py_Specialize_LoadGlobal` suffers from lock contention when acquiring the critical section mutexes for the `globals` and `builtins` dictionaries during bytecode specialization.
This PR skips LOAD_GLOBAL bytecode specialization if acquiring the two object mutexes would block.
(cherry picked from commit ac8ba0ca5a)
Co-authored-by: Peter Hawkins <phawkins@google.com>
* Run a job that the test suite with MSan to the CI (#158625)
* Run the test suite with MSan in CI
* Additional fixes
* Add `_Py_MSAN_UNPOISON_STRING`
* Apply Victor's suggestions
Co-authored-by: Victor Stinner <victor.stinner@gmail.com>
* Apply Victor's suggestions
Co-authored-by: Victor Stinner <victor.stinner@gmail.com>
---------
Co-authored-by: Victor Stinner <victor.stinner@gmail.com>
(cherry picked from commit b93fb19a6e)
* Mark bytes returned by getrandom as initialized for MSan
* Handle invalid non-ASCII struct formats in the fuzz harness
---------
Co-authored-by: Stan Ulbrych <stan@python.org>
Co-authored-by: Victor Stinner <victor.stinner@gmail.com>
gh-158446: Reject float format precision near INT_MAX (GH-158474)
Formatting a float or complex with a precision within about 1000 of
INT_MAX could crash or produce incorrect output.
PyOS_double_to_string() now raises ValueError("precision too big") for
such precisions, as the format string parsers already do for
precisions above INT_MAX.
The limit applies regardless of presentation type or value, so a few
calls that previously succeeded (inf, nan, or 'g' with such a
precision) now raise as well.
(cherry picked from commit b7b4f3ecf2)
Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
gh-157549: Use sysconf() for the page size in remote_debug.h (GH-157800)
getpagesize() is not an ISO C or POSIX function. Darwin hides the prototype under -std=c11, so the _remote_debugging build fails with -Werror=implicit-function-declaration. That is the failure in the reported x86_64-apple-darwin → arm64-apple-darwin cross build.
Call sysconf(_SC_PAGESIZE) instead, which is already how resource.getpagesize() works. Fall back to 4096 if sysconf() fails.
--without-remote-debug is unchanged. That flag only disables the target-side attach protocol (Py_REMOTE_DEBUG); it does not exclude the _remote_debugging client module.
(cherry picked from commit 9193375382)
Co-authored-by: ノウラ | Flare <nouraellm@gmail.com>
gh-128509: Use bytes singletons in marshal (GH-157398)
Replace soft deprecated PyBytes_FromStringAndSize(NULL, n) with
PyBytes_FromStringAndSize(str, n) so marshal can get bytes singleton.
(cherry picked from commit 658612ae77)
Co-authored-by: Victor Stinner <vstinner@python.org>
In the legacy Windows stdio mode PyConfig.stdio_encoding is now initialized
with the console output code page instead of the ANSI code page.
Only one encoding can be specified, so sys.stdin.encoding is still wrong if
the input code page differs from the output one.
(cherry picked from commit 0bb312a4ea)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
It was hard coded to map file descriptors 0, 1 and 2 to the console code
page, but a console can be opened as any file descriptor, and other
character devices, like NUL, are not consoles.
The file handle is now queried. The UTF-8 code page is also now reported
as "utf-8" instead of "cp65001".
(cherry picked from commit b4aea41d1c)
