gh-159603: Use pthread_attr_get_np() on FreeBSD (#159604)
Use pthread_attr_get_np() on FreeBSD to get the stack size (and guard
size) of the current thread.
Check if pthread_attr_get_np() function and <pthread_np.h> header are
available in configure.
Update test_json.test_recursion. On FreeBSD 16 with a default stack size of
512 MB, the json C implementation no longer raises RecursionError in
test_highly_nested_objects_encoding(). Reduce also max_depth from 5,000 to
2,000 so test_highly_nested_objects_encoding() of the Python implementation
only takes 1.6 seconds, instead of 12 seconds.
(cherry picked from commit 71ba39dc05)
gh-158121: Preserve instruction events when line monitoring is disabled (GH-158456)
(cherry picked from commit f0f76de0c8)
Co-authored-by: limadog9 <jakewallin42@gmail.com>
Fix typo in free-threaded GC prefetch check for 32-bit x86 MSVC.
(cherry picked from commit 1b015e66f0)
Co-authored-by: Pengyu Lee <lipengyu@kylinos.cn>
gh-159320: Enable stack size detection on Android (GH-159321)
Extend the use of pthread_getattr_np in stack size detection from any glibc linux to any glibc *or* android linux. Update tests for android's new recursion limit.
(cherry picked from commit 8ff41c2ae7)
Co-authored-by: T. Wouters <thomas@python.org>
Co-authored-by: Malcolm Smith <smith@chaquo.com>
A single-phase init function that creates its module using only the
last component of a dotted name (as pybind11 does for submodules) got
a module whose __name__ was the short name. Set the package context
while calling the init function, as the dynamic loader does, so the
module gets the full name from the spec.
(cherry picked from commit 04862280a2)
Co-authored-by: Itamar Oren <itamarost@gmail.com>
Preserve active attach waiters across successive stop-the-world pauses
so a tight collection loop cannot keep them from making progress.
Keep the waiter-aware resume operation local to pystate.c because this
branch does not have the newer BRC suspend and resume helpers.
(cherry picked from commit f156510960)
gh-157415: Fix data race in pthread wrapper (#157430)
Use free() instead of PyMem_RawFree() in pythread_wrapper() to avoid
a data race if another thread calls PyMem_SetAllocator() in parallel.
Reenable @nomemtest tests on TSAN.
(cherry picked from commit d1878ccb44)
Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>
* Expose two possible builtin JIT gaps
* Add trace and runtime guard for builtin dict
* Check _GUARD_BUILTINS_IS_CANONICAL once per frame
* Add check and fix for globals being folded
(cherry picked from commit 0906d2ab93)
Set elements are written in the order of their dumps. They were sorted
as (dump, element) pairs, so elements with the same dump were compared.
That failed for a set of several complex NaNs, which the compiler can
produce by constant folding, making the module impossible to marshal.
On this branch the failure was reported as MemoryError.
Sort (dump, index, element) tuples, in which the indexes are all
different. Elements with the same dump stay in the order of the set,
as float NaNs did before.
(cherry picked from commit 1967a87ad3)
Marshalling a set element can run Python code (__buffer__, an audit
hook) which changes the set. The number of elements was read before
the loop, so a set which grew was written past the end of the list and
one which shrank left NULL items in the list which was then sorted.
Raise RuntimeError("set changed size during iteration") instead.
(cherry picked from commit cd71799c47)
* [3.15] gh-153364: Make frame, coroutine, and task-waiter chain walks iterative and bounded (GH-153365) (#158813)
gh-153364: Make frame, coroutine, and task-waiter chain walks iterative and bounded (GH-153365)
* let me declare single limit
* use our new limit in process_frame_chain()
* add it in parse_async_frame_chain()
* parse_coro_chain()
* NEWS
* async in the message?
* test
* no race
* process_task_awaited_by
* process_task_awaited_by limit test
* NEWS
* MAX_TASK_WAITER_CHAIN_DEPTH
* TASK_WAITER_CHAIN_DEPTH in test
* TASK_WAITER_CHAIN_DEPTH 256
* prevent the drift with the comment
* better naming, better style
* MAX_TASK_WAITER_CHAIN_DEPTH comment
* task-waiter iterative bfs walk
* iterative coro-walk
* nicer news
* 1 << 14
* comment
* unused read_Py_ssize_t
* fix tombstones
* simplify
* correct msg
* better test
* news for tombstones
* left-over from when testing buggy version
* redundant new line
(cherry picked from commit e0861c6ae7)
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
(cherry picked from commit 9e401cf7f5)
* [3.15] gh-155811: Add a seqcount to `gc_stats` to prevent torn reads (GH-155828) (#158829)
* update_seq
* no need for XCHGL, MOVL is enough?
* gh-155811: Retry an inconsistent GC snapshot once
---------
(cherry picked from commit 5fecd448bb)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 028124039c)
* [3.15] gh-151292: `_remote_debugging`: Do not corrupt the binary file when hitting `OverflowError` (GH-152892) (#158830)
gh-151292: `_remote_debugging`: Do not corrupt the binary file when hitting `OverflowError` (GH-152892)
* the kolektor
* test
* better test
* news
* =Q, move const to the base, not self.running
* gh-151292: Track binary writer finalization state
---------
(cherry picked from commit f839c061d3)
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
(cherry picked from commit 8a7c23f316)
* [3.15] gh-158583: Fix uninitialized memory read in bytes.fromhex() (GH-158584) (#158691)
gh-158583: Fix uninitialized memory read in bytes.fromhex() (GH-158584)
(cherry picked from commit 9d22a5334b)
Co-authored-by: Victor Stinner <vstinner@python.org>
(cherry picked from commit 4f7af46e25)
* [3.15] gh-154194: Degrade frames in Tachyon instead of failing the sample (GH-154195) (#158831)
* gh-154194: Degrade frames in Tachyon instead of failing the sample (#154195)
* degrade gracefully
* news
* better NEWS wording
* do not raise on MAX_REMOTE_STR_READ
* bye MAX_REMOTE_STR_READ
* fix -m asyncio ps|pstree
* test truncation and linetable sentinel
* simpler
* simpler
* redundant now
* respect #157790 in the news
---------
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
(cherry picked from commit 7d25916b41)
* Preserve the stable ABI when creating fallback frame names
---------
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
(cherry picked from commit c27f494061)
* [3.15] Add MSan to CI (GH-158625) (#158832)
Run a job that the test suite with MSan to the CI (#158625)
* Run the test suite with MSan in CI
* Additional fixes
* Add `_Py_MSAN_UNPOISON_STRING`
* Apply Victor's suggestions
* Apply Victor's suggestions
---------
(cherry picked from commit b93fb19a6e)
Co-authored-by: Stan Ulbrych <stan@python.org>
Co-authored-by: Victor Stinner <victor.stinner@gmail.com>
(cherry picked from commit 1bc78dea18)
* [3.15] gh-156810: Write the profiler's collapsed-stack export as UTF-8 (GH-156811) (#156814)
gh-156810: Write the profiler's collapsed-stack export as UTF-8 (GH-156811)
(cherry picked from commit c3706f4c68)
Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
(cherry picked from commit fce28da9b7)
* [3.15] gh-158552: Wait for Windows threads to suspend before blocking sampling (GH-158802) (#158845)
gh-158552: Wait for Windows threads to suspend before blocking sampling (GH-158802)
* gh-158552: Wait for Windows threads to suspend before blocking sampling
* Use a named Windows thread enumeration status constant
(cherry picked from commit 1643525f0f)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
(cherry picked from commit 31288da21d)
* [3.15] gh-152721: Fix quadratic RLE replay time in the profiling binary reader (GH-152722) (#158850)
Backport of GH-152722.
Co-authored-by: tonghuaroot <tonghuaroot@gmail.com>
(cherry picked from commit ebaca2ac2d)
* [3.15] gh-156545: Fix flamegraph export RecursionError on deeply recursive programs (GH-156546) (#158851)
Backport of GH-156546.
Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
(cherry picked from commit 0ac7217040)
* [3.15] gh-158540: Add the profiled script's directory to sys.path (GH-158548) (#158844)
gh-158540: Add the profiled script's directory to sys.path (GH-158548)
* gh-158540: Add the profiled script's directory to sys.path
When a script is profiled with ``python -m profiling.sampling run
script.py`` from another directory, the script cannot import modules
placed next to it, because ``_sync_coordinator._execute_script()``
executes it with the working directory (added by
``_setup_environment()`` for the module case) as ``sys.path[0]``
instead of the script's own directory.
Make the script's directory importable in ``_execute_script()``,
matching the behavior of ``python script.py``.
Add a regression test that runs the coordinator on a script importing
a sibling module.
* Update _sync_coordinator.py Comment simplified
* gh-158540: Resolve symlinks when adding the script directory to sys.path
``python script.py`` resolves symlinks when computing ``sys.path[0]``, so
a script reached through a symlink (``link.py -> sub/where.py``) imports
modules from the real script's directory. Apply ``os.path.realpath()``
before taking the directory name, and make sure the result is placed at
the front of ``sys.path`` even if it was already listed.
Add a regression test for a symlinked script.
---------
(cherry picked from commit 3f02aab9fa)
Co-authored-by: he_tao <53343436+hetaozdh@users.noreply.github.com>
Co-authored-by: Eduardo Villalpando Mello <eduardo.villalpando.mello@gmail.com>
(cherry picked from commit f52d831397)
* [3.15] gh-153838: Skip non-regular source files in the heatmap exporter (GH-153839) (#158853)
Backport of GH-153839.
Co-authored-by: tonghuaroot <tonghuaroot@gmail.com>
(cherry picked from commit 48998df251)
* [3.15] gh-158539: Fix exception mode missing handlers in generators/coroutines (GH-158581) (#158852)
* [3.15] gh-158539: Fix exception mode missing handlers in generators/coroutines (GH-158581)
Backport of GH-158581.
Co-authored-by: LucasZhou <donghao.zhou@outlook.com>
* [3.15] gh-158539: Use portable static assertion messages
* [3.15] gh-158539: Keep layout assertions with debug-offset validation
* [3.15] gh-158539: Use the platform guard for in-process inspection tests
---------
Co-authored-by: LucasZhou <donghao.zhou@outlook.com>
(cherry picked from commit d625ecbb8d)
* [3.15] gh-158522: Fix truncated stack for a task whose coroutine recurses (GH-158526) (#158870)
Co-authored-by: Timofei Ivankov <128279579+deadlovelll@users.noreply.github.com>
(cherry picked from commit a904b397ed)
* gh-156545, gh-158539: Fix deep flamegraph export on small C stacks and macOS runtime lookup (#158874)
(cherry picked from commit 114de198c0)
Include the C-stack test helper from main, introduced by
ce5ae29ef9, which the regression test
requires but 3.15 does not yet provide.
---------
Co-authored-by: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com>
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Stan Ulbrych <stan@python.org>
Co-authored-by: Victor Stinner <victor.stinner@gmail.com>
Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
Co-authored-by: he_tao <53343436+hetaozdh@users.noreply.github.com>
Co-authored-by: Eduardo Villalpando Mello <eduardo.villalpando.mello@gmail.com>
Co-authored-by: LucasZhou <donghao.zhou@outlook.com>
Co-authored-by: Timofei Ivankov <128279579+deadlovelll@users.noreply.github.com>
* gh-157695: Parse also _Py_HAVE_xxx variables in sysconfig (#158926)
Add parse_config_h() tests to test_sysconfig.
(cherry picked from commit 978a8befa5)
* gh-158893: Make os.strerror() thread-safe (#158927)
Make os.strerror() thread-safe: use the reentrant strerror_r()
function if available.
* The configure script now checks if strerror_r() is supported.
* Add a stress test to test_free_threading.test_os (new module).
* Add a comment on decode_current_locale() assertion which fails if
the input string is mutated.
* Add an assertion to _Py_DecodeLocale() to detect if the input
string was mutated during the function call.
* gh-158893: Add internal _Py_strerror() function (#158982)
Add a new internal _Py_strerror() function to Python/fileutils.c. It
uses strerror_r() if available, or use strerror() otherwise.
Replace all strerror(code) calls with _Py_strerror(code).
(cherry picked from commit 15dd7356f3)
gh-152075: Avoid lock contention in _Py_Specialize_LoadGlobal under free threading (gh-153720)
Under high thread concurrency in free-threaded builds, `_Py_Specialize_LoadGlobal` suffers from lock contention when acquiring the critical section mutexes for the `globals` and `builtins` dictionaries during bytecode specialization.
This PR skips LOAD_GLOBAL bytecode specialization if acquiring the two object mutexes would block.
(cherry picked from commit ac8ba0ca5a)
Co-authored-by: Peter Hawkins <phawkins@google.com>
The TYPE_FROZENDICT reader reserved a reference slot but never filled it with
r_ref_insert, unlike TYPE_FROZENSET, so a frozendict referenced more than once
failed to load with ValueError.
(cherry picked from commit b6c11feb67)
Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
Loading a reference-tracked dictionary dereferenced a NULL pointer when
the allocation that registers it for back-references failed under low
memory. It now raises MemoryError, matching the tuple and list paths.
(cherry picked from commit 5b96d39147)
Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
It was hard coded to map file descriptors 0, 1 and 2 to the console code
page, but a console can be opened as any file descriptor, and other
character devices, like NUL, are not consoles.
The file handle is now queried. The UTF-8 code page is also now reported
as "utf-8" instead of "cp65001".
(cherry picked from commit b4aea41d1c)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
In the legacy Windows stdio mode PyConfig.stdio_encoding is now initialized
with the console output code page instead of the ANSI code page.
Only one encoding can be specified, so sys.stdin.encoding is still wrong if
the input code page differs from the output one.
(cherry picked from commit 0bb312a4ea)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
The __main__ module executed as a script or a command has __spec__ set to
None, so warn_explicit(module_globals=globals()) emitted a spurious
DeprecationWarning. It also raised ImportError when the loader was unable
to provide the source of the module: when the module was executed with -m
(the loader can only handle its own module name) or as a command (the
built-in importer has no source).
(cherry picked from commit f2eaf174b2)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
warn_explicit() computed the source line from the loader of the module
whose globals were passed as module_globals, but did not pass it to
WarningMessage. It also raised IndexError if lineno was out of the range
of the module source.
(cherry picked from commit c92e2fd8f9)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-158820: Add null checks to avoid segfaults with lazy imports at shutdown (GH-158821)
Destructors can run after finalization has cleared the interpreter's
lazy_modules set and lazy_pending_submodules dict. Declaring or
resolving a lazy import from one of them passed NULL to PySet_Add(),
PySet_Discard() or the pending-submodules lookup.
Treat the bookkeeping as a no-op once the state is gone, as the other
readers of this state already do.
(cherry picked from commit a6e636148e)
Co-authored-by: Brittany Reynoso <breynoso@meta.com>
Run a job that the test suite with MSan to the CI (#158625)
* Run the test suite with MSan in CI
* Additional fixes
* Add `_Py_MSAN_UNPOISON_STRING`
* Apply Victor's suggestions
* Apply Victor's suggestions
---------
(cherry picked from commit b93fb19a6e)
Co-authored-by: Stan Ulbrych <stan@python.org>
Co-authored-by: Victor Stinner <victor.stinner@gmail.com>
* gh-154194: Degrade frames in Tachyon instead of failing the sample (#154195)
* degrade gracefully
* news
* better NEWS wording
* do not raise on MAX_REMOTE_STR_READ
* bye MAX_REMOTE_STR_READ
* fix -m asyncio ps|pstree
* test truncation and linetable sentinel
* simpler
* simpler
* redundant now
* respect #157790 in the news
---------
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
(cherry picked from commit 7d25916b41)
* Preserve the stable ABI when creating fallback frame names
---------
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
* update_seq
* no need for XCHGL, MOVL is enough?
* gh-155811: Retry an inconsistent GC snapshot once
---------
(cherry picked from commit 5fecd448bb)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
