In a free-threaded build, if allocating the thread-local bytecode cache fails after the code-object cache has been created, the initializer destroys the first cache and returns an error without clearing its pointer. Failed construction then invokes the unwinder's deallocator, which destroys the same cache again and can crash the profiling process instead of propagating `MemoryError`.
(cherry picked from commit b577a5403a)
Co-authored-by: Jelle Zijlstra <jelle@openai.com>
* [3.15] gh-153364: Make frame, coroutine, and task-waiter chain walks iterative and bounded (GH-153365) (#158813)
gh-153364: Make frame, coroutine, and task-waiter chain walks iterative and bounded (GH-153365)
* let me declare single limit
* use our new limit in process_frame_chain()
* add it in parse_async_frame_chain()
* parse_coro_chain()
* NEWS
* async in the message?
* test
* no race
* process_task_awaited_by
* process_task_awaited_by limit test
* NEWS
* MAX_TASK_WAITER_CHAIN_DEPTH
* TASK_WAITER_CHAIN_DEPTH in test
* TASK_WAITER_CHAIN_DEPTH 256
* prevent the drift with the comment
* better naming, better style
* MAX_TASK_WAITER_CHAIN_DEPTH comment
* task-waiter iterative bfs walk
* iterative coro-walk
* nicer news
* 1 << 14
* comment
* unused read_Py_ssize_t
* fix tombstones
* simplify
* correct msg
* better test
* news for tombstones
* left-over from when testing buggy version
* redundant new line
(cherry picked from commit e0861c6ae7)
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
(cherry picked from commit 9e401cf7f5)
* [3.15] gh-155811: Add a seqcount to `gc_stats` to prevent torn reads (GH-155828) (#158829)
* update_seq
* no need for XCHGL, MOVL is enough?
* gh-155811: Retry an inconsistent GC snapshot once
---------
(cherry picked from commit 5fecd448bb)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 028124039c)
* [3.15] gh-151292: `_remote_debugging`: Do not corrupt the binary file when hitting `OverflowError` (GH-152892) (#158830)
gh-151292: `_remote_debugging`: Do not corrupt the binary file when hitting `OverflowError` (GH-152892)
* the kolektor
* test
* better test
* news
* =Q, move const to the base, not self.running
* gh-151292: Track binary writer finalization state
---------
(cherry picked from commit f839c061d3)
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
(cherry picked from commit 8a7c23f316)
* [3.15] gh-158583: Fix uninitialized memory read in bytes.fromhex() (GH-158584) (#158691)
gh-158583: Fix uninitialized memory read in bytes.fromhex() (GH-158584)
(cherry picked from commit 9d22a5334b)
Co-authored-by: Victor Stinner <vstinner@python.org>
(cherry picked from commit 4f7af46e25)
* [3.15] gh-154194: Degrade frames in Tachyon instead of failing the sample (GH-154195) (#158831)
* gh-154194: Degrade frames in Tachyon instead of failing the sample (#154195)
* degrade gracefully
* news
* better NEWS wording
* do not raise on MAX_REMOTE_STR_READ
* bye MAX_REMOTE_STR_READ
* fix -m asyncio ps|pstree
* test truncation and linetable sentinel
* simpler
* simpler
* redundant now
* respect #157790 in the news
---------
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
(cherry picked from commit 7d25916b41)
* Preserve the stable ABI when creating fallback frame names
---------
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
(cherry picked from commit c27f494061)
* [3.15] Add MSan to CI (GH-158625) (#158832)
Run a job that the test suite with MSan to the CI (#158625)
* Run the test suite with MSan in CI
* Additional fixes
* Add `_Py_MSAN_UNPOISON_STRING`
* Apply Victor's suggestions
* Apply Victor's suggestions
---------
(cherry picked from commit b93fb19a6e)
Co-authored-by: Stan Ulbrych <stan@python.org>
Co-authored-by: Victor Stinner <victor.stinner@gmail.com>
(cherry picked from commit 1bc78dea18)
* [3.15] gh-156810: Write the profiler's collapsed-stack export as UTF-8 (GH-156811) (#156814)
gh-156810: Write the profiler's collapsed-stack export as UTF-8 (GH-156811)
(cherry picked from commit c3706f4c68)
Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
(cherry picked from commit fce28da9b7)
* [3.15] gh-158552: Wait for Windows threads to suspend before blocking sampling (GH-158802) (#158845)
gh-158552: Wait for Windows threads to suspend before blocking sampling (GH-158802)
* gh-158552: Wait for Windows threads to suspend before blocking sampling
* Use a named Windows thread enumeration status constant
(cherry picked from commit 1643525f0f)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
(cherry picked from commit 31288da21d)
* [3.15] gh-152721: Fix quadratic RLE replay time in the profiling binary reader (GH-152722) (#158850)
Backport of GH-152722.
Co-authored-by: tonghuaroot <tonghuaroot@gmail.com>
(cherry picked from commit ebaca2ac2d)
* [3.15] gh-156545: Fix flamegraph export RecursionError on deeply recursive programs (GH-156546) (#158851)
Backport of GH-156546.
Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
(cherry picked from commit 0ac7217040)
* [3.15] gh-158540: Add the profiled script's directory to sys.path (GH-158548) (#158844)
gh-158540: Add the profiled script's directory to sys.path (GH-158548)
* gh-158540: Add the profiled script's directory to sys.path
When a script is profiled with ``python -m profiling.sampling run
script.py`` from another directory, the script cannot import modules
placed next to it, because ``_sync_coordinator._execute_script()``
executes it with the working directory (added by
``_setup_environment()`` for the module case) as ``sys.path[0]``
instead of the script's own directory.
Make the script's directory importable in ``_execute_script()``,
matching the behavior of ``python script.py``.
Add a regression test that runs the coordinator on a script importing
a sibling module.
* Update _sync_coordinator.py Comment simplified
* gh-158540: Resolve symlinks when adding the script directory to sys.path
``python script.py`` resolves symlinks when computing ``sys.path[0]``, so
a script reached through a symlink (``link.py -> sub/where.py``) imports
modules from the real script's directory. Apply ``os.path.realpath()``
before taking the directory name, and make sure the result is placed at
the front of ``sys.path`` even if it was already listed.
Add a regression test for a symlinked script.
---------
(cherry picked from commit 3f02aab9fa)
Co-authored-by: he_tao <53343436+hetaozdh@users.noreply.github.com>
Co-authored-by: Eduardo Villalpando Mello <eduardo.villalpando.mello@gmail.com>
(cherry picked from commit f52d831397)
* [3.15] gh-153838: Skip non-regular source files in the heatmap exporter (GH-153839) (#158853)
Backport of GH-153839.
Co-authored-by: tonghuaroot <tonghuaroot@gmail.com>
(cherry picked from commit 48998df251)
* [3.15] gh-158539: Fix exception mode missing handlers in generators/coroutines (GH-158581) (#158852)
* [3.15] gh-158539: Fix exception mode missing handlers in generators/coroutines (GH-158581)
Backport of GH-158581.
Co-authored-by: LucasZhou <donghao.zhou@outlook.com>
* [3.15] gh-158539: Use portable static assertion messages
* [3.15] gh-158539: Keep layout assertions with debug-offset validation
* [3.15] gh-158539: Use the platform guard for in-process inspection tests
---------
Co-authored-by: LucasZhou <donghao.zhou@outlook.com>
(cherry picked from commit d625ecbb8d)
* [3.15] gh-158522: Fix truncated stack for a task whose coroutine recurses (GH-158526) (#158870)
Co-authored-by: Timofei Ivankov <128279579+deadlovelll@users.noreply.github.com>
(cherry picked from commit a904b397ed)
* gh-156545, gh-158539: Fix deep flamegraph export on small C stacks and macOS runtime lookup (#158874)
(cherry picked from commit 114de198c0)
Include the C-stack test helper from main, introduced by
ce5ae29ef9, which the regression test
requires but 3.15 does not yet provide.
---------
Co-authored-by: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com>
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Stan Ulbrych <stan@python.org>
Co-authored-by: Victor Stinner <victor.stinner@gmail.com>
Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
Co-authored-by: he_tao <53343436+hetaozdh@users.noreply.github.com>
Co-authored-by: Eduardo Villalpando Mello <eduardo.villalpando.mello@gmail.com>
Co-authored-by: LucasZhou <donghao.zhou@outlook.com>
Co-authored-by: Timofei Ivankov <128279579+deadlovelll@users.noreply.github.com>
* gh-157695: Parse also _Py_HAVE_xxx variables in sysconfig (#158926)
Add parse_config_h() tests to test_sysconfig.
(cherry picked from commit 978a8befa5)
* gh-158893: Make os.strerror() thread-safe (#158927)
Make os.strerror() thread-safe: use the reentrant strerror_r()
function if available.
* The configure script now checks if strerror_r() is supported.
* Add a stress test to test_free_threading.test_os (new module).
* Add a comment on decode_current_locale() assertion which fails if
the input string is mutated.
* Add an assertion to _Py_DecodeLocale() to detect if the input
string was mutated during the function call.
* gh-158893: Add internal _Py_strerror() function (#158982)
Add a new internal _Py_strerror() function to Python/fileutils.c. It
uses strerror_r() if available, or use strerror() otherwise.
Replace all strerror(code) calls with _Py_strerror(code).
(cherry picked from commit 15dd7356f3)
gh-158552: Wait for Windows threads to suspend before blocking sampling (GH-158802)
* gh-158552: Wait for Windows threads to suspend before blocking sampling
* Use a named Windows thread enumeration status constant
(cherry picked from commit 1643525f0f)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Run a job that the test suite with MSan to the CI (#158625)
* Run the test suite with MSan in CI
* Additional fixes
* Add `_Py_MSAN_UNPOISON_STRING`
* Apply Victor's suggestions
* Apply Victor's suggestions
---------
(cherry picked from commit b93fb19a6e)
Co-authored-by: Stan Ulbrych <stan@python.org>
Co-authored-by: Victor Stinner <victor.stinner@gmail.com>
* gh-154194: Degrade frames in Tachyon instead of failing the sample (#154195)
* degrade gracefully
* news
* better NEWS wording
* do not raise on MAX_REMOTE_STR_READ
* bye MAX_REMOTE_STR_READ
* fix -m asyncio ps|pstree
* test truncation and linetable sentinel
* simpler
* simpler
* redundant now
* respect #157790 in the news
---------
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
(cherry picked from commit 7d25916b41)
* Preserve the stable ABI when creating fallback frame names
---------
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
gh-151292: `_remote_debugging`: Do not corrupt the binary file when hitting `OverflowError` (GH-152892)
* the kolektor
* test
* better test
* news
* =Q, move const to the base, not self.running
* gh-151292: Track binary writer finalization state
---------
(cherry picked from commit f839c061d3)
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* update_seq
* no need for XCHGL, MOVL is enough?
* gh-155811: Retry an inconsistent GC snapshot once
---------
(cherry picked from commit 5fecd448bb)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
gh-153364: Make frame, coroutine, and task-waiter chain walks iterative and bounded (GH-153365)
* let me declare single limit
* use our new limit in process_frame_chain()
* add it in parse_async_frame_chain()
* parse_coro_chain()
* NEWS
* async in the message?
* test
* no race
* process_task_awaited_by
* process_task_awaited_by limit test
* NEWS
* MAX_TASK_WAITER_CHAIN_DEPTH
* TASK_WAITER_CHAIN_DEPTH in test
* TASK_WAITER_CHAIN_DEPTH 256
* prevent the drift with the comment
* better naming, better style
* MAX_TASK_WAITER_CHAIN_DEPTH comment
* task-waiter iterative bfs walk
* iterative coro-walk
* nicer news
* 1 << 14
* comment
* unused read_Py_ssize_t
* fix tombstones
* simplify
* correct msg
* better test
* news for tombstones
* left-over from when testing buggy version
* redundant new line
(cherry picked from commit e0861c6ae7)
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
gh-151292: Change `total_samples:u32` in the binary format to `total_samples:u64` (GH-153425)
(cherry picked from commit 1d196507e6)
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
gh-151378: Fix unbounded memory growth when recording binary and the thread is sleeping (GH-152089)
(cherry picked from commit 639a5525fb)
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
gh-151613: Fix remote debugging frame cache ABA (#151614)
The remote debugging frame cache previously used only the last_profiled_frame address as its cache anchor. If a frame returned and a later frame reused the same _PyInterpreterFrame address, the profiler could accept a stale cache entry and splice parent frames from a different call chain into the current stack.
This adds a last_profiled_frame_seq counter next to last_profiled_frame, increments it when the anchor advances, stores it in frame cache entries, and validates cache hits against both the frame address and the sequence. Cache miss walks now copy stack chunks before storing new cache entries so stored continuations come from a stable snapshot. The new regression test exercises alternating call chains and checks that cached stacks never contain frames from both branches.
(cherry picked from commit 8cda6ae2f1)
gh-151424: Fix impossible stack traces in `RemoteUnwinder(..., cache_frames=True)` by copying chunks on cache miss (GH-151426)
(cherry picked from commit 6ce088e20a)
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
gh-150285: Fix too long docstrings in the _remote_debugging module (GH-150289)
(cherry picked from commit cdc499ae77)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-149474: use `Py_fopen` in `Binary{Reader,Writer}` for audit hook and path-like support (GH-149524)
(cherry picked from commit 354ef336e4)
Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
Compute ``final_depth`` in ``decode_stack_pop_push()`` and
``decode_stack_suffix()`` using ``uint64_t`` before validating it.
On 32-bit builds, using ``size_t`` arithmetic for ``keep + push`` can wrap
for large input values, causing the later bounds check to validate the wrong
final depth. Using a widened type keeps the validation aligned with the
actual result.
Treat the debug offset tables read from a target process as untrusted input
and validate them before the unwinder uses any reported sizes or offsets.
Add a shared validator in debug_offsets_validation.h and run it once when
_Py_DebugOffsets is loaded and once when AsyncioDebug is loaded. The checks
cover section sizes used for fixed local buffers and every offset that is
later dereferenced against a local buffer or local object view. This keeps
the bounds checks out of the sampling hot path while rejecting malformed
tables up front.
