When running profiling on Linux without sudo, attempts to read
process memory would fail with the misleading error 'Failed to find
the PyRuntime section in process <pid> on Linux platform'.
The actual issue is a permissions error because profiling was not
run with sudo. We were clearing the exception on Linux when trying
to read memory, instead, we should bubble up the permissions error
and show it properly.
(cherry picked from commit 0563890872)
Co-authored-by: ivonastojanovic <80911834+ivonastojanovic@users.noreply.github.com>
On DragonFly BSD lwp_gettid() is only unique within a process (the main
thread is always LWP 1), so combine it with the process id, like Solaris.
(cherry picked from commit 53597df11e)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In free-threading builds, concurrent calls to sys.getdlopenflags() and
sys.setdlopenflags() race on interp->imports.dlopenflags. Fix by
using FT_ATOMIC_LOAD_INT_RELAXED / FT_ATOMIC_STORE_INT_RELAXED in
_PyImport_GetDLOpenFlags and _PyImport_SetDLOpenFlags, consistent with
how analogous interpreter-state integer fields (lazy_imports_mode,
pystats_enabled) are protected.
Relaxed ordering is correct here: dlopenflags is a standalone config
integer with no ordering relationship to other memory.
(cherry picked from commit cde31ec135)
Co-authored-by: Jiucheng(Oliver) <git.jiucheng@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
gh-152132: Test Py_CompileString() in test_capi.test_run (#153470)
PyRun functions now raises ValueError if the start argument is
invalid.
Add Modules/_testlimitedcapi/run.c.
(cherry picked from commit d83d26724b)
gh-152132: Fix Py_RunMain() to return an exit code (#153446)
* Fix Py_RunMain() to return an exit code, rather than calling
Py_Exit(), when running a script, a command, or the REPL.
* _PyRun_SimpleFile() now logs errors to stderr, for example if setting
__main__.__file__ fails.
* Add tests on Py_RunMain() exitcode.
* Rename functions:
* _PyRun_SimpleStringFlagsWithName() => _PyRun_SimpleString()
* _PyRun_SimpleFileObject() => _PyRun_SimpleFile()
* _PyRun_AnyFileObject() => _PyRun_AnyFile()
* _PyRun_InteractiveLoopObject() => _PyRun_InteractiveLoop()
* Change _PyRun_SimpleString(), _PyRun_SimpleFile(), _PyRun_AnyFile() and
_PyRun_InteractiveLoop() return type to PyObject*.
* pymain_repl() now displays the error if PySys_Audit() or
import _pyrepl failed.
* PyRun_SimpleFileExFlags() and PyRun_AnyFileExFlags() now log
PyUnicode_DecodeFSDefault() error. So these functions can no longer
return -1 with an exception set.
(cherry picked from commit fac72f1ee9)
gh-152132: Test all PyRun functions in test_capi.test_run (GH-153352)
PyRun_InteractiveOneObjectEx() now raises TypeError if the second
parameter type is not str.
(cherry picked from commit cd9994e283)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-153300: Set global configuration variables in PyConfig_Set() (#153301)
PyConfig_Set() now also set global configuration variables. For
example, PyConfig_Set("inspect", value) now also sets Py_InspectFlag.
Use PyConfig_Set() in main.c to set the inspect flag. Python code can
now see the updated sys.flags.inspect value.
(cherry picked from commit 9fb713f1b8)
gh-153205: Set `MemoryError` in stackref array conversion (GH-153206)
(cherry picked from commit 08759ff101)
Co-authored-by: An Long <aisk@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: sobolevn <mail@sobolevn.me>
In `symtable_visit_type_param_bound_or_default()`, when a reserved name
(e.g. `__classdict__`) is used as a type parameter, `PyUnicode_FromFormat()`
is called to build the SyntaxError message. If the allocation fails and
returns NULL, the subsequent `PyErr_SetObject()` and `Py_DECREF()` calls
would dereference NULL, causing a segfault.
Fix by returning 0 immediately when `PyUnicode_FromFormat()` returns NULL.
This propagates the MemoryError set by `PyUnicode_FromFormat()`.
The bug was introduced in gh-128632 (commit 891c61c).
(cherry picked from commit 10ed03edf1)
Co-authored-by: Petr Vaganov <petrvaganoff@gmail.com>
* Remove test
---------
Co-authored-by: Petr Vaganov <petrvaganoff@gmail.com>
Co-authored-by: Stan Ulbrych <stan@python.org>
gh-151126: Fix a possible crash during the startup with no memory under `Py_STACKREF_DEBUG` (GH-152478)
(cherry picked from commit ecdef17730)
Co-authored-by: sobolevn <mail@sobolevn.me>
This mimics an optimization already present for the single-mutex critical section.
(cherry picked from commit c2ca7724af)
Co-authored-by: Daniele Parmeggiani <8658291+dpdani@users.noreply.github.com>
[3.15] gh-151546: Fix stack limits on musl (#151548) (#151583)
gh-151546: Fix stack limits on musl (#151548)
If the thread stack size is set by linker flags, pass the stack size
to Python/ceval.c via the new _Py_LINKER_THREAD_STACK_SIZE variable
to set Py_C_STACK_SIZE macro.
(cherry picked from commit 9a61d1c0c8)
(cherry picked from commit a5c5edddbc)
gh-151218: Replace sys.flags in PyConfig_Set() (#151402)
PyConfig_Set() and sys.set_int_max_str_digits() now replace
sys.flags (create a new object), instead of modifying sys.flags in-place.
Modifying sys.flags in-place can lead to data races when multiple
threads are reading or writing sys.flags in parallel.
Use _Py_atomic functions to get and set max_str_digits members.
(cherry picked from commit b16d23fc9f)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
[3.15] gh-151253: Dump the Python path configuration on _PyCodec_InitRegistry() failure (#151250) (#151269)
gh-151253: Dump the Python path configuration on _PyCodec_InitRegistry() failure (#151250)
If "import encodings" fails at Python startup, dump the Python path
configuration to help users debugging their configuration. The
encodings module is the first module imported during Python startup.
(cherry picked from commit 7b6e98911e)
(cherry picked from commit 10f616cf39)
gh-151278: Fix test_faulthandler on UBSan (#151279)
* Py_FatalError() no longer calls _PyFaulthandler_Fini() if it
doesn't hold the GIL.
* Skip test_faulthandler tests raising signals if run with UBSan.
* Enable test_faulthandler in GitHub Action "Reusable Sanitizer".
(cherry picked from commit e60c42dc3f)
gh-150700: Fix class-scope inline comprehensions when nested scopes reference `__class__` and friends (GH-150735)
* Fix class-scope inline comprehensions when nested scopes reference `__class__` and friends
In `inline_comprehension()`, when `__class__` / `__classdict__` /
`__conditional_annotations__` appears as `FREE` in a comprehension's
symbol table because a nested scope captured it (e.g. nested lambdas),
this name is still discarded from `comp_free` unconditionally.
This prevents `drop_class_free()` from seeing it, so the appropriate
`ste_needs_(...)` flag is never set on the enclosing class.
That leads to `codegen_make_closure()` throwing `SystemError` when it
couldn't find `__class__` / `__classdict__` /
`__conditional_annotations__` in the class's cellvars.
From now on we just discard from `comp_free` when no child scope
(e.g. a lambda) still needs the name as `FREE`. When a child scope does
need it, keep it in `comp_free` so `drop_class_free()` can set the
appropriate flag and the class creates the implicit cell.
* Fix tests
* Fix typo
* Fix formatting
* Add test checking validity of `__class__` returned
* Prefer 'used' to 'deferred'
(cherry picked from commit ce916dc506)
Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
gh-151112: Fix double free in `assemble_init` when out of memory (GH-151142)
(cherry picked from commit 580499177c)
Co-authored-by: Stan Ulbrych <stan@python.org>
gh-150319: Replace all documentation which says "See PEP 585" (#150325)
* Replace all documentation which says "See PEP 585"
The following classes in the stdlib get simple updates:
- array.array
- asyncio.Future
- asyncio.Task
- collections.defaultdict
- collections.deque
- contextvars.ContextVar
- contextvars.Token
- ctypes.Array
- os.DirEntry
- re.Match
- re.Pattern
- string.templatelib.Interpolation
- string.templatelib.Template
- types.MappingProxyType
- queue.SimpleQueue
- weakref.ref
The following classes are documented publicly as functions, and are
therefore updated internally (`__class_getitem__.__doc__`) but not in the
public docs:
- functools.partial
- itertools.chain
The following builtin types have updates to `__class_getitem__.__doc__`
but not to any documentation pages:
- BaseExceptionGroup
- coroutines (from generators)
- dict
- enumerate
- frozendict
- frozenset
- generators (and async generators)
- list
- memoryview
- set
- slice
- tuple
Special cases:
- union objects are now documented as "supporting class-level []",
rather than anything to do with generics.
- Templates might be generic over a single type (union, in theory) or
over a TypeVarTuple. As this is not currently fully settled, it is
marked with a comment and a mild hint that it is a single type is used
(namely, "type" is singular rather than "types", plural)
* Apply suggestions from code review
* Correct several class getitem docs
And expand the text for tuples.
* Add notes on generic typing of builtins
* Fix typo in tuple.__class_getitem__ docstring
* Typo fix: malformed refs
Fix `generic` links which weren't marked as `:ref:`.
* Strike unnecessary docs on generic-ness
* Apply suggestions from code review
These are applied at both the originally indicated locations and in the
corresponding docstring definitions.
* Update Doc/library/re.rst
* Update Objects/enumobject.c
* Remove tuple generic doc in 'stdtypes' page
This is covered in more detail in the cross-linked typing documentation.
The other copy of this documentation -- in the docstring for
`tuple.__class_getitem__` -- is left in place.
* Fix whitespace around new doc of generics
Per review, do not introduce or remove whitespace such that section
breaks are altered by the introduction of doc on various generic types.
In most cases, this is a removal of an extra line.
In one case (Arrays), it is the reintroduction of a line.
Additionally, two other minor fixes are included:
- incorrect indent on 'defaultdicts'
- make `mappingproxy.__class_getitem__.__doc__` consistent with other
mapping type generic docs
* Move placement of memoryview generic note
Previous placement was at the end of the main docstring, which is
consistent with other types but places it after a section on various
methods (which makes it read somewhat inconsistently). Moving it up
helps resolve.
* Ensure sphinxdoc does not start sentences lowercase
Lowercase class names at the start of sentences are marked out with the
`class` role. In the case of `deque`, documentation already refers to
these as `Deques`, so this form is preferred.
* Apply suggestions from code review
* Fix line endings and wrap more tightly
Line endings fixed by pre-commit ; also re-wrapped the MappingProxyType
text which was too long.
* Use 'ContextVars' style in sphinx doc
---------
Co-authored-by: Stephen Rosen <sirosen@globus.org>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Jelle Zijlstra <906600+JelleZijlstra@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
macOS 26 changed the default visibility of "dynamic" system messages. This
changes the logging strategy to tag all messages as "public" so they are
visible in the system log without special configuration.
(cherry picked from commit 71fc4c66d3)
Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
In free-threaded builds, concurrent calls to PyDict_AddWatcher, PyDict_ClearWatcher, PyDict_Watch, and PyDict_Unwatch can race on the shared callback array and the per-dict watcher tags. This change adds a mutex to serialize watcher registration and removal, atomic operations for tag updates, and atomic acquire/release synchronization for callback dispatch in _PyDict_SendEvent.
(cherry picked from commit 8a4895985f)
Co-authored-by: Alper <alperyoney@fb.com>
Fixes a sequential consistency bug whereby two threads that are deleting a struct member may observe both their deletions to be successful.
(cherry picked from commit 1bdfc0f253)
Co-authored-by: Daniele Parmeggiani <8658291+dpdani@users.noreply.github.com>
gh-149122: Fix segfault in compiler when certain builtin functions are passed a coroutine as arg (GH-149138)
(cherry picked from commit 16f292ef4e)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
Improve `hash()` builtin docstring with caveats.
Mention its return type and that the value can be expected to change between
processes (hash randomization).
Why? The `hash` builtin gets reached for and used by a lot of people whether it
is the right tool or not. IDEs surface docstrings and people use pydoc and
`help(hash)`.
(cherry picked from commit 665b7dfcfa)
gh-146455: Fix O(N²) in add_const() after constant folding moved to CFG (GH-146456)
The add_const() function in flowgraph.c uses a linear search over the
consts list to find the index of a constant. After gh-126835 moved
constant folding from the AST optimizer to the CFG optimizer, this
function is now called N times for N inner tuple elements during
fold_tuple_of_constants(), resulting in O(N²) total time.
Fix by maintaining an auxiliary _Py_hashtable_t that maps object
pointers to their indices in the consts list, providing O(1) lookup.
For a file with 100,000 constant 2-tuples:
- Before: 10.38s (add_const occupies 83.76% of CPU time)
- After: 1.48s
(cherry picked from commit 5d416324c5)
Co-authored-by: zSirius <107359899+zSirius@users.noreply.github.com>
