Always call pthread_attr_init() in hardware_stack_limits(), not only
on FreeBSD (if pthread_attr_get_np() is available).
Emscripten: add pthread_attr_init() workaround.
Use pthread_attr_get_np() on FreeBSD to get the stack size (and guard
size) of the current thread.
Check if pthread_attr_get_np() function and <pthread_np.h> header are
available in configure.
Update test_json.test_recursion. On FreeBSD 16 with a default stack size of
512 MB, the json C implementation no longer raises RecursionError in
test_highly_nested_objects_encoding(). Reduce also max_depth from 5,000 to
2,000 so test_highly_nested_objects_encoding() of the Python implementation
only takes 1.6 seconds, instead of 12 seconds.
Setting __name__, __qualname__, __doc__ or __module__ on a function
stopped the world in the free-threaded build since gh-154826 and
gh-154851, making functools.wraps() orders of magnitude slower with
multiple threads.
Use the per-object critical section and _PyObject_XSetRefDelayed()
instead, as gen_set_name() already does. Use acquire loads in the
readers and hold strong references across error reporting, tracing, and
eval-frame recording that may execute Python or detach the thread. The
__code__, __defaults__, __kwdefaults__ and __closure__ setters still
stop the world to clear func_version.
Document synchronization requirements for direct field access and the
lifetime of borrowed function attributes.
Extend the use of pthread_getattr_np in stack size detection from any glibc linux to any glibc *or* android linux. Update tests for android's new recursion limit.
Co-authored-by: Malcolm Smith <smith@chaquo.com>
* Import the module a lazy `import a.b as c` names
`import a.b as c` compiles to `IMPORT_NAME a.b` followed by `IMPORT_FROM b`.
Lazily, IMPORT_NAME leaves a placeholder holding "a.b", and IMPORT_FROM
rewrote it into the placeholder `lazy from a import b` produces. Reification
then imported `a` alone and read `b` off it, so the module `a.b` was never
imported under its own name: an attribute of the package shadowing it answered
instead, and `math.pi`, which no module backs, bound the float where the eager
statement raises ModuleNotFoundError.
Mark the dotted import on the placeholder and keep the whole name on it.
Reification imports that name and then walks its components with IMPORT_FROM,
which is what the eager statement does.
The test pinning `lazy import math.pi as pi` as working is inverted, since the
eager statement raises.
* Stop excluding test_trace from the lazy-imports-all run
It passes now that a lazy `import a.b as c` imports the module: the
KeyError on 'test.tracedmodules.testmod' came from the submodule never
being imported under its own name.
* Rename lz_submodule to lz_dotted_as and trim the comments
The flag means "bind the whole dotted name, not the root", which the old
name did not say, and import.c already has unrelated lazy_pending_submodules
machinery to be confused with.
* Chain lazy IMPORT_FROM placeholders instead of flagging dotted imports
Each deferred IMPORT_FROM off a placeholder without a fromlist now keeps
the previous placeholder in lz_from and the attribute name in lz_attr.
Reification walks back to the placeholder IMPORT_NAME left, runs that
import, and replays the lookups in order with _PyEval_ImportFrom, which
is what the eager bytecode does. This drops the lz_dotted_as flag and
also follows a custom __lazy_import__ that returns a placeholder for a
different module name.
* Chain every deferred IMPORT_FROM onto the previous placeholder
`lazy from a import b` now records its lookup the same way as `import
a.b as c`, so a placeholder holds either the module name and fromlist or
the previous placeholder and an attribute name, and reification has a
single path. The import passes only the name being resolved as the
fromlist, so accessing b still does not import the other names'
submodules.
* Treat an empty fromlist on a lazy import placeholder as no fromlist
__import__("a.b", fromlist=()) returns the top-level package `a`, the same
as fromlist=None, but the placeholder kept the empty tuple. Every consumer
of lz_attr then read it as a real fromlist: reification narrowed it to the
chained attribute and replayed the lookups on `a.b`, _PyEval_LazyImportFrom
took the attribute off sys.modules["a.b"], and the repr named `a.b.attr`.
_PyLazyImport_New already collapses None to NULL for exactly this reason,
so collapse an empty tuple there too and every site follows.
* gh-157757: Preserve empty fromlists for custom import hooks
---------
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
If the thread stack size is set by linker flags, pass the stack size
to Python/ceval.c via the new _Py_LINKER_THREAD_STACK_SIZE variable
to set Py_C_STACK_SIZE macro.
For AArch64 linux, reduces the total bytes in the code bodies from 489kb to 218kb.
Reduces the size of the stencils files from 394k lines to 167k lines.
* Make _Py_ReachedRecursionLimit inline again
* Remove _Py_MakeRecCheck replacing its use with _Py_ReachedRecursionLimit
* Move the check for C stack swtiching into _Py_CheckRecursiveCall
The function__entry and function__return probes stopped working in Python 3.11
when the interpreter was restructured around the new bytecode system. This change
restores these probes by adding DTRACE_FUNCTION_ENTRY() at the start_frame label
in bytecodes.c and DTRACE_FUNCTION_RETURN() in the RETURN_VALUE and YIELD_VALUE
instructions. The helper functions are defined in ceval.c and extract the
filename, function name, and line number from the frame before firing the probe.
This builds on the approach from https://github.com/python/cpython/pull/125019
but avoids modifying the JIT template since the JIT does not currently support
DTrace. The macros are conditionally compiled with WITH_DTRACE and are no-ops
otherwise. The tests have been updated to use modern opcode names (CALL, CALL_KW,
CALL_FUNCTION_EX) and a new bpftrace backend was added for Linux CI alongside
the existing SystemTap tests. Line probe tests were removed since that probe
was never restored after 3.11.
* Make the `PY_UNWIND` monitoring event available as a code-local
event to allow trapping on function exit events when an exception
bubbles up. This complements the PY_RETURN event by allowing to
catch any function exit event.
* Allow `PY_UNWIND` to be `DISABLE`d; disabling it disables the event for the whole code object.
* Do the above for `PY_THROW`, `RAISE`, `EXCEPTION_HANDLED`, and `RERAISE` events.
* Add FOR_ITER_VIRTUAL to specialize FOR_ITER for virtual iterators
* Add GET_ITER_SELF to specialize GET_ITER for iterators (including generators)
* Add GET_ITER_VIRTUAL to specialize GET_ITER for iterables as virtual iterators
* Add new (internal) _tp_iteritem function slot to PyTypeObject
* Put limited RESUME at start of genexpr for free-threading. Fix up exception handling in genexpr
Revert "GH-126910: Make `_Py_get_machine_stack_pointer` return the stack pointer (#147945)"
This reverts commit 255026d9ee,
which broke a tier-1 buildbot.
* Make _Py_get_machine_stack_pointer return the stack pointer (or close to it), not the frame pointer
* Make ``_Py_ReachedRecursionLimit`` inline again
* Remove ``_Py_MakeRecCheck`` relacing its use with ``_Py_ReachedRecursionLimit``
* Move stack swtiching check into ``_Py_CheckRecursiveCall``
Add special cases for classmethod and staticmethod descriptors in
_PyObject_GetMethodStackRef() to avoid calling tp_descr_get, which
avoids reference count contention on the bound method and underlying
callable. This improves scaling when calling classmethods and
staticmethods from multiple threads.
Also refactor method_vectorcall in classobject.c into a new _PyObject_VectorcallPrepend() helper so that it can be used by
PyObject_VectorcallMethod as well.
The positional arguments passed to _PyStack_UnpackDict are already
kept alive by the caller, so we can avoid the extra reference count
operations by using borrowed references instead of creating new ones.
This reduces reference count contention in the free-threaded build
when calling functions with keyword arguments. In particular, this
avoids contention on the type argument to `__new__` when instantiating
namedtuples with keyword arguments.
Stan Ulbrych