The build-helper programs (`Programs/_freeze_module` and `_bootstrap_python`) don't need to link extension modules that they don't need.
This PR makes sure the helper programs get only the modules they need to do their job, regardless of what the build configuration sets for the final built runtime and interpreter.
This follows the same pattern as what `PCbuild/_freeze_module.vcxproj` does with `PC/config_minimal.c`, and lets "extra" builtin modules (from `Setup.local` or `MODULE_BUILDTYPE=static`) depend on frozen modules or `getpath`.
A bare name or number followed by a token that cannot extend an expression now produces its AST node directly. This avoids the expression precedence chain for common arguments, assignments, and subscripts.
Rules declare the hook with a fastpath flag. The Rule constructor exposes memo and fastpath attributes, and the C generator emits the hook call.
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Add an @include meta to pegen: the rules of an included grammar file are
appended to the including file's rules. The C generator emits the rules
of each included file into a separate C file next to the main output.
Use it to move the invalid_* rules from Grammar/python.gram into
Grammar/invalid.gram, which generates Parser/parser_invalid.c.
Remove the unused analyze_deferred_refs() function that was completely
commented out. This function is not referenced anywhere in
the codebase and removing it improves code cleanliness.
Signed-off-by: Osama Abdelkader <osama.abdelkader@gmail.com>
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
Heap types have no static type object so switch from `Py_Is` to an
assert that the shadowed `tp_new` / `tp_init` function is the expected
one to validate the vectorcall is only used with the right type.
Heap types can have zero-argument `__new__` and `__init__` so add
support for that producing the same error message via a new helper
`_PyArg_NoPositionalStack()`.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
`KeywordCollectorVisitor` checked the quoted spelling of a keyword against
keys stored without quotes, so the check never matched and every
occurrence of a keyword in the grammar took a new token type. The 35
hard keywords were numbered between 500 and 721 instead of 500 to 534.
In framework builds, `libpython.dynamic` now joins `PYTHONFRAMEWORKPREFIX` with `LDLIBRARY` instead of `LIBDIR` with `LDLIBRARY`, so it points at the framework library that is actually installed. Other builds are unchanged.
`make_paths_relative()` now keeps the absolute `base_prefix` for computing the other paths, and only the stored `base_prefix` is made relative to the config file's directory. This is the same fix python-build-standalone carries in `patch-python-relative-build-details.patch` (by @geofft). That patch also changes the Makefile to generate relative paths by default; this PR leaves the Makefile alone.
gh-157361: Implement aiter() in Python instead of C
Move both forms of aiter() into the frozen _pybuiltins module and remove
the C async_callable_iterator types. The callable form now returns an
iterator whose __anext__() is a plain coroutine, so call-graph tools can
see through it into the awaited result of the callable.
Use pthread_attr_get_np() on FreeBSD to get the stack size (and guard
size) of the current thread.
Check if pthread_attr_get_np() function and <pthread_np.h> header are
available in configure.
Update test_json.test_recursion. On FreeBSD 16 with a default stack size of
512 MB, the json C implementation no longer raises RecursionError in
test_highly_nested_objects_encoding(). Reduce also max_depth from 5,000 to
2,000 so test_highly_nested_objects_encoding() of the Python implementation
only takes 1.6 seconds, instead of 12 seconds.
When a shared library is also built, point `libpython.static` at `LIBPL`,
the only place `make install` puts the static library, and don't list it for
framework builds, which don't install one. Static-only builds keep the
existing `LIBDIR` value, which is installed in that configuration.
The pegen entry points now take `pathlib.Path` objects instead of
strings, and the command-line tools (`python -m pegen`, `keywordgen`,
`first_sets`, `grammar_visualizer`) parse their path arguments into
`Path` objects.
The generators' `generate` and `prepare` take a `pathlib.PurePath`
since they only use the file's name (for the "generated by" comment
in the output) and callers pass names that are not files, such as
`"<string>"`.
This makes it so that in wasi builds if `-DPY_CALL_TRAMPOLINE` is passed, a call trampoline adaptor is used to prevent traps when there is a call signature mismatch between a C slot handler and the signature that the interpreter uses to call it.
This requires Clang >= 22. PEP 11 specifies that builds of Python 3.15 use WASI SDK version 33 which includes clang 23.1.0, so this could be used with Python 3.15 WASI builds but not earlier ones.
It requires a WebAssembly runtime that supports wasm-gc. Wasmtime has supported wasm-gc since version 27.0 released November 25, 2024. Some other runtimes still don't support it, but for those runtimes people can use builds with the trampoline disabled.
This also removed the Emscripten trampoline support for JS runtimes with no wasm-gc support. This removes a lot of implementation complexity and allows WASI and Emscripten to share code. The main concern is that it drops support for some very old iPhones, but every iPhone model released since 2018 is compatible with versions of Safari that support wasm-gc.
Tests were added that define handlers with various wrong numbers of arguments. The tests also run on non-webassembly platforms since they should work there as well.
* Add 7 INCREF/DECREF macros
* distinguish between 'const' and 'macro' in stable_abi.dat
The file now has the 'kind' from the toml, not the ReST role.
* Fix kind of PY_VECTORCALL_ARGUMENTS_OFFSET
* Make it explicit that macros aren't part of the ABI
* Add is_static_inline_function to the TOML file (these are treated as macros in the docs)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
* gh-157407: Use _PyUnicode_AsUTF8NoNUL for AC 's' and 'z' converters
The code was doing the same as the helper added in GH-111957 but inline.
That resulted in significantly more `.text` for most functions. Refactor
the Arguemnt Clinic converters code to deduplicate a bit while keeping
generated code close to original.
On my Linux x64 box this reduces `.text` by ~6200 bytes in a clang
release build (no PGO or LTO). Performance is flat.
The helper was already exported for `_sqlite3`, update the comment that
it is used in AC generated code as well.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Update Include/internal/pycore_unicodeobject.h
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
* Remove NEWS, add period
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
Py_GetVersion() now allocates memory on the heap, instead of using a
static buffer, to no longer truncate the version if it's longer than
299 bytes.
Update Py_GetCompiler() and Py_GetBuildInfo() tests: they are now
always a part of sys.version.
python-gdb.py is now able to format an invalid Unicode string:
render invalid characters as "\Uhhhhhhhh" instead of raising an
exception.
The PyUnicodeWriter API fills a UCS-4 buffer with 0xff byte pattern
to detect usage of uninitialized characters. It produces invalid
characters '\Uffffffff'.
PyBytesObjectPtr now iterates on bytes (list of integers), instead of
creating a temporary Unicode string. Add more bytes tests in test_gdb
pretty printer.
Remove the unused 'ch2' variable.
Check for parameters following a var-keyword parameter in one place
instead of three, and improve the error messages for a var-keyword
parameter used with non-positional parameters and for a var-positional
or var-keyword parameter in an optional group.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Such a combination is ambiguous: the number of arguments does not tell
whether the group is passed or all arguments belong to the var-positional
parameter.
* gh-136681: Drop _PyArg_Parser for positional-only defining-class methods
A method that takes a defining class and whose other parameters are all
positional-only (e.g. decimal.Decimal.from_float) was routed through
Argument Clinic's generic keyword path: it emitted a static _PyArg_Parser,
an (empty) keyword tuple and an _PyArg_UnpackKeywords() call, even though
it accepts no keyword arguments.
Such methods now go through parse_pos_only(): the wrapper still uses the
mandatory METH_METHOD|METH_FASTCALL|METH_KEYWORDS convention (the only one
that delivers the defining class), but it rejects keywords with
_PyArg_NoKwnames() and reads positional arguments directly, with no
parser, keyword tuple or argsbuf.
This converts 135 methods across 19 clinic headers, removing the static
_PyArg_Parser, the keyword arrays/tuples and the per-call
_PyArg_UnpackKeywords() + parser_init() overhead for each.
_PyArg_NoKwnames() is now exported (PyAPI_FUNC) so generated code in
shared extension modules can use it, like _PyArg_NoKeywords().
Error messages for these methods now match ordinary positional-only
functions ("expected ... arguments" / "takes no keyword arguments"
instead of the _PyArg_UnpackKeywords phrasing); affected stdlib tests
are updated accordingly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* gh-157407: Rename NEWS entry to the correct issue number
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
They are moved to Argument Clinic, so that pwd.getpwuid() and
grp.getgrgid() can declare the type of their parameter instead of
converting it in the "impl" function.
pwd.getpwuid() now raises OverflowError instead of KeyError for a user id
out of the range of uid_t, as grp.getgrgid() already did.
* Move pegen grammar analysis into its own module
* Record pegen helper rule kinds explicitly
* Introduce immutable C parser data and separate lowering
* Add C emitters with local rule and output state
* Route C generation through the prepared parser model
* Fix pegen import formatting for lint
* Preserve and test shared-prefix preparation after rebase
* gh-153569: move tokenizer input state and relocation into the reader
* gh-153569: borrow diagnostic lines through the source API
* gh-153569: group indentation and logical-line state
* gh-153569: remove unused tokenizer cursor and source lookup APIs
* gh-153569: report tokenizer diagnostics without rewinding the scanner
* gh-153569: use tokenizer views and remove obsolete API remnants
Itamar Oren