Release path-file read buffers on early return and the command string
when an audit hook rejects execution.
Backport to 3.15 and 3.14.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Py_GetVersion() now allocates memory on the heap, instead of using a
static buffer, to no longer truncate the version if it's longer than
299 bytes.
Update Py_GetCompiler() and Py_GetBuildInfo() tests: they are now
always a part of sys.version.
If a command line option wants to exit Python, move this option
processing from PyConfig_Read() to Py_Main(). So
Py_InitializeFromInitConfig() and Py_InitializeFromConfig() can no
longer return an exit code.
Deprecate PyInitConfig_GetExitCode(), PyStatus_Exit(), and PyStatus_IsExit()
functions, and PyStatus.exitcode member.
Renumber getopt.c long options to start at 1, instead of 0.
* Check for signals more often. Previously, a pending exception could
be removed by PyErr_Clear().
* Only call _PyInterpreterState_SetNotRunningMain() if
_PyInterpreterState_SetRunningMain() has been called.
* Add pymain_error() and pyrun_error(): write the error message to
sys.stderr instead of the C stream stderr.
* Convert _PyPathConfig_UpdateGlobal() PyStatus error to an
exception.
* Simplify pymain_run_startup(): avoid strerror() which is decoded
from the wrong encoding. Instead display the exception, Py_fopen()
raises an OSError with the path object (and can raise other exceptions).
* Add pymain_set_path0() function.
* Add _PySys_FormatV() function.
* Fix Py_RunMain() to return an exit code, rather than calling
Py_Exit(), when running a script, a command, or the REPL.
* _PyRun_SimpleFile() now logs errors to stderr, for example if setting
__main__.__file__ fails.
* Add tests on Py_RunMain() exitcode.
* Rename functions:
* _PyRun_SimpleStringFlagsWithName() => _PyRun_SimpleString()
* _PyRun_SimpleFileObject() => _PyRun_SimpleFile()
* _PyRun_AnyFileObject() => _PyRun_AnyFile()
* _PyRun_InteractiveLoopObject() => _PyRun_InteractiveLoop()
* Change _PyRun_SimpleString(), _PyRun_SimpleFile(), _PyRun_AnyFile() and
_PyRun_InteractiveLoop() return type to PyObject*.
* pymain_repl() now displays the error if PySys_Audit() or
import _pyrepl failed.
* PyRun_SimpleFileExFlags() and PyRun_AnyFileExFlags() now log
PyUnicode_DecodeFSDefault() error. So these functions can no longer
return -1 with an exception set.
PyConfig_Set() now also set global configuration variables. For
example, PyConfig_Set("inspect", value) now also sets Py_InspectFlag.
Use PyConfig_Set() in main.c to set the inspect flag. Python code can
now see the updated sys.flags.inspect value.
The `__main__` module imported in the `_pyrepl` module points to the `_pyrepl` module itself when the interpreter was launched without `-m` option and didn't execute a module,
while it's an unexpected behavior that `__main__` can be `_pyrepl` and relative imports such as `from . import *` works based on the `_pyrepl` module.
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Use an atomic operation when setting
`_PyRuntime.signals.unhandled_keyboard_interrupt`. We now only clear the
variable at the start of `_PyRun_Main`, which is the same function where
we check it.
This avoids race conditions where previously another thread might call
`run_eval_code_obj()` and erroneously clear the unhandled keyboard
interrupt.
- Add a helper to set an error from locale-encoded `char*`
- Use the helper for gdbm & dlerror messages
Co-authored-by: Victor Stinner <vstinner@python.org>
The existence of background threads running on a subinterpreter was preventing interpreters from getting properly destroyed, as well as impacting the ability to run the interpreter again. It also affected how we wait for non-daemon threads to finish.
We add PyInterpreterState.threads.main, with some internal C-API functions.
This change makes sure sys.path[0] is set properly for subinterpreters. Before, it wasn't getting set at all. This PR does not address the broader concerns from gh-109853.
Remove these private functions from the public C API:
* _PyRun_AnyFileObject()
* _PyRun_InteractiveLoopObject()
* _PyRun_SimpleFileObject()
* _Py_SourceAsString()
Move them to the internal C API: add a new pycore_pythonrun.h header
file. No longer export these functions.
Remove the following old functions to configure the Python
initialization, deprecated in Python 3.11:
* PySys_AddWarnOptionUnicode()
* PySys_AddWarnOption()
* PySys_AddXOption()
* PySys_HasWarnOptions()
* PySys_SetArgvEx()
* PySys_SetArgv()
* PySys_SetPath()
* Py_SetPath()
* Py_SetProgramName()
* Py_SetPythonHome()
* Py_SetStandardStreamEncoding()
* _Py_SetProgramFullPath()
Most of these functions are kept in the stable ABI, except:
* Py_SetStandardStreamEncoding()
* _Py_SetProgramFullPath()
Update Doc/extending/embedding.rst and Doc/extending/extending.rst to
use the new PyConfig API.
_testembed.c:
* check_stdio_details() now sets stdio_encoding and stdio_errors
of PyConfig.
* Add definitions of functions removed from the API but kept in the
stable ABI.
* test_init_from_config() and test_init_read_set() now use
PyConfig_SetString() instead of PyConfig_SetBytesString().
Remove _Py_ClearStandardStreamEncoding() internal function.
Deprecate global configuration variable like
Py_IgnoreEnvironmentFlag: the Py_InitializeFromConfig() API should be
instead.
Fix declaration of Py_GETENV(): use PyAPI_FUNC(), not PyAPI_DATA().
Add the -P command line option and the PYTHONSAFEPATH environment
variable to not prepend a potentially unsafe path to sys.path.
* Add sys.flags.safe_path flag.
* Add PyConfig.safe_path member.
* Programs/_bootstrap_python.c uses config.safe_path=0.
* Update subprocess._optim_args_from_interpreter_flags() to handle
the -P command line option.
* Modules/getpath.py sets safe_path to 1 if a "._pth" file is
present.
``pymain_run_python()`` now imports ``readline`` and ``rlcompleter``
before sys.path is extended to include the current working directory of
an interactive interpreter. Non-interactive interpreters are not
affected.
Also move imports of ``re`` and ``keyword`` module to top level so they
are materialized early, too. The ``keyword`` module is trivial and the
``re`` is already imported via ``inspect`` -> ``linecache``.
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code. It is still used in a number of non-builtin stdlib modules.
The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime. A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).
https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.
The core of the change is in:
* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers
I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings. That check is added to the PR CI config.
The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()). This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.
The following are not changed (yet):
* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init
https://bugs.python.org/issue46541
The getpath.py file is frozen at build time and executed as code over a namespace. It is never imported, nor is it meant to be importable or reusable. However, it should be easier to read, modify, and patch than the previous code.
This commit attempts to preserve every previously tested quirk, but these may be changed in the future to better align platforms.
* Move _PyObject_CallNoArgs() to pycore_call.h (internal C API).
* _ssl, _sqlite and _testcapi extensions now call the public
PyObject_CallNoArgs() function, rather than _PyObject_CallNoArgs().
* _lsprof extension is now built with Py_BUILD_CORE_MODULE macro
defined to get access to internal _PyObject_CallNoArgs().
Fix typo in the private _PyObject_CallNoArg() function name: rename
it to _PyObject_CallNoArgs() to be consistent with the public
function PyObject_CallNoArgs().
The coding cookie (ex: "# coding: latin1") is now ignored in the
command passed to the -c command line option.
Since pymain_run_command() uses UTF-8, pass PyCF_IGNORE_COOKIE
compiler flag to the parser.
pymain_run_python() no longer propages compiler flags between
function calls.
If config->run_filename doesn't exist, log the error into sys.stderr
using "%R" format, to escape properly unencodable characters (usually
with backslashreplace).
pymain_run_file() no longer encodes the filename: pass the filename
as an object to the new _PyRun_AnyFileObject() function.
Add new private functions:
* _PyRun_AnyFileObject()
* _PyRun_InteractiveLoopObject()
* _Py_FdIsInteractive()
