gh-156939: Fix struct.pack('0p', bytes) (GH-157071)
If the Pascal string is empty (size=0), do not write the size prefix.
Previously, a NUL byte was written outsize the buffer (buffer
overflow). In practice, the write remains into allocated memory
and is silently ignored: no memory is corrupted.
(cherry picked from commit 23525c90f5)
Co-authored-by: Victor Stinner <vstinner@python.org>
Normalize bytes format strings to str before using them as the cache key,
so that equal str and bytes formats no longer collide and get compared.
(cherry picked from commit 190d2ffbb9)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Document that 's' and 'p' accept bytes and bytearray.
* Fix some footnotes.
* Clarify that "string" is a byte string.
* Fix the module docstring.
(cherry picked from commit 3e5a3cb2bd)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-145633: Fix struct.pack('f') on s390x (GH-146422)
Use PyFloat_Pack4() to raise OverflowError.
Add more tests on packing/unpacking floats.
(cherry picked from commit 8de70b31c5)
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
Only set the format attribute after successful (re-)initialization.
(cherry picked from commit 3f33bf83e8)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
`_struct.c`: Fix UB from integer overflow in `prepare_s` (GH-145158)
Avoid possible undefined behaviour from signed overflow in `struct` module
As discovered via oss-fuzz.
(cherry picked from commit fd0400585e)
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
Update struct.__doc__: _Bool available unconditionally (GH-143716)
This amends commit a9296e7f3b.
(cherry picked from commit 31c81ab0a2)
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
gh-143674: Document F/D complex format characters in struct module (GH-143675)
Add documentation for the F (complex from two floats) and D (complex
from two doubles) format characters in the struct module docstring.
These format characters were implemented but not documented.
(cherry picked from commit 3e93225798)
Co-authored-by: Muneeb Ullah <moneebullah25@gmail.com>
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
gh-140260: fix data race in `_struct` module initialization with subinterpreters (GH-140909)
(cherry picked from commit 63548b3699)
Co-authored-by: Shamil <ashm.tech@proton.me>
According to the C standard, the memory representation of _Complex types
is equivalent to 2-element arrays. Unlike _Complex, arrays are always available.
- drop _complex.h header
- use appropriate real arrays to replace complex types
Co-authored-by: Lisandro Dalcin <dalcinl@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
Fix some UBSan failures for `PyStructObject` and `unpackiterobject`.
We also perform some cleanup by suppressing unused return values and renaming the
unused parameter in `METH_NOARGS` and getter methods to `dummy` and `closure`
respectively for semantic purposes.
memcopy'ing arbitrary values to _Bool variable triggers undefined
behaviour. Avoid this.
We assume that `false` is represented by all zero bytes.
Credits to Alex Gaynor.
Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Fix warnings when using -Wimplicit-fallthrough compiler flag.
Annotate explicitly "fall through" switch cases with a new
_Py_FALLTHROUGH macro which uses __attribute__((fallthrough)) if
available. Replace "fall through" comments with _Py_FALLTHROUGH.
Add _Py__has_attribute() macro. No longer define __has_attribute()
macro if it's not defined. Move also _Py__has_builtin() at the top
of pyport.h.
Co-Authored-By: Nikita Sobolev <mail@sobolevn.me>
This PR adds the ability to enable the GIL if it was disabled at
interpreter startup, and modifies the multi-phase module initialization
path to enable the GIL when loading a module, unless that module's spec
includes a slot indicating it can run safely without the GIL.
PEP 703 called the constant for the slot `Py_mod_gil_not_used`; I went
with `Py_MOD_GIL_NOT_USED` for consistency with gh-104148.
A warning will be issued up to once per interpreter for the first
GIL-using module that is loaded. If `-v` is given, a shorter message
will be printed to stderr every time a GIL-using module is loaded
(including the first one that issues a warning).
Revert commit c8c0afc713 (PR #94532),
which moved `struct.Struct` initialisation from `Struct.__init__` to `Struct.__new__`.
This caused issues with code in the wild that subclasses `struct.Struct`.
Remove <ctype.h> in C files which don't use it; only sre.c and
_decimal.c still use it.
Remove _PY_PORT_CTYPE_UTF8_ISSUE code from pyport.h:
* Code added by commit b5047fd019
in 2004 for MacOSX and FreeBSD.
* Test removed by commit 52ddaefb6b
in 2007, since Python str type now uses locale independent
functions like Py_ISALPHA() and Py_TOLOWER() and the Unicode
database.
Modules/_sre/sre.c replaces _PY_PORT_CTYPE_UTF8_ISSUE with new
functions: sre_isalnum(), sre_tolower(), sre_toupper().
Remove unused includes:
* _localemodule.c: remove <stdio.h>.
* getargs.c: remove <float.h>.
* dynload_win.c: remove <direct.h>, it no longer calls _getcwd()
since commit fb1f68ed7c (in 2001).
Here we are doing no more than adding the value for Py_mod_multiple_interpreters and using it for stdlib modules. We will start checking for it in gh-104206 (once PyInterpreterState.ceval.own_gil is added in gh-104204).
The implementation of __sizeof__() methods using _PyObject_SIZE() now
use an unsigned type (size_t) to compute the size, rather than a signed
type (Py_ssize_t).
Cast explicitly signed (Py_ssize_t) values to unsigned type
(Py_ssize_t).
This PR fixes undefined behaviour in the struct module unpacking support functions `bu_longlong`, `lu_longlong`, `bu_int` and `lu_int`; thanks to @kumaraditya303 for finding these.
The fix is to accumulate the bytes in an unsigned integer type instead of a signed integer type, then to convert to the appropriate signed type. In cases where the width matches, that conversion will typically be compiled away to a no-op.
(Evidence from Godbolt: https://godbolt.org/z/5zvxodj64 .)
To make the conversions efficient, I've specialised the relevant functions for their output size: for `bu_longlong` and `lu_longlong`, this only entails checking that the output size is indeed `8`. But `bu_int` and `lu_int` were used for format sizes `2` and `4` - I've split those into two separate functions each.
No tests, because all of the affected cases are already exercised by the test suite.
Make _struct.Struct a GC type
This fixes a memory leak in the _struct module, where as soon
as a Struct object is stored in the cache, there's a cycle from
the _struct module to the cache to Struct objects to the Struct
type back to the module. If _struct.Struct is not gc-tracked, that
cycle is never collected.
This PR makes _struct.Struct GC-tracked, and adds a regression test.
Replace "(PyCFunction)(void(*)(void))func" cast with
_PyCFunction_CAST(func).
Change generated by the command:
sed -i -e \
's!(PyCFunction)(void(\*)(void)) *\([A-Za-z0-9_]\+\)!_PyCFunction_CAST(\1)!g' \
$(find -name "*.c")
