522 Commits
Author SHA1 Message Date
1ndahous3 112b7efa80 Promote retained GC objects and batch generation updates (#8930)
Assisted-by: Codex:GPT-6
2026-10-02 01:42:09 +09:00
Jeong, YunWon cb0f3b012b Add a native _zoneinfo module (#8915)
* Add a native _zoneinfo module

ZoneInfo reads TZif tuples from zoneinfo._common.load_data. The stdlib
module implements utcoffset, dst, tzname, fromutc, and the zone cache.
whats_left compares that class. Cache tests no longer expect failure.

Assisted-by: Grok:grok-4.7

* Fix ZoneInfo layout and keep its cache on the module

utcoffset, dst, and tzname take the object. Zone data is boxed so the
payload lines up with tzinfo on 32-bit. The strong, weak, and timedelta
caches live on each interpreter's _zoneinfo module.

Assisted-by: Grok:grok-4.7
2026-10-01 11:32:32 +09:00
Jean-Marc Le Roux 3805038dec zlib: clear unconsumed_tail when a bounded decompress reaches stream end (#8908)
* zlib: clear unconsumed_tail when a bounded decompress reaches stream end

When a max_length-bounded Decompress.decompress() call reached the end
of the stream, the bytes after the stream went to unused_data, but
unconsumed_tail kept the previous call's tail, including input the call
had already consumed. A later flush() then moved those stale bytes into
unused_data.

Empty unconsumed_tail at end of stream, as CPython's
save_unconsumed_input() does since gh-158169. CPython releases before
that change report the bytes after the stream in unconsumed_tail, so
the snippet accepts either value when it runs under CPython.

Fixes #8906

Assisted-by: Claude Code:claude-opus-5-5

* zlib: bind the unconsumed input with an if expression

Choose between the empty slice and the rest of the input in one
expression instead of reassigning a mutable binding.

Assisted-by: Claude Code:claude-opus-5-5

* zlib: accept both valid tails in the snippet on every implementation

The snippet no longer branches on sys.implementation. RustPython and
CPython after gh-158169 give an empty tail; CPython before it gives the
bytes after the stream. The stale tail of gh-8906 fails either way.

Assisted-by: Claude Code:claude-opus-5-5
2026-09-30 22:15:40 +09:00
1ndahous3 e8919b9632 Preserve list mutation progress and release removed items outside locks (#8894)
* Preserve list mutation progress and release removed items outside locks

Assisted-by: Codex:GPT-6

* Trim excessive list extension reservations

Assisted-by: Codex:GPT-6
2026-09-30 22:14:47 +09:00
Sumi Jeong c9c78db91c Bound AST nesting so deep source raises instead of crashing (#8827)
* Bound AST nesting so deep source raises instead of crashing

Deeply nested source exhausted the native stack and killed the process
with SIGSEGV, which no Python-level `except` can catch.

The parser grows its own stack when it runs low on one, so it returns
trees deeper than the passes after it can walk. Those passes recurse
without a limit of their own, and the symbol table, which does apply
one, runs last. `compile(..., PyCF_ONLY_AST)` returns before reaching
it at all.

Apply that same limit to the tree right after it is parsed, and count
bracket nesting before parsing rather than after, as CPython's
tokenizer does. Checking afterwards built the tree first, and one
nested that deep exhausts the stack when it is dropped.

Fixes #7655

Assisted-by: Claude Code:Opus 5

* Return Result from too_deeply_nested_error

Lets the four call sites use `?` instead of matching on `Some` and
returning the error by hand, which drops 11 lines. `Result` is already
`#[must_use]`, so the attribute on the function goes away with it.

Assisted-by: Claude Code:Opus 5

* Return Result from pre_parse_source_error too

Same shape as the previous commit, for the other check this PR adds, so
the two are consistent with each other.

Assisted-by: Claude Code:Opus 5

* Drop exceeds_max_nesting for the check beside it

The escape-warning pass needs the bracket check but had no `SourceFile` to
build a `CompileError` from, so it got a `bool` wrapper of its own. Building
the `SourceFile` there lets it call `pre_parse_source_error` instead, which
is shaped like the other checks in that file, and costs nothing measurable.

Assisted-by: Claude Code:Opus 5
2026-09-30 22:13:15 +09:00
1ndahous3 74019cf627 Update concurrent.futures for Python 3.14 (#8904)
* Update concurrent.futures for Python 3.14

Assisted-by: Codex:GPT-6

* Enable relative import completion test

Assisted-by: Codex:GPT-6
2026-09-30 02:16:15 +09:00
1ndahous3 ea76251294 Cache parsed struct formats per interpreter (#8903)
* Cache parsed struct formats per interpreter

Assisted-by: Codex:GPT-6

* Preserve struct cache entries on concurrent misses

Assisted-by: Codex:GPT-6
2026-09-30 02:13:21 +09:00
Alessio Attilio d857eebdce fix(gc): traverse annotate name and qualname fields in PyFunction (#8839)
* fix(gc): traverse annotate name and qualname fields in PyFunction

* style: format extra_tests/snippets/stdlib_gc.py

Assisted-by: Antigravity:Gemini-3.8-Flash
2026-09-29 11:49:59 +09:00
1ndahous3 f909d2a30d Add native _datetime module (#8872)
* Add native _datetime module

Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Codex:GPT-6

* Use pymember and safe timezone storage in datetime

Assisted-by: Codex:GPT-6
2026-09-29 11:45:11 +09:00
1ndahous3 cc8b4d4f7e Compact sparse dictionary and set copies (#8879)
Assisted-by: Codex:GPT-6
2026-09-28 14:44:25 +00:00
1ndahous3 56eadb2997 Choose smaller difference scans and install completed intersections (#8881)
Assisted-by: Codex:GPT-6
2026-09-28 22:13:26 +09:00
1ndahous3 fd108237e4 Preallocate binary joins and preserve buffer lifetimes (#8880)
Assisted-by: Codex:GPT-6
2026-09-28 22:12:37 +09:00
1ndahous3 47eb9509b9 Reuse unchanged strings and fix mixed-representation stripping (#8878)
Assisted-by: Codex:GPT-6
2026-09-28 22:09:29 +09:00
1ndahous3 403a38cf49 Fix percent-format precision and integer conversion (#8876)
* Fix percent-format precision and integer conversion

Assisted-by: Codex:GPT-6

* Honor float subclass integer conversions in percent formatting

Assisted-by: Codex:GPT-6
2026-09-28 22:08:26 +09:00
1ndahous3 bb054f6e1b Release deque items outside locks during clear and assignment (#8875)
Assisted-by: Codex:GPT-6
2026-09-28 22:07:28 +09:00
1ndahous3 19e9b7c38a Preallocate dictionary storage in dict.fromkeys (#8874)
Assisted-by: Codex:GPT-6
2026-09-28 22:06:48 +09:00
1ndahous3 af37a76c1f Floor and ceil exact floats directly in math.floor/math.ceil (#8834)
Assisted-by: Claude Code:claude-opus-5-5
2026-09-28 21:56:46 +09:00
1ndahous3 d1327131a5 Specialize module attribute loads (#8864)
Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Codex:GPT-6
2026-09-28 21:55:11 +09:00
Alessio Attilio 7329839fb5 fix(core): align hashability of dict views, namespace, and typing builtins (#8844) 2026-09-28 21:53:58 +09:00
1ndahous3 afae370753 Fix array search equality and reentrant callbacks (#8870)
Assisted-by: Codex:GPT-6
2026-09-28 07:03:55 +09:00
1ndahous3 e2335f1ca9 Avoid redundant work in set difference operations (#8869)
Assisted-by: Codex:GPT-6
2026-09-28 06:53:13 +09:00
1ndahous3 456a8dc9db Match common encoding names before normalizing them (#8862)
Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Codex:GPT-6
2026-09-27 16:53:42 +00:00
1ndahous3 8c928c2444 Fix recursive sequence repetition fallback (#8865)
Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Codex:GPT-6
2026-09-27 16:44:38 +00:00
1ndahous3 15c99b32a6 Look up attributes without raising before calling __getattr__ (#8863)
Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Codex:GPT-6
2026-09-27 16:31:40 +00:00
Alessio Attilio b9fea9c7be fix(contextvars): isolate context stack per interpreter and thread on VirtualMachine (#8838)
* fix(contextvars): isolate context stack per interpreter and thread on VirtualMachine

* style: format and sort imports in extra_tests/snippets/stdlib_contextvars.py

Assisted-by: Antigravity:Gemini-3.8-Flash
2026-09-27 16:31:18 +00:00
1ndahous3 9fd6d76a39 Reuse stored hashes for frozenset hashing and in-place set union (#8859)
Assisted-by: Codex:GPT-6
2026-09-27 15:49:40 +00:00
1ndahous3 ad8bfdba4e Reuse stored hashes when merging dictionaries (#8858)
Assisted-by: Codex:GPT-6
2026-09-27 15:31:00 +00:00
1ndahous3 d31ccae04d Collect cycles through deque iterators and cmp_to_key wrappers (#8861)
Assisted-by: Codex:GPT-6
2026-09-27 15:14:17 +00:00
Alessio Attilio 65647e99ee fix(csv): quote non-numeric objects in writer QUOTE_NONNUMERIC mode (#8843)
* fix(csv): quote non-numeric objects in writer QUOTE_NONNUMERIC mode

* style: format extra_tests/snippets/stdlib_csv.py

Assisted-by: Antigravity:Gemini-3.8-Flash
2026-09-28 00:13:35 +09:00
1ndahous3 f24b0d896b Stream deque inputs and release evicted items outside locks (#8860)
Assisted-by: Codex:GPT-6
2026-09-28 00:05:12 +09:00
1ndahous3 1a259e494f Read live list elements during repr without copying the list (#8853)
Assisted-by: Codex:GPT-6
2026-09-27 20:50:22 +09:00
1ndahous3 360803ff47 Reuse set hashes and fix stored key comparison order (#8852)
Assisted-by: Codex:GPT-6
2026-09-27 20:49:26 +09:00
1ndahous3 afe106c89c Collect functools cycles and release replaced state outside locks (#8851)
Assisted-by: Codex:GPT-6
2026-09-27 20:47:33 +09:00
1ndahous3 90afe00d6c Fix dict view comparisons and avoid temporary sets (#8849)
Assisted-by: Codex:GPT-6
2026-09-27 13:40:51 +09:00
1ndahous3andJeong, YunWon 04d5e12ead Raise MemoryError on impossible allocations and reuse list/tuple buffers (#8840)
* Raise MemoryError instead of aborting on impossible allocations

Assisted-by: Claude Code:claude-opus-5-5

* use CPython 3.14's integer digit limit for left shifts

* size the pow memory check by what malachite allocates

* Make sequence allocations fallible and reuse deallocation buffers

Assisted-by: Codex:GPT-6

---------

Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
2026-09-27 13:39:29 +09:00
1ndahous3 e2ca20882f Return at once when repeating an empty sequence (#8842)
* Return at once when repeating an empty sequence

Assisted-by: Claude Code:claude-opus-5-5

* use sys.maxsize for the empty array repeat check
2026-09-27 09:55:40 +09:00
1ndahous3 eabba7c1b7 Match CPython's overflow limits in math.perm and math.comb (#8841)
Assisted-by: Claude Code:claude-opus-5-5
2026-09-27 09:54:54 +09:00
1ndahous3 373b0da216 Add native _collections._tuplegetter and _collections._count_elements (#8835)
* Add native _collections._tuplegetter and _collections._count_elements

Assisted-by: Claude Code:claude-opus-5-5

* ruff + fixed unittest

* use PySsize

* remove docstring comment

* cover "get()" overriding
2026-09-27 09:52:24 +09:00
1ndahous3 32ca8aa6f5 Track deque mutations and release removed items outside locks (#8847)
Assisted-by: Codex:GPT-6
2026-09-27 09:50:09 +09:00
Jeong, YunWonandRoshan Ramani 5c875d322f Call __index__ in default integer argument conversion (#8828)
* Accept `__index__` objects for integer arguments

CPython converts these nine arguments with `__index__`, RustPython
required an exact `int`. Uses the existing `ArgSize`/`ArgPrimitiveIndex`
converters at each call site rather than widening the blanket `int`
conversion, so argument paths CPython keeps strict (ctypes pointers,
thread handles) are unaffected, and the range check happens while the
argument is bound rather than in the method body.

Also widens `maxsplit` before adding one. `sys.maxsize` was already
reachable there with a plain `int` and overflowed in a debug build.

Assisted-by: Claude Code:claude-opus-5

* Add snippet for `__index__` argument conversion

Expectations verified against CPython 3.14.

Assisted-by: Claude Code:claude-opus-5

* Call __index__ in primitive integer conversion; add ArgStrictInt

The TryFromObject impls for isize, usize and the fixed-width integer
types now take an int (or subclass) as-is and otherwise call __index__,
instead of accepting only int instances.

ArgStrictInt<T> keeps the old int-only conversion. The ctypes address
and handle arguments (from_address, PyObj_FromPtr, dlsym, dlclose,
call_function, call_cdeclfunction, non-Windows FreeLibrary) use it.
_thread._make_thread_handle checks for an int itself and raises
"ident must be an integer".

Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7

* Accept __index__ where arguments were bound as int objects

chr, bin, math.factorial, zlib.crc32/adler32, binascii.crc32/crc_hqx,
the start/stop of sequence index(), the select event mask and a
__len__ result took an int object directly and rejected other objects
with __index__. They now go through __index__.

math.ldexp still accepts only an int for its exponent and raises
"Expected an int as second argument to ldexp." otherwise.

Extend protocol_index_argument.py with the newly accepted calls, an int
subclass whose __index__ is not called, and the sites that still
reject __index__.

Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7

* Replace ArgPrimitiveIndex and ArgSize with PySsize/PySize aliases

The primitive integer conversions call __index__ themselves, so
ArgPrimitiveIndex<T> and its alias ArgSize behaved the same as T. They
are removed. Arguments that used ArgSize or ArgPrimitiveIndex<isize>
now take PySsize (= isize), ArgPrimitiveIndex<usize> takes PySize
(= usize), and the other ArgPrimitiveIndex<T> take T. Those with a
literal fallback take it as a `default`; those where a missing argument
is a distinct state stay OptionalArg.

list.insert is a #[pymethod] again instead of going through a wrapper.

Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7

---------

Co-authored-by: Roshan Ramani <roshanramani.dev@gmail.com>
2026-09-27 01:22:19 +09:00
Kwak Byoung Min 5f18b9cd56 Support Python 3.14 complex values in struct formats (#8829)
Accept F/D through the existing buffer packing implementation and restore
its complex round-trip test. Fix exact __complex__ results warning as
subclasses so valid conversions also work with warnings treated as errors.

Codex assisted with implementation, regression coverage, and review.

Tested: CPython 3.14.7 comparison; struct/memoryview/complex/cmath; workspace and C-API tests; Clippy; pytest; prek
Assisted-by: Codex:gpt-6-astra
2026-09-26 22:57:24 +09:00
1ndahous3 57e826affb Look up str.translate tables without raising on misses (#8831)
Assisted-by: Claude Code:claude-opus-5-5
2026-09-26 21:42:22 +09:00
1ndahous3 8fa504249e Speed up audit events and exec/eval; share audit hooks across threads (#8824)
* Speed up exec/eval and audited builtins when no audit hooks are set

Assisted-by: Claude Code:claude-opus-5-5

* move audit hooks from `VirtualMachine` to `PyGlobalState`

* addaudithook only appends once the lock

* sync with main
2026-09-26 21:40:17 +09:00
1ndahous3 206b090358 Join lists and tuples directly in str.join (#8826)
Assisted-by: Claude Code:claude-opus-5-5
2026-09-26 10:42:54 +09:00
1ndahous3 b10c4d6975 Dispatch unary operators and abs() through number slots (#8825)
Assisted-by: Claude Code:claude-opus-5-5
2026-09-26 10:41:42 +09:00
sean abreau f3adc69754 Fix string addition and sequence subclass dispatch (#8822)
Use sequence concatenation for str and clear inherited concat slots when Python subclasses override __add__. Cover direct calls, reflected dispatch, and dynamic overrides against CPython.

Codex assisted with investigation, implementation, tests, validation, and the PR description.

Assisted-by: Codex:gpt-6
2026-09-26 09:16:40 +09:00
sean abreau 5d15791e11 Accept keyword arguments in warnings.warn_explicit (#8814)
Allow the four required arguments to be passed by position or keyword, matching CPython and fixing pytest Node.warn calls. Add regression coverage for mixed calls, warning instances, and invalid argument binding.

Codex implemented the fix and regression tests, ran validation, and drafted this commit and the PR description. The contributor manually verified the warning reproducer.

Assisted-by: Codex:GPT-6
2026-09-25 09:45:19 +09:00
Jeong, YunWon 7cd4a54e1c Compose native text signatures at compile time from FromArgs (#8807)
* Compose native text signatures at compile time from FromArgs

FromArgs gains a PARAMS const describing its Python parameters.
The derive builds the whole internal doc, `name(sig)\n--\n\ndoc`,
in a const context from those params, and `__text_signature__` /
`__doc__` parse it from the doc. The macro-time string path and the
runtime sig_parts composition are removed.

Signatures follow clinic conventions: `$module` for module functions,
Rust literal defaults rendered as Python reprs, other defaults as
`<unrepresentable>`, and a new `#[pyarg(py_default = "...")]` for the
few fields whose Rust default differs from the Python one. The
`#[pyarg]` keys are documented on the FromArgs derive.

Module functions keep their module object and return it from
`__self__`. PyNativeFunction now traverses its references.

os.stat, os.mkdir, os.listdir, os.open, dict.fromkeys, round, sorted,
int.to_bytes and bytes.hex take arguments matching their signatures.

Assisted-by: Grok:grok-4.7
Assisted-by: Claude:claude-opus-5-5

* Update builtin_signature snippet and fix mkdir on Windows

round and sum now report their FromArgs parameters, so the snippet
asserts their signatures instead of their absence. sum's start is a
PyObjectRef defaulting to int 0 with py_default "0". mkdir reads
dir_fd only on platforms that use it.

Assisted-by: Claude:claude-opus-5-5

* Remove expectedFailure from two signature tests

test_cmp_to_signature and test_autospec_on_bound_builtin_function
now pass. The TestCmpToKeyC override existed only for the marker.

Assisted-by: Claude:claude-opus-5-5
2026-09-24 20:53:28 +09:00
Jeong, YunWon 27647e1e4a Add _thread.get_native_id (#8754)
Return the kernel thread id (gettid / pthread_threadid_np /
GetCurrentThreadId). threading.get_native_id and Thread.native_id
follow once the function exists.

Assisted-by: Grok:4.6
2026-09-20 18:38:28 +09:00
hyoinandout c857f3acfd mmap: correct find/rfind for empty subsequences (#8749)
get_find_range clamped end up to start, so an inverted range such as
(2, 1) became the empty-but-valid range (2, 2). A real subsequence then
found nothing there, which looked right, but the empty subsequence
matched and both methods reported a hit where CPython reports -1.

rfind also returned the start of the range for an empty subsequence,
because its body was copied from find and the returned edge was never
adjusted. An empty subsequence matches at the edge the scan begins
from, so a reverse scan must report the end.

Dropping the clamp lets an inverted range stay inverted and be rejected
explicitly. find and rfind now share find_inner, which keeps the two
directions from drifting apart again, and it checks the mmap is open
before inspecting the subsequence, so find(b"") on a closed mmap raises
ValueError instead of returning a position.

  m = mmap.mmap(-1, 3); m[:] = b"abc"
  m.rfind(b"")        # was 0, now 3
  m.rfind(b"", 1, 2)  # was 1, now 2
  m.find(b"", 2, 1)   # was 2, now -1
  m.rfind(b"", 2, 1)  # was 2, now -1

Verified on macOS arm64 against CPython 3.14.7 with a 19-case table
covering both methods over empty and non-empty subsequences, clamped,
negative and inverted ranges: 8 mismatches before, none after. The
added snippet assertions fail on the unpatched build and pass on
CPython 3.14.7 as well. test_mmap, cargo fmt --check and
cargo clippy -p rustpython-stdlib -D warnings are clean.

Fixes #8722

Assisted-by: Claude Code:claude-opus-5
2026-09-20 16:01:10 +09:00