35 Commits
Author SHA1 Message Date
Jeong, YunWonandLee Dogeon afad92bf1b Add OrderedDict, marshal, inspect, and chflags bitflagset (#8780)
* Implement _collections.OrderedDict

Extracted from #6799: native OrderedDict as a dict subclass,
iterator mutation tracking, and mapping-view set operations.

* Add OrderedDict, warnings source lines, and socket sendmsg

Implement _collections.OrderedDict as a dict subclass with a
linked list of insertion-order nodes.

Pack struct F/D as native float/double complex pairs, and report
unexpected __replace__ field names as a list repr.

Register encodings.latin_1 at startup. Reject NUL in input()
prompts. Copy non-exact type dicts in mapping order.

Fetch warning source lines through _bless_my_loader and unicode
splitlines. Keep the last warnings.filters list after deletion.

Snapshot sendmsg sequences before converting buffers, parse
ancillary ints with __index__, and dispatch AF_CAN by protocol.

Store sys.trace and sys.profile on each ThreadSlot so
_settraceallthreads/_setprofileallthreads are visible to
sys.gettrace/getprofile on other threads.

Use <frozen importlib._bootstrap> as frozen importlib
co_filename. Keep HEAPTYPE socket objects GC-tracked.

Assisted-by: Grok:grok-4.6

* Load marshal via readinto and dump sets in dumps order

marshal.load fills a scratch buffer with readinto on a
contiguous writable memoryview, and rejects a returned length
other than the requested size.

set and frozenset dumps write elements in sorted(v,
key=marshal.dumps) order.

Assisted-by: Grok:grok-4.6

* Add classmethod_descriptor and ABC IS_ABSTRACT flags

Give METH_CLASS methods a distinct classmethod_descriptor type.
Store Py_TPFLAGS_IS_ABSTRACT from __abstractmethods__ and look
up that attribute only in the type's own dict.

Expose defaultdict.default_factory as a member descriptor.
Inherit bool's unused number slots from int. Mark set and
frozenset __contains__ as METH_COEXIST.

Parse classmethod, staticmethod, object.__new__, and heap type
text signatures. Register interactive source in linecache.

Assisted-by: Grok:grok-4.6

* address review: odict reuse, descr call, new wrapper

Reuse odict node slots and walk the live list in next_after.
Require both | operands to be dicts. Drop the odict iterator
on GC clear. Bind classmethod descriptors through descr_get
and update IS_ABSTRACT under the type lock. Give odict keys
and items sq_contains. Walk tp_new_wrapper past new_wrapper
so HEAPTYPE socket subclasses can call socket.__new__.
Split FreeBSD Bluetooth protocol constants from Linux values.

Assisted-by: Grok:grok-4.6

* Represent chflags bits with bitflagset

Assisted-by: Grok:grok-4.6

* Represent access and mode bits with bitflagset

Assisted-by: Grok:grok-4.6

* Continue unfinished REPL suites as incomplete input

Assisted-by: Grok:grok-4.6

* Add pickle and os.stat inspect signatures

Give _pickle.Pickler the clinic constructor text signature and
accept fix_imports and buffer_callback as positional-or-keyword.
Allow #[pyfunction(text_signature)] and use it so os.stat reports
(path, *, dir_fd=None, follow_symlinks=True). Uncomment import
_pickle and drop the pickle inspect expectedFailures.

Assisted-by: Grok:grok-4.6

* Add Unpickler, dump/load, and lstat signatures

Give _pickle Unpickler, dump, dumps, load, and loads clinic
text signatures, and os.lstat (path, *, dir_fd=None).

Assisted-by: Grok:grok-4.6

* Complete OrderedDict with insertion-order links

Move the native OrderedDict into ordered_dict.rs and track order
with a linked list of nodes, keeping mapping-view set operations.

Assisted-by: Grok:grok-4.6

* address CI: stat casts, HCI_DATA_DIR, passing xfails

Compare stat permission bits as Mode so Linux clippy does not
flag a u32-to-u32 cast. Export HCI_DATA_DIR with the Bluetooth
socket option values. Drop expectedFailure from tests that now
pass.

Assisted-by: Grok:grok-4.6

* Fix wasm teardown deadlock and bluetooth constants

Assisted-by: Grok:grok-4.6

* Clear process-wide eval breaker bits in STW test

Assisted-by: Grok:grok-4.6

* Drop exclusive-held panic that tripped wasm clippy

Assisted-by: Grok:grok-4.6

* Fix OrderedDict contains and export Windows BDADDR

Assisted-by: Grok:grok-4.6

* Add cmp_to_key and hash-once LRU cache

_functools.cmp_to_key now returns KeyWrapper. LRU lookups store the
key hash on _lru_list_elem so __hash__ is not called again on hits
or eviction.

Assisted-by: Grok:grok-4.6

* Allow subclass keywords on list, tuple, and float

list.__init__ rejects keywords only when tp_new is still list's.
tuple/float.__new__ reject them only when tp_init is still the base's.

Assisted-by: Grok:grok-4.6

* Fix CI lint, test re-export, and structseq members

Drop the duplicate clear_eval_breaker_for_test re-export.
Format structseq imports. Expose struct sequence fields as
readonly members so assignment raises "readonly attribute".

Assisted-by: Grok:grok-4.6

* Allow subclass keywords on itertools and filter

Positional-only constructors ignore extra keywords when a subclass
has replaced tp_init, matching list/tuple/float.

Assisted-by: Grok:grok-4.6

* Remove text_signature derive parameter

Drop the string override on #[pyfunction] and #[pyclass]. Signatures
are generated from the Rust parameter list only. Restore inspect
expectedFailures that depended on the override.

Assisted-by: Grok:grok-4.6

* Rename set contains_method to __contains__

Keep a borrowed contains helper for slots and C-API. The coexist
pymethod is named __contains__.

Assisted-by: Grok:grok-4.6

* Use MonitoringEvent instead of EVENT_* aliases

Drop the unused MonitoringEvents bitflags and the EVENT_PY_START
style mask constants. Call sites use the enum and .mask().

Assisted-by: Grok:grok-4.6

* Store warnings filters as PyListRef

The list already serializes its elements. Drop the extra mutex around
the cached list pointer.

Assisted-by: Grok:grok-4.6

* Re-export clear_eval_breaker_for_test after rebase

Keep the STW test helper visible without bringing back stop_bit_set.

Assisted-by: Grok:grok-4.6

* Fix groupby reentrancy and islice arguments

Give _grouper its own tgtkey and take currvalue after the key
comparison. Accept __index__ on islice bounds, reject step 0, and
drain the source up to stop.

Assisted-by: Grok:grok-4.6

* SETREF warnings filters, onceregistry, and defaultaction

Keep the last list, dict, or str after the module attributes are
deleted by swapping the interpreter cache pointer. Do not store a
None fallback for _warnings_context.

Assisted-by: Grok:grok-4.6

* Iterate OrderedDict inner dict via next_entry

&PyDict is not an iterator after the borrowed-ref signature change.

Assisted-by: Grok:grok-4.6

* Remove expectedFailure from testBluetoothConstants

Linux now exports BTPROTO_HCI and the rest of the set.

Assisted-by: Grok:grok-4.6

* Borrow get_filter arguments as PyObject

Match the warn_explicit call site that passes borrowed
handles after the internal-helper borrow change.

Assisted-by: Grok:grok-4.6

* Fix bool slots and SemHandle safety docs

Leave unused bool number slots unset so inspect reports
__add__ on int. Document SemHandle::from_raw safety.

Assisted-by: Grok:grok-4.6

* Keep sendmsg errors and drop duplicate linecache

Replace sendmsg iterable TypeError only. Register REPL
source in linecache once per compiled input.

Assisted-by: Grok:grok-4.6

* Nest REPL parse or-patterns for clippy

Assisted-by: Grok:grok-4.6

* Mark pydoc os.stat signature test expectedFailure

Assisted-by: Grok:grok-4.6

---------

Co-authored-by: Lee Dogeon <dev.moreal@gmail.com>
2026-09-24 12:21:26 +09:00
Jeong, YunWon 259ce8ce66 Fix the itertools.tee leak and race, the contextvars and generator races, _asyncio's InvalidStateError lookup, and _imp's frozen data (#8518)
* _imp: read get_frozen_object's data as a whole marshal value

The data argument went through deserialize_code(), which reads a code body
without the type byte in front of it, so nothing marshal.dumps() produces was
accepted. Read it with marshal.loads() and require a code object back.

Assisted-by: Claude

* _imp: bind find_frozen's arguments with FromArgs

The arity check was done by hand on a FuncArgs. Take the arguments through a
FromArgs struct instead, which makes withdata keyword-only, and fill in the
data it asks for: the frozen encoding is not marshal, so the code is
re-serialized into what get_frozen_object() reads back.

Assisted-by: Claude

* itertools: give tee's shared buffer a Python type

The buffer was a PyRc<PyItertoolsTeeData>, which is not a Python object, so the
collector could not walk into it and any cycle running through a tee was
uncollectable. Make it the _tee_dataobject type with a traverse, held by
PyRef, and split the rest along the same lines: tee() is a function, _tee is
the iterator type it builds, and _tee takes a single iterable rather than
returning a tuple from __new__. _tee is weak-referenceable, tee() rejects a
negative n with ValueError and reserves its tuple fallibly.

test_itertools.test_tee passes now; its expectedFailure marker is removed.

Assisted-by: Claude

* contextvars: hold the shared context state under locks

The variable map was a RefCell, the enter flag, the context index, the token's
used flag and the variable hash were Cells, and each carried an unsafe impl
Sync. A Context or ContextVar shared between threads overlapped their borrows
and panicked. The map and the per-variable cache are now PyMutex, the flags and
the index are atomics, entering a context is a compare_exchange, and the three
unsafe impl Sync are gone.

The cache also stopped being read through AtomicCell::as_ptr, which raced a
concurrent store on a value holding a PyObjectRef.

Values displaced from the map or the cache are dropped after the lock is
released: __del__ can come straight back into the same context, and the locks
are not reentrant.

Assisted-by: Claude

* generator: read the frame state under the running claim

send(), send_none(), throw() and close() read `closed` and `frame.lasti()`
before `running` was compare_exchanged, so the frame they went on to resume
could be one another thread had already advanced. A resume that decided from
`lasti() == 0` that the generator had not started pushes no value onto the
value stack, and the code after the yield pops one, which underflows the stack.

The compare_exchange now hands back a guard, taken before those reads and
released after maybe_close(), so the generator is retired while it is still
claimed.

Assisted-by: Claude

* itertools: claim a tee's position and its buffer as one step

`_tee::next` read `index` and moved it on afterwards, and
`_tee_dataobject::get_item` released `running` before the value it fetched
from the source was cached.

Two callers on one `_tee` then read the same index, hand out the same value
twice and advance past a value that was never cached, which leaves `index`
past `values.len()` and indexes the buffer out of bounds. Two callers at the
same index on separate tees each fetch a value from the source, and one of the
two is dropped without reaching a caller.

Both claims now cover the read and the update.

Assisted-by: Claude

* _asyncio: require InvalidStateError to be a type

new_invalid_state_error() called whatever `asyncio.exceptions.InvalidStateError`
names and unwrapped the downcast of the result, so a future asked for a result
it does not have panicked once that attribute was rebound to something that is
not an exception:

    asyncio.InvalidStateError = lambda *args: 42
    _asyncio.Future(loop=object()).result()

The type is looked up the way get_cancelled_error_type() looks its own up, and
raised with new_exception_msg; a lookup that does not produce an exception type
falls back to RuntimeError, as the other arms already did.

Assisted-by: Claude
2026-08-14 16:05:30 +09:00
이광효 1178841eab itertools: defer dropwhile predicate validation (#8504)
Store the `dropwhile` predicate as a Python object and call it while
advancing the iterator. This defers callable validation until the predicate
is first needed, matching CPython for empty input while preserving exception
propagation during iteration.

Remove the now-passing `test_dropwhile` expected-failure marker.

Assisted-by: Codex:gpt-5.6-sol
2026-08-13 04:32:35 +09:00
William Goode 267c7916b1 Fix map no iterables (#8478)
* map: raise TypeError when called with no iterables

Assisted-by: Claude Code:claude-fable-5

* add test for map with no iterables

Assisted-by: Claude Code:claude-fable-5

* removed expectedFailures

* removed test for map constructed with no iterables - covered by existing in test_itertools
2026-08-11 21:16:10 +09:00
정지웅 755a2b7d26 itertools: fix count() repr to show float step 1.0 (#8261)
Assisted-by: Claude Code:claude-opus-4-8
2026-07-13 14:12:59 +09:00
Shahar Naveh b42bfdc93d Update test_itertools.py from 3.14.4 (#7618) 2026-04-17 20:06:48 +09:00
Shahar Naveh 056cca4476 Update test_itertools.py from 3.14.3 (#7130) 2026-02-14 20:00:17 +09:00
Jeong, YunWon db01a1d653 Remove pickle from itertools 2026-01-17 19:21:11 +09:00
Terry Tianlin LuanandJeong, YunWon 609dbb1439 Downgraded skips in tests (#6716)
* Downgraded skips in tests

* Fixed failing tests

* Fixed test_ftplib + test_socket + test_ssl + test_threaded_import failures

* Removed comments about which tests are run in which environment

* Addressed PR comments

* Annotated skips on failing tests

* Removed unneeded tests

* Removed unneeded sys import from test_ftplib

* Added annotation to test_ftplib

* Readded skipIf to test_cleanup_with_symlink_modes with a more general ENV_POLLUTING_TESTS_WINDOWS

* Addressed PR comments

* Made changes to minimize diff in PR

* Apply suggestion from @youknowone

---------

Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
2026-01-15 21:43:22 +09:00
Shahar Naveh da71b92dd3 Pickle warning for itertools (#6129) 2025-09-07 16:10:22 +09:00
Shahar Naveh 8b6e1e398b Update test_itertools.py to 3.13.7 (#6122)
* Update `test_itertools.py` to 3.13.7

* Apply patch where test name was changed

* Fix some failing tests
2025-09-02 20:19:57 +09:00
Jiseok CHOI 8c4c63673e fix(itertools): add re-entrancy guard to tee object (#5931)
* fix(itertools): add re-entrancy guard to tee object

* apply feedback PyRwLock -> PyMutex & remove AtomicCell<bool> lock field
2025-07-10 18:11:24 +09:00
Shahar Naveh 18fdf85510 Fix itertools.pairwise (#5884)
* Fix pairwise hangs

* Don't skip failing tests;)

* Drop lock of read, not write
2025-07-02 21:10:24 +09:00
Hunter Harloff f6569313d7 Move test_itertools to CPython 3.12 Spec 2024-04-06 19:14:58 +09:00
Amuthan Mannar 9241e2e5d5 [VM] Object pickling implementation for product object (python itertools) (#5089)
* Implemented __reduce__, __setstate__ in product object
2023-10-12 20:12:33 +09:00
Ivan GromovandDimitrisJim cbaed46fb4 Implement itertools.accumulate.__reduce__ and __setstate__ (#4434)
Co-authored-by: DimitrisJim <d.f.hilliard@gmail.com>
2023-06-12 19:26:03 +09:00
Andrey Maltsev 30718a3be5 Update test_itertools.py from Cpython v3.11.2 (#4836)
* Update test_itertools.py from Cpython v3.11.2
2023-04-04 17:40:23 +03:00
Dan Näsman 78586f0275 Chain reduce (#4232) 2022-10-21 15:04:25 +09:00
Dan Nasman f6e423c479 add reduce and setstate for zip_longest 2022-10-19 20:59:05 +03:00
dvermd 164e343517 add takewhile.__reduce__ 2022-10-01 15:31:03 +02:00
jeffwang0516 e5437e4514 remove expected failures for related unittest 2022-08-02 01:15:09 +08:00
CPython developers c737713c9e Update test_itertools from CPython 3.10.5 2022-07-18 02:32:42 +09:00
siontama 60e3ce2dd8 Add basetype flag to itertools module 2022-07-16 00:58:42 +09:00
Padraic Fanning 9de8274780 Unmark passing test 2022-07-03 14:39:36 -04:00
Alexander Scharinger 50dba866fc test: itertools.compress 2022-06-16 10:59:25 +02:00
Alexander Scharinger 4f87f875fc Add filterfalse.__reduce__ 2022-05-26 00:07:17 +02:00
Hyun Park dfa4bdc695 add filter.__reduce__ (#3732) 2022-05-21 19:08:20 +09:00
Padraic Fanning d5aa68d7b5 Clean up skips in test_itertools 2021-10-15 00:09:50 +09:00
Jim Fasarakis-Hilliard 8a967e84c6 Fix arguments for islice. 3 args are allowed. 2021-07-28 11:35:43 +09:00
jfh ab382dbc84 Add fixes for itertools.repeat. 2021-07-25 22:13:27 +03:00
rickygao 52242d4b5d copy the latest test_itertools.py from CPython 2021-02-17 12:56:46 +08:00
rickygao c0e8b8d8bc follow the latest signature itertools.accumulate(iterable[, func, *, initial=None]) 2021-02-16 14:37:06 +08:00
clemado1 fb18a8d66e Fix panic in itertools pruduct 2020-08-20 11:52:31 +09:00
Ben Lewis 2d6a1af8e2 Fix/mark itertools tests. 2020-07-27 12:42:44 +12:00
Ben Lewis 611b4b4907 Add test_itertools.py from CPython 3.8.3. 2020-07-27 12:39:33 +12:00