mirror of
https://github.com/RustPython/RustPython.git
synced 2026-10-29 03:04:38 +08:00
stack/python-3.15-8950
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 youknowdot
|
519370bece |
Adapt essential snippet contracts to Python 3.15 final
Preserve existing coverage while following verified final behavior for script metadata, builtin signatures and loading, AST fixture construction, memoryview diagnostics, mmap search bounds, and struct format errors. The unchanged final comparator exposes the earlier expectations; focused native baseline failures establish the required implementation changes. Assisted-by: OpenAI Codex:model identifier unavailable |
||
|
youknowdot
|
04d9990616 |
Add the eager-runtime hook required by Python 3.15 importlib (#8946)
* Add the eager-runtime importlib lazy-attribute hook Expose _imp._set_lazy_attributes(modobj, name, /), used by CPython 3.15 importlib after eager loading. Match its empty pending-lazy-module registry behavior, including name type and string-subclass hashing checks. This does not implement lazy imports or change the target version or bundled libraries. Assisted-by: Codex:model-version-unavailable * Consolidate lazy import hook tests into stdlib_imp Move the distinct direct-hook argument and hashing contracts into the existing _imp snippet and remove the separate file. Preserve every pre-existing assertion and the CPython version guard; canonical 3.15 lazy-import tests already cover feature-level registry and reification behavior. Assisted-by: Codex:model-version-unavailable |
||
|
Jeong, YunWon
|
259ce8ce66 |
Fix the itertools.tee leak and race, the contextvars and generator races, _asyncio's InvalidStateError lookup, and _imp's frozen data (#8518)
* _imp: read get_frozen_object's data as a whole marshal value
The data argument went through deserialize_code(), which reads a code body
without the type byte in front of it, so nothing marshal.dumps() produces was
accepted. Read it with marshal.loads() and require a code object back.
Assisted-by: Claude
* _imp: bind find_frozen's arguments with FromArgs
The arity check was done by hand on a FuncArgs. Take the arguments through a
FromArgs struct instead, which makes withdata keyword-only, and fill in the
data it asks for: the frozen encoding is not marshal, so the code is
re-serialized into what get_frozen_object() reads back.
Assisted-by: Claude
* itertools: give tee's shared buffer a Python type
The buffer was a PyRc<PyItertoolsTeeData>, which is not a Python object, so the
collector could not walk into it and any cycle running through a tee was
uncollectable. Make it the _tee_dataobject type with a traverse, held by
PyRef, and split the rest along the same lines: tee() is a function, _tee is
the iterator type it builds, and _tee takes a single iterable rather than
returning a tuple from __new__. _tee is weak-referenceable, tee() rejects a
negative n with ValueError and reserves its tuple fallibly.
test_itertools.test_tee passes now; its expectedFailure marker is removed.
Assisted-by: Claude
* contextvars: hold the shared context state under locks
The variable map was a RefCell, the enter flag, the context index, the token's
used flag and the variable hash were Cells, and each carried an unsafe impl
Sync. A Context or ContextVar shared between threads overlapped their borrows
and panicked. The map and the per-variable cache are now PyMutex, the flags and
the index are atomics, entering a context is a compare_exchange, and the three
unsafe impl Sync are gone.
The cache also stopped being read through AtomicCell::as_ptr, which raced a
concurrent store on a value holding a PyObjectRef.
Values displaced from the map or the cache are dropped after the lock is
released: __del__ can come straight back into the same context, and the locks
are not reentrant.
Assisted-by: Claude
* generator: read the frame state under the running claim
send(), send_none(), throw() and close() read `closed` and `frame.lasti()`
before `running` was compare_exchanged, so the frame they went on to resume
could be one another thread had already advanced. A resume that decided from
`lasti() == 0` that the generator had not started pushes no value onto the
value stack, and the code after the yield pops one, which underflows the stack.
The compare_exchange now hands back a guard, taken before those reads and
released after maybe_close(), so the generator is retired while it is still
claimed.
Assisted-by: Claude
* itertools: claim a tee's position and its buffer as one step
`_tee::next` read `index` and moved it on afterwards, and
`_tee_dataobject::get_item` released `running` before the value it fetched
from the source was cached.
Two callers on one `_tee` then read the same index, hand out the same value
twice and advance past a value that was never cached, which leaves `index`
past `values.len()` and indexes the buffer out of bounds. Two callers at the
same index on separate tees each fetch a value from the source, and one of the
two is dropped without reaching a caller.
Both claims now cover the read and the update.
Assisted-by: Claude
* _asyncio: require InvalidStateError to be a type
new_invalid_state_error() called whatever `asyncio.exceptions.InvalidStateError`
names and unwrapped the downcast of the result, so a future asked for a result
it does not have panicked once that attribute was rebound to something that is
not an exception:
asyncio.InvalidStateError = lambda *args: 42
_asyncio.Future(loop=object()).result()
The type is looked up the way get_cancelled_error_type() looks its own up, and
raised with new_exception_msg; a lookup that does not produce an exception type
falls back to RuntimeError, as the other arms already did.
Assisted-by: Claude
|
||
|
Jeong, YunWon
|
d5d19e0611 |
Fix 33 reproduced fuzzing and static-review defects (#8514)
* mmap: guard move() against dest/src past the mapping size
An offset larger than the mapping made `size - dest` underflow, producing
an out-of-range slice index panic. Check the high side first, as write() does.
Assisted-by: Claude
* exceptions: keep ImportError name/path/name_from out of an empty instance dict
ImportError.__reduce__ unwrapped get_arg(0), which is None when the exception
was constructed with no positional argument, aborting on pickle.dumps(ImportError()).
Fall back to the full args tuple there, and expose name/path/name_from as class
attributes defaulting to None so a bare ImportError() reduces to (cls, ()).
Assisted-by: Claude
* asyncio: stop unwrapping the _current_tasks downcast in current_task()
_asyncio._current_tasks is a reassignable module attribute; the three sibling
functions already degrade gracefully when it is not a dict, current_task() did not.
Assisted-by: Claude
* asyncio: raise TypeError when FutureIter.throw()'s exception class returns a non-exception
exc_type.call() goes through a Python-controlled __new__, so its result can be
any object; the downcast was unwrapped. Report it as a TypeError instead.
Assisted-by: Claude
* sequence: use a fallible reservation for sequence repetition
The guard only rejects a repeat whose per-element size crosses MAX_MEMORY_SIZE,
so (1,) * (10**12) still reached Vec::with_capacity and aborted in the allocator.
Reserve fallibly and surface a MemoryError.
Assisted-by: Claude
* collections: reject an oversized deque repetition with MemoryError
deque * sys.maxsize reached the allocator and aborted with a capacity overflow.
Apply the MAX_MEMORY_SIZE guard the other sequences already carry.
Assisted-by: Claude
* itertools: raise OverflowError for an out-of-range r argument
combinations/combinations_with_replacement/permutations narrowed r with
to_usize().unwrap(), so r=2**64 panicked instead of raising.
Assisted-by: Claude
* math: stream the generic sumprod path instead of collecting both iterables
The big-int path collected each argument into a Vec before multiplying, so an
unbounded iterable exhausted memory. Advance both iterators in lockstep and
accumulate, reporting a length mismatch when only one is exhausted.
Assisted-by: Claude
* _imp: raise TypeError for a second positional argument to find_frozen
withdata is keyword-only and unimplemented; passing it positionally hit an
unimplemented!() and aborted.
Assisted-by: Claude
* _typing: check _idfunc arity before indexing args
_typing._idfunc() with no argument indexed args[0] out of bounds.
Assisted-by: Claude
* exceptions: require a sequence for the ExceptionGroup excs argument
The argument was collected before being validated, so an unbounded iterable
such as itertools.count() exhausted memory. Reject a non-sequence up front.
Assisted-by: Claude
* mmap: treat an inverted find/rfind range as empty
find(b"x", 5, 2) built a slice whose start exceeded its end and panicked.
Assisted-by: Claude
* collections: give deque and defaultdict a GC traverse
Neither type opted into traversal, so a reference cycle through a deque or a
defaultdict was never collected.
Assisted-by: Claude
* itertools: opt the iterator types into GC traverse
cycle and its siblings hold Python references but declared no traverse, so a
cycle built through one of them leaked.
Assisted-by: Claude
* _ctypes: mask an out-of-range int instead of panicking
c_char_p(2**64) and pointer item assignment called .expect() on the narrowing
conversion. Wrap the value to the target width, which is what the C
implementation stores.
Assisted-by: Claude
* hashlib: import _hashlib when a hash module is loaded
The hash object type is a static type owned by _hashlib; calling _md5.md5()
without _hashlib imported hit an uninitialized static type and panicked.
Assisted-by: Claude
* _csv: fall back to the built-in dialect defaults when none is registered
The dialect table is empty until csv.py registers 'excel', so _csv.reader([])
unwrapped a missing entry and panicked.
Assisted-by: Claude
* builtins: reject surrogates in compile()/eval() source instead of panicking
expect_str() panics on a str containing surrogates; convert with try_as_utf8
so eval(chr(0xd800)) raises.
Assisted-by: Claude
* _suggestions: require a list for _generate_suggestions candidates
The argument was collected before validation, so an unbounded iterable
exhausted memory.
Assisted-by: Claude
* lzma: size the filter chain before consuming it
filters= was collected into a Vec before the length check, so an unbounded
iterable exhausted memory. Take the length through the sequence protocol first.
Assisted-by: Claude
* classmethod: opt into GC traverse
staticmethod already declares traverse; classmethod did not, so a cycle through
the wrapped callable leaked.
Assisted-by: Claude
* posix: reject unbounded iterables in posix_spawn and setgroups
argv, setsigdef, setsigmask and setgroups bound an ArgIterable and collected it
before validating, so an infinite generator exhausted memory. Require a
list/tuple for argv, validate signals while streaming, and take setgroups
through the sequence protocol.
Assisted-by: Claude
* _ctypes: size Array slice assignment and _argtypes_ before collecting
Both eagerly collected their argument, so an unbounded iterable exhausted
memory before the length check ran.
Assisted-by: Claude
* sys: propagate the breakpointhook warning failure
warn() was unwrapped, so an unimportable $PYTHONBREAKPOINT under
-W error panicked instead of raising.
Assisted-by: Claude
* structseq: require the sequence argument when constructing a struct sequence
A no-argument construction produced an empty backing tuple, and reading any
named field then indexed out of bounds.
Assisted-by: Claude
* Guard the hash slot dispatch against unbounded recursion
`PyObject::hash` invoked the type's hash slot directly, so an element-wise
`__hash__` following a deeply nested object graph recursed one native frame
per level and overflowed the stack. Wrap the dispatch in `with_recursion`,
matching the repr and rich-compare dispatches in the same file, so
`hash(x)` on a nested tuple/GenericAlias/slice raises `RecursionError`.
Assisted-by: Claude
* genericalias: guard the __parameters__ walk against unbounded recursion
`make_parameters_from_slice` recursed into every list/tuple argument with
nothing counting the frames, so `list[L]` for a self-referential or deeply
nested `L` overflowed the native stack at subscript time. Wrap the descent
in `with_recursion`, which makes the walk fallible; `PyGenericAlias::new`,
`from_args` and `make_parameters` now return `PyResult` and every caller
propagates.
Assisted-by: Claude
* Fix the type confusion in PyAtomicRef's Debug impl
`PyAtomicRef<T>` stores a pointer to a `Py<T>`, as `Deref`, `load_raw`,
`swap` and `Drop` all read it, but `Debug` cast it to a bare `T` and
formatted the object header as payload bytes. For `PyFunction`, whose
`code: PyAtomicRef<PyCode>` has a pointer-chasing `Debug`, that
dereferenced header words and segfaulted.
Cast to `PyObject` instead, which is what `Drop` already does and which
also covers the `PyAtomicRef<PyObject>` and `PyAtomicRef<Option<T>>`
instantiations that have no `Py<T>`.
`_asyncio._enter_task` reached this through `{:?}` in its "Cannot enter
into task" message; format the two tasks with their Python repr, which is
what the message is meant to show.
Assisted-by: Claude
* _sre: disallow instantiating Match
`Match` inherited `object.__new__`, so `M.__new__(M)` produced an instance
whose `regs`/`string`/`pattern` were never filled in by a match run; the
mapping subscript path read them and dereferenced garbage. The type has no
public constructor, so mark it `DISALLOW_INSTANTIATION`, which makes
`M.__new__(M)` raise `TypeError: cannot create 're.Match' instances`.
Assisted-by: Claude
* utils: return the empty repr instead of asserting a non-empty collection
`collection_repr` took the first element with an `.expect()` justified by the
caller's preceding non-empty check. Another thread clearing the collection
between that check and the iteration made the iterator yield nothing and
panicked the worker. Fall back to the caller-supplied empty form, which is
the text those callers already produce for an empty collection.
Assisted-by: Claude
* itertools: advance cycle's index atomically
`cycle.__next__` did `fetch_add(1)` and then reset the index to 0 in a
separate store, so two threads replaying the saved items could both read an
index past the end of `saved` and panic on the slice access. Do the advance
and the wrap in one `fetch_update`.
Assisted-by: Claude
* _ctypes: reject a float argument to a foreign call without argtypes
`conv_param`, the conversion used when `argtypes` is not set, converted its
argument with `try_int`, which goes through `__int__` and so accepts a
float. `libc.strlen(1.5)` therefore passed 1 where the callee expects a
`char *` and the callee dereferenced it. Match `ConvParam`, which does a
`PyLong_Check` and converts nothing: take the branch only for an `int` (or
a subclass, so `True` still converts), and let a float fall through to
"Don't know how to convert parameter".
The branch below it converted a float to a C double, but `try_int` claimed
every float before it could run, so it was dead; `CArgValue::Double`
existed only for that branch and both go. Typed doubles are unaffected —
they travel as `CArgValue::Typed` with code 'd'.
Assisted-by: Claude
* Report the iterator itself from PyIter's traverse
`Traverse for PyIter<O>` delegated to the inherent `PyObject::traverse` of
the object it wraps, so it reported that iterator's referents instead of
the iterator. The iterator's own reference to those referents was then
never subtracted during the collector's reference-subtraction pass, the
referents kept a non-zero gc_refs, and every object reachable from them
was classified as a root.
Any cycle running through a type with a `PyIter` field therefore survived
collection: `map`, `filter`, `zip`, `enumerate`, `reversed` and the
`itertools` iterators all leaked, while the same cycle through a `list`,
`tuple` or `list_iterator` collected. Report the wrapped object, as the
`PyObjectRef`, `PyRef<T>` and `PyStackRef` impls do.
`itertools.tee` still leaks: its shared buffer is a `PyRc<PyItertoolsTeeData>`
rather than a Python object, so the collector cannot see through it.
Assisted-by: Claude
* Remove the obsolete expectedFailure on test_code_module.test_unicode_error
Compiling a source string containing a lone surrogate now raises
UnicodeEncodeError, so the test passes.
Assisted-by: Claude
* itertools: reserve the combination indices fallibly
`combinations` and `combinations_with_replacement` built their index vector
with an infallible allocation, so an `r` that passes the ssize_t check but
does not fit in memory aborted the process instead of raising MemoryError.
Assisted-by: Claude
* Apply the struct sequence constructor's dict argument
`structseq(sequence, dict)` discarded its second argument, so the hidden
fields past `n_sequence_fields` — `tm_zone`, `st_atime` and friends — were
always None when constructed directly or restored from a `(sequence, dict)`
pickle, and a non-dict second argument was accepted silently.
Take the dict, require it to be a dict, and fill the hidden slots the
sequence did not cover from it. A key that names a field the sequence
already supplied, or no field at all, is now a
"got duplicate or unexpected field name(s)" TypeError instead of being
dropped. Both arguments are bindable by name, as `sequence` and `dict`.
`os.stat_result` and `os.statvfs_result` did not accept a second argument
at all; they and `time.struct_time` now share the parsing.
Assisted-by: Claude
* _imp: report the argument count in find_frozen's arity error
Assisted-by: Claude
* Add regression tests for the reproduced crashers
One case per catalog entry, each asserting the behavior the fix produces:
recursion guards, the memory-unsafety sites, the overflow and unbounded
allocation guards, the eager-collection rejections, the unwrap sites, and
the cycles the collector now breaks. Every expected value was checked
against CPython 3.14.
Assisted-by: Claude
* Tolerate a changed-size RuntimeError in the set repr stress test
The mutator and reader threads race on purpose; a "changed size during
iteration" RuntimeError is a valid outcome of that race and should not fail
the test. The panic it guards against is not.
Assisted-by: Claude
* Move the crash regression tests into per-module snippets
crash_regressions.py collected every reproduced crasher in one file. Split it
into the snippet for the module each case exercises, and add stdlib_gc.py,
stdlib_asyncio.py, stdlib_lzma.py, stdlib_threading_set_repr.py and
stdlib_threading_itertools_cycle.py for the cases with no existing home.
The suite runs every snippet under the host CPython too, so the checks only
RustPython raises are guarded by sys.implementation.name: the hash and
__parameters__ recursion depth, and the deque repeat overflow. The float
ctypes argument accepts either TypeError or ctypes.ArgumentError.
Assisted-by: Claude
|
||
|
Rex Ledesma
|
4e2e0b41c6 |
chore: add ruff format --check (#5774)
* chore: add `ruff format --check` * fix tests |
||
|
Jeong YunWon
|
8c38a8381c | os is frozen in 3.11 | ||
|
Jeong YunWon
|
d22446d1dd | imp.py -> stdlib_imp.py |