* Implement PEP 688 __buffer__ and __release_buffer__
A Python class could not export a buffer: the slot machinery had no
bf_getbuffer or bf_releasebuffer, and every consumer acquired buffers as
PyBUF_FULL_RO through a module of PyBUF_* constants.
Add both slots. PyBuffer::release now runs a Python __release_buffer__
before the exporter's own release, once per acquisition, which PyBuffer
tracks with an `acquired` flag that clones do not inherit. An export made
by a Python __buffer__ is held by a _buffer_wrapper payload that counts its
exports and drops the returned memoryview with the last one, and the view
handed to __release_buffer__ is a _buffer_window that owns no export, so
releasing it inside the hook is inert instead of re-entering it.
Replace the PyBUF_* constants with a BufferFlags bitflags type whose
composite requests are supersets of the simpler ones, so `contains` answers
the REQ_* questions, and pass the request to PyBuffer::from_object. Each
consumer now asks for what its counterpart asks for: y* arguments for
SIMPLE, w* for WRITABLE, BytesIO.write for CONTIG_RO, bytes(), bytearray()
and memoryview() for FULL_RO. memoryview checks the request in
memory_getbuf, and array.array and mmap.mmap expose __release_buffer__.
Test buffer support with PyObject::check_buffer (PyObject_CheckBuffer)
instead of attempting an acquisition, so an exception raised by __buffer__
is no longer reported as the object not being bytes-like, and a __buffer__
with side effects runs once. PyBytesInner becomes a y* conversion as a
result: bytes and bytearray methods no longer accept iterables of ints, and
find, index, count and __contains__ take the arguments
parse_args_finds_byte and bytes_contains describe.
A view exports its start offset in the descriptor rather than in its
window, which fixes a panic when collecting from a negative-stride view.
BytesIO.write rechecks closed after acquiring its buffer, which __buffer__
can close in between.
Assisted-by: Claude Code:claude-opus-5
* Rework the buffer protocol around managed exports and view offsets
Give `PyBuffer` the `_PyManagedBufferObject` shape: one `bf_getbuffer`
acquisition is shared by every handle taken from it, cloning takes another
share instead of re-acquiring, and the exporter's release runs once when the
last share goes away. Remove `retain`, the unsafe `drop_without_release`, the
three `impl Drop`s and the `ManuallyDrop` that stood in for this. Add
`abort_acquisition` so a failed request does not run `bf_releasebuffer`.
Move the view start into `BufferDescriptor::offset`, the `Py_buffer.buf`
analogue, and drop the separate `start` fields on `PyMemoryView` and
`PyBufferWrapper`. Slicing goes through `SaturatedSlice::adjust_indices_start`,
which reproduces `PySlice_AdjustIndices` and keeps the adjusted start.
Fix `zip_eq` to take its contiguous fast path only when both last dimensions
are contiguous, and make `for_each_segment` and `zip_eq` handle zero-length and
zero-dimensional views.
Add `BufferDescriptor::projected` so a request without `PyBUF_ND`,
`PyBUF_STRIDES` or `PyBUF_FORMAT` receives a correspondingly reduced
descriptor, and reject a request without `PyBUF_INDIRECT` against an exporter
that has suboffsets.
Copy the source first in `memoryview` slice assignment when both sides reach
the same root exporter.
Hold the export across the resize in `bytearray.extend`, take `y*` in
`marshal.loads`, stop probing the buffer protocol in `FsPath`, rewrite `ord`
over the concrete string types, fold `array`'s buffer slot into one
`slot_as_buffer`, take `w*`/`y*` in `_overlapped`, and thread the new `offset`
field through the `_ctypes` descriptors.
Assisted-by: Claude
* Re-check a memoryview after a conversion and tighten cast and release
`pack_single` and `unpack_single` addressed the buffer with a position taken
before `__index__` ran, so releasing the view from that conversion read or wrote
outside the exporter's storage, panicking when it had also shrunk. Check the
released flag again once the conversion is done, as `CHECK_RELEASED_AGAIN` does.
Reject a cast to a format that is not a single native format character with an
optional `@` in front of it. An empty format reached a division by its item size
of zero. get_native_fmtchar
Report a second `__release_buffer__` on the same view as a `ValueError` rather
than accepting it, and check that the view belongs to the object first; the
silent case is a view that exports nothing. wrap_releasebuffer
Compare against another memoryview by reading its view where it lies instead of
acquiring a buffer from it, so the restricted view handed to
`__release_buffer__` compares equal rather than unequal in one direction only.
memory_richcompare
Name the type in the unraisable an exception from `__release_buffer__` reports,
as `releasebuffer_call_python` does, instead of reporting the exporter object.
Assisted-by: Claude
* Re-check the destination of a memoryview slice assignment
Acquiring the source runs `__buffer__`, which can release the destination view,
so check the released flag again once the source is in hand and before the
structures are compared, as `copy_single` does.
Build the sliced destination as a view that counts as no export, the way a
`Py_buffer dest = *view` copy does. Holding one kept the exporter unresizable
for the length of the assignment, so a source that released the view and then
resized the exporter met a `BufferError` instead of the assignment reporting the
released view.
Assisted-by: Claude
* Answer the memoryview contiguity getsets with the layout
`f_contiguous` reported the row-major answer for one dimension and False
for any more, so a view laid out both ways, such as one of shape (1, 8),
reported False while `check_buffer_request` accepted a `F_CONTIGUOUS`
request on it. `contiguous` answered row-major order alone.
Move the Fortran-order test from `PyMemoryView` to `BufferDescriptor`,
next to the row-major one it differs from only in iteration order, and
answer all three getsets with it.
Assisted-by: Claude
CPython's _abc._abc_init validates that __abc_tpflags__ doesn't combine
Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING (Modules/_abc.c::_abc_init).
RustPython's check_abc_tpflags copied the bits onto slots.flags without
that validation, so a class declaring both produced an inconsistent
type silently.
Validate the masked __abc_tpflags__ value against COLLECTION_FLAGS and
return a String error (which the existing new_heap_inner caller turns
into a TypeError) when both bits are present. Move the inheritance-skip
check after the self-attrs branch so a child overriding __abc_tpflags__
is still validated even when the parent already set one of the bits.
- Updated Lib/collections from CPython v3.14.3
- Preserved RustPython's fallback implementation for defaultdict
- Added test for update_reentrant_add_clears_counter
- Minor formatting fixes in test file
Co-authored-by: CPython Developers <>
This pull request updates `types` module to v3.14.2. While doing it, it
fixes also async-related feature. This pull request's base is generated
by #6827.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Generators that act as iterable coroutines are now recognized as
awaitable, improving async behavior.
* Yield-from and await interactions now handle coroutine-iterable
sources more consistently.
* **Bug Fixes**
* Reduces spurious TypeError cases when awaiting or yielding from
wrapped coroutine-like generators.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This test will not work until the `__buffer__` and `__release_buffer__`
methods are implemented on the appropriate builtin types, which is
outside the current scope.
The original hash algorithm just XOR'd all the hashes of the elements of
the set, which is problematic. The CPython algorithm is required to pass
the tests.
- Replace `PyFrozenSet::hash` with CPython's algorithm
- Remove unused `hash_iter_unorded` functions
- Add `frozenset` benchmark
- Enable tests
- Lower performance expectations on effectiveness test
- Adjust `slot::hash_wrapper` so that it doesn't rehash the computed
hash value in the process of converting PyInt to PyHash.
Python does not define `list().__bool__`, `dict().__bool__`, and
`str().__bool__`, and some tests were failing because they were
defined.
I could not find any references to them and deleting them doesn't
seem to break anything.
Clean implementation of changes in PR #3371 based on feedback.
Copies from [CPython tag `v3.9.7` and adds back custom RustPython changes where needed for:
- `Lib/collections/__init__.py`
- `Lib/test/test_collections.py`
Closes: #3371
Jeong, YunWon