* Skip Python-defined classes when collecting builtin type docs
generate.py recorded every object.__subclasses__() entry as builtins.<name>
after importing the whole stdlib, so classes written in Python (tkinter,
unittest, tarfile, ...) ended up in the doc database. Classes that are
mutable heap types without C method descriptors and whose module is not a
C extension are now skipped, and their entries are removed from
data.inc.rs.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Store database docstrings xz-compressed behind an optional doc feature
- rustpython-doc's build script compresses all database docstrings into
one xz blob with an offset index; identical texts share one offset.
- Macros emit blob references for database docs; vm decompresses the
blob on first access.
- New `doc` cargo feature (default on) in rustpython, rustpython-vm and
rustpython-stdlib. Without it the blob and xz are not linked and
database-backed __doc__ is None; __text_signature__ is unaffected.
- Native types without a docstring get __doc__ = None in their dict.
- Workflows that build with --no-default-features enable `doc` where
tests or releases need docstrings.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Raise MemoryError instead of aborting on impossible allocations
Assisted-by: Claude Code:claude-opus-5-5
* use CPython 3.14's integer digit limit for left shifts
* size the pow memory check by what malachite allocates
* Make sequence allocations fallible and reuse deallocation buffers
Assisted-by: Codex:GPT-6
---------
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
* Address builtin members by field offset
`#[pymember]` now declares a field of the payload and generates a
PyMemberDef whose offset is the payload offset plus the field offset.
The getter/setter function form and `MemberAccess::Func` are removed;
builtin, slot, and C-API members share one get/set path. The macro
checks the field type against the member type.
Converted members: property fget/fset/fdel/__doc__, function
__globals__/__closure__/__builtins__/__doc__/__module__, staticmethod
and classmethod __func__/__wrapped__, complex real/imag (T_DOUBLE),
frame f_trace_lines (T_BOOL), defaultdict default_factory, getset and
member descriptor __name__/__objclass__, module __dict__.
frame.f_trace_opcodes is a getset.
Writable member fields are atomic pointer cells. The instance dict
pointer is an atomic cell at a fixed offset in front of the object,
and dict presence follows the type's HAS_DICT flag. The pickle
basicsize check no longer counts dict, weakref, and slot storage,
which live outside the payload.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Address member review findings
- `#[pymember]` keeps leading `#[cfg]` attributes on the generated
registration, and malformed `field`/`offset` values are compile
errors.
- The `descriptor` module and the items the expansion names are public.
- Writable double members are an `AtomicU64` cell, loaded and stored
with relaxed atomics. Readonly double members stay plain `f64` reads.
- `LOAD_ATTR_SLOT` is not specialized for `Py_AUDIT_READ` members.
- `defaultdict.__reduce__` passes no factory argument when
`default_factory` is None.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Declare members on payload fields
`#[pymember]` goes on a payload struct field, or on the struct with an
explicit `offset` when the storage is not a payload field. The struct
`#[pyclass]` builds `PyClassDef::MEMBERS`, a static `PyMemberSpec`
table, and `extend_class` registers one member descriptor per entry.
`path` addresses a subfield, one field may carry several members, and a
`#[cfg]` on the field gates its entry. The impl-side const form is
removed and now a compile error.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Split nullable and non-null atomic cells
The nullable cell API (`new_empty`, `load_owned`, `store`,
`compare_exchange_empty`) moves from `PyAtomicRef<PyObject>` to
`PyAtomicRef<Option<PyObject>>`, and the typed `PyAtomicRef<Option<T>>`
shares it. The non-null `PyAtomicRef<PyObject>` gets `load_owned` and
`store` that return the object directly.
Slot cells, property fget/fset/fdel/__doc__, and defaultdict
default_factory are nullable cells. staticmethod and classmethod
callable are non-null cells and are read without `unwrap`. The
instance dict cell is `PyAtomicRef<Option<PyDict>>`, so reading it no
longer downcasts.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Drop the unused cold trace_lines flag
`FrameColdData::trace_lines` had no writer after `f_trace_lines`
became a frame object member. A frame without a frame object traces
lines by default.
Assisted-by: Claude:claude-opus-5-5
* Compute payload offsets from the merged Py header
`PyInner` is now `Py`, so the payload offset and the dict member
offset are measured from `Py<T>`.
Assisted-by: Claude:claude-opus-5-5
* Expose descriptor names and owner types as members
method_descriptor, classmethod_descriptor, and wrapper_descriptor
expose `__objclass__` and `__name__` as readonly members, and
method-wrapper exposes `__self__` as a readonly member. The separate
`objclass` field of method and classmethod descriptors always equaled
`common.typ` and is removed.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Expose remaining builtin tp_members as members
Attributes stored in a payload field become readonly or writable
member descriptors: bound method __func__/__self__, code
co_argcount/co_posonlyargcount/co_kwonlyargcount/co_stacksize (Py_T_UINT),
generic alias, coroutine and async generator members, interpolation and
template, range and slice start/stop/step, type __base__, union, and the
typing TypeVar, ParamSpec, TypeVarTuple, and TypeAliasType members.
`MemberKind` gains `Int` and `Uint` with the `PyMember_GetOne`/`SetOne`
range checks.
Attributes computed on access or kept behind a lock stay getsets.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Spell a docless member as doc = false
`#[pymember(doc = false)]` stores no docstring, matching the
`doc = "..."` form, and replaces the separate `no_doc` flag.
Assisted-by: Claude:claude-opus-5-5
* Make pymember readonly by default
`#[pymember]` declares a readonly member unless `writable` is given.
The `readonly` option is removed and every declaration is converted.
Assisted-by: Claude:claude-opus-5-5
* Check the receiver of method descriptors
Binding or calling a method_descriptor with an object that is not an
instance of the owner type raises TypeError before the native function
runs. Static and class methods are exempt.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Expose exception attributes as members
BaseException.__suppress_context__, StopIteration.value, SystemExit.code,
ImportError, NameError, AttributeError, OSError, SyntaxError,
BaseExceptionGroup and the UnicodeEncode/Decode/TranslateError attributes
are member descriptors backed by payload fields. ImportError, NameError,
AttributeError and SyntaxError gain payloads for their fields, and the
methods that read those attributes read the fields. Add the Py_ssize_t
member kind for UnicodeError start and end.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Avoid same-type casts in unsigned member conversion
On targets where c_ulong is u32, casting through c_ulong was a no-op
cast that clippy rejects.
Assisted-by: Claude:claude-opus-5-5
* Prefer an explicit pymember doc over the doc table
doc = "text" is used as the docstring without consulting the stored
attribute documentation; only an omitted doc looks it up. Drop the member
doc strings that repeated the stored documentation.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Infer pymember kinds from field types
MemberLayout gives the member kind of each accepted field type, and the
macro reads it from the field or path subfield, so type = "bool" and
the other numeric spellings are gone. type only selects object_ex.
Writable doubles use a new AtomicF64 cell instead of AtomicU64 bits.
MemberLayout and MemberCell replace the per-kind marker traits.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Read nullable atomic cells only through load_owned
PyAtomicRef<Option<T>> and PyAtomicRef<Option<PyObject>> lose to_owned,
to_owned_ordering and try_to_owned; load_owned, which retries the
incref until it applies to the pointer still in the cell, is the one
owned read. Callers that kept a deref borrow across VM calls now take a
strong reference.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Inherit static type slots from the whole MRO
extend_class copied slots only from the direct bases, and tp_init is
copied only from a base that defines it, so TabError missed the
SyntaxError init that IndentationError inherits. Walk the MRO the same
way init_slots does.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Let Rust doc comments win over stored docstrings
A /// comment on a Python-exposed item is its docstring; the
rustpython-doc entry is used only when there is none. #[pymember] reads
the field's /// comment instead of doc = "...". Doc comments that were
shadowed by a stored docstring become // comments, so no docstring
changes. AGENTS.md describes the rule.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Adapt members to upstream changes
_collections._tuplegetter.__doc__ is a writable nullable member on the
payload field. The impl-side const form no longer compiles.
Assisted-by: Grok:4.7
* Drop docstring text left as comments and inline downcast_cell
Comments that only repeated an item's docstring are removed; developer
notes stay as // comments. The typed cell methods call
downcast_unchecked directly, each with its own safety note.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Read sqlite row factories through load_owned
The nullable cell no longer has to_owned; the sqlite module is built only
with the sqlite feature, so the default build did not catch it.
Assisted-by: Claude:claude-opus-5-5
Interned strings live in the process-wide Context and cache their hash.
Each top-level interpreter built its own HashSecret from its hash_seed,
so a second interpreter with a different seed looked up cached hashes
computed with the first secret and failed bootstrap with
`KeyError: _frozen_importlib`.
The secret is now a process-wide OnceLock set by the first top-level
interpreter; later seeds are ignored. The per-interpreter hash_secret
field and the check in VirtualMachine::new are removed.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Read PyTuple payloads through as_slice()
- Add Py<PyTuple<R>>::as_slice(); it is generic over R, so typed tuples use
the same view.
- Callers reach slice methods, indexing and slice arguments through
as_slice().
- PyTuple's own sequence and mapping slots and marshal's set_marshal_item reach
the remaining PyTuple methods through the payload field.
- The Py<T>::payload field is pub(crate).
Assisted-by: Claude Code:claude-opus-5-5
* Read PyBytes payloads through as_bytes()
- Callers read bytes through Py<PyBytes>::as_bytes().
- bytes.rs reaches PyBytesInner and the remaining PyBytes methods through
the payload field; fspath and os call payload.contains_nuls().
- DictKey for [u8] falls back to the PyObject key_eq; it previously resolved
to its own key_eq through Deref.
Assisted-by: Claude Code:claude-opus-5-5
* Copy capi tuple arguments with as_slice().to_vec()
Assisted-by: Claude Code:claude-opus-5-5
* Read sqlite3 Row tuples through as_slice()
Assisted-by: Claude Code:claude-opus-5-5
* Read platform-specific tuples and bytes through as_slice() and as_bytes()
Assisted-by: Claude Code:claude-opus-5-5
* Read PyInt and PyFloat payloads through as_bigint() and to_f64()
Add `Py<PyInt>::as_bigint()`, `Py<PyInt>::try_to_primitive()` and
`Py<PyFloat>::to_f64()`, which forward to the payload methods of the same
name. Reads of the `value` field and primitive conversions go through
these views; other PyInt and PyFloat methods are called through the
`payload` field inside rustpython-vm and through payload() outside it.
Assisted-by: Claude Code:claude-opus-5-5
* Read PyComplex payloads through as_complex()
Add Py<PyComplex>::as_complex() and use it at the call sites that
read the payload directly.
Assisted-by: Claude Code:claude-opus-5-5
* Move PyInt conversion helpers to Py<PyInt>
try_to_i64_fast(), to_str_radix_10(), as_u32_mask(), as_u64_mask(),
try_to_primitive() and try_to_primitive_raw() only read the integer
value, so they are defined on Py<PyInt> and callers no longer go through
the payload. PyRange computes its protocol length from the BigInt
directly instead of building a PyInt payload.
Assisted-by: Claude Code:claude-opus-5-5
* Share one conversion between try_to_primitive() and its OverflowError variant
Py<PyInt>::try_to_primitive() and try_to_primitive_in_range(), renamed
from try_to_primitive_raw(), both call to_primitive(). They differ only
in how a negative value for an unsigned type is reported: ValueError for
try_to_primitive(), OverflowError for try_to_primitive_in_range(). The
target type is bound with `for<'a> TryFrom<&'a BigInt>`, so the methods
no longer tie a lifetime to &self.
Assisted-by: Claude Code:claude-opus-5-5
* Move PyInt bitwise operators to Py<PyInt>
__xor__, __or__, __and__ and their int_op() helper are defined on
Py<PyInt>. PyBool calls them without going through the payload.
Assisted-by: Claude Code:claude-opus-5-5
* Move PyBytes operators to Py<PyBytes>
__add__, __contains__, __getitem__ and __mod__ are defined on
Py<PyBytes>. The subscript, contains and remainder slots call them
without going through the payload.
Assisted-by: Claude Code:claude-opus-5-5
* Move PyInt general_op() and modpow() to Py<PyInt>
The power slot calls them on Py<PyInt> without going through the
payload.
Assisted-by: Claude Code:claude-opus-5-5
* Accept `__index__` objects for integer arguments
CPython converts these nine arguments with `__index__`, RustPython
required an exact `int`. Uses the existing `ArgSize`/`ArgPrimitiveIndex`
converters at each call site rather than widening the blanket `int`
conversion, so argument paths CPython keeps strict (ctypes pointers,
thread handles) are unaffected, and the range check happens while the
argument is bound rather than in the method body.
Also widens `maxsplit` before adding one. `sys.maxsize` was already
reachable there with a plain `int` and overflowed in a debug build.
Assisted-by: Claude Code:claude-opus-5
* Add snippet for `__index__` argument conversion
Expectations verified against CPython 3.14.
Assisted-by: Claude Code:claude-opus-5
* Call __index__ in primitive integer conversion; add ArgStrictInt
The TryFromObject impls for isize, usize and the fixed-width integer
types now take an int (or subclass) as-is and otherwise call __index__,
instead of accepting only int instances.
ArgStrictInt<T> keeps the old int-only conversion. The ctypes address
and handle arguments (from_address, PyObj_FromPtr, dlsym, dlclose,
call_function, call_cdeclfunction, non-Windows FreeLibrary) use it.
_thread._make_thread_handle checks for an int itself and raises
"ident must be an integer".
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Accept __index__ where arguments were bound as int objects
chr, bin, math.factorial, zlib.crc32/adler32, binascii.crc32/crc_hqx,
the start/stop of sequence index(), the select event mask and a
__len__ result took an int object directly and rejected other objects
with __index__. They now go through __index__.
math.ldexp still accepts only an int for its exponent and raises
"Expected an int as second argument to ldexp." otherwise.
Extend protocol_index_argument.py with the newly accepted calls, an int
subclass whose __index__ is not called, and the sites that still
reject __index__.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Replace ArgPrimitiveIndex and ArgSize with PySsize/PySize aliases
The primitive integer conversions call __index__ themselves, so
ArgPrimitiveIndex<T> and its alias ArgSize behaved the same as T. They
are removed. Arguments that used ArgSize or ArgPrimitiveIndex<isize>
now take PySsize (= isize), ArgPrimitiveIndex<usize> takes PySize
(= usize), and the other ArgPrimitiveIndex<T> take T. Those with a
literal fallback take it as a `default`; those where a missing argument
is a distinct state stay OptionalArg.
list.insert is a #[pymethod] again instead of going through a wrapper.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
---------
Co-authored-by: Roshan Ramani <roshanramani.dev@gmail.com>
Accept F/D through the existing buffer packing implementation and restore
its complex round-trip test. Fix exact __complex__ results warning as
subclasses so valid conversions also work with warnings treated as errors.
Codex assisted with implementation, regression coverage, and review.
Tested: CPython 3.14.7 comparison; struct/memoryview/complex/cmath; workspace and C-API tests; Clippy; pytest; prek
Assisted-by: Codex:gpt-6-astra
* Resolve class attribute docs at macro time and drop rustpython-doc from vm
- #[pyclass] and #[pystruct_sequence] emit a sorted PyClassDef::ATTR_DOCS
table from the doc database; attribute and slot docs look up that table.
- rustpython-vm no longer depends on rustpython-doc.
- _operator.attrgetter, itemgetter, methodcaller and object get
constructor Args with FromArgs::PARAMS, replacing the hardcoded class
signature docs in class.rs.
- methodcaller() with no arguments raises "methodcaller needs at least one
argument, the method name".
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Drop default values that equal the field type's default
Bare `default` on integer and bool fields already renders the zero value
in the text signature, so `default = 0` and `default = false` are
replaced with bare `default`.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Return the empty str singleton from Context::new_str
Assisted-by: Claude:claude-opus-5-5
* Speed up exec/eval and audited builtins when no audit hooks are set
Assisted-by: Claude Code:claude-opus-5-5
* move audit hooks from `VirtualMachine` to `PyGlobalState`
* addaudithook only appends once the lock
* sync with main
* Remove unused PyLease
PyLease wrapped a read guard over the object's type when `typ` was a
`PyRwLock<PyTypeRef>`. Its last users, `class()` and `lease_class()`,
went away when `typ` became a `PyAtomicRef<PyType>` (#3828). Its field
is private and it has no constructor, so it cannot be built anywhere.
Assisted-by: Claude Code:claude-opus-5-5
* Skip test_large_content_length_truncated on macOS
The test sets the CGI handler timeout to 1ms. That timeout also bounds
reading the request line, so when the client has not sent the request
within 1ms the server logs "Request timed out" and closes the
connection. The client then gets ConnectionResetError or
BrokenPipeError. This reproduces on macOS with both RustPython and
CPython 3.14.6. The skip was removed in #8759 after Linux-only
repeats.
Assisted-by: Claude Code:claude-opus-5-5
* Render CPython defaults in generated text signatures
Arguments whose default rendered as `<unrepresentable>`, while the
same callable has a representable default in CPython, now render that
default:
- Where the Rust value can be a literal, the argument becomes a plain
field with `default = <literal>`.
- The remaining cases keep their Rust default and add `py_default`
with a comment. These are wrappers such as ArgIntoFloat or
ArgIntoBool, values that need the VM, enums, and Option where None
means another value.
Byte string literals render as Python bytes. A function registered
under several names gets an internal doc per name, so its __doc__ no
longer carries the signature header.
Removed expectedFailure from test_pydoc tests that now pass.
Assisted-by: Grok:grok-4.7
Assisted-by: Claude:claude-opus-5-5
* Match native callable signatures to CPython names and kinds
Rename parameters and adjust positional-only / keyword-only kinds and
defaults of native functions and methods so their generated text
signatures match CPython. Remove expectedFailure markers from tests
that now pass.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Fix Windows console read size and keep type.__prepare__ permissive
Assisted-by: Claude:claude-opus-5-5
* Derive signature defaults from typed Rust values
Param.default is now a DefaultRepr rendered by the const writer.
Literals, integer and bool constants become typed defaults, and other
default expressions use the type's const fn py_default. A bare
`optional` renders as None, so `default = None` rewrites are reverted.
Address review comments: zlib max_length defaults, get_ca_certs
binary_form keyword, getpeercert der, WASM ssl certificate keywords,
fstatvfs fd-only, and _sre endpos shown as sys.maxsize.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Generate class constructor text signatures at compile time
A #[pyclass] impl composes the type's internal doc from the Args of
its Initializer or Constructor, so type.__text_signature__ and
inspect.signature work for native classes. __doc__ stays the plain
doc. Struct sequences get (iterable=(), /) and __reduce__ (self, /).
Hand-parsed constructors are converted to FromArgs structs, and
_weakref.proxy is split from ProxyType.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Remove expectedFailure from test_hmac tests that now pass
Assisted-by: Claude:claude-opus-5-5
* Match remaining native signatures and count missing CPython signatures as matches
Give lock.__enter__ and RLock.__enter__ their own no-argument methods,
add FromArgs for start_new_thread and start_joinable_thread keeping
their errors, take RTLD_GLOBAL/RTLD_LOCAL from libc, and align names,
kinds and <unrepresentable> defaults where CPython's signature is
invalid (bytes.hex, reduce, warn, link, setlocale, add_done_callback,
...). getaddrinfo accepts keyword arguments. whats_left treats a
callable CPython has no signature for as matching.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Drop unfulfilled dead_code expectation on os.link dir_fd args
Assisted-by: Claude:claude-opus-5-5
* Distinguish OptionalArg and Option in signature defaults
An `optional` field's signature default now comes from its type:
Option<T> renders None, while OptionalArg<T> (a missing argument that
is not any Python value) renders <unrepresentable>. Fields where a
missing argument behaves exactly like None become Option<T>; fields
whose missing state differs keep OptionalArg. Redundant
py_default = "<unrepresentable>" entries are removed.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Remove expectedFailure from test_rlcompleter.test_attr_matches
Assisted-by: Claude:claude-opus-5-5
* Use Option for optional arguments whose missing and None states match
OptionalOption and OptionalArg<Option<T>> fields that treat a missing
argument like None become Option<T> with #[pyarg(optional)], which renders
=None in the text signature. Plain optional parameters of that kind move
into small FromArgs structs. Uses where missing and None differ (reduce
initial, StringIO newline, TextIOWrapper.reconfigure, set_asyncgen_hooks)
keep OptionalOption.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Render ::NAME defaults by name and restore named constant defaults
- default = ::NAME writes NAME into the text signature and stores
Into::into(NAME) as the Rust value.
- A one-field tuple-struct argument pattern such as Fildes(fd) names the
parameter after its binding, so fcntl and termios keep destructuring.
- zlib, lzma, syslog, mmap and select defaults use their named constants
again instead of literals or <unrepresentable>.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Name FromArgs fields after their Python parameters
Drop #[pyarg(name = ...)] where the Python name is a valid Rust field name
and rename the field instead. The option stays only for keywords (fn, loop,
final, type, self) and the non-snake_case memLevel.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Build literal defaults from Python values and type <unrepresentable>
- default = LIT on a non-primitive field converts the literal through
ToPyObject and TryFromObject when the argument is missing, so object
fields no longer need a vm expression plus a matching py_default.
- A bare default on a primitive integer, bool or float field renders its
zero value.
- DefaultRepr::Unrepresentable replaces the <unrepresentable> text, and
py_default = "<unrepresentable>" is a compile error.
- pyexpat.ParserCreate distinguishes a missing intern from intern=None,
mmap AccessMode renders its value, and sqlite blobopen names rowid.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Resolve native docstrings from the doc database at compile time
- rustpython-doc stores its table as a sorted const slice with const fn
lookups (get, get_attr, class_attr_doc) instead of a phf map, and the
update-doc-db workflow emits that form.
- Class, method, getset, module function and struct sequence docs, and
per-class slot wrapper docs, are chosen by const lookups in the derive
output. The database text wins over Rust doc comments.
- Macros reach the table through rustpython_vm::__exports, so only the vm
depends on rustpython-doc.
- Remove expectedFailure markers from doctest and pydoc tests that pass.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Keep time.gmtime, localtime and ctime without a text signature
Their [seconds] parameter has no text signature, and unittest.mock relies
on inspect.signature failing for them. Take OptionalOption again so the
default stays unrepresentable, and treat None like a missing argument.
Assisted-by: Claude:claude-opus-5-5
Use sequence concatenation for str and clear inherited concat slots when Python subclasses override __add__. Cover direct calls, reflected dispatch, and dynamic overrides against CPython.
Codex assisted with investigation, implementation, tests, validation, and the PR description.
Assisted-by: Codex:gpt-6
Rename the object layout struct `PyInner<T>` to `Py<T>` and make it the
public type, replacing the `#[repr(transparent)]` newtype
`Py<T>(PyInner<T>)`. Header fields stay `pub(super)`. `PyObject` now
wraps `Py<Erased>`.
- Drop the `.0` hop on `Py<T>` accesses.
- Remove the delegating `Traverse for Py<T>` impl; the former
`Traverse for PyInner<T>` impl now serves `Py<T>`.
- Remove `Debug for PyInner<T>`; `debug_obj` writes the
`[PyObject ...]` form itself.
- Use `Py::from_payload_ptr` in `PyWeak::drop_inner` and drop its
now-stale `allow(dead_code)`.
Assisted-by: Claude Code:claude-opus-5-5
* Reuse helpers where appropriate
* Implement a helper for data that doesn't follow the pattern of the
rest of the Unicode files
* Use BTreeMap, which is deduped by key and sorted instead of sorting
a Vec
* Add payload views and Py handle shortcuts
PyTuple::payload_slice, PyBytes::payload_bytes, and PyStr::payload_wtf8
expose the payload data. Common payload methods are forwarded from Py<T>.
Assisted-by: Grok CLI:grok-4.7
* Call PyTuple::as_slice from payload_slice
`&*self` tripped clippy borrow_deref_ref and needless_borrow.
Assisted-by: Grok CLI:grok-4.7
* Use Element.is_empty in the truth-value check
len() != 0 trips clippy::len_zero once is_empty is available.
Assisted-by: Grok CLI:grok-4.7
* Drop redundant payload view aliases
as_bytes, as_wtf8, and as_slice already forward to the payload methods.
Assisted-by: Grok CLI:grok-4.7
* Forward more payload methods from Py handles
Frame iframe, bytearray buffer, set add, exception args and traceback,
and length or lookup helpers are available on Py<T>.
Assisted-by: Grok CLI:grok-4.7
* Read type slots through payload in macros
Generated subclass checks and MRO slot inheritance no longer use Deref
to reach PyType.slots or attributes.
Assisted-by: Grok CLI:grok-4.7
* Index a few tuples through as_slice
These Py<PyTuple> indexes no longer rely on Deref to [T].
Assisted-by: Grok CLI:grok-4.7
* Read type slots through Py<PyType>::slots
obj.class().slots and builtin type slots now call slots() instead of
the payload field.
Assisted-by: Grok CLI:grok-4.7
* Read type version and heap layout through Py<PyType>
Inline-cache checks use tp_version_tag(). Heap-type weakref checks use
heaptype_ext().
Assisted-by: Grok CLI:grok-4.7
* Call slots() on Py type handles
Type creation, type methods, and frame specialization read slots through
Py<PyType> instead of the payload field.
Assisted-by: Grok CLI:grok-4.7
* Use as_bytes, as_slice, and traceback accessors
Bytes lengths go through as_bytes(). Tuple len, emptiness, and
iteration go through as_slice(). Exception handles use traceback().
Assisted-by: Grok CLI:grok-4.7
* Drop shortcuts that only forward payload methods
Tuple length and iteration go through as_slice. Dict and list expose
len. Set add stays on the payload.
Assisted-by: Grok CLI:grok-4.7
* Drop dict and list len shortcuts
len without is_empty fails clippy. Tuple element lookup uses
as_slice().get instead of iter().nth.
Assisted-by: Grok CLI:grok-4.7
Allow the four required arguments to be passed by position or keyword, matching CPython and fixing pytest Node.warn calls. Add regression coverage for mixed calls, warning instances, and invalid argument binding.
Codex implemented the fix and regression tests, ran validation, and drafted this commit and the PR description. The contributor manually verified the warning reproducer.
Assisted-by: Codex:GPT-6
* ucd: Parse Unihan data for 3.2.0; unmask test
Python's 3.2.0 path uses the Unihan data for supplementary numeric
values. The Unihan data isn't bundled with the "full" data which means
we have to parse it ourselves. The full data is large, so I compressed
it with zlib.
AI disclosure: I asked an LLM to elaborate on why the full Unicode 3.2.0
data precludes \u5793. It explained that the code point is part of
Unihan. I wrote the parser myself. I linted the zlib snippet with AI.
Assisted-by: GPT-5.6: Luna
* Use spare_capacity_mut + set_len over transmute
crates/vm/build.rs wrote every environment variable of the build
process into OUT_DIR/env_vars.rs, and _sysconfigdata included that file
into build_time_vars. All build-machine environment variables ended up
as string literals in the binary and were readable at runtime through
sysconfig.get_config_vars().
Remove the dump and the include, and drop the now unused itertools
build-dependency. build_time_vars keeps only the explicit entries.
Fixes#8802
Assisted-by: Claude:claude-fable-5-1
* Keep slot values freelist-eligible
Member cells now sit in the object allocation ahead of the extension
header, so a slot read is one pointer instead of a boxed slice. A
value stored in a slot is not marked published. With threading, only
the value that leaves the slot is, so its free can wait out a reader.
Without threading a slot read is a single incref.
Assisted-by: Grok:4.7
* Address slot members by byte offset
A slot member's offset is the byte distance from the object to its
inline pointer cell. Loads and stores, including the specialized
instructions, use that address. A relative C offset is resolved by
adding the object header and the type's basic size.
Assisted-by: Grok:4.7
* Keep base slot offsets stable on subclasses
Cell 0 sits directly in front of the extension header. Further slots
extend away from the object, so a member offset computed for a base
class still addresses the same cell on a subclass.
Assisted-by: Grok:4.7
* Pack slot cells flush against the extension header
On 32-bit targets an odd slot count is not a multiple of ObjExt's
alignment. The padding now sits in front of the cells, so cell 0
stays immediately before the header. The non-threading slot load is
also the function's return value.
Assisted-by: Grok:4.7
* Place weakref lists in front of slot cells
A subclass that adds a weakref list no longer inserts it between
ObjExt and the object, so inherited member offsets stay valid.
Assisted-by: Grok:4.7
* address review: negative defaults, to_bytes signed, get_self, fstat
- python_default_repr renders negated int/float literals and looks
through parentheses and groups.
- int.to_bytes signed is ArgIntoBool again, so any object is accepted
by truthiness; its signature default stays False via py_default.
- PyNativeFunction::get_self falls back to the module object.
- os.fstat has its own FromArgs with a single fd parameter instead of
reusing stat's arguments.
Assisted-by: Claude:claude-opus-5-5
* address review: literal default rendering and fstat supports sets
- Literal defaults are rendered by python_literal_repr, which recurses
through negation, parentheses and groups, so `-(1)` renders `-1`.
- Float literals without a dot or exponent, such as `1f64`, render
with a trailing `.0`.
- os.fstat is removed from supports_fd, supports_dir_fd and
supports_follow_symlinks.
Assisted-by: Claude:claude-opus-5-5
* Store instance slots as atomic object pointers
PyMemberDef now carries type, offset, and flags. Slot values are
atomic PyObject pointers with try-incref reads, so a concurrent
store cannot free the memory under a reader. Objects placed in a
slot stay published until QSBR reclaims them.
Assisted-by: Grok:4.7
* Collapse nested slot load condition
Assisted-by: Grok:4.7
* Reject over-long digit strings before parsing
A pure ASCII digit string longer than the int digit limit now fails
from a wide digit check. The denial-of-service test no longer spends
the failure path scanning and parsing the whole literal.
Assisted-by: Grok:4.7
* Collapse nested digit-limit condition
Assisted-by: Grok:4.7
* Restore MemberKind for member type codes
The VM stores the member type as an enum whose discriminants match
the C type codes. The C API still reads the integer and converts it
at the boundary. Flags stay integers.
Assisted-by: Grok:4.7
* Mark MemberKind::from_i32 as must_use
Assisted-by: Grok:4.7
* Compose native text signatures at compile time from FromArgs
FromArgs gains a PARAMS const describing its Python parameters.
The derive builds the whole internal doc, `name(sig)\n--\n\ndoc`,
in a const context from those params, and `__text_signature__` /
`__doc__` parse it from the doc. The macro-time string path and the
runtime sig_parts composition are removed.
Signatures follow clinic conventions: `$module` for module functions,
Rust literal defaults rendered as Python reprs, other defaults as
`<unrepresentable>`, and a new `#[pyarg(py_default = "...")]` for the
few fields whose Rust default differs from the Python one. The
`#[pyarg]` keys are documented on the FromArgs derive.
Module functions keep their module object and return it from
`__self__`. PyNativeFunction now traverses its references.
os.stat, os.mkdir, os.listdir, os.open, dict.fromkeys, round, sorted,
int.to_bytes and bytes.hex take arguments matching their signatures.
Assisted-by: Grok:grok-4.7
Assisted-by: Claude:claude-opus-5-5
* Update builtin_signature snippet and fix mkdir on Windows
round and sum now report their FromArgs parameters, so the snippet
asserts their signatures instead of their absence. sum's start is a
PyObjectRef defaulting to int 0 with py_default "0". mkdir reads
dir_fd only on platforms that use it.
Assisted-by: Claude:claude-opus-5-5
* Remove expectedFailure from two signature tests
test_cmp_to_signature and test_autospec_on_bound_builtin_function
now pass. The TestCmpToKeyC override existed only for the marker.
Assisted-by: Claude:claude-opus-5-5
add_note, __reduce__, and __setstate__ take &Py<BaseException>.
dict.__ror__ takes &Py<PyDict>. Callers drop the clone made for add_note.
Assisted-by: Grok CLI:grok-4.7
* Pass owned iterables through; host enumerate on Py<T>
dict update, OrderedDict, defaultdict, itertools, and pickle pass an
owned object to PyIter::try_from_object.
#[pyclass(...)] impl Py<T> now emits PyClassImpl for T. enumerate's
methods live on that impl. A bare impl Py<T> stays an extension.
Assisted-by: Grok CLI:grok-4.7
* Host Self-free pyclass impls on Py<T>
filter, Ellipsis, NotImplemented, bool, Placeholder, and NoDefault
do not use Self as the payload type, so their pyclass impl can be Py<T>.
Assisted-by: Grok CLI:grok-4.7
GET_ITER, GET_YIELD_FROM_ITER, mapping keys(), and builtin iter()
already hold a PyObjectRef. get_iter() cloned it before try_from_object.
Assisted-by: Grok CLI:grok-4.7
* Borrow tp_descr_get/tp_init arguments and by-value public helpers
- DescrGetFunc and GetDescriptor::descr_get take
(&PyObject, Option<&PyObject>, Option<&PyObject>); impls call
.to_owned() where they store or return an argument.
- InitFunc takes &PyObject and Initializer::init takes &Py<Self>.
- VirtualMachine::get_attribute_opt, print_exception,
call_get_descriptor(_specific) and other helpers flagged by
clippy::needless_pass_by_value take &PyObject / &Py<T>; callers in
crates/, examples/ and tests/ drop the clones made for the call.
Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Grok CLI:grok-4.7
* Pass the borrowed zelf to internal_close in WindowsConsoleIO init
Assisted-by: Claude Code:claude-opus-5-5
* Fix borrowed signatures missed by the rebase
OrderedDict.init, classmethod_descriptor.descr_get, and the warnings
get_attribute_opt callers still used owned handles.
Assisted-by: Grok CLI:grok-4.7
* Implement _collections.OrderedDict
Extracted from #6799: native OrderedDict as a dict subclass,
iterator mutation tracking, and mapping-view set operations.
* Add OrderedDict, warnings source lines, and socket sendmsg
Implement _collections.OrderedDict as a dict subclass with a
linked list of insertion-order nodes.
Pack struct F/D as native float/double complex pairs, and report
unexpected __replace__ field names as a list repr.
Register encodings.latin_1 at startup. Reject NUL in input()
prompts. Copy non-exact type dicts in mapping order.
Fetch warning source lines through _bless_my_loader and unicode
splitlines. Keep the last warnings.filters list after deletion.
Snapshot sendmsg sequences before converting buffers, parse
ancillary ints with __index__, and dispatch AF_CAN by protocol.
Store sys.trace and sys.profile on each ThreadSlot so
_settraceallthreads/_setprofileallthreads are visible to
sys.gettrace/getprofile on other threads.
Use <frozen importlib._bootstrap> as frozen importlib
co_filename. Keep HEAPTYPE socket objects GC-tracked.
Assisted-by: Grok:grok-4.6
* Load marshal via readinto and dump sets in dumps order
marshal.load fills a scratch buffer with readinto on a
contiguous writable memoryview, and rejects a returned length
other than the requested size.
set and frozenset dumps write elements in sorted(v,
key=marshal.dumps) order.
Assisted-by: Grok:grok-4.6
* Add classmethod_descriptor and ABC IS_ABSTRACT flags
Give METH_CLASS methods a distinct classmethod_descriptor type.
Store Py_TPFLAGS_IS_ABSTRACT from __abstractmethods__ and look
up that attribute only in the type's own dict.
Expose defaultdict.default_factory as a member descriptor.
Inherit bool's unused number slots from int. Mark set and
frozenset __contains__ as METH_COEXIST.
Parse classmethod, staticmethod, object.__new__, and heap type
text signatures. Register interactive source in linecache.
Assisted-by: Grok:grok-4.6
* address review: odict reuse, descr call, new wrapper
Reuse odict node slots and walk the live list in next_after.
Require both | operands to be dicts. Drop the odict iterator
on GC clear. Bind classmethod descriptors through descr_get
and update IS_ABSTRACT under the type lock. Give odict keys
and items sq_contains. Walk tp_new_wrapper past new_wrapper
so HEAPTYPE socket subclasses can call socket.__new__.
Split FreeBSD Bluetooth protocol constants from Linux values.
Assisted-by: Grok:grok-4.6
* Represent chflags bits with bitflagset
Assisted-by: Grok:grok-4.6
* Represent access and mode bits with bitflagset
Assisted-by: Grok:grok-4.6
* Continue unfinished REPL suites as incomplete input
Assisted-by: Grok:grok-4.6
* Add pickle and os.stat inspect signatures
Give _pickle.Pickler the clinic constructor text signature and
accept fix_imports and buffer_callback as positional-or-keyword.
Allow #[pyfunction(text_signature)] and use it so os.stat reports
(path, *, dir_fd=None, follow_symlinks=True). Uncomment import
_pickle and drop the pickle inspect expectedFailures.
Assisted-by: Grok:grok-4.6
* Add Unpickler, dump/load, and lstat signatures
Give _pickle Unpickler, dump, dumps, load, and loads clinic
text signatures, and os.lstat (path, *, dir_fd=None).
Assisted-by: Grok:grok-4.6
* Complete OrderedDict with insertion-order links
Move the native OrderedDict into ordered_dict.rs and track order
with a linked list of nodes, keeping mapping-view set operations.
Assisted-by: Grok:grok-4.6
* address CI: stat casts, HCI_DATA_DIR, passing xfails
Compare stat permission bits as Mode so Linux clippy does not
flag a u32-to-u32 cast. Export HCI_DATA_DIR with the Bluetooth
socket option values. Drop expectedFailure from tests that now
pass.
Assisted-by: Grok:grok-4.6
* Fix wasm teardown deadlock and bluetooth constants
Assisted-by: Grok:grok-4.6
* Clear process-wide eval breaker bits in STW test
Assisted-by: Grok:grok-4.6
* Drop exclusive-held panic that tripped wasm clippy
Assisted-by: Grok:grok-4.6
* Fix OrderedDict contains and export Windows BDADDR
Assisted-by: Grok:grok-4.6
* Add cmp_to_key and hash-once LRU cache
_functools.cmp_to_key now returns KeyWrapper. LRU lookups store the
key hash on _lru_list_elem so __hash__ is not called again on hits
or eviction.
Assisted-by: Grok:grok-4.6
* Allow subclass keywords on list, tuple, and float
list.__init__ rejects keywords only when tp_new is still list's.
tuple/float.__new__ reject them only when tp_init is still the base's.
Assisted-by: Grok:grok-4.6
* Fix CI lint, test re-export, and structseq members
Drop the duplicate clear_eval_breaker_for_test re-export.
Format structseq imports. Expose struct sequence fields as
readonly members so assignment raises "readonly attribute".
Assisted-by: Grok:grok-4.6
* Allow subclass keywords on itertools and filter
Positional-only constructors ignore extra keywords when a subclass
has replaced tp_init, matching list/tuple/float.
Assisted-by: Grok:grok-4.6
* Remove text_signature derive parameter
Drop the string override on #[pyfunction] and #[pyclass]. Signatures
are generated from the Rust parameter list only. Restore inspect
expectedFailures that depended on the override.
Assisted-by: Grok:grok-4.6
* Rename set contains_method to __contains__
Keep a borrowed contains helper for slots and C-API. The coexist
pymethod is named __contains__.
Assisted-by: Grok:grok-4.6
* Use MonitoringEvent instead of EVENT_* aliases
Drop the unused MonitoringEvents bitflags and the EVENT_PY_START
style mask constants. Call sites use the enum and .mask().
Assisted-by: Grok:grok-4.6
* Store warnings filters as PyListRef
The list already serializes its elements. Drop the extra mutex around
the cached list pointer.
Assisted-by: Grok:grok-4.6
* Re-export clear_eval_breaker_for_test after rebase
Keep the STW test helper visible without bringing back stop_bit_set.
Assisted-by: Grok:grok-4.6
* Fix groupby reentrancy and islice arguments
Give _grouper its own tgtkey and take currvalue after the key
comparison. Accept __index__ on islice bounds, reject step 0, and
drain the source up to stop.
Assisted-by: Grok:grok-4.6
* SETREF warnings filters, onceregistry, and defaultaction
Keep the last list, dict, or str after the module attributes are
deleted by swapping the interpreter cache pointer. Do not store a
None fallback for _warnings_context.
Assisted-by: Grok:grok-4.6
* Iterate OrderedDict inner dict via next_entry
&PyDict is not an iterator after the borrowed-ref signature change.
Assisted-by: Grok:grok-4.6
* Remove expectedFailure from testBluetoothConstants
Linux now exports BTPROTO_HCI and the rest of the set.
Assisted-by: Grok:grok-4.6
* Borrow get_filter arguments as PyObject
Match the warn_explicit call site that passes borrowed
handles after the internal-helper borrow change.
Assisted-by: Grok:grok-4.6
* Fix bool slots and SemHandle safety docs
Leave unused bool number slots unset so inspect reports
__add__ on int. Document SemHandle::from_raw safety.
Assisted-by: Grok:grok-4.6
* Keep sendmsg errors and drop duplicate linecache
Replace sendmsg iterable TypeError only. Register REPL
source in linecache once per compiled input.
Assisted-by: Grok:grok-4.6
* Nest REPL parse or-patterns for clippy
Assisted-by: Grok:grok-4.6
* Mark pydoc os.stat signature test expectedFailure
Assisted-by: Grok:grok-4.6
---------
Co-authored-by: Lee Dogeon <dev.moreal@gmail.com>
Internal functions that took PyObjectRef or PyRef<T> by value but only
borrowed it (reported by clippy::needless_pass_by_value) now take
&PyObject or &Py<T>. Callers pass a borrow instead of an owned handle;
61 caller-side .clone()/.to_owned() calls made only to pass the argument
are removed. Where a converted body already cloned the parameter, that
.clone() became .to_owned().
Functions with #[py*] attributes, trait methods, and functions whose
signature is fixed by a function pointer, getset or slot wrapper type are
unchanged.
PyFloat_FromString in crates/capi passes the borrowed object to
parse_float_from_string without taking an owned reference.
Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Grok CLI:grok-4.7
* host_env: rebuild a unix SemHandle from its stored integer
Windows already has SemHandle::from_raw. Unix callers that keep the
semaphore integer and close it themselves need the same constructor.
Drop still closes, so that caller forgets the rebuilt handle.
Assisted-by: Grok
* Update crates/host_env/src/multiprocessing.rs
* host_env: publish remaining Windows socket constants
Windows SOMAXCONN follows winsock2.h (0x7fffffff), not the Winsock 1.1
value windows-sys still exports. The socket module takes the signed
INADDR group constants and the other Winsock names host_env already
defines.
Assisted-by: Grok
* test_socket: expect Bluetooth constants to pass on Windows
The Windows names this test reads are published now, so the blanket
expectedFailure is an unexpected success there. Other platforms still
miss BTPROTO_HCI and the rest of the non-Windows set.
Assisted-by: Grok
Jiseok CHOI