* Use bitflagset for type flags
PyTypeFlags is defined with bitflagset! and the atomic tp_flags word with
atomic_bitflagset!, replacing the hand-written PyAtomicTypeFlags.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Use bitflagset 0.0.4 from its git main branch
bitflagset 0.0.4 is not on crates.io yet. Patch it to the git main
branch in the workspace and in the example projects that depend on
rustpython-vm, since each of them is a separate workspace.
Assisted-by: Claude:claude-opus-5-5
* Depend on crates.io bitflagset 0.0.4
bitflagset 0.0.4 is published. Drop the git patches from the workspace
and the separate example-project workspaces.
Assisted-by: Grok:4.7
* Keep empty patch tables in example workspaces
The example workspaces already have an empty [patch.crates-io] table.
Dropping the bitflagset git line leaves that table in place.
Assisted-by: Grok:4.7
* Call bitflagset methods directly for type flags
Drop the plain-set has_feature wrapper and the intersects,
is_created_with_flags, DEFAULT, and heap_type_flags aliases.
Single-bit updates use insert, and flag combinations use from_slice.
Assisted-by: Grok:4.7
* OR paired type flags in one update
HAS_WEAKREF with MANAGED_WEAKREF, and HAS_DICT with MANAGED_DICT
(and INLINE_VALUES when present), are applied with one |=.
Assisted-by: Grok:4.7
* Store heap type flag sets as constants
HEAPTYPE | BASETYPE, and the dict and immutable variants built from
it, are const values instead of from_slice calls at each use.
Assisted-by: Grok:4.7
* Test SEQUENCE and MAPPING in one load
PyTypeFlags::COLLECTION is the two-bit mask. Callers check it with
one is_disjoint instead of two contains loads.
Assisted-by: Grok:4.7
* Fix clippy 1.99 lints in the vm
Pass closure arguments by value where clippy reports a needless
borrow, drop the unfulfilled redundant_else expectation on sum, and
return the HRESULT after todo!().
Assisted-by: Grok:4.7
* Add a native _zoneinfo module
ZoneInfo reads TZif tuples from zoneinfo._common.load_data. The stdlib
module implements utcoffset, dst, tzname, fromutc, and the zone cache.
whats_left compares that class. Cache tests no longer expect failure.
Assisted-by: Grok:grok-4.7
* Fix ZoneInfo layout and keep its cache on the module
utcoffset, dst, and tzname take the object. Zone data is boxed so the
payload lines up with tzinfo on 32-bit. The strong, weak, and timedelta
caches live on each interpreter's _zoneinfo module.
Assisted-by: Grok:grok-4.7
Clippy doesn't run on everything which is why I keep accidentally
uncovering new lints to fix. I updated CI to run clippy on all features,
except tk, as well as all binaries, examples, and tests. That should
cover everything.
Tk currently doesn't build (at least for me), so that feature is still
disabled in CI.
While here, fix some of the lints that are now raised.
Some of the FFI wrappers take in a byte slice that's presumed to be NUL
capped. It's more ergonomic and safer to take in a &CStr which is
impossible to get wrong because it's validated at the margins.
Additionally, there were a few areas where a NUL was embedded directly
into the source (i.e. "COMMIT(NUL)" which displayed as "COMMIT^@"). This
had the interesting effect of breaking grep on the source code.
* zlib: clear unconsumed_tail when a bounded decompress reaches stream end
When a max_length-bounded Decompress.decompress() call reached the end
of the stream, the bytes after the stream went to unused_data, but
unconsumed_tail kept the previous call's tail, including input the call
had already consumed. A later flush() then moved those stale bytes into
unused_data.
Empty unconsumed_tail at end of stream, as CPython's
save_unconsumed_input() does since gh-158169. CPython releases before
that change report the bytes after the stream in unconsumed_tail, so
the snippet accepts either value when it runs under CPython.
Fixes#8906
Assisted-by: Claude Code:claude-opus-5-5
* zlib: bind the unconsumed input with an if expression
Choose between the empty slice and the rest of the input in one
expression instead of reassigning a mutable binding.
Assisted-by: Claude Code:claude-opus-5-5
* zlib: accept both valid tails in the snippet on every implementation
The snippet no longer branches on sys.implementation. RustPython and
CPython after gh-158169 give an empty tail; CPython before it gives the
bytes after the stream. The stale tail of gh-8906 fails either way.
Assisted-by: Claude Code:claude-opus-5-5
* Bound AST nesting so deep source raises instead of crashing
Deeply nested source exhausted the native stack and killed the process
with SIGSEGV, which no Python-level `except` can catch.
The parser grows its own stack when it runs low on one, so it returns
trees deeper than the passes after it can walk. Those passes recurse
without a limit of their own, and the symbol table, which does apply
one, runs last. `compile(..., PyCF_ONLY_AST)` returns before reaching
it at all.
Apply that same limit to the tree right after it is parsed, and count
bracket nesting before parsing rather than after, as CPython's
tokenizer does. Checking afterwards built the tree first, and one
nested that deep exhausts the stack when it is dropped.
Fixes#7655
Assisted-by: Claude Code:Opus 5
* Return Result from too_deeply_nested_error
Lets the four call sites use `?` instead of matching on `Some` and
returning the error by hand, which drops 11 lines. `Result` is already
`#[must_use]`, so the attribute on the function goes away with it.
Assisted-by: Claude Code:Opus 5
* Return Result from pre_parse_source_error too
Same shape as the previous commit, for the other check this PR adds, so
the two are consistent with each other.
Assisted-by: Claude Code:Opus 5
* Drop exceeds_max_nesting for the check beside it
The escape-warning pass needs the bracket check but had no `SourceFile` to
build a `CompileError` from, so it got a `bool` wrapper of its own. Building
the `SourceFile` there lets it call `pre_parse_source_error` instead, which
is shaped like the other checks in that file, and costs nothing measurable.
Assisted-by: Claude Code:Opus 5
* Report errors from #[pyclass] trait items
The trait branch of impl_pyclass_impl built its own ImplContext, so the
errors collected while extracting trait items were never emitted: an
invalid #[pymethod] or #[pygetset] in a #[pyclass] trait was left out of
the class without a compile error. The trait branch now uses the shared
context whose errors are emitted.
Assisted-by: Claude Code:claude-opus-5-5
* Pass the object to #[pymethod] and #[pygetset] receivers outside vm
Outside crates/vm, where impl Py<T> cannot be written, #[pymethod] and
#[pygetset] receivers that took &self become zelf: &Py<Self> inside the
existing impl T blocks. Where Rust code calls one of these methods on a
payload value, the logic stays in a payload method and the Python method
forwards to it.
Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Reject payload receivers on #[pymethod] and #[pygetset]
A #[pymethod] or #[pygetset] item with a self receiver is now a compile
error when self is the payload: in an impl for T rather than Py<T> or
PyRef<T>, and in a #[pyclass] trait. The error points to taking
zelf: &Py<Self> or zelf: PyRef<Self>, or defining the item in impl Py<T>.
Assisted-by: Claude Code:claude-opus-5-5
* Pass the object to the layout test getter
The subclass layout test's extra getter is a #[pygetset] on the
payload. Take zelf: &Py<Self> so the receiver is the object.
Assisted-by: Grok:4.7
Mask sys.executable, sys.version_info, the install scheme, and set order in signatures. Compare decimal and zoneinfo with the pure-Python fallback. Print the referent type's module in weakref repr.
Assisted-by: Grok:grok-4.7
In crates/vm, #[pyclass] impl blocks whose Python methods or getset
descriptors took &self are moved to impl Py<T>, so those items receive the
object instead of the payload. Non-Python helpers stay in impl T. Where Rust
code calls one of these methods on a payload value, the logic stays in a
payload method and the Python method forwards to it.
#[pymethod] and #[pygetset] items in the vm #[pyclass] traits (the dict view
traits and the _io buffered mixins) take zelf: &Py<Self> instead of &self.
Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Set builtin __doc__, __module__, and datetime signatures
Expose empty doc-db member entries as ''. Heap __dict__ and __weakref__
getsets carry their descriptor docs. Member repr uses the qualified
type name.
__module__ is collections for the deque iterators, _csv for Dialect,
ctypes for ArgumentError and CField, _lzma for LZMAError, operator for
the getter types, pickle for PickleBuffer, _ssl for MemoryBIO,
SSLSession, _SSLContext and _SSLSocket, weakref for the proxy types,
and posix (nt on Windows) for times_result and uname_result.
unicodedata's copied UCD functions set __module__ to unicodedata.
_testinternalcapi function docs and weakproxy.__str__ use the stored
text. datetime replace defaults render as unchanged, __replace__ is
($self, /, **changes) and binds through replace, now defaults tz to
None, and fromisoformat and tzinfo methods name the parameter object.
Assisted-by: Grok:grok-4.7
* Move doctext helpers above the test module
db_slice and docs sat after mod tests.
Remove expectedFailure from test_html_doc, test_text_doc,
test_help_output_redirect, test_method_aliases, and test_long_signatures.
Assisted-by: Grok:grok-4.7
- Problem: without the threading feature, sending a non-contiguous or
non-byte-format memoryview through _interpchannels loses its data or
metadata on the receiving end
- Cause: the no-threading path copied only the contiguous bytes and
discarded the buffer's descriptor, reconstructing everything as flat bytes
- Fix: copy the whole backing memory plus its BufferDescriptor, and
reconstruct with that descriptor instead of a byte-only default
Assisted-by: Claude Code:claude-sonnet-5
Signed-off-by: Jiwoo Ahn <ikwydls1314@gmail.com>
* fix(gc): traverse annotate name and qualname fields in PyFunction
* style: format extra_tests/snippets/stdlib_gc.py
Assisted-by: Antigravity:Gemini-3.8-Flash
* Record native docstrings the doc DB generator skipped
- Stop ignoring `__dir__`; skip `__base__`, `__bases__` and `__module__` instead.
- Recurse into native classes whose metaclass is not `type`, and also key a
class stored under another attribute name by its own `__name__`.
- Collect native objects exposed by pure-Python modules, such as the codec
error handlers, under their own `__module__` and `__name__`.
- Key types found through `object.__subclasses__()` by their own module as
well as `builtins`.
- Record getset/member descriptors named like class metadata
(`coroutine.__name__`), and native types a module exposes only through an
instance (`sys.flags`).
- Treat a class as native only when it owns its C descriptors, so an enum
holding a copied `str.__format__` is skipped.
- Keep docstrings unstripped, and import every module in sorted order before
walking them.
- Apply the resulting key delta, measured on macOS with CPython 3.14.7, to
data.inc.rs.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Stop Rust comments on Python items from becoming __doc__
Use `//` instead of `///` on Python-exposed items so their comments are not
published as `__doc__`.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Match native names, signatures and doc lookup
- Rename the weakref type to `weakref.ReferenceType` and define
`__class_getitem__` on `_ctypes.Array` instead of its metaclass.
- Give each name of a multi-name `#[pymodule(sub)]` function
(`posix.remove`/`unlink`) its database doc.
- Fill methods from `with(...)` traits (dict view `__reversed__`,
`isdisjoint`) with the implementing class's database doc.
- Resolve database docs for `#[pymodule]` classes at macro time, and for the
codec error handlers and `_lzma.LZMAError` in const context.
- Add `Context::new_exception_type_with_doc`.
- Match `posix_spawn` parameter names and defaults, make `dict_keys` and
`dict_items` unhashable, add `_CData.__ctypes_from_outparam__`, and omit
the Windows-only lines of `sys.__doc__` on other platforms.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Give struct sequence fields their database docs
`new_readonly_tuple_member` takes the field doc, looked up in the class's
attribute doc table through the new `class_attr_item_doc`.
Assisted-by: Claude:claude-opus-5-5
* Look up pymodule(sub) docs by an explicit module name
posix and nt share one doc entry. A sub module sets name to the
Python module it is merged into, and doc lookup uses that name.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Record pending class doc bodies on PyMethodDef
The pymethod derive sets doc_body_pending when a method has no Rust
doc body and expansion did not resolve a database span.
concat_with_attr_docs copies that span from the class table and
clears the flag. Other constructors leave the flag false.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Name argument fields after their Python parameters
PosixSpawnArgs.args is renamed to argv, and nt's follow_symlinks field
drops a name override equal to the field name.
Assisted-by: Claude:claude-opus-5-5
type_slot_names reads __slotnames__ from the type's own dict before
calling copyreg._slotnames, and rejects a cached value that is not a
list or None. object_getstate_default no longer replaces errors from
type_slot_names with TypeError("cannot pickle object"), so a
RecursionError raised there propagates.
Assisted-by: Claude:claude-opus-5-5
* Address review findings on member descriptors
Member definitions carry an internal `PY_ATOMIC` flag taken from
`MemberLayout::ATOMIC`, and loads use an atomic access only for atomic
storage; extension members never set it. `XMLParser` publishes its new
state, `entity` and `target` only after every handler lookup succeeds.
`subs_parameters` reads type names only for the error. The unraisable
hook prints `<unknown>` when `__module__` fails or is not a str. The
builtin function `__reduce__` decides on the bound object alone.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Fix member layout assertions on 32-bit targets
The tp_flags assertion compared AtomicU64 with u64 alignment, which differs
on i686. The kevent assertion required usize, isize, c_ulong and c_long to be
64-bit. Both now check only what the member cells need.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Use bitflags for member definition flags
PY_READONLY, PY_AUDIT_READ, PY_RELATIVE_OFFSET and PY_ATOMIC become the
repr(transparent) PyMemberFlags. C extension flags are read with
from_bits_retain.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
Use `//` instead of `///` on Python-exposed items so their comments are not
published as `__doc__`.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* fix(contextvars): isolate context stack per interpreter and thread on VirtualMachine
* style: format and sort imports in extra_tests/snippets/stdlib_contextvars.py
Assisted-by: Antigravity:Gemini-3.8-Flash
* Read type.__module__ from the dict only for heap types
A static type's `__module__` is the text before the last `.` of its name,
or `builtins`. A heap type reads its dict and raises `AttributeError`
when the entry is missing; `type` and `object` reprs ignore that error.
`type()` no longer stores `__module__ = None` when the caller's globals
have no `__name__`.
`#[pyexception(module = ...)]` now reaches the type name, and static
types that relied on a dict `__module__` (ast nodes, pyexpat, ssl,
_hashlib and _interpchannels errors, ExceptionGroup) carry their module.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Expose more builtin attributes as member descriptors
`code.co_flags`, `co_nlocals`, `co_filename`, `super.__self__`,
`__self_class__`, `__thisclass__`, `traceback.tb_frame`, `tb_lasti`,
`type.__itemsize__` and `builtin_function_or_method.__module__` are now
`#[pymember]` fields instead of getsets. `super` keeps its three
references in atomic cells instead of a lock, `tb_lasti` is an `i32`,
`co_nlocals` is stored at construction, and the builtin function
`__module__` is a writable object cell.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Expose stdlib attributes as member descriptors
Decompressor `eof`/`needs_input`/`unused_data` (bz2, lzma, zlib, plus
lzma `check`), `ContextVar.name`, `functools.partial` `func`/`args`/
`keywords`, socket `family`/`type`/`proto`, `select.kevent` fields,
`SemLock` `handle`/`kind`/`maxvalue`/`name`, `Pickler.bin`/`fast`,
`xmlparser.intern`, ElementTree `XMLParser.entity`/`target`,
`UCD.unidata_version`, `_ExceptHookArgs` fields, multibytecodec
`stream`, and `_io` `Buffered*.raw` and `TextIOWrapper`
`buffer`/`encoding`/`line_buffering`/`write_through` are now
`#[pymember]` fields instead of getsets.
Values that changed under a lock move to atomic cells on the payload.
New member kinds `Short`, `Long`, `String`, `UShort`, `ULong`,
`LongLong` and `ULongLong` access exactly the width of their C type.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Match bare super and static type itemsize
`super.__new__(super)` leaves `__thisclass__`, `__self__` and
`__self_class__` empty, and a super with no type takes the
zero-argument path when bound.
Static types carry their `tp_itemsize` (`type`, `memoryview`, `code`,
`frame`, generators, `_sre` objects, `_ctypes` thunks), inherit it from
their direct base when they leave it at 0, and a variable-size base
rejects any nonempty `__slots__` and a weakref slot.
`_thread._ExceptHookArgs` is a struct sequence.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
* Expose SimpleNamespace.__dict__, __basicsize__ and __flags__ as members
`SimpleNamespace.__dict__` addresses the instance dict cell the same way
`module.__dict__` does. `slots.basicsize` now holds the full
`tp_basicsize` (header plus payload), so `__basicsize__` reads it
directly. Type flags are one atomic word: `__abstractmethods__` and
`_abc` set their bits in it in place, the separate `abc_tpflags` word is
gone, and `__flags__` reads the word as an unsigned long member.
Assisted-by: Grok:4.7
Assisted-by: Claude:claude-opus-5-5
Jeong YunWon