mirror of
https://github.com/RustPython/RustPython.git
synced 2026-10-29 03:04:38 +08:00
dependabot/pre_commit/https-/github.com/astral-sh/ruff-pre-commit-0.17.1
122
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Bas Schoenmaeckers
|
5dcef1aa6d | Add subclass type flags (#9327) | ||
|
youknowdotandyouknowdot
|
afe5d9030a |
Separate constructor and instance vectorcall dispatch (#9320)
Keep constructor fast paths separate from per-instance vectorcall support. Track automatic eligibility without discarding raw support required by PyVectorcall_Call, and recheck eligibility in retained callable wrappers. Implementation and source review were AI-assisted. The model version identifier is unavailable; automated checks are not human verification. Assisted-by: OpenAI Codex:model version unavailable Co-authored-by: youknowdot <336763669+youknowdot@users.noreply.github.com> |
||
|
youknowdotandCodex
|
ec299f3844 |
Avoid attaching C API worker templates under the interpreter lock (#9318)
Keep worker VM cloning separate from attachment and add a native structural regression test. Assisted-by: OpenAI Codex Co-authored-by: Codex <codex@openai.com> |
||
|
youknowdot
|
1ba1107115 |
Align ctypes call ownership and C error state with Python 3.15 (#9218)
* Align ctypes call ownership and C error state with Python 3.15 Keep native raised errors separate from handled exceptions and preserve them across teardown callbacks. Import the related canonical ctypes call and type contracts. Assisted-by: OpenAI Codex: model version unavailable * Enumerate ctypes shared libraries natively on Linux and Android Copy loader-owned names before Python filesystem decoding and update the related canonical ctypes utility and discovery tests from CPython 3.15. Preserve existing platform paths and dllist test markers. Assisted-by: OpenAI Codex: model version unavailable * Enable canonical dllist system enumeration coverage on Linux Regenerate the pinned test through update_lib patches with reviewed marker data. Preserve other platform conditions and the unavailable extension-fixture guard. Assisted-by: OpenAI Codex: model version unavailable * Match Python float and long-double C argument representations Use the existing Python float formatter for float and double C argument representations. Keep long-double arguments on the canonical generic-address representation path. Preserve the unchanged canonical parameter repr assertions. Assisted-by: OpenAI Codex: model version unavailable |
||
|
Bas Schoenmaeckers
|
2ae701e825 |
Add a test for PyType_FromSpec (#9258)
|
||
|
Bas Schoenmaeckers
|
fb9c099511 |
Flatten Py_slot_subslots and Py_tp_slots in PySlot iterator (#9246)
* Flatten `Py_slot_subslots` and `Py_tp_slots` in `PySlot` iterator * Review * Add support for metaclass slots |
||
|
Bas Schoenmaeckers
|
5a051f9442 |
Implement basic PyUnicodeWriter (#9216)
* Implement basic `PyUnicodeWriter` * Add tests * Review * Allow surrogate codepoints in PyUnicodeWriter_WriteChar * use try_reserve |
||
|
Bas Schoenmaeckers
|
c08375baeb |
Implement PyType_FromSpec & PyType_FromMetaclass (#9213)
* Implement `PyType_FromSpec` & `PyType_FromMetaclass` * Review |
||
|
youknowdotandJeong YunWon
|
3f55cf4af0 |
Implement basic REPL execution and fix piped stdin handling (#9163)
* Preserve original main VM through C API execution
Run and finalize the original main VM while keeping the registered attachment template available without holding its mutex across Python callbacks.
Assisted-by: OpenAI Codex:model version unavailable
* Make unraisable reporting robust at hook boundaries
Keep the existing message API and traceback formatting while handling missing, disabled, audited and failing hooks without panicking.
Assisted-by: OpenAI Codex:model version unavailable
* Separate interactive warning collection from compilation
Retain ordinary warning behavior and typed module APIs while exposing record-local parser warnings and post-warning code generation. Add a source-only lazy future-import guard.
Assisted-by: OpenAI Codex:model version unavailable
* Evaluate native stdin with shared VM record semantics
Use one evaluator for basic REPL and root transports, preserve effective inspect snapshots and future flags, and carry owned host exit status through original-main finalization.
Assisted-by: OpenAI Codex:model version unavailable
* Refresh canonical REPL tests and retire fixed failures
Run update_lib migrate from pinned CPython 3.15.0, preserving three remaining platform/runtime markers and retiring the two traceback/startup failures verified on the native evaluator.
Assisted-by: OpenAI Codex:model version unavailable
* Keep the root compiler dependency for development targets
The native evaluator now compiles through the VM. Only root benchmarks and the parse_folder example use rustpython-compiler directly, so declare it as a dev dependency while preserving VM compiler features.
Assisted-by: OpenAI Codex:model version unavailable
* Validate explicit conditional failure marker retirement
Require exact selector and original decorator evidence for each conditional expected-failure marker retired by migrate. Preserve undeclared guards and reject missing, duplicate, modified or unconsumed declarations.
Assisted-by: OpenAI Codex:model version unavailable
* Refresh canonical stdin and exception audit tests
Migrate the pinned CPython 3.15.0 modules and their canonical helper fixtures through update_lib. Retire exactly four failure markers proven obsolete by hosted CI, preserving all other existing guards.
Assisted-by: OpenAI Codex:model version unavailable
* Synchronize runpy warning attribution and document remaining capabilities
Migrate whole canonical runpy.py to pass module names to compile. Use update_lib to annotate only reproduced missing-capability cases: absent _testcapi import-event tests, unsupported startup/import-time features, and the deferred frozen-loader module-attribution caller.
Assisted-by: OpenAI Codex:model version unavailable
* Propagate interrupted native stdin reads to signal dispatch
Preserve the shared input buffer and partial bytes while returning EINTR to the VM before retrying. Exercise the transport deterministically and verify a returning SIGINT handler while pipe stdin stays open.
Assisted-by: OpenAI Codex:model version unavailable
* Reject statements after a compound statement in single mode
single_mode_multiple_statements_error now reports "invalid syntax" at the
statement that follows a compound first statement, and interactive_input
runs it right after parsing as compile() does.
The lazy `from __future__` import check moved from lazy_future_import_error
into unsupported_grammar_error, so it applies to every compile mode.
Assisted-by: Claude Code:claude-opus-5-5
* Remove passing expected failure from test_runpy
test_run_module_filter_syntax_warnings_by_module passes.
Assisted-by: Claude Code:claude-opus-5-5
* Restructure stdin execution after Modules/main.c and pythonrun.c
- src/lib.rs follows pymain_run_python: pymain_run_stdin, pymain_repl,
pymain_run_startup, pymain_run_interactive_hook and pymain_run_pyrepl
return exit codes instead of raising SystemExit.
- VirtualMachine gains run_any_stdin (_PyRun_AnyFile),
run_interactive_loop (_PyRun_InteractiveLoop) and run_simple_stdin;
run_shell feeds rustyline lines into run_interactive_loop, and
sys._baserepl calls run_any_stdin.
- print_uncaught_exception (PyErr_Print) replaces
report_uncaught_exception, and handle_system_exit_and_keyboard_interrupt
replaces handle_reported_exception.
- An unhandled KeyboardInterrupt is recorded and the process exits
through SIGINT after finalization.
- Interpreter::run_main replaces run_with_exit_status and InterpreterExit;
rustpython_capi::run_main_interpreter takes an exit-code callback.
- config_set_inspect audits cpython.PyConfig_Set and rebuilds sys.flags
through set_flag_obj (_PySys_SetFlagObj); the inspect state is an
AtomicBool and dict replace_by_identity is removed.
- add_main_module (PyImport_AddModuleRef("__main__")) is used by the
interactive and simple-file paths.
- The interactive parser warning helpers take their state directly.
Assisted-by: Claude Code:claude-opus-5-5
* Exit through Py_Exit from PyErr_Print and run the basic REPL via stdin
- print_uncaught_exception (PyErr_Print) calls exit_process (Py_Exit) for a
SystemExit when inspect mode is off, instead of returning the exception.
run_interactive_loop no longer checks SystemExit itself and flushes after
printing an error; sys._baserepl returns None.
- exit_process finalizes through VirtualMachine::finalize_ex, which holds
the Py_FinalizeEx steps split out of Interpreter::finalize, and exits
through host_env::os::exit_with_cleanup (C exit()).
- The basic REPL on a terminal (PYTHON_BASIC_REPL, or _pyrepl missing)
runs run_any_stdin instead of the rustyline shell.
- run_shell saves the history after every line.
Assisted-by: Claude Code:claude-opus-5-5
* Display exceptions through PyErr_Display and track the main thread running __main__
- display_exception (PyErr_Display) and display_exception_to (_PyErr_Display)
try traceback._print_exception_bltin before the native printer and flush
the file. PyErr_Print's excepthook fallback, print_exception and
sys.excepthook use them. A missing sys.stderr is reported with
dump_object (PyObject_Dump); a None sys.stderr prints nothing.
- Paths gains program_name (argv[0]) from getpath. The "can't open file"
message uses it with strerror(errno) and goes to sys.stderr.
- running_main holds the ident of the thread running __main__.
pymain_run_python sets it around running the main code, Py_Exit clears it,
the fork child reinitializes it, and _interpreters.is_running reads it
instead of treating the main interpreter as always running.
- Interpreter::run_main clears the unhandled KeyboardInterrupt flag before
running and applies exit_sigint after finalizing; finalize no longer
does.
- run_interactive_loop returns a SystemExit raised outside inspect mode
again and no longer flushes after printing an error; run_shell saves the
history once after the loop.
- pymain_set_inspect reports its error through print_uncaught_exception.
Assisted-by: Claude Code:claude-opus-5-5
* Treat a parse error at the end of the source as incomplete input
A parser error that starts at the end of the source is reported as
_IncompleteInputError when incomplete input is allowed, so the
interactive loop and codeop keep reading after `try:` and its block or
after a decorator line.
Assisted-by: Claude Code:claude-opus-5-5
* Place syntax errors on tokens made up at the end of the input
A parse error on the NEWLINE, DEDENT or ENDMARKER the tokenizer makes up
at the end of the input now carries an EofToken: the number of lines
read and the last line. SyntaxError takes its lineno and text from it,
and its offset is that line's length with end_offset -1, or 0 for both
when the error is raised at the token itself (the generic syntax error
and a missing forced `:` or `(`). Exec input that does not end with a
newline is checked with one appended.
Assisted-by: Claude Code:claude-opus-5-5
* Slice the source with get when checking an indented block error at EOF
Assisted-by: Claude Code:claude-opus-5-5
* Rename the host exits after C exit() and fix wasi and Windows builds
- host_env::os::exit now calls the C exit() on every platform; on Windows
std::process::exit ends in ExitProcess, so it flushes the Rust stdout
and calls libc::exit there. The former exit, which calls _exit(), is
now exit_immediately, used by os._exit, faulthandler and
posixsubprocess.
- errno::strerror_string is available on wasi.
- PyGlobalState::reinit_running_main is built only where its fork caller
is.
Assisted-by: Claude Code:claude-opus-5-5
---------
Co-authored-by: Jeong YunWon <jeong@youknowone.org>
|
||
|
Jeong, YunWon
|
2dbe992438 |
Remove ucd_3_2_0 feature from rustpython-capi (#9153)
Drop the capi [features] table added in #9011 along with the root ucd_3_2_0 forwarding to rustpython-capi and the default-features = false on the workspace rustpython-capi dependency. The root ucd_3_2_0 feature still enables rustpython-stdlib/ucd_3_2_0. Assisted-by: Claude Code:claude-opus-5-5 |
||
|
Bas Schoenmaeckers
|
88f5189a93 |
Update pyo3 to 0.30 (#9152)
* Update `pyo3` to `0.30` * Fix cfunction with no module as self |
||
|
Joshua Megnauth
|
a2c11aa51d |
Optional Unicode 3.2.0 but enabled by default (#9011)
Useful for smaller builds, like WebAssembly. The UCD 3.2.0 path is only directly used in a handful of places in Python. Assisted-by: GPT-5.6-Luna |
||
|
|
28393e932d |
Implement PEP 814 frozendict (#8951)
* Restore consolidated frozendict implementation on current main Preserve reverse dictionary iterator fixes and the 3.15 frame imports. Keep library refreshes separate pending the runtime dependency sequencing decision. Assisted-by: OpenAI Codex:model identifier unavailable * Support scoped canonical test updates in update_lib Index conditional test classes and methods without crossing namespaces, reject ambiguous targets, and preserve inherited override indentation. Add hash-pinned whole-test copying that verifies execution context and exact source bytes, with focused Python fixtures. Assisted-by: OpenAI Codex:model identifier unavailable * Align frozendict copy support and tests with CPython 3.15 Import exact-final copy.py with update_lib; defer Mapping ABC registration to the later library stage. Generate only proven marker corrections, retain observed debug recursion-limit expected failures, and copy the complete canonical builtin-doc discovery test using hash-pinned update_lib copy-test. Assisted-by: OpenAI Codex:model identifier unavailable * Preserve decorator rationale during canonical test copying Reject differing decorator source blocks even when their ASTs match. Cover inline and multiline marker comments, guarded async methods, and CLI no-write behavior. Assisted-by: OpenAI Codex:model identifier unavailable * Finish frozendict with borrowed PyAnyDict views Use the landed PyRef target model for a transparent dict/frozendict header view, retaining objects explicitly with PyRef<PyAnyDict>. Keep targets sealed and preserve mutable-only entry points. Borrow frame globals from the object header rather than movable reference wrappers. Unify exact-TypeError key diagnostics with final CPython semantics, including callback failure propagation and no explicit cause. Keep the Python regressions in builtin_dict and test reference layout/ownership in Rust. Preserve existing canonical library updates. Assisted-by: OpenAI Codex:model identifier unavailable * Align frozendict with CPython and use CPython 3.15 tests - Name the target type in unhashable key errors through PyDict::update_arg (dict_update_arg), replacing the separate PyFrozenDict::merge_into copy of the merge logic. - Split the | operator into dict_or (_PyDict_Or) and frozendict_or. - Return the argument from frozendict(frozendict) only when the called type is frozendict. - Call through the vectorcall slot in PyCallable::invoke when there are no keyword arguments, instead of special-casing frozendict. - Add ensure_builtins (_PyEval_EnsureBuiltins) for eval/exec, and insert __builtins__ for source strings regardless of globals type. - Read __warningregistry__ and __name__ from stored globals entries and accept only None or str for the module name in setup_context. - Remove the unused PyFrozenDict::from_object. - Migrate test_dict.py, mapping_tests.py, test_copy.py and test_builtin.py to v3.15.0 with update_lib, and add test_import/data/syntax_warnings.py. - Drop builtin_dict.py frozendict tests covered by those modules, keeping the descriptor and namespace integration tests. Assisted-by: Claude Code:claude-opus-5-5 * Update crates/vm/src/builtins/dict.rs --------- Co-authored-by: Jeong YunWon <jeong@youknowone.org> Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com> |
||
|
youknowdot
|
8b66b7c48f |
Unify PyObjectRef ownership with extensible PyRef views (#9080)
Represent PyObjectRef through PyRef<PyObject> while preserving typed ownership, payload projection, and drop behavior. Use the sealed target selector for borrowed set/frozenset union views and typed tuple elements. Keep tuple storage erased behind invariant typed slice views, with checked conversions and reference-count ownership tests. Preserve exact tuple identity for normalized nonempty full slices. Integrate the ownership contract into the public object module documentation and retain focused Rust ownership/layout tests without modifying bundled Lib sources. Spell the set/frozenset union owner directly as PyRef<PyAnySet> without a second owned alias; retain the sealed internal view registration. Assisted-by: OpenAI Codex:model identifier unavailable |
||
|
youknowdot
|
29f819ec40 |
Implement the stable ABI bytes formatting functions (#9060)
Assisted-by: OpenAI Codex:model identifier unavailable |
||
|
youknowdot
|
d674ea11e3 |
Drop the legacy abi3-only PyCFunction_Call export (#9059)
Follow up on #9055: RustPython targets abi3t, which does not require this legacy binary-compatibility entry point. Assisted-by: OpenAI Codex:model identifier unavailable |
||
|
youknowdot
|
bc5b9abd75 |
Export the stable ABI PyCFunction_Call shim (#9055)
Assisted-by: OpenAI Codex:model identifier unavailable |
||
|
이광효andJeong YunWon
|
a8e4bde885 |
Reject unsupported native keywords before argument binding (#8957)
* Reject unsupported native keywords before argument binding Add explicit keyword dispatch metadata independently of inferred Rust calling flags. Reject nonempty keywords before binding and conversion, while preserving descriptor self checks and keyword-capable calls. Annotate 31 builtins and list.append/list.clear, derive C-API policies from ml_flags, and group dynamic registration metadata in MethodDefSpec. Add regression coverage and remove the id doctest expected-failure marker. Validation: workspace and C-API tests/clippy, VM without doc, keyword regressions, and test_call/test_extcall/test_builtin/test_list passed. The full snippet suite had 475 passes and 7 failures: six CPython-side expectation mismatches and one missing optional SQLite module. Assisted-by: Codex:GPT-6 * Infer native keyword dispatch from argument metadata Replace no_keywords annotations with registration-time inference from FromArgs::PARAMS, including nested parameter metadata. Forward arguments with keyword-capable parameters to the binder, preserving min/max body validation and descriptor self checks. Add regressions for raw arguments, keyword argument structs, and list.sort. Tested: C-API tests, workspace and C-API Clippy, VM without doc, both keyword regression snippets, and test_call/test_extcall/test_builtin/test_list. Workspace tests reported eight _opcode snapshot failures. The full snippet suite reported 477 passes and seven failures: six CPython-side failures and one RustPython failure due to the missing optional SQLite module. Assisted-by: Codex:GPT-6 * Simplify native function registration metadata Reuse SimpleItemMeta for pyfunction attributes and remove FunctionItemMeta. Remove MethodDefSpec and restore the original new_method_def arguments with an additional keyword_dispatch parameter, updating its callers. Consolidate keyword dispatch regressions in syntax_function_args.py, preserving their assertions and placing them before builtin shadowing. Validation: workspace and C-API tests/clippy, VM without doc, the combined snippet, and test_extcall/test_builtin/test_list/test_call passed. The full snippet suite reported 477 passes and seven existing failures: six CPython-side failures and one missing optional SQLite module. Assisted-by: Codex:GPT-6 * Reject native keywords by calling convention flags Remove `KeywordDispatch`, the `keyword_dispatch` field of `PyMethodDef`, `PyMethodDef::with_keyword_dispatch`, `function::keyword_dispatch` and the derive macros' `keyword_dispatch_tokens`. Native functions and method descriptors reject nonempty keywords when `flags` lacks `KEYWORDS`, and report the unqualified name when `flags` contains `VARARGS`. `Context::new_method_def` takes `(name, f, flags, doc)` again. Add a snippet for `int.__new__` called with a keyword argument. Assisted-by: Claude Code:claude-opus-5-5 --------- Co-authored-by: Jeong YunWon <jeong@youknowone.org> |
||
|
Bas Schoenmaeckers
|
e6cbf3628d |
Add buffer support to c-api (#8071)
* Add buffer support to c-api * Review * Refactor PyBuffer * Manual implement `PyBuffer_GetPointer` * Review * Store in 1 Box |
||
|
Bas Schoenmaeckers
|
5b3e2a40e2 |
Add recursion guard to the c-api (#8796)
* Add recursion guard to the c-api * Do not change with_recursion as this has side effects |
||
|
Bas Schoenmaeckers
|
ada43c4f82 |
Add PyABIInfo_Check to the c-api (#8941)
|
||
|
Bas Schoenmaeckers
|
d81b7c2cb1 |
Add basic c-variadic functions to the c-api (#8933)
* Add basic c-variadic functions to the c-api * Review |
||
|
Jeong, YunWon
|
1f83b1593f |
Fix CI failures and bump version to 0.6.1 (#8940)
* Use AtomicPyTypeFlags in PyType_FromSlots PyType_FromSlots still imported the old PyAtomicTypeFlags name. Assisted-by: Grok:4.7 * Pass CPython test args as a bash array Assisted-by: Grok:4.7 * Bump workspace version to 0.6.1 Assisted-by: Grok:4.7 * Restore the CPython test step in ci.yaml Assisted-by: Grok:4.7 |
||
|
Alessio Attilio
|
ce3f671e10 |
feat(capi): implement PyModule_Add* and module helpers from modsupport.h (#8927)
Assisted-by: Gemini:gemini-3.7-flash |
||
|
Bas Schoenmaeckers
|
b3c84c0fc7 |
Add first draft of PyType_FromSlots (#8918)
|
||
|
Jeong, YunWon
|
1d7d704414 |
Use bitflagset for type flags (#8873)
* Use bitflagset for type flags PyTypeFlags is defined with bitflagset! and the atomic tp_flags word with atomic_bitflagset!, replacing the hand-written PyAtomicTypeFlags. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Use bitflagset 0.0.4 from its git main branch bitflagset 0.0.4 is not on crates.io yet. Patch it to the git main branch in the workspace and in the example projects that depend on rustpython-vm, since each of them is a separate workspace. Assisted-by: Claude:claude-opus-5-5 * Depend on crates.io bitflagset 0.0.4 bitflagset 0.0.4 is published. Drop the git patches from the workspace and the separate example-project workspaces. Assisted-by: Grok:4.7 * Keep empty patch tables in example workspaces The example workspaces already have an empty [patch.crates-io] table. Dropping the bitflagset git line leaves that table in place. Assisted-by: Grok:4.7 * Call bitflagset methods directly for type flags Drop the plain-set has_feature wrapper and the intersects, is_created_with_flags, DEFAULT, and heap_type_flags aliases. Single-bit updates use insert, and flag combinations use from_slice. Assisted-by: Grok:4.7 * OR paired type flags in one update HAS_WEAKREF with MANAGED_WEAKREF, and HAS_DICT with MANAGED_DICT (and INLINE_VALUES when present), are applied with one |=. Assisted-by: Grok:4.7 * Store heap type flag sets as constants HEAPTYPE | BASETYPE, and the dict and immutable variants built from it, are const values instead of from_slice calls at each use. Assisted-by: Grok:4.7 * Test SEQUENCE and MAPPING in one load PyTypeFlags::COLLECTION is the two-bit mask. Callers check it with one is_disjoint instead of two contains loads. Assisted-by: Grok:4.7 * Fix clippy 1.99 lints in the vm Pass closure arguments by value where clippy reports a needless borrow, drop the unfulfilled redundant_else expectation on sum, and return the HRESULT after todo!(). Assisted-by: Grok:4.7 |
||
|
Jeong, YunWon
|
8f84fea377 |
Fix 32-bit member layout assertions and address member descriptor review (#8882)
* Address review findings on member descriptors Member definitions carry an internal `PY_ATOMIC` flag taken from `MemberLayout::ATOMIC`, and loads use an atomic access only for atomic storage; extension members never set it. `XMLParser` publishes its new state, `entity` and `target` only after every handler lookup succeeds. `subs_parameters` reads type names only for the error. The unraisable hook prints `<unknown>` when `__module__` fails or is not a str. The builtin function `__reduce__` decides on the bound object alone. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Fix member layout assertions on 32-bit targets The tp_flags assertion compared AtomicU64 with u64 alignment, which differs on i686. The kevent assertion required usize, isize, c_ulong and c_long to be 64-bit. Both now check only what the member cells need. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Use bitflags for member definition flags PY_READONLY, PY_AUDIT_READ, PY_RELATIVE_OFFSET and PY_ATOMIC become the repr(transparent) PyMemberFlags. C extension flags are read with from_bits_retain. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 |
||
|
Jeong, YunWon
|
31963fa767 |
Expose more attributes as member descriptors (#8857)
* Read type.__module__ from the dict only for heap types A static type's `__module__` is the text before the last `.` of its name, or `builtins`. A heap type reads its dict and raises `AttributeError` when the entry is missing; `type` and `object` reprs ignore that error. `type()` no longer stores `__module__ = None` when the caller's globals have no `__name__`. `#[pyexception(module = ...)]` now reaches the type name, and static types that relied on a dict `__module__` (ast nodes, pyexpat, ssl, _hashlib and _interpchannels errors, ExceptionGroup) carry their module. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Expose more builtin attributes as member descriptors `code.co_flags`, `co_nlocals`, `co_filename`, `super.__self__`, `__self_class__`, `__thisclass__`, `traceback.tb_frame`, `tb_lasti`, `type.__itemsize__` and `builtin_function_or_method.__module__` are now `#[pymember]` fields instead of getsets. `super` keeps its three references in atomic cells instead of a lock, `tb_lasti` is an `i32`, `co_nlocals` is stored at construction, and the builtin function `__module__` is a writable object cell. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Expose stdlib attributes as member descriptors Decompressor `eof`/`needs_input`/`unused_data` (bz2, lzma, zlib, plus lzma `check`), `ContextVar.name`, `functools.partial` `func`/`args`/ `keywords`, socket `family`/`type`/`proto`, `select.kevent` fields, `SemLock` `handle`/`kind`/`maxvalue`/`name`, `Pickler.bin`/`fast`, `xmlparser.intern`, ElementTree `XMLParser.entity`/`target`, `UCD.unidata_version`, `_ExceptHookArgs` fields, multibytecodec `stream`, and `_io` `Buffered*.raw` and `TextIOWrapper` `buffer`/`encoding`/`line_buffering`/`write_through` are now `#[pymember]` fields instead of getsets. Values that changed under a lock move to atomic cells on the payload. New member kinds `Short`, `Long`, `String`, `UShort`, `ULong`, `LongLong` and `ULongLong` access exactly the width of their C type. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Match bare super and static type itemsize `super.__new__(super)` leaves `__thisclass__`, `__self__` and `__self_class__` empty, and a super with no type takes the zero-argument path when bound. Static types carry their `tp_itemsize` (`type`, `memoryview`, `code`, `frame`, generators, `_sre` objects, `_ctypes` thunks), inherit it from their direct base when they leave it at 0, and a variable-size base rejects any nonempty `__slots__` and a weakref slot. `_thread._ExceptHookArgs` is a struct sequence. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Expose SimpleNamespace.__dict__, __basicsize__ and __flags__ as members `SimpleNamespace.__dict__` addresses the instance dict cell the same way `module.__dict__` does. `slots.basicsize` now holds the full `tp_basicsize` (header plus payload), so `__basicsize__` reads it directly. Type flags are one atomic word: `__abstractmethods__` and `_abc` set their bits in it in place, the separate `abc_tpflags` word is gone, and `__flags__` reads the word as an unsigned long member. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 |
||
|
Jeong, YunWon
|
be23b2c562 |
Store database docstrings xz-compressed behind an optional doc feature (#8836)
* Skip Python-defined classes when collecting builtin type docs generate.py recorded every object.__subclasses__() entry as builtins.<name> after importing the whole stdlib, so classes written in Python (tkinter, unittest, tarfile, ...) ended up in the doc database. Classes that are mutable heap types without C method descriptors and whose module is not a C extension are now skipped, and their entries are removed from data.inc.rs. Assisted-by: Claude:claude-opus-5-5 Assisted-by: Grok:grok-4.7 * Store database docstrings xz-compressed behind an optional doc feature - rustpython-doc's build script compresses all database docstrings into one xz blob with an offset index; identical texts share one offset. - Macros emit blob references for database docs; vm decompresses the blob on first access. - New `doc` cargo feature (default on) in rustpython, rustpython-vm and rustpython-stdlib. Without it the blob and xz are not linked and database-backed __doc__ is None; __text_signature__ is unaffected. - Native types without a docstring get __doc__ = None in their dict. - Workflows that build with --no-default-features enable `doc` where tests or releases need docstrings. Assisted-by: Claude:claude-opus-5-5 Assisted-by: Grok:grok-4.7 |
||
|
Jeong, YunWon
|
bdc9246b37 |
Address builtin members by field offset (#8816)
* Address builtin members by field offset `#[pymember]` now declares a field of the payload and generates a PyMemberDef whose offset is the payload offset plus the field offset. The getter/setter function form and `MemberAccess::Func` are removed; builtin, slot, and C-API members share one get/set path. The macro checks the field type against the member type. Converted members: property fget/fset/fdel/__doc__, function __globals__/__closure__/__builtins__/__doc__/__module__, staticmethod and classmethod __func__/__wrapped__, complex real/imag (T_DOUBLE), frame f_trace_lines (T_BOOL), defaultdict default_factory, getset and member descriptor __name__/__objclass__, module __dict__. frame.f_trace_opcodes is a getset. Writable member fields are atomic pointer cells. The instance dict pointer is an atomic cell at a fixed offset in front of the object, and dict presence follows the type's HAS_DICT flag. The pickle basicsize check no longer counts dict, weakref, and slot storage, which live outside the payload. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Address member review findings - `#[pymember]` keeps leading `#[cfg]` attributes on the generated registration, and malformed `field`/`offset` values are compile errors. - The `descriptor` module and the items the expansion names are public. - Writable double members are an `AtomicU64` cell, loaded and stored with relaxed atomics. Readonly double members stay plain `f64` reads. - `LOAD_ATTR_SLOT` is not specialized for `Py_AUDIT_READ` members. - `defaultdict.__reduce__` passes no factory argument when `default_factory` is None. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Declare members on payload fields `#[pymember]` goes on a payload struct field, or on the struct with an explicit `offset` when the storage is not a payload field. The struct `#[pyclass]` builds `PyClassDef::MEMBERS`, a static `PyMemberSpec` table, and `extend_class` registers one member descriptor per entry. `path` addresses a subfield, one field may carry several members, and a `#[cfg]` on the field gates its entry. The impl-side const form is removed and now a compile error. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Split nullable and non-null atomic cells The nullable cell API (`new_empty`, `load_owned`, `store`, `compare_exchange_empty`) moves from `PyAtomicRef<PyObject>` to `PyAtomicRef<Option<PyObject>>`, and the typed `PyAtomicRef<Option<T>>` shares it. The non-null `PyAtomicRef<PyObject>` gets `load_owned` and `store` that return the object directly. Slot cells, property fget/fset/fdel/__doc__, and defaultdict default_factory are nullable cells. staticmethod and classmethod callable are non-null cells and are read without `unwrap`. The instance dict cell is `PyAtomicRef<Option<PyDict>>`, so reading it no longer downcasts. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Drop the unused cold trace_lines flag `FrameColdData::trace_lines` had no writer after `f_trace_lines` became a frame object member. A frame without a frame object traces lines by default. Assisted-by: Claude:claude-opus-5-5 * Compute payload offsets from the merged Py header `PyInner` is now `Py`, so the payload offset and the dict member offset are measured from `Py<T>`. Assisted-by: Claude:claude-opus-5-5 * Expose descriptor names and owner types as members method_descriptor, classmethod_descriptor, and wrapper_descriptor expose `__objclass__` and `__name__` as readonly members, and method-wrapper exposes `__self__` as a readonly member. The separate `objclass` field of method and classmethod descriptors always equaled `common.typ` and is removed. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Expose remaining builtin tp_members as members Attributes stored in a payload field become readonly or writable member descriptors: bound method __func__/__self__, code co_argcount/co_posonlyargcount/co_kwonlyargcount/co_stacksize (Py_T_UINT), generic alias, coroutine and async generator members, interpolation and template, range and slice start/stop/step, type __base__, union, and the typing TypeVar, ParamSpec, TypeVarTuple, and TypeAliasType members. `MemberKind` gains `Int` and `Uint` with the `PyMember_GetOne`/`SetOne` range checks. Attributes computed on access or kept behind a lock stay getsets. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Spell a docless member as doc = false `#[pymember(doc = false)]` stores no docstring, matching the `doc = "..."` form, and replaces the separate `no_doc` flag. Assisted-by: Claude:claude-opus-5-5 * Make pymember readonly by default `#[pymember]` declares a readonly member unless `writable` is given. The `readonly` option is removed and every declaration is converted. Assisted-by: Claude:claude-opus-5-5 * Check the receiver of method descriptors Binding or calling a method_descriptor with an object that is not an instance of the owner type raises TypeError before the native function runs. Static and class methods are exempt. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Expose exception attributes as members BaseException.__suppress_context__, StopIteration.value, SystemExit.code, ImportError, NameError, AttributeError, OSError, SyntaxError, BaseExceptionGroup and the UnicodeEncode/Decode/TranslateError attributes are member descriptors backed by payload fields. ImportError, NameError, AttributeError and SyntaxError gain payloads for their fields, and the methods that read those attributes read the fields. Add the Py_ssize_t member kind for UnicodeError start and end. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Avoid same-type casts in unsigned member conversion On targets where c_ulong is u32, casting through c_ulong was a no-op cast that clippy rejects. Assisted-by: Claude:claude-opus-5-5 * Prefer an explicit pymember doc over the doc table doc = "text" is used as the docstring without consulting the stored attribute documentation; only an omitted doc looks it up. Drop the member doc strings that repeated the stored documentation. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Infer pymember kinds from field types MemberLayout gives the member kind of each accepted field type, and the macro reads it from the field or path subfield, so type = "bool" and the other numeric spellings are gone. type only selects object_ex. Writable doubles use a new AtomicF64 cell instead of AtomicU64 bits. MemberLayout and MemberCell replace the per-kind marker traits. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Read nullable atomic cells only through load_owned PyAtomicRef<Option<T>> and PyAtomicRef<Option<PyObject>> lose to_owned, to_owned_ordering and try_to_owned; load_owned, which retries the incref until it applies to the pointer still in the cell, is the one owned read. Callers that kept a deref borrow across VM calls now take a strong reference. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Inherit static type slots from the whole MRO extend_class copied slots only from the direct bases, and tp_init is copied only from a base that defines it, so TabError missed the SyntaxError init that IndentationError inherits. Walk the MRO the same way init_slots does. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Let Rust doc comments win over stored docstrings A /// comment on a Python-exposed item is its docstring; the rustpython-doc entry is used only when there is none. #[pymember] reads the field's /// comment instead of doc = "...". Doc comments that were shadowed by a stored docstring become // comments, so no docstring changes. AGENTS.md describes the rule. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Adapt members to upstream changes _collections._tuplegetter.__doc__ is a writable nullable member on the payload field. The impl-side const form no longer compiles. Assisted-by: Grok:4.7 * Drop docstring text left as comments and inline downcast_cell Comments that only repeated an item's docstring are removed; developer notes stay as // comments. The typed cell methods call downcast_unchecked directly, each with its own safety note. Assisted-by: Grok:4.7 Assisted-by: Claude:claude-opus-5-5 * Read sqlite row factories through load_owned The nullable cell no longer has to_owned; the sqlite module is built only with the sqlite feature, so the default build did not catch it. Assisted-by: Claude:claude-opus-5-5 |
||
|
Jeong, YunWon
|
dc3b93abfb |
Read tuple, bytes, int, float and complex payloads through views on Py<T> (#8819)
* Read PyTuple payloads through as_slice() - Add Py<PyTuple<R>>::as_slice(); it is generic over R, so typed tuples use the same view. - Callers reach slice methods, indexing and slice arguments through as_slice(). - PyTuple's own sequence and mapping slots and marshal's set_marshal_item reach the remaining PyTuple methods through the payload field. - The Py<T>::payload field is pub(crate). Assisted-by: Claude Code:claude-opus-5-5 * Read PyBytes payloads through as_bytes() - Callers read bytes through Py<PyBytes>::as_bytes(). - bytes.rs reaches PyBytesInner and the remaining PyBytes methods through the payload field; fspath and os call payload.contains_nuls(). - DictKey for [u8] falls back to the PyObject key_eq; it previously resolved to its own key_eq through Deref. Assisted-by: Claude Code:claude-opus-5-5 * Copy capi tuple arguments with as_slice().to_vec() Assisted-by: Claude Code:claude-opus-5-5 * Read sqlite3 Row tuples through as_slice() Assisted-by: Claude Code:claude-opus-5-5 * Read platform-specific tuples and bytes through as_slice() and as_bytes() Assisted-by: Claude Code:claude-opus-5-5 * Read PyInt and PyFloat payloads through as_bigint() and to_f64() Add `Py<PyInt>::as_bigint()`, `Py<PyInt>::try_to_primitive()` and `Py<PyFloat>::to_f64()`, which forward to the payload methods of the same name. Reads of the `value` field and primitive conversions go through these views; other PyInt and PyFloat methods are called through the `payload` field inside rustpython-vm and through payload() outside it. Assisted-by: Claude Code:claude-opus-5-5 * Read PyComplex payloads through as_complex() Add Py<PyComplex>::as_complex() and use it at the call sites that read the payload directly. Assisted-by: Claude Code:claude-opus-5-5 * Move PyInt conversion helpers to Py<PyInt> try_to_i64_fast(), to_str_radix_10(), as_u32_mask(), as_u64_mask(), try_to_primitive() and try_to_primitive_raw() only read the integer value, so they are defined on Py<PyInt> and callers no longer go through the payload. PyRange computes its protocol length from the BigInt directly instead of building a PyInt payload. Assisted-by: Claude Code:claude-opus-5-5 * Share one conversion between try_to_primitive() and its OverflowError variant Py<PyInt>::try_to_primitive() and try_to_primitive_in_range(), renamed from try_to_primitive_raw(), both call to_primitive(). They differ only in how a negative value for an unsigned type is reported: ValueError for try_to_primitive(), OverflowError for try_to_primitive_in_range(). The target type is bound with `for<'a> TryFrom<&'a BigInt>`, so the methods no longer tie a lifetime to &self. Assisted-by: Claude Code:claude-opus-5-5 * Move PyInt bitwise operators to Py<PyInt> __xor__, __or__, __and__ and their int_op() helper are defined on Py<PyInt>. PyBool calls them without going through the payload. Assisted-by: Claude Code:claude-opus-5-5 * Move PyBytes operators to Py<PyBytes> __add__, __contains__, __getitem__ and __mod__ are defined on Py<PyBytes>. The subscript, contains and remainder slots call them without going through the payload. Assisted-by: Claude Code:claude-opus-5-5 * Move PyInt general_op() and modpow() to Py<PyInt> The power slot calls them on Py<PyInt> without going through the payload. Assisted-by: Claude Code:claude-opus-5-5 |
||
|
Jeong, YunWon
|
0ebc6570e3 |
Add payload views and Py handle shortcuts (#8812)
* Add payload views and Py handle shortcuts PyTuple::payload_slice, PyBytes::payload_bytes, and PyStr::payload_wtf8 expose the payload data. Common payload methods are forwarded from Py<T>. Assisted-by: Grok CLI:grok-4.7 * Call PyTuple::as_slice from payload_slice `&*self` tripped clippy borrow_deref_ref and needless_borrow. Assisted-by: Grok CLI:grok-4.7 * Use Element.is_empty in the truth-value check len() != 0 trips clippy::len_zero once is_empty is available. Assisted-by: Grok CLI:grok-4.7 * Drop redundant payload view aliases as_bytes, as_wtf8, and as_slice already forward to the payload methods. Assisted-by: Grok CLI:grok-4.7 * Forward more payload methods from Py handles Frame iframe, bytearray buffer, set add, exception args and traceback, and length or lookup helpers are available on Py<T>. Assisted-by: Grok CLI:grok-4.7 * Read type slots through payload in macros Generated subclass checks and MRO slot inheritance no longer use Deref to reach PyType.slots or attributes. Assisted-by: Grok CLI:grok-4.7 * Index a few tuples through as_slice These Py<PyTuple> indexes no longer rely on Deref to [T]. Assisted-by: Grok CLI:grok-4.7 * Read type slots through Py<PyType>::slots obj.class().slots and builtin type slots now call slots() instead of the payload field. Assisted-by: Grok CLI:grok-4.7 * Read type version and heap layout through Py<PyType> Inline-cache checks use tp_version_tag(). Heap-type weakref checks use heaptype_ext(). Assisted-by: Grok CLI:grok-4.7 * Call slots() on Py type handles Type creation, type methods, and frame specialization read slots through Py<PyType> instead of the payload field. Assisted-by: Grok CLI:grok-4.7 * Use as_bytes, as_slice, and traceback accessors Bytes lengths go through as_bytes(). Tuple len, emptiness, and iteration go through as_slice(). Exception handles use traceback(). Assisted-by: Grok CLI:grok-4.7 * Drop shortcuts that only forward payload methods Tuple length and iteration go through as_slice. Dict and list expose len. Set add stays on the payload. Assisted-by: Grok CLI:grok-4.7 * Drop dict and list len shortcuts len without is_empty fails clippy. Tuple element lookup uses as_slice().get instead of iter().nth. Assisted-by: Grok CLI:grok-4.7 |
||
|
Jeong, YunWon
|
d474353be5 |
Address slot members by byte offset (#8811)
* Keep slot values freelist-eligible Member cells now sit in the object allocation ahead of the extension header, so a slot read is one pointer instead of a boxed slice. A value stored in a slot is not marked published. With threading, only the value that leaves the slot is, so its free can wait out a reader. Without threading a slot read is a single incref. Assisted-by: Grok:4.7 * Address slot members by byte offset A slot member's offset is the byte distance from the object to its inline pointer cell. Loads and stores, including the specialized instructions, use that address. A relative C offset is resolved by adding the object header and the type's basic size. Assisted-by: Grok:4.7 * Keep base slot offsets stable on subclasses Cell 0 sits directly in front of the extension header. Further slots extend away from the object, so a member offset computed for a base class still addresses the same cell on a subclass. Assisted-by: Grok:4.7 * Pack slot cells flush against the extension header On 32-bit targets an odd slot count is not a multiple of ObjExt's alignment. The padding now sits in front of the cells, so cell 0 stays immediately before the header. The non-threading slot load is also the function's return value. Assisted-by: Grok:4.7 * Place weakref lists in front of slot cells A subclass that adds a weakref list no longer inserts it between ObjExt and the object, so inherited member offsets stay valid. Assisted-by: Grok:4.7 |
||
|
Bas SchoenmaeckersandJeong, YunWon
|
a1971c1d9d |
Unify PyObject dereferencing in c-api (#8292)
* Add `FfiPtrExt` for uniform pointer dereferences * Panic on NULL ptrs * capi: import type_name from core, not std Assisted-by: Grok:4.6 * Use `assume_owned_or_err` in PyGetSetDef * Update latest code --------- Co-authored-by: Jeong, YunWon <jeong@youknowone.org> |
||
|
Bas Schoenmaeckers
|
3272bee78b |
Add PyNumber_AsSsize_t to the c-api (#8801)
|
||
|
Jeong, YunWon
|
2619bdcca5 | Fix hasattr (#8809) | ||
|
Jeong, YunWon
|
155cea292f |
Store instance slots as atomic object pointers (#8804)
* Store instance slots as atomic object pointers PyMemberDef now carries type, offset, and flags. Slot values are atomic PyObject pointers with try-incref reads, so a concurrent store cannot free the memory under a reader. Objects placed in a slot stay published until QSBR reclaims them. Assisted-by: Grok:4.7 * Collapse nested slot load condition Assisted-by: Grok:4.7 * Reject over-long digit strings before parsing A pure ASCII digit string longer than the int digit limit now fails from a wide digit check. The denial-of-service test no longer spends the failure path scanning and parsing the whole literal. Assisted-by: Grok:4.7 * Collapse nested digit-limit condition Assisted-by: Grok:4.7 * Restore MemberKind for member type codes The VM stores the member type as an enum whose discriminants match the C type codes. The C API still reads the integer and converts it at the boundary. Flags stay integers. Assisted-by: Grok:4.7 * Mark MemberKind::from_i32 as must_use Assisted-by: Grok:4.7 |
||
|
Jeong, YunWon
|
0940849029 |
Borrow tp_descr_get/tp_init arguments and by-value public helpers (#8803)
* Borrow tp_descr_get/tp_init arguments and by-value public helpers - DescrGetFunc and GetDescriptor::descr_get take (&PyObject, Option<&PyObject>, Option<&PyObject>); impls call .to_owned() where they store or return an argument. - InitFunc takes &PyObject and Initializer::init takes &Py<Self>. - VirtualMachine::get_attribute_opt, print_exception, call_get_descriptor(_specific) and other helpers flagged by clippy::needless_pass_by_value take &PyObject / &Py<T>; callers in crates/, examples/ and tests/ drop the clones made for the call. Assisted-by: Claude Code:claude-opus-5-5 Assisted-by: Grok CLI:grok-4.7 * Pass the borrowed zelf to internal_close in WindowsConsoleIO init Assisted-by: Claude Code:claude-opus-5-5 * Fix borrowed signatures missed by the rebase OrderedDict.init, classmethod_descriptor.descr_get, and the warnings get_attribute_opt callers still used owned handles. Assisted-by: Grok CLI:grok-4.7 |
||
|
Jeong, YunWonandLee Dogeon
|
afad92bf1b |
Add OrderedDict, marshal, inspect, and chflags bitflagset (#8780)
* Implement _collections.OrderedDict Extracted from #6799: native OrderedDict as a dict subclass, iterator mutation tracking, and mapping-view set operations. * Add OrderedDict, warnings source lines, and socket sendmsg Implement _collections.OrderedDict as a dict subclass with a linked list of insertion-order nodes. Pack struct F/D as native float/double complex pairs, and report unexpected __replace__ field names as a list repr. Register encodings.latin_1 at startup. Reject NUL in input() prompts. Copy non-exact type dicts in mapping order. Fetch warning source lines through _bless_my_loader and unicode splitlines. Keep the last warnings.filters list after deletion. Snapshot sendmsg sequences before converting buffers, parse ancillary ints with __index__, and dispatch AF_CAN by protocol. Store sys.trace and sys.profile on each ThreadSlot so _settraceallthreads/_setprofileallthreads are visible to sys.gettrace/getprofile on other threads. Use <frozen importlib._bootstrap> as frozen importlib co_filename. Keep HEAPTYPE socket objects GC-tracked. Assisted-by: Grok:grok-4.6 * Load marshal via readinto and dump sets in dumps order marshal.load fills a scratch buffer with readinto on a contiguous writable memoryview, and rejects a returned length other than the requested size. set and frozenset dumps write elements in sorted(v, key=marshal.dumps) order. Assisted-by: Grok:grok-4.6 * Add classmethod_descriptor and ABC IS_ABSTRACT flags Give METH_CLASS methods a distinct classmethod_descriptor type. Store Py_TPFLAGS_IS_ABSTRACT from __abstractmethods__ and look up that attribute only in the type's own dict. Expose defaultdict.default_factory as a member descriptor. Inherit bool's unused number slots from int. Mark set and frozenset __contains__ as METH_COEXIST. Parse classmethod, staticmethod, object.__new__, and heap type text signatures. Register interactive source in linecache. Assisted-by: Grok:grok-4.6 * address review: odict reuse, descr call, new wrapper Reuse odict node slots and walk the live list in next_after. Require both | operands to be dicts. Drop the odict iterator on GC clear. Bind classmethod descriptors through descr_get and update IS_ABSTRACT under the type lock. Give odict keys and items sq_contains. Walk tp_new_wrapper past new_wrapper so HEAPTYPE socket subclasses can call socket.__new__. Split FreeBSD Bluetooth protocol constants from Linux values. Assisted-by: Grok:grok-4.6 * Represent chflags bits with bitflagset Assisted-by: Grok:grok-4.6 * Represent access and mode bits with bitflagset Assisted-by: Grok:grok-4.6 * Continue unfinished REPL suites as incomplete input Assisted-by: Grok:grok-4.6 * Add pickle and os.stat inspect signatures Give _pickle.Pickler the clinic constructor text signature and accept fix_imports and buffer_callback as positional-or-keyword. Allow #[pyfunction(text_signature)] and use it so os.stat reports (path, *, dir_fd=None, follow_symlinks=True). Uncomment import _pickle and drop the pickle inspect expectedFailures. Assisted-by: Grok:grok-4.6 * Add Unpickler, dump/load, and lstat signatures Give _pickle Unpickler, dump, dumps, load, and loads clinic text signatures, and os.lstat (path, *, dir_fd=None). Assisted-by: Grok:grok-4.6 * Complete OrderedDict with insertion-order links Move the native OrderedDict into ordered_dict.rs and track order with a linked list of nodes, keeping mapping-view set operations. Assisted-by: Grok:grok-4.6 * address CI: stat casts, HCI_DATA_DIR, passing xfails Compare stat permission bits as Mode so Linux clippy does not flag a u32-to-u32 cast. Export HCI_DATA_DIR with the Bluetooth socket option values. Drop expectedFailure from tests that now pass. Assisted-by: Grok:grok-4.6 * Fix wasm teardown deadlock and bluetooth constants Assisted-by: Grok:grok-4.6 * Clear process-wide eval breaker bits in STW test Assisted-by: Grok:grok-4.6 * Drop exclusive-held panic that tripped wasm clippy Assisted-by: Grok:grok-4.6 * Fix OrderedDict contains and export Windows BDADDR Assisted-by: Grok:grok-4.6 * Add cmp_to_key and hash-once LRU cache _functools.cmp_to_key now returns KeyWrapper. LRU lookups store the key hash on _lru_list_elem so __hash__ is not called again on hits or eviction. Assisted-by: Grok:grok-4.6 * Allow subclass keywords on list, tuple, and float list.__init__ rejects keywords only when tp_new is still list's. tuple/float.__new__ reject them only when tp_init is still the base's. Assisted-by: Grok:grok-4.6 * Fix CI lint, test re-export, and structseq members Drop the duplicate clear_eval_breaker_for_test re-export. Format structseq imports. Expose struct sequence fields as readonly members so assignment raises "readonly attribute". Assisted-by: Grok:grok-4.6 * Allow subclass keywords on itertools and filter Positional-only constructors ignore extra keywords when a subclass has replaced tp_init, matching list/tuple/float. Assisted-by: Grok:grok-4.6 * Remove text_signature derive parameter Drop the string override on #[pyfunction] and #[pyclass]. Signatures are generated from the Rust parameter list only. Restore inspect expectedFailures that depended on the override. Assisted-by: Grok:grok-4.6 * Rename set contains_method to __contains__ Keep a borrowed contains helper for slots and C-API. The coexist pymethod is named __contains__. Assisted-by: Grok:grok-4.6 * Use MonitoringEvent instead of EVENT_* aliases Drop the unused MonitoringEvents bitflags and the EVENT_PY_START style mask constants. Call sites use the enum and .mask(). Assisted-by: Grok:grok-4.6 * Store warnings filters as PyListRef The list already serializes its elements. Drop the extra mutex around the cached list pointer. Assisted-by: Grok:grok-4.6 * Re-export clear_eval_breaker_for_test after rebase Keep the STW test helper visible without bringing back stop_bit_set. Assisted-by: Grok:grok-4.6 * Fix groupby reentrancy and islice arguments Give _grouper its own tgtkey and take currvalue after the key comparison. Accept __index__ on islice bounds, reject step 0, and drain the source up to stop. Assisted-by: Grok:grok-4.6 * SETREF warnings filters, onceregistry, and defaultaction Keep the last list, dict, or str after the module attributes are deleted by swapping the interpreter cache pointer. Do not store a None fallback for _warnings_context. Assisted-by: Grok:grok-4.6 * Iterate OrderedDict inner dict via next_entry &PyDict is not an iterator after the borrowed-ref signature change. Assisted-by: Grok:grok-4.6 * Remove expectedFailure from testBluetoothConstants Linux now exports BTPROTO_HCI and the rest of the set. Assisted-by: Grok:grok-4.6 * Borrow get_filter arguments as PyObject Match the warn_explicit call site that passes borrowed handles after the internal-helper borrow change. Assisted-by: Grok:grok-4.6 * Fix bool slots and SemHandle safety docs Leave unused bool number slots unset so inspect reports __add__ on int. Document SemHandle::from_raw safety. Assisted-by: Grok:grok-4.6 * Keep sendmsg errors and drop duplicate linecache Replace sendmsg iterable TypeError only. Register REPL source in linecache once per compiled input. Assisted-by: Grok:grok-4.6 * Nest REPL parse or-patterns for clippy Assisted-by: Grok:grok-4.6 * Mark pydoc os.stat signature test expectedFailure Assisted-by: Grok:grok-4.6 --------- Co-authored-by: Lee Dogeon <dev.moreal@gmail.com> |
||
|
Jeong, YunWon
|
30ff0329a2 |
Borrow handles in internal helpers that only borrowed them (#8798)
Internal functions that took PyObjectRef or PyRef<T> by value but only borrowed it (reported by clippy::needless_pass_by_value) now take &PyObject or &Py<T>. Callers pass a borrow instead of an owned handle; 61 caller-side .clone()/.to_owned() calls made only to pass the argument are removed. Where a converted body already cloned the parameter, that .clone() became .to_owned(). Functions with #[py*] attributes, trait methods, and functions whose signature is fixed by a function pointer, getset or slot wrapper type are unchanged. PyFloat_FromString in crates/capi passes the borrowed object to parse_float_from_string without taking an owned reference. Assisted-by: Claude Code:claude-opus-5-5 Assisted-by: Grok CLI:grok-4.7 |
||
|
Bas Schoenmaeckers
|
15c9bdd48f |
Impl PyBytes_FromString PyBytes_FromObject PyBytes_AsStringAndSize (#8795)
* Impl `PyBytes_FromString` `PyBytes_FromObject` `PyBytes_AsStringAndSize` * Do not copy in PyBytes_FromObject when already a bytes obj |
||
|
Bas Schoenmaeckers
|
43a595ed65 |
Add SeqIter and CallIter support to c-api (#8794)
|
||
|
Jeong, YunWon
|
803520930a |
capi: add PyFrame_GetLocals, GetGlobals, GetBuiltins (#8761)
Wire the C-API getters to frame.f_locals, f_globals, and f_builtins, and cover write-through vs snapshot locals in a capi test. Assisted-by: Grok:4.6 |
||
|
Jeong, YunWon
|
7bc3ef6c0a |
ExceptionGroup fields, OSError use-init, and live f_lineno (#8738)
* Add ExceptionGroup fields and OSError use-init BaseExceptionGroup stores msg, excs, and excs_str; args keep the original sequence. OSError defers parse when a subclass overrides only __init__. characters_written can be deleted. Exception args/traceback/cause/context reject invalid setattr/delattr. UnicodeError.object is a deletable field. except* remainder projection copies traceback/context/cause. SyntaxError.__init__ clears omitted location attributes. Assisted-by: Grok:grok-4.6 * Fix OSError payload init and UnicodeError construction new_payload_exception now runs slot_new so BlockingIOError parses characters_written. new_exception falls back to invoke_exception for extra-payload types. Mixed tab/space UnexpectedIndentation is TabError. AttributeError pickle omits obj. Assisted-by: Grok:grok-4.6 * address review: classify TabError from error line Assisted-by: Grok:grok-4.6 * Add new_simple_exception; fill SyntaxError and f_lineno new_exception constructs extra-payload types; BaseException-sized types use new_simple_exception. Non-UTF-8 SyntaxError sets location and text. Codegen errors load text via ProgramText. Keep Missing parentheses capitalization. f-string adjacent atoms report a missing comma. f_lineno may be None; lasti is not restored on unwind. Assisted-by: Grok:grok-4.6 * address review: skip f-string continuation keywords Assisted-by: Grok:grok-4.6 * Fix clippy map_unwrap_or in Non-UTF-8 offset Assisted-by: Grok:grok-4.6 * Rename encode error ctor and exception setters new_unicode_encode_error_real is new_unicode_encode_error. set___traceback__/__cause__/__context__ are pygetset setters; Rust uses set_traceback/set_cause/set_context. Assisted-by: Grok:grok-4.6 * Use set_context in native pickle Assisted-by: Grok:grok-4.6 * Collapse nested if in asyncio traceback restore Assisted-by: Grok:grok-4.6 * Use current instruction for f_lineno addr2line lasti is stored as the next instruction index. addr2line of that byte offset is past RETURN_VALUE, so f_lineno was None and stdlib_threading_current_frames hung on leftover workers. Assisted-by: Grok:grok-4.6 * Look up format args, dir class, and frame builtins Consume printf args before rejecting unknown conversions. object.__dir__ uses getattr of __class__/__dict__. _PyEval_GetBuiltin looks up iter/__import__ on the frame. Assisted-by: Grok:grok-4.6 * Fix builtin lookups after rebase and unxfail passing tests Assisted-by: Grok:grok-4.6 * Restore RERAISE lasti and SyntaxError locations RERAISE with oparg restores lasti from the exception table before unwind, so leftover f_lineno matches the original raise. Handler lookup still uses the raising instruction. Indentation errors outrank print/exec missing-parentheses. BOM-with-non-utf8, bytes literals, and global+nonlocal now report the same offsets as the parser/symtable. object.__dir__ keeps string keys; type.__type_params__ skips the getset wrapper stored on builtin types. Assisted-by: Grok:grok-4.6 * Unxfail syntax/import tests and bind jump locals IMPORT_NAME passes fromlist None instead of an empty tuple. Jumping f_lineno binds None into unbound localsplus slots. str lstrip/rstrip copy subclasses to exact str when unchanged. Assisted-by: Grok:grok-4.6 * Fix import_from callers and rename frame lineno sqlite3.iterdump now passes fromlist as a PyObject. FrameObject::lineno is the i32 helper; f_lineno is the getset. Assisted-by: Grok:grok-4.6 * Pass sqlite fromlist without Into conversion from_list.into() could not infer Into<PyObjectRef> at the import_from call. The typed tuple already converts. Assisted-by: Grok:grok-4.6 |
||
|
Bas Schoenmaeckers
|
da694300b8 |
Add Py_IncRef, Py_DecRef, Py_XNewRef (#8726)
|
||
|
Bas Schoenmaeckers
|
dacf514210 |
Add long export/writer to c-api (#8161)
* Add long export/writer c-api * Add `PyLong_AsNativeBytes` * Fix PyLongWriter_Create errors * Set PyLongWriter_Create digits to null on err |
||
|
Roshan Ramani
|
1345a82443 |
Report bytes out-of-range elements as CPython does (#8679)
* Report `bytes` out-of-range elements as CPython does `collect_bytes` is shared by the `bytes` and `bytearray` entry points and hardcoded one element error, so `bytes([256])` said "byte must be in range(0, 256)" where CPython says "bytes". Every `bytearray` path does say "byte", so only the one caller was wrong. The element error is now a parameter, alongside the type error this function already varies per caller. Verified against CPython 3.14 for all seven paths. Assisted-by: Claude Code:claude-opus-5 * Keep the bytearray wording in PyByteArray_FromObject It called `bytes_from_object`, so the previous commit made it report "bytes must be in range(0, 256)" where CPython's `PyByteArray_FromObject` says "byte". It is the C-level bytearray constructor, so it now uses `bytearray_from_object`, which carries that wording and the constructor's unsized iteration. Assisted-by: Claude Code:claude-opus-5 |
||
|
Jeong, YunWon
|
2c8d3ce50b |
Implement PEP 667 frame locals semantics (#8602)
* Implement PEP 667 frame locals semantics Assisted-by: Codex:GPT-5 * Fix C API test CString import Assisted-by: Codex:GPT-5 * Address PEP 667 review findings Assisted-by: Codex:GPT-5 |
||
|
Jeong, YunWon
|
819b4b86fd |
Fix malformed Unicode error constructors (#8497)
* Fix malformed Unicode error constructors Replace message-only Unicode decode and encode errors with fully initialized exceptions, preserve source objects and failure ranges, and remove the obsolete constructors. Assisted-by: Codex:gpt-5 * Fix Windows Unicode CI regressions Assisted-by: Codex:gpt-5 |
||
|
Jeong, YunWon
|
11de045ef8 |
Per-interpreter runtime state and a process-wide GC stop-the-world (#8517)
* vm: add per-interpreter runtime state and interpreter registry - vm/runtime.rs: process-global interpreter registry (monotonic ids, weak entries), InterpreterWhence/InterpreterInfo, process main id recording via main_interpreter_id(), a threading-gated owner map (store_owned_interpreter/ take_owned_interpreter/is_owned_interpreter/owned_interpreter_count), and the SUPPORTS_ISOLATED_INTERPRETERS constant. - PyGlobalState gains interpreter_id/whence/is_main and is_main_interpreter(); PyConfig/Settings derive Clone so a subinterpreter can clone parent config. - Interpreter: id()/whence()/is_main()/is_process_main(), create_subinterpreter() and create_owned_subinterpreter(); unregister on Drop. - thread.rs: per-interpreter thread slots (INTERP_THREAD_SLOTS), slot swap when switching interpreters on one OS thread, cleanup keyed by interpreter id. - Install signal handlers and init the main-thread ident only on the main interpreter; _thread._is_main_interpreter reflects the current interpreter. - sys.implementation.supports_isolated_interpreters reads the constant. - Guard the registry static for non-threading builds where rc::Weak is !Send. Assisted-by: Claude Code:claude-opus-4-8 * gc: stop every interpreter while collecting The generation lists are process-global, so a collection reads and frees objects owned by every interpreter. CollectStopTheWorld stopped only the collecting interpreter, leaving other interpreters' threads free to mutate the same object graph during the reference-subtraction, reachability and snapshot phases. Stop all live interpreters instead, in runtime id order, and restart them in reverse. The global `collecting` mutex serializes collectors process-wide, so no second collector takes these exclusions in another order; fork acquires a single interpreter's exclusion, so the orders cannot cycle. - runtime: add live_interpreter_states(), ordered by interpreter id. - StopTheWorldState methods take &PyGlobalState instead of &VirtualMachine, so an interpreter's world can be stopped without a VM for it; update the call sites in frame, _thread, posix, faulthandler and capi. - Document in gc_state() that the collector is process-wide: gc.disable(), thresholds, gc.garbage and gc.get_objects() observe process-wide state, and a per-interpreter collector additionally needs untrack_object (called from default_dealloc with no VM in scope) routed to the owning interpreter. Tests: stop_the_world_parks_threads_of_another_interpreter asserts a thread entered in one interpreter parks another interpreter's threads (it fails without this change), plus a collect-while-another-interpreter-churns test. Assisted-by: Claude Code:claude-opus-4-8 * vm: scope the interpreter registry per thread without threading The registry held `rc::Weak` in a process-global `OnceLock` and covered the resulting `!Send`/`!Sync` with an `unsafe impl` justifying it as "non-threading builds are single-threaded". That is not this codebase's model: `static_cell!` is thread-local without the `threading` feature precisely so each OS thread can own its own `Context::genesis()` and `GcState`, so two threads could reach the same `Rc` counts through the registry. Use `static_cell!` for the registry as well, matching `gc_state()`, and drop the `unsafe impl`. Ids are consequently unique per registry rather than per process in non-threading builds, which is documented on `alloc_interpreter_id`. Move the recorded main interpreter id into the registry so it follows the same scoping instead of living in a separate global `OnceLock`. Also make `CollectStopTheWorld::new` accumulate into a live guard: it built a bare `Vec` and only moved it into the restarting `Drop` type after stopping every interpreter, so an unwind partway through the loop left the already stopped interpreters parked forever with their exclusion held. Assisted-by: Claude Code:claude-opus-4-8 * vm: keep the interpreter registry usable across bootstrap, drop and fork Three gaps where the registry did not describe the interpreters that actually exist, each of which hides an interpreter from the collector's stop-the-world. Register before `initialize()`. Registration ran as the last step of `initialize_vm`, so the whole bootstrap — which executes Python bytecode and allocates GC-tracked objects — was invisible to `live_interpreter_states()`. It still cannot run any earlier than this: the init hooks take `PyRc::get_mut` on the state, which fails as soon as the registry holds a weak reference to it. Stop unregistering in `Interpreter::drop`. The handle does not decide the interpreter's lifetime — every `ThreadedVirtualMachine` from `new_thread()` holds its own `PyRc<PyGlobalState>` — so an interpreter with running workers disappeared from the registry while its threads kept mutating the object graph. The entries are weak, so lifetime is already correct without the removal; dead entries are now reaped when registering instead. Interpreters are consequently released rather than unregistered at a fixed point, so the two tests asserting disappearance now wait for it: a collection in progress legitimately holds a reference to every live interpreter. Repair other interpreters after fork. `py_os_after_fork_child` only fixed the forking interpreter, leaving every other one with slots for threads that did not survive (still ATTACHED if they were running bytecode) plus locks and stop-the-world flags held by them. Since a collection stops all interpreters, the child's first collection would wait for threads that no longer exist. Reset their locks, stop-the-world state and thread tables, drop this thread's cached slots for them, and reinit the registry's own locks first, since enumerating interpreters now takes them. Tests: test_gc, test_threading and test_fork1 pass, as do the vm tests in both the threading and default configurations. Assisted-by: Claude Code:claude-opus-4-8 * vm: attach and detach thread slots when switching interpreters `enter_vm` decided whether to attach from `was_outermost` (an empty VM_STACK), which held while a thread could only ever be in one interpreter. With a slot per (thread, interpreter) pair, entering interpreter B from a thread already inside interpreter A's section switched CURRENT_THREAD_SLOT to B's slot but attached nothing: the thread then ran B's bytecode with B's slot DETACHED while A's slot stayed ATTACHED. A collector stopping B force-parks the DETACHED slot and concludes B is stopped, and then walks the object graph this thread is still mutating. Pair the attach/detach with the slot switch instead (≈ `_PyThreadState_Swap`): `begin_interpreter_section` detaches the enclosing interpreter's slot, makes the target slot current and attaches it, and `end_interpreter_section` undoes that and re-attaches the enclosing interpreter. Both live in `set_current_vm`, which every path making a VM current already goes through, so `enter_vm` and `VmBootstrapGuard` no longer track outermost-ness themselves. `nested_enter_of_subinterpreter_is_stoppable` covers this: it runs a subinterpreter nested inside the parent's section and asserts the sub's threads park when the sub's world is stopped. It fails with the previous attach-at-outermost-only behavior. Assisted-by: Claude Code:claude-opus-4-8 * vm: list only the current interpreter's subclasses Interpreters share the context, so `class Foo(int)` in one of them pushes onto the same `int.subclasses` every other one reads, and `int.__subclasses__()` returned types no other interpreter can reach. Record the creating interpreter on `HeapTypeExt` and filter `__subclasses__` by it, the way `lookup_tp_subclasses` reads `tp_subclasses` out of per-interpreter state for static builtin types. Types built before any interpreter exists — the ones the shared context creates, including the exception hierarchy — carry no id and stay visible to every interpreter, which is what `_PyStaticType_InitBuiltin` produces by registering the builtin subclass links once per interpreter. The other walks over `subclasses` (version-tag invalidation, abc flag propagation, mro updates, slot propagation) are left as they are: each starts from a type being mutated, so from a heap type, whose subclasses all live in the interpreter that created it. `subinterpreter_subclasses_are_scoped_to_their_interpreter` covers this and fails without the filter. Assisted-by: Claude Code:claude-opus-5 * gc: give each interpreter its own collector state The generation lists stay process-wide, because an object is untracked from `default_dealloc`, where no interpreter is in scope to route to. What changes is that a collection no longer acts on every interpreter's objects, and the gc module no longer reports one interpreter's state to another. `track_object` stamps the running interpreter into a new `gc_owner` word on the object header, and a collection takes as candidates only the objects carrying its own tag plus the ones carrying none. `gc_owner` fits in the padding the header's alignment already forces, so objects do not grow; an assertion on the header size keeps it that way. Objects allocated with no interpreter running — everything the shared context builds — carry no owner and stay candidates for every interpreter, which is where they were before. Objects that outlive the interpreter that tracked them are adopted the same way by the next full collection, rather than being left to a collector that will never come. `enabled`, the thresholds, the debug flags, the statistics, `gc.garbage` and `gc.callbacks` move onto `PyGlobalState` — the last two off the shared `Context` — so `gc.disable()`, `gc.set_threshold()`, `gc.get_stats()`, `gc.get_objects()` and `gc.garbage` describe the interpreter that asks. The occupancy counts behind `gc.get_count()` and `gc.get_freeze_count()` stay process-wide: they measure how full the shared lists are. Their decrements are saturating now, since a collection zeroes the generations it emptied while another interpreter's objects are still sitting in them. Stop-the-world still stops every interpreter. Unowned objects are candidates and any interpreter can incref one, so the refcounts a collection reads are only stable while all of them are parked. This also fixes a deadlock it exposed: `CollectStopTheWorld` dropped its references to the stopped interpreters while the collection still held the generation read locks, so releasing the last reference to one — which frees its objects, and so untracks them — waited for a write lock behind that read lock. The references are now held until the guard itself drops. `collections_only_reach_the_collecting_interpreter` and `get_objects_only_reports_the_calling_interpreter` cover this and both fail without the owner check. Assisted-by: Claude Code:claude-opus-5 * _queue, _thread: detach before taking locks held across waits `_queue.Semaphore` holds its mutex across the `allow_threads` condvar wait, and `join_internal` holds a thread handle's completion mutex the same way. Stop-the-world can stop a thread while it holds either one. The remaining acquisitions ran attached, so a thread blocking on such a mutex had no safepoint left to reach: the stop never completed, and the holder was never resumed to release it. Route those acquisitions through helpers that detach first. The fork-child reinit paths keep their direct locks. Assisted-by: Claude * _io, _winapi: detach on the remaining stopped-holdable lock takes `TextIOWrapper.__repr__` took `data` directly while every other method takes it through `lock_opt`, which detaches. `Overlapped` holds `inner` across the `allow_threads` in `GetOverlappedResult`, and all four of its takes were direct. A thread stopped by stop-the-world can be holding either mutex, so taking one while attached left the blocked thread with no safepoint to reach. Assisted-by: Claude * gc: size the interpreter owner tag to the header padding `gc_owner` was a u32. With 4-byte pointers its alignment pushed it out of the padding that follows the gc bits and generation, growing every object by a word and tripping the `SIZEOF_PYOBJECT_HEAD` assertion on 32-bit targets. Introduce `GcOwner = u16`, which the `repr(C)` layout places at offset 10 on 32-bit and 18 on 64-bit, leaving the header at 6 words on both. Tags now run out after 65535 interpreters; `alloc_owner` already falls back to `GC_NO_OWNER`, so an interpreter past that collects as it did before tagging. Also widen three test deadlines that measure liveness, not speed. Assisted-by: Claude * vm: reuse cleared frame blocks and shorten interpreter hot paths - `datastack` remembers the most recently popped frame block. `push_frame` reports an exact LIFO reuse, and `setup_datastack_frame` then skips zero-filling localsplus. - Small-int loads push the context's cached int as a borrowed stack reference instead of taking a new one. - Calls to jitted functions go straight to `execute_call_vectorcall`. - Binary-op specialization reads ints through the new `PyInt::try_to_i64_fast` instead of the generic primitive conversion, and `try_to_bool` moves its non-bool path into a `#[cold]` helper. - Dict caches read an entry through a keys-version stamp (`get_index_if_keys_version`) rather than an entry-index hint. - Frame publishing caches a pointer to `ThreadSlot::top_iframe` in a thread-local `Cell` instead of borrowing `CURRENT_THREAD_SLOT`. * gc, vm: address review notes on owner tags and interpreter docs Look up retired owner tags with a sorted binary search instead of a linear scan, which every scanned object paid for once per dropped interpreter. Drop the claim that clearing a tag frees it for reuse; `alloc_owner` only ever hands out new tags. Also correct the tag space it mentions, which is 16-bit since the tag was sized to the header padding. Document `is_main` as "top-level interpreter" rather than "the process main": every top-level interpreter sets it, and only the first registered one becomes the main `main_interpreter_id` reports. Assert membership rather than an exact owned-interpreter count delta; the owned table is process-global and other tests store into it in parallel. Assisted-by: Claude * vm: gate interpreter registration on stop-the-world admission A collection snapshots the registry, stops the interpreters it found, and then reads tracked objects with their threads parked. An interpreter that registered after the snapshot was taken was absent from it, so nothing stopped it and its bootstrap ran Python — allocating and mutating the shared generation lists — underneath that scan. Registration and the stop now share a process-global gate: the collection holds it from before the snapshot until the restart, and registration takes it around the registry insert. The insert runs detached, since a thread that waited for the gate, or re-attached while holding it, would leave the stop it waits for no safepoint to complete at. Assisted-by: Claude * gc: keep the tracking counters off the barrier path Every tracked allocation and every free went through sequentially consistent counter updates, and each free through a `fetch_update` CAS loop. The counters drive only the gen0 threshold and `gc.get_count()`, and the generation locks — not the counters — order the list changes they describe, so they are relaxed now and the decrement is a load plus a conditional `fetch_sub`. `is_enabled` and `threshold`, both read once per allocation, are relaxed for the same reason: an allocation racing `gc.disable()` may use either value. Drop `alloc_count`, which nothing has ever read. Assisted-by: Claude * dict: settle lookups and iteration steps under one read guard A lookup probed under a read guard, dropped it with the matched entry in hand, and then took the lock again to re-find the entry. `lookup_extract` reads the entry while the probe still holds the guard when key identity settles the match, which is the case that cannot run Python. Dict and set iterators took one lock to compare the size and another to read the entry, then cloned both the key and the value even though a keys or values view keeps only one of them. `next_entry_checked` does the size check and the read under one guard, and clones through a per-view projection. A store that missed its inline-cache hint re-probed the dict afterwards only to recover the entry index the store had just computed; `unchecked_push` reports that index instead. Assisted-by: Claude * vm: build a call's argument vector once The specialized CALL handlers collected the positional arguments into one vector and then copied them into a second one to put `self` in front, so every specialized builtin, method-descriptor, class and non-Python call paid two allocations, a copy and two frees. The arguments are already laid out on the value stack in vectorcall order, so `take_call_args` fills one vector by index — the shape `execute_call_vectorcall` already used. `vectorcall_native_function` and the keyword path of `vectorcall_function` then cloned that vector again to build `FuncArgs`; both now move it in through `from_vectorcall_owned`, as the other vectorcall slots do. Assisted-by: Claude * vm: reach an instance dict without cloning it The specialized attribute instructions cloned the instance dict — a rwlock round-trip plus a refcount round-trip — for two things that only look at it: `LoadAttrMethodLazyDict` asking whether the dict exists, and the keys-version stamp check that is the whole of `shadowing_instance_attr`'s fast path. Both now read it borrowed, through `has_instance_dict` / `with_instance_dict`. `generic_getattr_opt` probed the dict with the name's `&Wtf8`, which hashes the name on every lookup and can never match a key by pointer. Passing the `Py<PyStr>` uses the string's cached hash and the interned-key identity check, and drops an allocation when a stored key is not an exact `str`. Assisted-by: Claude * vm: shorten the per-instruction safepoint and the call preamble The dispatch loop asks once per instruction whether stop-the-world wants this thread, and that read went through `CURRENT_THREAD_SLOT` — a `RefCell` borrow, so two stores to thread-local memory around an `Option` test. Cache the `stop_requested` pointer in a plain `Cell` at the same three places the frame pointers are cached, and the safepoint becomes one relaxed load. `lasti` is advanced from the index the loop just read, rather than reloaded to increment it, and `Resume` reads `quickened` before swapping it, so a call to an already-quickened code object costs a load instead of an atomic read-modify-write. An exact-args vectorcall to a Python function built a heap `FrameObject` where the equivalent `invoke` path uses a data stack frame; it now does the same when tracing is off. Assisted-by: Claude * vm: give KwArgs a zero-sized hasher `KwArgs::default()` is built for every call, keyword-less ones included, and it seeded a `RandomState` each time — a thread-local read and 16 bytes in every `FuncArgs`. Keyword names come from the program text, so the map now uses `BuildHasherDefault`, whose `Default` is a zero-init. Assisted-by: Claude * vm: promote borrowed stack refs before a yield `yield <small int>` tripped the "borrowed refs on stack at yield point" assertion: `LoadSmallInt` pushes a borrowed ref, and a yield saves the stack with the frame, which is exactly what that assertion forbids. Promote the stack first, so a suspended frame owns everything it holds. Assisted-by: Claude * vm: run the exact-args vectorcall on a heap frame again `vectorcall_function` took the data stack path for an exact-args call whenever tracing was off. A data stack frame that materializes — one traceback entry is enough — copies its localsplus into the `FrameObject`, so every value it holds gains a reference for as long as the frame runs, and a tail call holds the callee's function object until the caller's next call or return. Both are reachable without this hunk, through the `CallPyExactArgs` specialization, but routing the vectorcall entry point through the data stack made them apply from a call site's first execution: test_enumerate, test_memoryio, test_sys, test_tempfile and test_traceback all read a refcount or a finalization that moved. Assisted-by: Claude * vm: release a tail call's function at the callee's return The trampoline moved the callee's function object into the caller's `SuspendedFrame` and dropped it only once the caller had run again, so a function entered through a tail call kept one reference past its return: `sys.getrefcount` on it read one high for the caller's next stretch of bytecode, and anything the function reached stayed alive that much longer. The callee's frame is released before the `ReturnValue` or `Unwind` action is formed, and a materialized frame object holds its own references, so the owner is dropped where the suspended caller is popped instead. Assisted-by: Claude * vm: stop copying a running frame's locals when it materializes `materialize_slow` cloned the source frame's fast locals into the new `FrameObject`, so every value in a data stack frame gained a reference lasting as long as that frame object: one traceback entry through a frame kept all of its locals alive, and `sys.getrefcount` read one high for each of them. Nothing read that copy while the frame ran. `find_live_source_iframe` already routes reads to the live frame, and `exit_iframe` fills the slots from it at return, so the copy now starts empty as `materialize_slow_chain`'s does, and `snapshot_to_heap` goes with its last caller. Two readers had to be brought to the live frame the same way. `has_active_hidden_locals` read the frame object's own slots, which let `locals()` in a class body write an inlined comprehension's hidden variable into the class namespace; it now shares `live_fastlocals` with the other two. Another thread's frame has no live frame to reach from here, so the six cross-thread sites fill the copy under stop-the-world through `materialize_with_locals`. Assisted-by: Claude * vm: key data stack frame reuse on the exact frame size push_frame and pop_frame compared 16-byte-aligned sizes, so two frames whose sizes differ by less than the alignment matched as a reuse. The caller reads that as "every slot was cleared by the previous frame", but the larger frame's tail slots were never touched by it. Assisted-by: Claude * vm: store the frame object payload address in the cross-thread slot set_current_frame and reinit_frame_slot_after_fork wrote the address of the Py<FrameObject>, while the cross-thread readers hand that pointer to Py::from_payload_ptr, which subtracts the object header from it. Both writers now store the payload address. Assisted-by: Claude * vm: copy a foreign thread's frames instead of linking them sys._current_frames(), _thread._current_exceptions() and the cross-thread f_back path handed back frame objects still linked to the running frames. The owning thread wrote its locals into those same buffers at exit_iframe while the reader was cloning out of them. materialize_detached and materialize_detached_chain copy the values under stop-the-world and leave nothing for the owner to write into; materialize_with_locals is gone. FrameColdData gains attached_tid, the thread still running the frame a frame object was materialized from, which exit_iframe clears once it has written the values in. check_locals_access now rejects a read from any other thread while it is set; the FrameOwner test could not, because materialize_slow forces FrameObject ownership. Also removes with_iframe, which had no callers. Assisted-by: Claude * vm: resolve the two rustdoc links this branch added MAIN_INTERPRETER_ID is not in scope in interpreter.rs, and new_thread only exists in a threading build, so neither link resolved. Assisted-by: Claude |