mirror of
https://github.com/RustPython/RustPython.git
synced 2026-10-28 18:59:04 +08:00
stack/python-3.15-8950
87
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jeong, YunWon
|
7cd4a54e1c |
Compose native text signatures at compile time from FromArgs (#8807)
* Compose native text signatures at compile time from FromArgs FromArgs gains a PARAMS const describing its Python parameters. The derive builds the whole internal doc, `name(sig)\n--\n\ndoc`, in a const context from those params, and `__text_signature__` / `__doc__` parse it from the doc. The macro-time string path and the runtime sig_parts composition are removed. Signatures follow clinic conventions: `$module` for module functions, Rust literal defaults rendered as Python reprs, other defaults as `<unrepresentable>`, and a new `#[pyarg(py_default = "...")]` for the few fields whose Rust default differs from the Python one. The `#[pyarg]` keys are documented on the FromArgs derive. Module functions keep their module object and return it from `__self__`. PyNativeFunction now traverses its references. os.stat, os.mkdir, os.listdir, os.open, dict.fromkeys, round, sorted, int.to_bytes and bytes.hex take arguments matching their signatures. Assisted-by: Grok:grok-4.7 Assisted-by: Claude:claude-opus-5-5 * Update builtin_signature snippet and fix mkdir on Windows round and sum now report their FromArgs parameters, so the snippet asserts their signatures instead of their absence. sum's start is a PyObjectRef defaulting to int 0 with py_default "0". mkdir reads dir_fd only on platforms that use it. Assisted-by: Claude:claude-opus-5-5 * Remove expectedFailure from two signature tests test_cmp_to_signature and test_autospec_on_bound_builtin_function now pass. The TestCmpToKeyC override existed only for the marker. Assisted-by: Claude:claude-opus-5-5 |
||
|
Jeong, YunWonandLee Dogeon
|
afad92bf1b |
Add OrderedDict, marshal, inspect, and chflags bitflagset (#8780)
* Implement _collections.OrderedDict Extracted from #6799: native OrderedDict as a dict subclass, iterator mutation tracking, and mapping-view set operations. * Add OrderedDict, warnings source lines, and socket sendmsg Implement _collections.OrderedDict as a dict subclass with a linked list of insertion-order nodes. Pack struct F/D as native float/double complex pairs, and report unexpected __replace__ field names as a list repr. Register encodings.latin_1 at startup. Reject NUL in input() prompts. Copy non-exact type dicts in mapping order. Fetch warning source lines through _bless_my_loader and unicode splitlines. Keep the last warnings.filters list after deletion. Snapshot sendmsg sequences before converting buffers, parse ancillary ints with __index__, and dispatch AF_CAN by protocol. Store sys.trace and sys.profile on each ThreadSlot so _settraceallthreads/_setprofileallthreads are visible to sys.gettrace/getprofile on other threads. Use <frozen importlib._bootstrap> as frozen importlib co_filename. Keep HEAPTYPE socket objects GC-tracked. Assisted-by: Grok:grok-4.6 * Load marshal via readinto and dump sets in dumps order marshal.load fills a scratch buffer with readinto on a contiguous writable memoryview, and rejects a returned length other than the requested size. set and frozenset dumps write elements in sorted(v, key=marshal.dumps) order. Assisted-by: Grok:grok-4.6 * Add classmethod_descriptor and ABC IS_ABSTRACT flags Give METH_CLASS methods a distinct classmethod_descriptor type. Store Py_TPFLAGS_IS_ABSTRACT from __abstractmethods__ and look up that attribute only in the type's own dict. Expose defaultdict.default_factory as a member descriptor. Inherit bool's unused number slots from int. Mark set and frozenset __contains__ as METH_COEXIST. Parse classmethod, staticmethod, object.__new__, and heap type text signatures. Register interactive source in linecache. Assisted-by: Grok:grok-4.6 * address review: odict reuse, descr call, new wrapper Reuse odict node slots and walk the live list in next_after. Require both | operands to be dicts. Drop the odict iterator on GC clear. Bind classmethod descriptors through descr_get and update IS_ABSTRACT under the type lock. Give odict keys and items sq_contains. Walk tp_new_wrapper past new_wrapper so HEAPTYPE socket subclasses can call socket.__new__. Split FreeBSD Bluetooth protocol constants from Linux values. Assisted-by: Grok:grok-4.6 * Represent chflags bits with bitflagset Assisted-by: Grok:grok-4.6 * Represent access and mode bits with bitflagset Assisted-by: Grok:grok-4.6 * Continue unfinished REPL suites as incomplete input Assisted-by: Grok:grok-4.6 * Add pickle and os.stat inspect signatures Give _pickle.Pickler the clinic constructor text signature and accept fix_imports and buffer_callback as positional-or-keyword. Allow #[pyfunction(text_signature)] and use it so os.stat reports (path, *, dir_fd=None, follow_symlinks=True). Uncomment import _pickle and drop the pickle inspect expectedFailures. Assisted-by: Grok:grok-4.6 * Add Unpickler, dump/load, and lstat signatures Give _pickle Unpickler, dump, dumps, load, and loads clinic text signatures, and os.lstat (path, *, dir_fd=None). Assisted-by: Grok:grok-4.6 * Complete OrderedDict with insertion-order links Move the native OrderedDict into ordered_dict.rs and track order with a linked list of nodes, keeping mapping-view set operations. Assisted-by: Grok:grok-4.6 * address CI: stat casts, HCI_DATA_DIR, passing xfails Compare stat permission bits as Mode so Linux clippy does not flag a u32-to-u32 cast. Export HCI_DATA_DIR with the Bluetooth socket option values. Drop expectedFailure from tests that now pass. Assisted-by: Grok:grok-4.6 * Fix wasm teardown deadlock and bluetooth constants Assisted-by: Grok:grok-4.6 * Clear process-wide eval breaker bits in STW test Assisted-by: Grok:grok-4.6 * Drop exclusive-held panic that tripped wasm clippy Assisted-by: Grok:grok-4.6 * Fix OrderedDict contains and export Windows BDADDR Assisted-by: Grok:grok-4.6 * Add cmp_to_key and hash-once LRU cache _functools.cmp_to_key now returns KeyWrapper. LRU lookups store the key hash on _lru_list_elem so __hash__ is not called again on hits or eviction. Assisted-by: Grok:grok-4.6 * Allow subclass keywords on list, tuple, and float list.__init__ rejects keywords only when tp_new is still list's. tuple/float.__new__ reject them only when tp_init is still the base's. Assisted-by: Grok:grok-4.6 * Fix CI lint, test re-export, and structseq members Drop the duplicate clear_eval_breaker_for_test re-export. Format structseq imports. Expose struct sequence fields as readonly members so assignment raises "readonly attribute". Assisted-by: Grok:grok-4.6 * Allow subclass keywords on itertools and filter Positional-only constructors ignore extra keywords when a subclass has replaced tp_init, matching list/tuple/float. Assisted-by: Grok:grok-4.6 * Remove text_signature derive parameter Drop the string override on #[pyfunction] and #[pyclass]. Signatures are generated from the Rust parameter list only. Restore inspect expectedFailures that depended on the override. Assisted-by: Grok:grok-4.6 * Rename set contains_method to __contains__ Keep a borrowed contains helper for slots and C-API. The coexist pymethod is named __contains__. Assisted-by: Grok:grok-4.6 * Use MonitoringEvent instead of EVENT_* aliases Drop the unused MonitoringEvents bitflags and the EVENT_PY_START style mask constants. Call sites use the enum and .mask(). Assisted-by: Grok:grok-4.6 * Store warnings filters as PyListRef The list already serializes its elements. Drop the extra mutex around the cached list pointer. Assisted-by: Grok:grok-4.6 * Re-export clear_eval_breaker_for_test after rebase Keep the STW test helper visible without bringing back stop_bit_set. Assisted-by: Grok:grok-4.6 * Fix groupby reentrancy and islice arguments Give _grouper its own tgtkey and take currvalue after the key comparison. Accept __index__ on islice bounds, reject step 0, and drain the source up to stop. Assisted-by: Grok:grok-4.6 * SETREF warnings filters, onceregistry, and defaultaction Keep the last list, dict, or str after the module attributes are deleted by swapping the interpreter cache pointer. Do not store a None fallback for _warnings_context. Assisted-by: Grok:grok-4.6 * Iterate OrderedDict inner dict via next_entry &PyDict is not an iterator after the borrowed-ref signature change. Assisted-by: Grok:grok-4.6 * Remove expectedFailure from testBluetoothConstants Linux now exports BTPROTO_HCI and the rest of the set. Assisted-by: Grok:grok-4.6 * Borrow get_filter arguments as PyObject Match the warn_explicit call site that passes borrowed handles after the internal-helper borrow change. Assisted-by: Grok:grok-4.6 * Fix bool slots and SemHandle safety docs Leave unused bool number slots unset so inspect reports __add__ on int. Document SemHandle::from_raw safety. Assisted-by: Grok:grok-4.6 * Keep sendmsg errors and drop duplicate linecache Replace sendmsg iterable TypeError only. Register REPL source in linecache once per compiled input. Assisted-by: Grok:grok-4.6 * Nest REPL parse or-patterns for clippy Assisted-by: Grok:grok-4.6 * Mark pydoc os.stat signature test expectedFailure Assisted-by: Grok:grok-4.6 --------- Co-authored-by: Lee Dogeon <dev.moreal@gmail.com> |
||
|
Jeong, YunWonandLee Dogeon
|
d02d81e740 |
perf: VM and runtime hot paths from pyperformance (#8737)
* time: use CLOCK_UPTIME_RAW for monotonic/perf_counter on Apple
CLOCK_MONOTONIC keeps advancing during system sleep on macOS, unlike
CPython's mach_absolute_time()-backed perf_counter/monotonic. Benchmarks
that spanned a laptop sleep recorded thousands of seconds for a single
iteration (pyperformance float/fannkuch showed ~1000x slowdowns).
Use CLOCK_UPTIME_RAW (equivalent to mach_absolute_time) on Apple targets
and report it as such in time.get_clock_info().
Assisted-by: Claude Code:claude-fable-5-1
* vm: derive f_lineno lazily instead of tracking prev_line per instruction
The dispatch loop wrote prev_line (a locations lookup plus a Cell store)
on every instruction only so f_lineno could read it. f_lineno now derives
the line from lasti and code.locations, prev_line is only maintained on
the cold tracing path for 'line' event de-duplication, the two use_tracing
reads per instruction are merged into one, and the pre-dispatch lasti
reload is replaced by the value just stored.
When a trace function is installed on an already-running frame (bdb
stepping back into an untraced caller, frame.f_trace assignment, a local
trace returned from the 'call' event on a resumed generator) prev_line is
synced from lasti so no spurious 'line' event fires.
Assisted-by: Claude Code:claude-fable-5-1
* vm: coalesce per-call thread-locals into one FRAME_SLOT_CACHE
set_current_frame ran on every frame enter/exit and touched three separate
thread_local! statics (each a _tlv_get_addr call on macOS). Merge them into
a single thread-local struct so the hot path does one TLS access.
Assisted-by: Claude Code:claude-fable-5-1
* vm: fold stop/finalizing checks into EVAL_BREAKER
eval_breaker_tripped ran three checks per bytecode: the finalizing flag,
a thread-local stop_requested lookup, and the EVAL_BREAKER load. Add
STOP_BIT and FINALIZING_BIT so the per-instruction check is a single
relaxed atomic load; the slow path (check_signals) still re-derives the
exact per-thread answer. The bits are set alongside the per-thread flags
and cleared under the stop-the-world exclusion.
Assisted-by: Claude Code:claude-fable-5-1
* gc: set the TRACKED bit with a plain store for freshly allocated objects
gc_bits is zero for a fresh allocation (both the freelist reuse path and
PyInner::new zero it), so track_new_object can initialise the bit with a
relaxed store instead of a fetch_or. Re-tracking resurrected objects keeps
the read-modify-write variant.
Assisted-by: Claude Code:claude-fable-5-1
* vm: skip thread exception publish when pushing/popping a None exc slot
Generator/coroutine resume pushes and pops an exc_info slot on every
send. When that slot is None the value returned by topmost_exception
cannot change (it skips None entries), so the thread-local exception
publish in push_exception/pop_exception is unnecessary.
Assisted-by: Claude Code:claude-fable-5-1
* vm: allocation-free keyword-call fast path and super() MRO walk
Keyword calls to plain functions where every parameter is filled exactly
once are reordered in place into the fast-locals layout (no IndexMap, no
keyword name clones); any mismatch falls back to the existing slow path
so error messages are unchanged. super().attr now walks the type's MRO
by reference instead of cloning every class into two Vecs per lookup.
Assisted-by: Claude Code:claude-fable-5-1
* builtins: cheaper enumerate and zip iteration
enumerate kept its counter as a BigInt behind a write lock and did bignum
arithmetic on every item; use a machine integer that promotes to BigInt
only for huge/negative starts or on overflow. zip pre-sizes its per-item
Vec so the common two-iterable case allocates once.
Assisted-by: Claude Code:claude-fable-5-1
* _asyncio: cache module objects and use native set/dict operations
Task registration re-resolved _asyncio module attributes and re-imported
contextvars on every task; the eager path additionally went through generic
call_method for set add/discard. Cache the module objects per thread
(validated by identity), split the contextvars cache from the task
containers, and use PySet/PyDict methods directly.
Assisted-by: Claude Code:claude-fable-5-1
* perf: use mimalloc as global allocator for rustpython CLI
Adds mimalloc (0.1.52, default-features = false) as the #[global_allocator]
for the rustpython binary, gated behind a new `mimalloc` feature enabled
by default on non-wasm targets (dep declared under
target.'cfg(not(target_arch = "wasm32"))'.dependencies, so wasm builds
never pull it in regardless of default-features).
Profiling a `t = (a, b, c)` loop showed malloc/free at ~40% of samples on
macOS with the system allocator. Switching to mimalloc gives consistent
wins across small-object-heavy workloads, medians over 5-9 interleaved
A/B rounds (release build, this machine):
tuple (5M `(a,b,c)`) 1.020s -> 0.863s (1.18x)
4-thread tuple (5M total) 1.643s -> 1.382s (1.19x)
class instances (2M) 0.874s -> 0.797s (1.10x)
dict-heavy (200k x 20) 2.196s -> 1.992s (1.10x)
string-heavy (join x 20) 0.662s -> 0.592s (1.12x)
nqueens (pyperformance) 0.327s -> 0.286s (1.14x)
chaos (pyperformance) 0.204s -> 0.191s (1.06x)
richards (pyperformance) 0.154s -> 0.151s (1.02x)
go (pyperformance) 0.411s -> 0.405s (1.01x)
listcomp / fib ~neutral (<=1%)
Peak RSS (/usr/bin/time -l): instances workload roughly unchanged
(29.0MB -> 29.1MB); dict-heavy workload actually drops (155MB -> 106MB),
likely because mimalloc reclaims freed pages more aggressively than the
system allocator across repeated large-dict churn.
`cargo build` and `cargo build --release` are warning-free with the
default feature set, and `cargo build --release --no-default-features
--features stdlib,threading,importlib,stdio,host_env` (mimalloc
excluded) also builds cleanly. CI's wasm32-wasip1 and other
--no-default-features build matrices in .github/workflows already omit
`mimalloc` from their feature lists and target wasm via the
cfg(not(wasm32)) dependency gate, so no workflow changes needed.
`rustpython -m test -j4 test_gc test_weakref test_threading test_dict
test_list test_tuple test_str test_bytes test_memoryview test_ctypes
test_mmap test_array` passes identically (2541 tests, 241 skipped) on
both the baseline and mimalloc binaries.
Assisted-by: Claude Code:claude-fable-5-1
* vm: cap dict/set table shrink and compact holes on resize
asyncio's eager task path (and its per-task bookkeeping sets in general)
repeatedly adds and discards distinct task objects from a handful of
long-lived sets (_eager_tasks, _scheduled_tasks). Two dict_inner issues
turned that steady-state churn into the dominant cost of async_tree_eager:
1. resize() picked a target size of `used * 2` with no floor, so once a
churny set settled around 1-2 live entries it shrank to a table that
the very next insert would already overflow, forcing a resize on
almost every single add. Clamping to the original minimum table size
(8) restores the headroom dict/set start with.
2. `entries` only ever grows (holes from deletions are never reclaimed),
so resize()'s rebuild pass scanned every entry ever inserted into the
dict's lifetime, not just the live ones - a set with a stable working
set size but heavy churn grew this scan, and the dict's memory, without
bound. resize() now compacts holes out of `entries` first, matching
CPython's compacting hash table and keeping the shape-stamp fast path
(which already requires a hole-free layout) reachable more often.
Measured with a standalone async_tree micro-benchmark (6 levels x 6
branches, pyperformance's bm_async_tree "none"/"eager" shape) run inside
one process for 10 asyncio.run() iterations, comparing medians of
interleaved binary runs on an otherwise idle machine:
- eager: ~2.91s -> ~0.86s median (about 3.4x)
- none: ~1.98s -> ~1.23s median (about 1.6x)
A single fresh-process run (no in-process accumulation) still improves
by roughly 15-20% for eager, since the resize-storm from point 1 hits
even a single run's worth of task churn.
test_asyncio (34 files, all passing), test_dict, test_set, test_weakref,
test_weakset, test_ordered_dict, test_gc, test_descr, test_class, and
`cargo test -p rustpython-vm -p rustpython-stdlib` all pass.
Assisted-by: Claude Code:claude-fable-5-1
* _functools: add native _lru_cache_wrapper
RustPython had no native _functools._lru_cache_wrapper, so
functools.lru_cache always fell back to the pure-Python implementation
in Lib/functools.py (RLock, linked-list bookkeeping, closures). That
fallback dominates typing.Protocol's __instancecheck__ path: it calls
inspect.getattr_static, whose _shadowed_dict is itself lru_cache'd, so
every isinstance() check against a runtime-checkable protocol paid for
several pure-Python cache-wrapper calls plus lock acquisitions.
Add a native pyclass mirroring CPython's C _lru_cache_wrapper: a
reentrant lock scoped only around the cache lookup/insert steps (not
held across the wrapped call, matching CPython's threading behavior),
LRU ordering piggybacked on the cache dict's own insertion order
(move-to-end via del+reinsert, oldest evicted from the front) so no
separate key-comparison structure is needed, and __copy__/__deepcopy__
returning self to match CPython since __reduce__ (used only for
pickling) requires __qualname__, which wrapped non-function callables
such as functools.partial don't have.
Measured with a standalone script reproducing pyperformance's
bm_typing_runtime_protocols isinstance() loop (300 loops, medians of
5 interleaved runs): 0.260s -> 0.182s, about 1.43x faster. In
isolation, a cache-hit through lru_cache went from ~0.31s to ~0.07s
for 200k calls, now faster than calling the underlying function
uncached (matching CPython's C accelerator).
Verified with test_typing, test_abc, test_inspect, test_functools,
test_weakref, test_isinstance, and test_descr (all pass, matching the
pre-existing baseline, including test_functools's already-present
TestLRUC suite which now actually exercises the native implementation).
Assisted-by: Claude Code:claude-fable-5-1
* sre_engine: ASCII fast path for \d \s \w unicode category checks
is_uni_digit/is_uni_space/is_uni_alnum went through icu_properties'
GeneralCategory table lookup for every character, even the common
ASCII case. Add a `ch < 0x80` fast path that resolves directly via
std's ascii predicates, falling back to the full table only for
non-ASCII code points (Latin-1 supplement chars like 'A-umlaut' still
route through the table since they are alphanumeric but not ASCII).
Measured on a tight `re.findall(r"\w+", ...)` / `re.sub` loop over
mostly-ASCII text (interleaved A/B, 4 rounds, medians):
re.sub: 1.178s -> 1.134s (~3.7% faster)
re.findall: 1.110s -> 1.062s (~4.3% faster)
Assisted-by: Claude Code:claude-fable-5-1
* vm: skip VM attach for no-op generator/coroutine __del__
Generator, coroutine, and async-generator dealloc always went through
drop_slow_inner's call_slot_del, which unconditionally attaches to a VM
(with_vm -> begin_interpreter_section -> per-interpreter thread-slot
hashmap lookup) before running del(), even though del() itself is a
documented no-op once the object is already closed or running - the
common case for a generator/coroutine consumed to completion.
Add PyTypeSlots::del_needed, an optional VM-free fast predicate checked
before call_slot_del/with_vm; generator, coroutine, and async_generator
install a check mirroring their own del() no-op branches, letting
drop_slow_inner and try_call_finalizer skip the VM attach entirely for
finished objects.
Standalone microbenchmarks (median of 5 interleaved runs):
- create+drain a 3-yield generator, 2M times: 1.099s -> 0.997s (-9%)
- pyperformance bm_generators shape (yield-from tree traversal): -4%
- pyperformance bm_coroutines shape (recursive fibonacci coroutines): -13%
Assisted-by: Claude Code:claude-fable-5-1
* _asyncio: skip full __import__ dispatch on module-cache hot path
with_asyncio_cache/with_contextvars_cache called vm.import("_asyncio"/
"contextvars", 0) on every Task/Future registration just to validate
their module-identity cache, even on the (overwhelming) hit path. That
still pays the full __import__ builtin dispatch (attribute lookup on
builtins, FuncArgs construction, native slot_call, ImportArgs parsing,
import_module_level) instead of a cheap sys.modules lookup. Added
fast_import_cached(), which checks sys.modules directly and only falls
back to vm.import on a miss (module missing/reloaded).
Measured with a standalone asyncio_tree-style microbenchmark
(6 levels x 6 branches, several asyncio.run() iterations per process,
median of 5 interleaved A/B rounds):
none: 1.045s -> 0.895s (~14% faster)
eager: 0.716s -> 0.591s (~17% faster)
eager_tg: 1.023s -> 0.849s (~17% faster)
test_asyncio (all files), test_contextvars (module absent from this
checkout, pre-existing), test_weakref, test_weakset all pass/unchanged.
Assisted-by: Claude Code:claude-fable-5-1
* vm: skip VM-attach section in with_vm when already on the current interpreter
Object teardown (drop_slow_inner -> call_slot_del, try_call_finalizer,
weakref callbacks) calls with_vm(), which unconditionally went through
set_current_vm -> begin_interpreter_section: an INTERP_THREAD_SLOTS
HashMap lookup plus an Arc clone, even in the overwhelmingly common
case where the dropping thread is already attached to the exact
interpreter that owns the object (a __del__ object or weakref target
going out of scope mid-bytecode).
with_vm now checks whether the object's owning interpreter is already
the one on top of VM_STACK. Only the top of VM_STACK is ever ATTACHED
on a thread (every push is preceded by an attach, every pop restores
the enclosing attach), so when it matches we call f(vm) directly and
skip begin_interpreter_section/set_current_vm entirely. The rare
not-attached/different-interpreter cases (nested subinterpreters,
thread outside any VM, shutdown) fall through to the original
attach/detach path unchanged.
Interleaved release-build measurements (5 rounds, medians), plain-Rust
micro-scripts, no pyperf:
- 2M create+drop of a __del__ object: ~0.55s -> ~0.46s (~16% faster)
- 2M create+drop with a weakref callback: ~1.14s -> ~1.06s (~7% faster)
- 2M io.StringIO() create+drop: ~0.734s -> ~0.697s (~5% faster)
- fib(30) and a 5M-iteration tuple-unpack loop: unchanged (noise-level)
`sample` profiles of the __del__ benchmark confirm
begin_interpreter_section/ensure_thread_slot frames disappear after
the change.
Verified: `cargo test -p rustpython-vm --release`, and
`-m test -j4 test_gc test_weakref test_finalization test_threading
test_thread test_concurrent_futures test_os test_signal test_generators
test_io test_atexit test_sys` (test_interpreters module absent in this
tree, pre-existing). Also a 4-thread stress script hammering __del__
object creation against a concurrent gc.collect() loop for 10s, and a
threading+queue handoff dropping a __del__ object after cross-thread
transfer, both pass.
Assisted-by: Claude Code:claude-fable-5-1
* vm: general sys.modules fast path for vm.import
VirtualMachine::import always resolved and called builtins.__import__
(FuncArgs construction, native slot_call, ImportArgs::from_args, then
import_module_level) even when the module was already cached in
sys.modules. ~63 Rust stdlib call sites hit this on hot paths
(copyreg, pickle's reduce_newobj, warnings, asyncio, etc).
Add VirtualMachine::try_import_cached: for a plain absolute import
(level 0, empty from-list), if builtins.__import__ is still the
original import function (compared by identity against vm.import_func)
and the module -- or, for a dotted name, its top-level package -- is
already present and fully initialized in sys.modules, return it
directly, skipping the __import__ dispatch and import_module_level
entirely. Falls through to the existing slow path whenever the module
is uncached, still initializing, or __import__ has been overridden by
user code, so test_import/test_importlib/test_builtin overrides still
work.
Exposed import::is_module_initializing (refactored out of
import_ensure_initialized) so the fast path can check
__spec__._initializing the same way the slow path does.
Removes _asyncio's local fast_import_cached workaround (added in
cb2064f0e) now that vm.import itself has the same fast path generally.
Measured (interleaved A/B, 5 rounds, pickle.dumps of a plain object,
which hits copyreg import via reduce_newobj): ~2% wall-clock
improvement per call (baseline ~4.75s / 100k dumps vs ~4.66s
candidate), consistent across all rounds. Verified: overridden
__import__ still invoked, a replaced sys.modules is read through the
same accessor, and a self-importing/initializing module is not handed
out early -- all match baseline behavior.
test suite (test_import test_importlib test_builtin test_pkg
test_pkgutil test_runpy test_zipimport test_site test_warnings
test_copy test_pickle test_asyncio.test_tasks test_threaded_import
test_sys): 13/14 files pass; test_threaded_import fails identically on
baseline (ModuleNotFoundError, pre-existing, unrelated to this change).
Assisted-by: Claude Code:claude-fable-5-1
* _asyncio: bypass generic context.run dispatch in task_eager_start
task_eager_start ran the first step via context.run(coro.send, None),
which builds a bound `send` method object and then re-enters the
generic Callable/FuncArgs dispatch just to invoke context.run's own
generic call. Since eager tasks in async_tree_eager complete almost
entirely synchronously (no event-loop scheduling needed), this
per-task dispatch overhead dominated. Enter/exit the native PyContext
directly and call coro.send via a single call_method instead,
falling back to the generic path for non-native Context subclasses.
async_tree_eager median: ~705ms -> ~697ms across 5 interleaved A/B
rounds (~1% eager-only win); async_tree (none) unaffected (noise-level
delta). test_asyncio (all files), test_weakref, test_weakset pass;
pre-existing test_contextvars module-not-found failure unchanged from
baseline.
Assisted-by: Claude Code:claude-fable-5-1
* vm: drop scopeguard::defer! from generator/coroutine resume hot path
resume_gen_frame restored owner/exc-slot/frame-chain state via a
scopeguard::defer! closure that ran on every exit (Ok, Err, or panic).
Since the closure's body is the function's only tail expression, a
plain captured result plus explicit cleanup (the same shape with_frame
already uses for the analogous call path) covers both normal exit
paths identically; only the recursion-depth decrement keeps a
scopeguard, matching with_frame's precedent that this codebase does
not otherwise guarantee generator/frame state consistency across a
Rust panic (panic=unwind is not used to implement any Python-level
control flow here).
Delegation-chain benchmark (chain.py, forwarding N values through a
yield-from chain) measures a small but consistent win: ~2-5% less
wall time per forwarded value at depths 1/4/16 across interleaved A/B
rounds. Single-generator create+finish cost is a wash.
Assisted-by: Claude Code:claude-fable-5-1
* vm: run generator bodies through the call trampoline
`Py<FrameObject>::resume` built its `ExecutingFrame` with
`tailcall_enabled: false`, because a `TailCall` result had nowhere to go:
the trampoline was reachable only from `run_frame_fast`, whose entry frame
lives on the data stack and can never yield. So every ordinary Python call
made from inside a generator or coroutine body re-entered `run_frame_fast`
recursively instead of being flattened.
Generalize the trampoline to drive a heap-resident entry frame whose
enter/exit bookkeeping belongs to its caller, and whose `Yield` is a valid
terminal result rather than a panic. `FrameKind` now says, per frame, what
the trampoline owes it: `Entry` (exit it, caller frees the storage),
`GenEntry` (`resume_gen_frame` linked it and will unlink it; a `Yield` is
the trampoline's result) or `Callee` (exit it and free its storage). The
three near-identical dispatch arms collapse into one `Step::Finished` /
`Step::Deliver` loop.
The suspended-frame stack moves onto the VM and is reused across
invocations: a generator body enters the trampoline once per resume, so a
`Vec::with_capacity(8)` there cost a malloc/free per yield (+27 ns/resume
when first measured). Invocations nest strictly LIFO, so each owns the
region above the length it finds on entry.
Neutral on its own — a generator body's first call was the only one not
already flattened, since the callee's own frame runs under the trampoline
either way. It is what lets SEND/FOR_ITER hand a generator frame to the
same loop. Measured (chain of `yield from`, 1-yield generator create+finish,
bm_generators, bm_coroutines, async_tree none/eager): within noise, except
generator create+finish 625 ns -> 603 ns.
Assisted-by: Claude Code:claude-fable-5-1
* vm: run a `yield from` chain in one eval loop
`SEND` on a suspended generator recursed: `Coro::send` -> `resume_gen_frame`
-> a fresh `ExecutingFrame::run` per level, and the value came back the same
way. Measured at ~55 ns per level of delegation, against CPython's ~5.8 ns.
Only about 5 ns of that is the resume bookkeeping itself (measured by
stubbing out the claim, the exception slot, the owner swap and the recursion
checks). The rest is what a delegating frame does per value: leave and
re-enter the eval loop, and dispatch `RESUME`, `JUMP_BACKWARD_NO_INTERRUPT`,
`SEND` and `YIELD_VALUE` — four instructions that, in the canonical
`yield from` / `await` shape, only hand a value along.
So the trampoline now runs the chain itself. `SendGen` parks its frame and
returns `ExecutionResult::GenResume`; the trampoline claims and links the
sub-generator (`coroutine::flat_resume_enter`, the first half of what
`Coro::send` does around a resume) and runs its frame in the same loop. On
the way down, every frame it finds already suspended at a `yield from` whose
delegate can be resumed is parked without being run at all
(`yield_from_delegate`); on the way back up, a yielded value is re-yielded
for each such frame by parking its `lasti` past the `YIELD_VALUE`
(`park_after_yield_from`) instead of running it. A whole chain therefore
costs one claim/link/unlink per level and no instruction dispatch at all,
with `Py<FrameObject>::resume` extending the same treatment to the outermost
level, which is where `next()` and asyncio's task step enter a chain.
Each level is still claimed, linked and unlinked exactly as a recursive
`Coro::send` would, so `gi_running`, `f_back`, `gi_frame`, `gi_yieldfrom`,
`sys._getframe`, tracebacks, `throw`, `close`, PEP 479 and the
`RecursionError` depth are unchanged; `resume_gen_frame` and the flat path
share `gen_frame_link`/`gen_frame_unlink` and the invariants are documented
on `FlatResume`. Everything else keeps the recursive path: tracing or
monitoring on (both would see the skipped instructions), `throw`/`close`,
async generators, subclasses, non-generator iterators, `FOR_ITER`, a `SEND`
that is not a `yield from`, and a delegate that has not started or is not
itself delegating — the last two because one lone level costs more to hand
to the trampoline than to send into from the eval loop the caller is in.
Measured (median of 5, ns/value or seconds):
chain of `yield from`, depth 16 911 -> 479 -47%
chain of `yield from`, depth 4 212 -> 158 -26%
bm_generators 0.272 -> 0.187 -31%
richards 0.772 -> 0.771 ~0
bm_coroutines 0.0129 -> 0.0130 +0.8%
async_tree none ~neutral
chain depth 1 (no delegation) 81.1 -> 83.6 +3.1%
fib(24), calls through trampoline 0.0180 -> 0.0183 +1.7%
Per level of delegation: 55.3 ns -> 26.4 ns (CPython 3.14: 5.8 ns).
Assisted-by: Claude Code:claude-fable-5-1
* vm: reset the oparg prefix when a signal unwinds a frame
An `EXTENDED_ARG` leaves its prefix in `arg_state` for the instruction that
follows it. The signal path in the eval loop unwinds to a handler and
`continue`s straight back to the top, skipping the loop's own
`arg_state.reset()`, so the handler's first instruction was decoded with the
interrupted instruction's prefix still applied. Every other path that jumps
somewhere new — the `set_f_lineno` restart just above — resets first.
Reachable whenever a signal arrives on the instruction right after an
`EXTENDED_ARG` in a frame that has a handler: a `KeyboardInterrupt` into a
`yield from` chain panicked with `InvalidBytecode` in roughly half of the
runs of a 16-deep chain, and now does not in ten out of ten.
Assisted-by: Claude Code:claude-fable-5-1
* bytecode: fetch a code unit with one 16-bit acquire load
The dispatch loop read the opcode with an Acquire `AtomicU8` load and the
arg byte with a separate Relaxed one. The acquire load sits between the
two, so LLVM had to re-load `self.code` and `code.instructions` for the
second access: four extra instructions and two extra dependent loads on
the instruction-fetch path of every bytecode instruction.
`CodeUnits::read_unit` reads both bytes as a single aligned Acquire
`AtomicU16`, which pairs with `replace_op`'s Release store exactly as
`read_op` did. `CodeUnit` gains `align(2)` so the 16-bit access is
guaranteed aligned rather than relying on the allocator.
5.91 -> 5.76 ns/bytecode-instruction on a straight-line
LOAD_FAST/STORE_FAST loop (-2.5%); range loop -2%.
Assisted-by: Claude Code:claude-fable-5-1
* vm: compute the cache-entry count after dispatch, not before
`op.cache_entries()` is a byte-table lookup used only to skip inline
cache units after an instruction falls through. Computing it before the
handler kept it live across the whole (very large) inlined match, so
every bytecode instruction paid an address materialization, the table
load, a stack spill and a reload of the count.
Moving it next to its only use lets it be computed on the fall-through
path with the value in a register.
Measured together with the following commit (the eval-breaker outline):
5.76 -> 5.55 ns/bytecode-instruction on a straight-line
LOAD_FAST/STORE_FAST loop, range loop -1.5%, fib(27) -2.5%.
Assisted-by: Claude Code:claude-fable-5-1
* vm: move the eval-breaker body out of the dispatch loop
The eval-breaker check itself stays inline on the per-instruction path
(moving it off regresses badly), but everything it guards -- the
finalizing/main-thread check, stop-the-world suspend, the QSBR
checkpoint, signal delivery and the scheduled GC -- was inlined into the
loop body too. That code is never executed on the hot path yet its
address materializations and live values competed for registers with
every opcode handler, so the loop head spilled more than it needed to.
`eval_breaker_work` is `#[cold] #[inline(never)]`, leaving a single
`ldrb`/`cbz` pair inline for the common "nothing pending" case.
Measured with the preceding commit: 5.76 -> 5.55 ns/bytecode-instruction
on a straight-line LOAD_FAST/STORE_FAST loop.
Assisted-by: Claude Code:claude-fable-5-1
* vm: always inline the eval stack push/pop primitives
`pop_value`, `pop_value_opt`, `top_value` and friends are a handful of
instructions each, but `ExecutingFrame::run` is ~50 KB of inlined opcode
handlers, so LLVM's inline budget refused them and emitted real calls:
STORE_FAST and POP_TOP each began with `bl ...pop_value`, which also
forced spills around the call. `#[inline(always)]` puts the four or five
instructions they actually are back into the handlers.
5.55 -> 5.29 ns/bytecode-instruction on a straight-line
LOAD_FAST/STORE_FAST loop (-4.8%); nbody -3.5%, fib(27) -3.5%,
`s += i` loop -3.5%, `for _ in range(n)` -4%.
Assisted-by: Claude Code:claude-fable-5-1
* vm: carry the instruction fetch state in loop locals
Instruction fetch went through memory three times per bytecode
instruction: `self.lasti` (a pointer field) was re-loaded, the program
counter was stored to it and read back at the top of the next iteration,
and `self.code` -> `code.instructions` was re-loaded to address the code
unit. LLVM cannot keep any of them in registers because the handlers
write through `&mut self`, so the fetch chain started with a
store-to-load round trip followed by two dependent pointer loads.
`lasti_cell` and `instructions` are now copied into locals once (both are
plain shared references with the frame's lifetime), and the program
counter is carried in `idx`: after a handler that did not jump the next
index is already known from `lasti_before` plus the cache count. The
frame's `lasti` is still stored before every instruction, so tracebacks,
`f_lineno`, signals and `sys._getframe` see exactly what they did; only
the read-back is gone. The paths that unwind (signal, exception handler,
`set_f_lineno`) re-read it, since they jump to a new position.
Also collapses the eval-breaker `if`, which clippy flagged after the
previous commit moved its body out of line.
range loop -3.3%, richards -3.1%, nbody -2.9%, `s += i` loop -2.2%,
go -1.7%, raytrace -1.2%; mean -1.1% over the 13-benchmark set. The
straight-line LOAD_FAST/STORE_FAST micro is unchanged at 5.28
ns/bytecode-instruction (it has no cache entries and never jumps, so it
was already the best case for the old code).
Assisted-by: Claude Code:claude-fable-5-1
* bytecode: hoist the instruction array base out of the dispatch loop
Addressing a code unit still went `self.code` -> `code.instructions` ->
the `Box<[CodeUnit]>` data pointer on every instruction, two dependent
loads before the acquire load that fetches the unit itself, because the
handlers write through `&mut self` and LLVM has to assume the fields
could change.
The instruction array is allocated once when the code object is built
and never reallocated -- specialization rewrites units in place -- so
`CodeUnits::units_ptr` may be read once before the loop and reused.
`read_unit_from` is the same 16-bit acquire load taken from that base;
`read_unit` now delegates to it, and the index is unchecked exactly as
it was before.
nqueens -1.9%, raytrace -1.6%, range loop -1.3%, deltablue -1.1%,
nbody -0.8%, dispatch micro -0.7%; mean -0.7% over the 13-benchmark set.
Assisted-by: Claude Code:claude-fable-5-1
* common: an immortal bit in the refcount word
Atomic reference counting is about a third of what a bytecode costs now
that the dispatch loop is trimmed: built `--no-default-features`, where
`PyAtomic` is a `Cell`, the straight-line dispatch micro runs at
3.5 ns/instruction against 5.4. Every `TO_BOOL`/`POP_JUMP_IF_*` pair and
every `None` or small-int push pays a relaxed `ldadd` and a `Release`
`ldaddl` on an object that can never die.
PEP 683's answer is to say so in the word. A fourth flag bit, `IMMORTAL`,
takes a bit from the strong count (leaving 60 on a 64-bit target, 28 on a
32-bit one, both far past what a program reaches); `make_immortal` sets it
and parks the count at CPython's `_Py_IMMORTAL_REFCNT`, and `inc`,
`inc_by`, `dec` and `safe_inc` read the bit and return without touching
the counter. The parked count is what keeps the rest of the tree honest
without a special case: it is far too large for a `strong_count() == 1`
in-place-mutation fast path to fire on, and `start_gc_refs` already reads
a count that large as `GC_REACHABLE`, so an immortal object is a permanent
root that keeps its referents alive.
Two details pay for the branch the mortal path now runs, which cost 4% on
object-heavy benchmarks before they went in:
* `leak` sets `IMMORTAL` alongside `LEAKED`. An interned object could
already never reach zero, so `dec` no longer needs the test it made on
the result of the decrement: the immortal test at the top decides both.
* `inc`'s three exceptional cases — a destructed word, a count at the
ceiling, a count at zero — fold into one comparison against the masked
word, with the handling moved to a cold helper.
What is left is one instruction on `dec` and roughly nothing on `inc`, and
the mortal path measures at parity.
Assisted-by: Claude Code:claude-fable-5-1
* vm: make the singletons, small ints, interned strings and static types immortal
Everything the `Context` builds once and keeps — `None`, `True`, `False`,
`Ellipsis`, `NotImplemented`, the empty tuple, frozenset, bytes and str,
the -5..256 int cache and the interned one-character strings — is now
marked immortal, as is every static type object, which its `static_cell`
holds for the life of the process. So is every entry of the string pool:
`mark_intern` already stopped those refcounts from reaching zero, so
immortality frees no memory that would otherwise have been freed; it only
takes the atomic read-modify-write off every incref and decref of an
attribute name, a dict key or a docstring.
Straight-line dispatch micro (200 LOAD_FAST/STORE_FAST pairs over cached
ints, median of 5 interleaved runs): 5.39 -> 4.01 ns/instruction, -26%.
A four-thread `if flag:` loop, where the contention on the shared bool was
real: 2.36 s -> 2.14 s, -10%.
pyperformance bench functions run standalone, medians of 5 interleaved
rounds against a matched build: fannkuch -7.8%, nqueens -3.8%, chaos
-3.0%, raytrace +1.0%, go +1.2%, richards +4.2%, nine of them together
-2.9%. Per-benchmark deltas under about 4% sit inside this tree's
build-to-build ThinLTO layout noise, which reaches that size between two
builds of identical source.
Observable: `sys.getrefcount` of an immortal object reports 4294967295,
CPython's `_Py_IMMORTAL_REFCNT`, and stops moving — what
`test_sys.test_refcount` expects of `None`. `is_interned`, and the dict
pointer-equality key fast path behind it, keep answering from the separate
`LEAKED` bit.
Assisted-by: Claude Code:claude-fable-5-1
* codecs: fast-path utf-8 decode, skipping the codec registry round trip
bytes.decode('utf-8') / str(bytes, 'utf-8', ...) always went through the
generic codec registry: a per-call attribute lookup for
_is_text_encoding, then a full call into the pure-Python
encodings.utf_8.decode wrapper, which called back into the native
utf_8_decode and unpacked a 2-tuple result.
CPython's PyUnicode_Decode special-cases a handful of built-in encoding
names (utf-8 among them) and decodes them directly, bypassing the codec
registry entirely -- a codecs.register() override does not affect
str(b"...", "utf-8") there either. This adds the same fast path here:
recognized utf-8 aliases go straight to the native decoder.
Measured with a pure-Python `str(bytes, 'utf-8', 'surrogatepass')` loop:
~723ns -> ~401ns per call (1.8x). On the pyperformance unpickle
benchmark (which decodes short strings this way for every
SHORT_BINUNICODE opcode): ~14.4ms -> ~13.2ms per loop, an ~8.3% win.
Assisted-by: Claude Code:claude-fable-5-1
* codecs: fast-path utf-8/ascii/latin-1 encode and ascii/latin-1 decode
Extend the utf-8 decode fast path from 3fe684629 to the symmetric encode
direction (str.encode()/bytes(s, enc)), and add the same registry-skipping
fast path for ascii and latin-1 encode/decode, all of which are simple
native codecs.rs functions already. For utf-8 encode with no surrogates in
the string (the common case), encoding is just a copy of the string's
existing UTF-8 bytes, so that's the fastest path; strings containing
surrogates and non-strict error handlers still go through the native
encoder + ErrorsHandler, matching custom codecs.register_error handlers.
Measured (median of interleaved runs vs baseline, 2M iterations):
str.encode("utf-8"): 351ns -> 195ns (-44%)
str.encode(): 449ns -> 291ns (-35%)
bytes(s, "utf-8"): 542ns -> 382ns (-30%)
str.encode("ascii"): 436ns -> 302ns (-31%)
str.encode("latin-1"): 487ns -> 333ns (-32%)
bytes.decode("ascii"): 519ns -> 359ns (-31%)
bytes.decode("latin-1"):590ns -> 417ns (-29%)
bytes.decode("utf-8")/str(b,"utf-8"): unchanged (already fast-pathed)
Assisted-by: Claude Code:claude-fable-5-1
* vm: make the runtime safe for borrowed stack entries
`PyStackRef` has carried a borrow tag since the stack was unified with the
fastlocals, but only `LOAD_SMALL_INT` -- which borrows immortal `Context`
objects -- ever set it, so nothing in the runtime had to be right about
borrowed entries. Turning `LOAD_FAST_BORROW` into a real borrow makes three
places load-bearing, and this fixes them ahead of the switch.
`Traverse for PyStackRef` handed every entry to the tracer, borrowed ones
included. The cycle collector subtracts one from an object's `gc_refs` per
edge it is shown, so a borrowed entry -- which owns no strong count -- would
have pushed a live object's count to zero and freed it out from under the
frame. Borrowed entries are now skipped; nothing becomes unreachable, because
the fastlocals slot the entry borrows from is on the same traversal.
`park_after_yield_from` stands in for a `YIELD_VALUE` that never runs, so the
`promote_stack()` in that handler never ran either; it now promotes there too,
and the debug assertion after it stops being vacuous.
`STORE_FAST`, `DELETE_FAST` and the fused stores now call
`LocalsPlus::debug_audit_local_release`, which in debug builds panics if the
slot is about to drop the last count of an object a borrowed entry still
points at -- the one shape codegen's `optimize_load_fast` exists to prevent,
and the one that would be a use-after-free if the analysis or a modelled stack
effect were wrong.
`BORROW_LOCAL_LOADS` (off) is the switch for the borrow itself, and
`LOAD_FAST_BORROW`/`LOAD_FAST_BORROW_LOAD_FAST_BORROW` now go through
`push_local`, which is a plain clone while it is off. The invariant the switch
depends on is written out where `PyStackRef` is defined.
No measurable change: with the switch off the only behavioral difference is
the traversal fix, which no current bytecode can reach.
Assisted-by: Claude Code:claude-fable-5-1
* vm: read consumed stack entries in place instead of owning them
`pop_value()` turns whatever is on the stack into an owned `PyObjectRef`, so
an instruction that only reads its operand still pays a reference count round
trip -- and, once `LOAD_FAST_BORROW` starts pushing borrows, would pay an
atomic increment to promote a borrow it is about to throw away. This moves
every consumer that never needs ownership onto `pop_stackref()`, which hands
back the tagged entry and costs nothing for a borrow.
Discarding pops (62 of them, across the `LOAD_ATTR`, `TO_BOOL`, `BINARY_OP`,
`COMPARE_OP`, `CONTAINS_OP` and `CALL` specializations, which read their
operands through `top_value()`/`nth_value()` and then drop them) become
`pop_stackref()`. So do the read-only operands of `POP_TOP`, `POP_JUMP_IF_*`,
`TO_BOOL`, `BINARY_OP`, `COMPARE_OP`, `CONTAINS_OP`, `IS_OP`, `STORE_ATTR`,
`DELETE_ATTR`, `STORE_SUBSCR`, `DELETE_SUBSCR` and `UNPACK_SEQUENCE`.
Two protocol entry points had to move from an owned receiver to `&PyObject` to
make that reachable: `try_to_bool`, which never used the ownership it took,
and `rich_compare`, which forwards to the `&self` `_cmp` anyway. Both keep
working unchanged for `PyObjectRef` callers through deref.
`LOAD_ATTR`'s method specializations popped the receiver and pushed it
straight back below the bound function; they now move the entry across with
its tag intact, which is how CPython keeps a method call's receiver borrowed
from `LOAD_ATTR` all the way through `CALL`. The plain attribute read reads
through the receiver rather than owning it.
Neutral while `BORROW_LOCAL_LOADS` is off -- for an owned entry
`pop_stackref()` and `pop_value()` compile to the same thing.
Assisted-by: Claude Code:claude-fable-5-1
* vm: turn LOAD_FAST_BORROW into an actual borrow
`LOAD_FAST_BORROW` accounts for 21.5% of executed instructions on
nbody+richards+nqueens, and until now it did exactly what `LOAD_FAST` does:
an atomic increment on the way onto the stack and, from the instruction that
consumed it, an atomic release decrement on the way off. Neither is needed.
The fastlocals slot holds a count for as long as the entry is on the stack --
codegen's `optimize_load_fast` only rewrites a `LOAD_FAST` when it can prove
the entry is consumed before anything can release that slot -- so the entry
can be a borrow that owns nothing, which is what CPython 3.14 does with a
tagged `_PyStackRef`.
Flipping `BORROW_LOCAL_LOADS` on is the whole change; the two preceding
commits are what make it safe (cycle-collector traversal, yield-point
promotion, a debug audit of the invariant) and what make it pay (consumers
that read through the entry rather than promoting it back to owned).
Medians of 7 interleaved rounds, this machine, against the pre-series binary:
straight-line while loop 6.32 -> 6.01 ns/instruction (-5.0%)
x = a + b loop -6.3% if flag: loop -3.8%
attribute-read loop -5.2% method-call loop -3.3%
subscript loop -6.7% fib(27) -0.8%
nbody -2.6% fannkuch -1.7% nqueens +1.3% richards -3.0%
deltablue -2.3% go -3.0% raytrace -1.7% float -3.2%
chaos -2.6% spectral_norm -2.5% mean -2.1%
About half of that is the consumer rewrite alone (mean -1.0% with the switch
still off) and the rest is the borrow. nqueens is the one benchmark that goes
the other way, by less than the roughly 4% of build-to-build layout noise this
tree has, and it moves with the consumer rewrite rather than the switch.
Full suite green (418 files, 42,110 tests). A build with debug assertions on
runs the suite with no audit failure, and trips the audit exactly when
`LOAD_FAST` is forced to borrow in a shape codegen refuses to convert
(`f(x, (x := None))`). Adversarial scripts -- swap loops and reassignment
under `__del__` tracking, argument evaluation that deletes its sibling,
`a[a.pop()]`, closures over a rebound cell, generators dropped while
suspended, tracebacks whose `f_locals` are read after unwinding, `settrace`
writing `f_locals` mid-expression, a method receiver whose local is cleared
while the call is being built, `__add__`/`__lt__`/`__bool__` that clear the
local their operand came from, threads sharing objects through locals, and
`gc.collect()` from a finalizer -- all agree with CPython 3.13 under
`gc.set_threshold(1, 1, 1)`, in both builds.
Assisted-by: Claude Code:claude-fable-5-1
* vm: exact-dict fast path for DICT_MERGE and CALL_FUNCTION_EX kwargs
`g(**d)` on an exact dict compiled to BUILD_MAP+DICT_MERGE+CALL_FUNCTION_EX
and paid for two Python-level `keys()`/`__getitem__` round trips per key at
both the DICT_MERGE handler and collect_ex_args' iterate_mapping_keys, even
though the source is a plain dict. Add an exact-dict (downcast + type
check, so subclasses still go through the mapping protocol) fast path to
both sites that iterates entries natively via PyDict::items_vec, matching
CPython's PyDict_Check fast paths in PyDict_Merge / CALL_FUNCTION_EX.
Measured (1M calls, release build): g(**d) 1320ns -> 665ns, f(**d)
1470ns -> 900ns, c.m(**d) 1690ns -> 1135ns, f(x=1,**d) 1600ns -> 1035ns,
f(**d1,**d2) 1700ns -> 1013ns. bm_django_template ~7% faster,
bm_argparse (subparsers) ~7% faster.
Assisted-by: Claude Code:claude-fable-5-1
* vm: skip the recursion guard hashing exact ints inside a tuple
tuple.__hash__ hashed every element through PyObject::hash, which wraps
each call in with_recursion's native-stack-depth check. That guard exists
for elements whose hash can recurse into arbitrary Python code, but
int.__hash__ is a leaf computation - it can neither recurse nor call back
into Python - so the check is pure overhead for the overwhelmingly common
case of tuples of plain ints (e.g. namedtuple-shaped dict keys).
pyperformance's mdp benchmark builds huge tables keyed by tuples of deeply
nested namedtuples whose leaves are ints; profiling with `sample` showed
PyObject::hash/with_recursion/tuple's slot_hash dominating the CPU time.
Hashing exact-int elements directly via hash::hash_bigint cuts mdp's
wall time from ~6.79s to ~4.81s (median of 5 interleaved runs), about 29%
faster, with no change to hash values or recursion-guard behavior for any
other element type.
Assisted-by: Claude Code:claude-fable-5-1
* vm: wire slot descriptors from __slots__ into C-level type slots
Classes that put a dunder name (e.g. "__setitem__") into __slots__ get
a per-instance member descriptor for it, which is a legitimate way to
override a special method per-instance (chameleon's tal.RepeatDict does
exactly this). PyType::new_verbose ran init_slots() to compute
mp_ass_subscript/mp_subscript/etc. before these member descriptors were
added to the type's attribute namespace, so the slot lookup never saw
them and __setitem__/__getitem__ silently fell back to "does not
support item assignment".
Fix by calling update_slot for each dunder-shaped __slots__ member
right after it is installed on the type, mirroring what
type.__setattr__ already does for regular attribute updates.
Assisted-by: Claude Code:claude-fable-5-1
* frame: stop allocating cold data to close a generator
`clear_locals_and_stack`, which runs whenever a generator or coroutine
comes to an end, reached for the frame's lazily-allocated `FrameColdData`
through `cold()`. That allocates it on first access, so every finishing
generator paid a `Box<FrameColdData>` malloc and free just to find
`f_extra_locals`, `f_locals_cache` and `f_overwritten_fast_locals` all
empty -- cold data is only ever populated by tracing and frame
introspection. Ask for it with `cold_opt()` instead and skip the three
reads when it was never allocated.
Medians of interleaved A/B runs, ns per generator create+finish over a
plain call: `for _ in g(): pass` 265 -> 231 (-13%), a coroutine that
completes synchronously 642 -> 617 (-4%), `sum(x for x in range(3))`
1033 -> 997 (-3.5%).
Assisted-by: Claude Code:claude-fable-5-1
* function: bind exact args straight into a generator's frame
Calling a generator, coroutine or async generator function took the
general argument-binding path even when the call site was the simplest
possible one. `invoke_exact_args_slots` and the vectorcall fast path both
recognised the shape -- every parameter positional, filled exactly once
-- and then handed generator-like code back to `invoke()`, which built a
`FuncArgs` (a `Vec` plus an `IndexMap`) only for
`fill_locals_from_args_inner` to walk varargs packing, keyword matching
and defaults that this shape cannot reach.
Add `make_generator_exact_args`, the counterpart of
`prepare_exact_args_frame` for the one call shape it refuses: it builds
the heap frame the generator will own, moves the arguments straight into
its fastlocals and returns the generator. Both fast paths now use it. The
vectorcall one matters on its own for generator expressions, whose fresh
`MAKE_FUNCTION` on every evaluation keeps them out of the call-site
specialization that would otherwise catch them.
Medians of interleaved A/B runs, ns per create+finish over a plain call:
`for _ in g(): pass` 231 -> 218 (-6%), a synchronously-completing
coroutine 616 -> 598 (-3%), `sum(x for x in range(3))` 1007 -> 967 (-4%).
Assisted-by: Claude Code:claude-fable-5-1
* frame: drain a dead frame in place instead of moving it out
Three allocations and a large move on the path a finishing generator
takes to release its frame.
`Traverse::clear` opened with `self.iframe.get_mut().take()`, moving the
whole `InterpreterFrame` -- a couple of hundred bytes -- out of its slot
and onto the stack, only to drop it there. Drain it through `as_mut()`
and empty the slot afterwards.
`LocalsPlus::clear_into` collected the evaluation stack into a `Vec` that
existed only to be extended into the caller's; push into that one
directly. And `clear_stack_and_cells` built (and dropped) a `Vec` for a
stack it had not looked at -- a frame that reached its `return`, which is
every generator that simply finished, left nothing there. Look first.
Medians of 5 interleaved rounds, ns per create+finish: `for _ in g():
pass` 307 -> 297 (-3.4%), a synchronously-completing coroutine 600 -> 585
(-2.5%), `list(g())` 565 -> 558; a plain call is unchanged.
Assisted-by: Claude Code:claude-fable-5-1
* vm: don't raise a TypeError to ask an iterator for its length
`length_hint_opt` opened with `iter.length()`, whose whole job on a type
with no length slot is to build the `TypeError` that says so -- which
this caller then catches, recognises and discards. Every iterator and
every generator takes that path, and `list()` calls it on whatever it is
handed, so a `list(gen)` paid for an exception object per call.
`length_opt` answers the same question with `None`. CPython's
`PyObject_LengthHint` gates the call on the slots for the same reason.
Median of 5 interleaved rounds: `list(g())` x200k for a 1-yield
generator 556 -> 407 ns (-27%). pyperformance generators, coroutines,
comprehensions and async_tree are unchanged, as are nqueens, richards
and chaos.
Assisted-by: Claude Code:claude-fable-5-1
* gc: track a generator and its frame in one pass
Making a generator tracks two objects that are born together and die
together -- the generator and the frame it owns -- and each one took the
gen0 list lock and a thread-local read of the running interpreter for
itself. Give the pair one entry point: `PyGenerator`, `PyCoroutine` and
`PyAsyncGen` opt out of tracking at allocation, the way frames already
do, and `make_generator_or_coro` links both into gen0 under a single
lock, with one owner read and one collection check.
Medians of 5 interleaved rounds: pyperformance coroutines -2.5%,
generators -0.9%; `for _ in g(): pass` over a plain call 205 -> 200 ns.
Assisted-by: Claude Code:claude-fable-5-1
* dict_inner: fast-path exact-int equality in key_eq
Mirrors CPython's lookdict: two exact `int` keys now compare by BigInt
value directly instead of going through the generic rich-compare
dispatch (identical_or_equal -> bool_eq -> rich_compare_bool -> __eq__
slot lookup). Falls back to bool_eq whenever either side isn't exactly
`int` (subclasses may override __eq__), so behavior is unchanged there.
Measured (min-of-6 rounds, 2M-iteration micros, this tree's build has
~4% layout noise so min is used as the noise floor):
- dict_int_lookup (`d[i]`, int keys): 70.9ns -> 62.0ns (-12.5%)
- dict_str_lookup / dict_tuple_lookup / set_str_contains: unchanged
(within noise), confirming no regression on non-int keys.
pyperformance sanity checks (single round each, base vs candidate):
bm_mdp median ~5027ms/loop vs ~5003ms/loop (no regression, hashing
isn't the dominant cost there); bm_nqueens, bm_go, bm_deltablue,
bm_hexiom, bm_chaos all within ~1% (noise).
Hash/equality values cross-checked bit-identical against CPython 3.14
for 0, 1, -1, True, False, 2**61-1, 2**61, -(2**63), 2**100, -(2**100),
and int/float/bool/Decimal/Fraction(1) hash-equality invariants.
Also tried an exact-type fast dispatch in PyObject::hash (skipping the
slots.hash vtable call + with_recursion for exact int/str) - it only
moved hash_small_int/hash_large_int by ~2%, under the 3% bar, so it was
not committed.
Tests: `rustpython -m test -j4 test_hash test_int test_long test_tuple
test_dict test_set test_dictviews test_ordered_dict test_collections
test_fractions test_numeric_tower test_float test_bool test_str
test_userdict test_weakref` all pass (1796 run, 72 skipped). `cargo
test -p rustpython-vm --release -p rustpython-common` passes. clippy
clean on the touched lines.
Assisted-by: Claude Code:claude-fable-5-1
* vm: let the modulus type handle a three-argument pow
CPython's ternary dispatch offers the operation to the third operand's type
whenever its slot differs from the two already tried, including the case where
neither operand had one. The port required both of those slots to exist, so
`pow(10, 2, Decimal(7))` never reached `Decimal` and raised TypeError.
Assisted-by: Claude Code:claude-fable-5-1
* vm: add a feature-gated opcode-pair histogram
Counts `(prev_op, op)` in the dispatch loop and dumps the nonzero pairs at
exit, so superinstruction and specialization candidates can be ranked from
what pyperformance-shaped code actually dispatches rather than guessed at.
Entirely behind the non-default `opcode-histogram` feature: the module is
`#[cfg]`-ed out and the loop's two-line hook with it, so a default build is
byte-identical to one without the patch.
Assisted-by: Claude Code:claude-fable-5-1
* vm: let boolean producers run the branch that follows them
`TO_BOOL*`, `COMPARE_OP*`, `CONTAINS_OP*` and `IS_OP` materialised a bool
only for the `POP_JUMP_IF_*` one dispatch later to pop and test it, and the
compiler's `NOT_TAKEN` marker cost a third dispatch on the fall-through edge.
Generalise the `COMPARE_OP_INT`-only jump fusion so every boolean producer
resolves the jump itself, and step over `NOT_TAKEN` from there and from the
plain `POP_JUMP_IF_*` handlers. Both back off to ordinary dispatch when the
successor is an instrumented opcode or while tracing, so `sys.monitoring`
branch events and `sys.settrace` opcode events are unaffected.
Dispatches over 13 standalone pyperformance benchmarks: -5.0% total
(richards -11.7%, unpickle_pure_python -8.9%, sympy -8.4%, hexiom -8.1%,
django_template -7.1%, go -6.1%; nbody unchanged). Times, best of 7
interleaved rounds: deltablue -4.0%, hexiom -3.5%, go -3.4%,
unpickle_pure_python -3.3%, richards -3.2%, raytrace -3.0%, nqueens -1.9%,
geomean -1.0%; a branch-heavy microbenchmark -4.7% against a straight-line
control at 0%.
Assisted-by: Claude Code:claude-fable-5-1
* Adapt yield-from resume to optional generator frames
Coro.frame is now PyAtomicRef<Option<FrameObject>>, so the flattened
resume path uses frame_opt/clear_except_code instead of a direct frame.
Assisted-by: Grok:4.6
* Fix clippy and cr_origin after try_to_bool/f_lineno changes
try_to_bool now takes &self, so drop the leftover clones. Derive
cr_origin lines from lasti instead of prev_line, and satisfy
-Dwarnings on the immortal-refcount and del_needed types.
Assisted-by: Grok:4.6
* Convert fd filenames without the os ToPyObject impl
host_env-free builds (sandbox check, wasm) do not compile os.rs, so
OsPathOrFd::filename cannot use Borrowed's ToPyObject. Use as_raw()
instead, which is the same conversion.
Assisted-by: Grok:4.6
* Gate fd path helpers on the host_env feature
WASM clippy builds rustpython-vm without host_env, so OsPathOrFd and
the fspath/fd helpers are unused and fail -Dwarnings.
Assisted-by: Grok:4.6
* Drop redundant pickle try_to_bool clones after rebase
Assisted-by: Grok:4.6
* Document why enumerate keeps a Counter beside BigInt
Assisted-by: Grok:4.6
* Move fd path helpers into ospath_fd under host_env
Assisted-by: Grok:4.6
* Group threading eval-breaker bits in one module
Assisted-by: Grok:4.6
* Make InterpreterFrame construction unsafe
run_iframe stays safe: a live frame already has valid identity pointers.
Assisted-by: Grok:4.6
* Move OsPathOrFd re-export to the top of ospath.rs
Assisted-by: Grok:4.6
---------
Co-authored-by: Lee Dogeon <dev.moreal@gmail.com>
|
||
|
Jeong, YunWonandLee Dogeon
|
358f6a875a |
native _json, _pickle, pyexpat, and _elementtree (#8741)
* _json: add native make_encoder (c_make_encoder)
json.encoder fell back to the pure-Python _make_iterencode because _json
had no make_encoder, making json.dumps ~25x slower than CPython. Implement
the C encoder semantics natively: str/None/bool/int/float/list/tuple/dict
ordering, key coercion and skipkeys, sort_keys via the VM's fallible sort
(comparison errors propagate instead of panicking), circular reference
detection, indent handling, allow_nan, default() fallback with recursion
guard, and CPython's add_note annotations. Removes the now-passing
expectedFailure markers in test_json.
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: add UseForeignDTD, fix silently swallowed handler exceptions
genshi's XML template loader (bm_genshi) called xmlparser.UseForeignDTD(),
which pyexpat.rs didn't implement, raising AttributeError. Add it as an
accepted no-op: the pyexpat module here wraps the pure-Rust xml-rs crate
(not real libexpat), which has no hook for splicing in a foreign DTD.
Root-causing the resulting bug also uncovered why genshi's template
generate()/render() produced empty output instead of erroring: do_parse's
invoke_handler swallowed every exception raised by a Python callback
(handler.call(...).ok()), and genshi's expat callbacks fail immediately
by reading `parser.CurrentLineNumber`/`CurrentColumnNumber`, which
pyexpat.rs never exposed. The AttributeError was silently discarded,
so no start/end/text events ever reached genshi's queue.
Fix both: propagate handler exceptions out of Parse()/ParseFile() (matching
libexpat/CPython semantics), and add CurrentLineNumber/CurrentColumnNumber/
CurrentByteIndex as read-only attributes backed by xml-rs's per-event
TextPosition (byte index is derived from it and is exact for ASCII
documents). This also fixes test_minidom's
testExceptionOnSpacesInXMLNSValue, previously an expected failure because
the ValueError from an invalid xmlns value was swallowed the same way.
PositionTest in test_pyexpat.py checks byte-exact expat semantics (token
*start* position) that xml-rs's *current parse cursor* position can't
reproduce exactly, so that one test is now marked an expected failure.
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: support multi-chunk Parse()/ParseFile() feeds
bm_xml_etree failed with AssertionError: missing end tags from
TreeBuilder.close(), because xml.etree.ElementTree reads and feeds large
documents to XMLParser.feed() in ~64KB chunks (and xml.sax's
IncrementalParser and ElementTree's XMLPullParser do the same). Each
Parse()/ParseFile() call built a brand-new xml-rs EventReader from only
the bytes passed in that call, discarding all parser state between calls.
When a chunk boundary fell in the middle of an element, the fragment
parsed by itself was invalid/incomplete XML, so xml-rs silently errored
out (matching this module's existing "ignore stricter xml-rs errors"
behavior) and the corresponding start/end events were dropped entirely
-- not just delayed.
xml-rs has no API to resume a parser across separate Read sources, so
instead each call now appends the new chunk to a persistent buffer and
re-parses the buffer as a whole, replaying only the events at or beyond
the count already dispatched in previous calls (dispatched_events). This
also makes position tracking (CurrentLineNumber/Column/ByteIndex) correct
regardless of how the input was chunked. Trade-off: each Parse() call is
now O(size fed so far), so a document delivered in many small chunks is
parsed several times over; acceptable for correctness but noticeably
slower than a true incremental parser for pathological chunk sizes.
This also fixes several previously-xfailed tests that depend on
incremental feeding: test_xml_etree.py's XMLPullParserTest (test_events,
test_feed_while_iterating, test_simple_xml, test_simple_xml_chunk_22,
test_simple_xml_with_ns) and ElementTreeTest.test_parseliteral, plus
test_sax.py's ExpatReaderTest (test_expat_entityresolver_default,
test_expat_incremental, test_expat_incremental_reset) -- their
expectedFailure markers are removed. test_simple_xml_chunk_1/_chunk_5
(1- and 5-byte chunks) still fail and remain xfail: at that granularity
xml-rs's stricter-than-expat tokenizer errors on some intermediate
prefixes that libexpat tolerates.
Assisted-by: Claude Code:claude-fable-5-1
* test: drop stale expectedFailure markers fixed by the pyexpat changes
test_plistlib's four invalid-document tests and test_xmlrpc's
test_loads_unsupported pass now that pyexpat no longer swallows
handler exceptions and keeps parser state across chunked feeds.
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: drive a single background EventReader for true incremental Parse()
8802451f3 fixed multi-chunk Parse()/ParseFile() feeds by accumulating every
fed byte and re-parsing the whole buffer from scratch on each call, which is
O(total bytes) per chunk -- quadratic in document size. xml-rs's EventReader
has no API to pause/resume across separate Read sources, so instead one
EventReader is now driven to completion on a dedicated background thread fed
through a condvar-backed queue (FeedBuffer/FeedReader); Parse()/ParseFile()
push their chunk, wake the thread, and drain (on the calling thread only)
whatever events that unblocks, stopping once the thread reports it is
blocked waiting for more input. Handlers always run on the calling thread.
A handler exception stops dispatch without tearing down the stream, so the
next Parse() resumes where it left off, mirroring libexpat. CurrentByteIndex
is now computed incrementally per line (with an amortized per-query cursor,
needed since real documents are often a single very long line) instead of
scanning the whole buffer. ParserCreate() never spawns a thread; one is
spawned lazily by the first Parse()/ParseFile() call, and dropping the
parser always stops and joins it.
Fixes test_xml_etree{,_c}'s test_events_comment, previously xfailed because
the old whole-buffer replay lost top-level comment events across feeds.
Measured (M4 Pro, release build):
- Scaling (ElementTree.parse, newline-separated doc): new backend 1/4/16 MB
= 0.15/0.61/2.35s (linear); CPython 3.14 same file = 0.015/0.055/0.23s.
- pyperformance's bm_xml_etree doc (156KB, effectively one line): old
backend single etree.parse() = 1.99s, single iterparse() = 3.53s; new
backend = 0.025s / 0.031s (~80-110x). Full bench_parse/iterparse/generate/
process (30/10/10/1 reps) now finish in ~1.5s/0.6s/0.4s/0.3s instead of
timing out past pyperformance's 400s limit.
- 10k ParserCreate() (never fed / partially fed / fully fed and dropped):
0.32s, no residual native threads, no RSS growth.
test -j4 test_pyexpat test_xml_etree test_xml_etree_c test_sax test_minidom
test_xml_dom_minicompat test_pulldom test_plistlib test_xmlrpc test_html
test_htmlparser test_threading: all pass (1350 tests, 89 skipped).
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: add a Sync fallback backend for targets without threads
2f58e9531 made pyexpat always drive its EventReader on a background
thread (ParserStream::spawn), panicking via .expect() if the thread
couldn't be spawned. pyexpat is registered unconditionally, including
on wasm32, where std::thread::spawn always fails at runtime, so the
first Parse() call panicked there.
Introduce a Backend enum: Backend::Threaded (the existing ParserStream)
on targets that can spawn threads, and a new Backend::Sync, the
accumulate-and-reparse implementation from 8802451f3, used on wasm32
and as a runtime fallback if spawning a thread ever fails. Both share
LineTracker for identical CurrentLineNumber/CurrentColumnNumber/
CurrentByteIndex and handler/error semantics.
Fixed two correctness bugs while porting the Sync backend forward:
marking the parser permanently "finished" as soon as a from-scratch
reparse happened to reach EndDocument (which can be a false positive
on a not-yet-complete buffer, e.g. right before epilog content
arrives) rather than only on the caller's real isfinal signal; and
counting that same EndDocument towards the dispatched-event index,
which could make a real event land on an index a later, longer reparse
had already skipped past. Verified by force-selecting Backend::Sync
under a test-only env var (removed) and diffing its output against
Backend::Threaded for the xml/sax/etree/plistlib/xmlrpc test suites
and a 2MB document fed in 1KB chunks.
Assisted-by: Claude Code:claude-fable-5-1
* _pickle: native Pickler, Unpickler and PickleBuffer
Adds `crates/stdlib/src/pickle.rs`, a native `_pickle` implementing
Pickler, Unpickler, PickleBuffer, PicklerMemoProxy/UnpicklerMemoProxy,
dump/dumps/load/loads and the PickleError family, so `Lib/pickle.py`
stops falling back to `_Pickler`/`_Unpickler`.
Protocols 0-5 are supported and produce byte-identical output to the
pure-Python pickler, including framing for protocol >= 4, memoization,
`fast` mode, dispatch_table, reducer_override, persistent_id /
persistent_load, find_class with `_compat_pickle` fix_imports, buffer
callbacks and out-of-band protocol 5 buffers. This also turns on
`pickle.PickleBuffer`, which RustPython did not have before.
Speedups over the pure-Python implementation (best of 5, us/op):
pickle 835 -> 21 (39x), pickle_list 525 -> 4.4 (119x),
pickle_dict 4018 -> 32 (125x), unpickle 630 -> 24 (26x),
unpickle_list 317 -> 5.0 (63x). CPython 3.14's C accelerator runs the
same workloads 1.5-2.9x faster than this.
test_pickle now runs 1008 tests (was 470) since the C-implementation
test classes are enabled; test_pickle, test_pickletools,
test_picklebuffer are green and 22 obsolete expectedFailure markers are
removed.
Assisted-by: Claude Code:claude-fable-5-1
* _pickle: raise the pickle.find_class audit event
Unpickler.find_class now raises `sys.audit('pickle.find_class', module,
name)` like CPython, so audit hooks can veto global lookups
(test_audit.test_pickle). Also drops four expectedFailure markers that
the native module makes pass: test_pydoc's builtin-method description
tests, test_memoryview.test_picklebuffer_reference_loop and
test_pyclbr.test_others.
Full suite: 456 test files / 42,648 tests green.
Assisted-by: Claude Code:claude-fable-5-1
* _pickle: avoid temporary buffers on the save fast paths
Opcode plus payload is now appended straight to the output buffer
(`write_pair`) instead of building a `Vec` per int, str, bytes and memo
opcode, `save_tuple` no longer copies the element slice, and the
`when serializing ...` note text is only formatted when an error is
actually raised.
bm_pickle "pickle" workload: 21.4 -> 17.0 us/op.
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: make the threaded backend fork-safe
fork() only keeps the calling thread. test_platform's
test_mac_ver_with_fork crashed the child with SIGTRAP: plistlib parses
a plist through pyexpat (spawning a background parser thread) before
os.fork(), then parses again in the child. Bisecting away every other
explanation (a plain threading.Thread survives the same fork fine,
since it goes through this VM's own fork-safe thread bookkeeping)
showed the crash is not from touching the *inherited* ParserStream
(already joined by refcounting before the fork in this repro) but from
spawning a *brand new* parser thread from the child, which reliably
crashed during the new thread's own startup.
Register a pthread_atfork child hook (lazily, the first time a parser
thread is spawned) that sets a permanent, process-wide flag the
instant this process is ever the child of a fork(). Once set:
- ParserStream::try_spawn refuses to spawn any further thread and
Backend::new falls back to Backend::Sync instead, exactly as on
targets that cannot spawn threads at all.
- Feeding an inherited Backend::Threaded raises a clear ExpatError
instead of touching its FeedBuffer/channel (whose mutex/condvar may
look held by a thread that no longer exists), and dropping it leaks
its JoinHandle/buffer/channel via ManuallyDrop rather than joining or
destroying them, since even destroying a POSIX mutex that looks held
is undefined behavior.
The parent is unaffected: the pthread_atfork hook only runs in the
child. Verified with a 200-iteration fork stress test (fresh parsers
and parsers forked mid-parse) and the full test_platform/xml/fork/
subprocess/multiprocessing_fork test modules.
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: implement name interning via self.intern, matching libexpat
self.intern (an always-present dict by default) was stored but never
consulted; every StartElement/EndElement/attribute name allocated a
fresh PyStr even when the same tag or attribute name repeated across
many sibling events. Wire it up the way libexpat does
(PyDict_SetDefault(self->intern, name, name)): element and attribute
*names* are now memoized so a repeated name reuses the same PyStr
object (and its cached hash) instead of reallocating and rehashing.
Attribute values are left out of the cache, same as libexpat.
This fixes test_pyexpat's previously-xfailed InterningTest.test, which
asserts object identity across repeated StartElementHandler/
EndElementHandler calls.
Measured on the pyperformance xml_etree parse/iterparse/generate/
process variants: no measurable wall-clock change (the dominant cost
there is Python-level TreeBuilder/Element bytecode execution and
thread hand-off, not name allocation), so this is a correctness/
feature-completion fix rather than a throughput win for that
benchmark; kept because it is a real, previously-inert compatibility
gap and is otherwise free.
Assisted-by: Claude Code:claude-fable-5-1
* _elementtree: native Element, SubElement and ParseError
Adds crates/stdlib/src/elementtree.rs providing the accelerator names
xml/etree/ElementTree.py picks up at import time. Element carries the
full API (tag/text/tail/attrib, indexing and slicing, find*/iter/
itertext, copy/deepcopy/pickle state) with CPython's lazily created
attrib dict and deferred text join, is subclassable, weak-referenceable
and GC-traversable, and drops every displaced object after releasing its
lock so a __del__ can re-enter safely.
xml_etree, best of 3 (s): parse 1.64 -> 1.14, iterparse 0.68 -> 0.54,
generate 0.44 -> 0.25, process 0.27 -> 0.16.
Assisted-by: Claude Code:claude-fable-5-1
* _elementtree: native TreeBuilder and XMLParser
TreeBuilder keeps CPython's element/data/tail bookkeeping in Rust,
including the pull-parser event hooks _setevents installs. XMLParser
drives a pyexpat parser whose handlers are this type's own native
methods, so a start or end tag reaches the builder without a Python
frame: the universal-name cache, attribute dict and TreeBuilder call all
happen in Rust when the target is our own builder, while foreign targets
still go through the documented start/end/data/comment/pi protocol.
ElementTree.parse() also picks up the _parse_whole fast path.
xml_etree, best of 3 (s): parse 1.14 -> 0.53, iterparse 0.54 -> 0.28,
generate 0.25 -> 0.24, process 0.16 -> 0.16. Against the pure-Python
baseline that is parse 3.1x, iterparse 2.4x, generate 1.8x, process
1.7x.
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: stop rescanning the whole line per position query
LineTracker converted a TextPosition column to a byte offset by
validating and decoding its entire buffered line tail on every query,
and appended fed bytes to that tail one at a time. On a document that is
one long line -- what xml.etree.tostring() writes, and what the
xml_etree benchmark parses -- the tail is the whole read-ahead, so a
per-event lookup became quadratic. Walk only the characters the query
actually asks for, over the deque's two halves in place, and record fed
bytes a line at a time. Byte indices are unchanged.
xml_etree, best of 3 (s): parse 0.52 -> 0.38, iterparse 0.28 -> 0.25.
Assisted-by: Claude Code:claude-fable-5-1
* _elementtree: charge Element.__deepcopy__ to the recursion budget
A chain of elements is copied by native recursion, so a deep enough tree
overflowed the real stack instead of raising. Wrap the copy in
with_recursion, as CPython does, and drop the skip on
test_deeply_nested_deepcopy for the accelerated run.
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: don't let a stale NeedMore end a Parse() early
The parser thread reports NeedMore when it finds the feed buffer empty
and parks. If it reached that point before the chunk Parse() was about
to push became visible -- which it does whenever the thread happens to
be scheduled first, so more often under load -- pump() consumed that
report as if it answered the new chunk and handed control back to Python
having dispatched nothing, silently losing the whole document.
Stamp each NeedMore with the push count observed when the buffer was
found empty and ignore reports older than the push being pumped. The
push notifies the condvar under the same mutex the report was sent
from, so the parser thread is guaranteed to wake and report again.
Fixes an intermittent failure of test_pyexpat's InterningTest under
parallel load (about 5 losses per 20000 parses before, none after).
Assisted-by: Claude Code:claude-fable-5-1
* Restore xml.etree test markers instead of skipTest in bodies
Assisted-by: Grok:4.6
* Clone state before try_to_bool in pickle load_build
try_to_bool takes the object by value, and load_build still needs the
state dict afterwards.
Assisted-by: Grok:4.6
* Drop decorative section-separator comments
Assisted-by: Grok:4.6
* Represent pickle opcodes as an enum
Assisted-by: Grok:4.6
* Use full paths for thread-only types in pyexpat
Assisted-by: Grok:4.6
---------
Co-authored-by: Lee Dogeon <dev.moreal@gmail.com>
|
||
|
Jeong, YunWon
|
300d8bdc02 |
Honor inline values, custom mro, and type lookup (#8729)
* Add inline values flags and special-method lookup lookup_maybe_method clears AttributeError from descr_get. slot_tp_richcompare treats a miss as NotImplemented. hackcheck rejects object.__setattr__ on types. Heap types with a managed dict and itemsize 0 get INLINE_VALUES; _testinternalcapi.has_inline_values reports the valid bit. super is a base type; subclass __get__ binds the instance. __classcell__/__qualname__ may share a name with a slot. Unset member access uses the type's fully qualified name. Assisted-by: Grok:grok-4.6 * Call custom mro and descr_get for type.__new__ type.mro() computes C3 from the current bases. A metaclass other than type supplies the MRO via mro(). super() compares obj.__class__ to the instance type. slot_tp_new looks up __new__ through descr_get. is_subtype walks the stored MRO without an identity shortcut. __slots__ names are sorted so __class__ assignment matches. Assisted-by: Grok:grok-4.6 * Expose _testinternalcapi recursion and rare events get_recursion_depth feeds test.support.infinite_recursion. SPECIALIZATION_* and SHARED_KEYS_MAX_SIZE match the adaptive counters. Rare-event hooks increment on __class__/__bases__/function/builtins writes. Assisted-by: Grok:grok-4.6 * Add remaining _testinternalcapi code and locale APIs Expose get_co_localskinds, get_code_var_counts, verify_stateless_code, code_returns_only_none, normalize_path, EncodeLocaleEx, DecodeLocaleEx, and run_in_subinterp_with_config. Mark unimplemented compiler IR and specialization monitoring tests as expectedFailure. Assisted-by: Grok:grok-4.6 * Fix lint for _testinternalcapi identifiers Add CPython identifier spellings to the cspell dictionary and apply the rustfmt signature that CI rewrote. Assisted-by: Grok:grok-4.6 * Expand _testinternalcapi with remaining test APIs Add constants, compiler_cleandoc, dict_getitem_knownhash, time helpers, interpreter/frame accessors, configs, static types, a persistent hamt map, and GC/size counters used by the test suite. Assisted-by: Grok:grok-4.6 * Expose compiler IR APIs in _testinternalcapi Add InstructionSequence and compiler_codegen, optimize_cfg, and assemble_code_object. Drop expectedFailure marks on tests that pass. Assisted-by: Grok:grok-4.6 * Match optimize_cfg consts and addop locations Write optimized consts back into the caller's list, mark SETUP_* pseudo-ops as having an argument, and store addop locations in instruction-sequence field order. Drop expectedFailure marks on tests that pass. Assisted-by: Grok:grok-4.6 * Fix hamt API, SETUP_* HAS_ARG, and FOR_ITER events Honor explicit pseudo flags when generating opcode metadata so SETUP_FINALLY/CLEANUP/WITH keep HAS_ARG after regeneration. Complete the _testinternalcapi hamt mapping: identity on set/delete, richcompare, weakrefs, subscript, contains, and length-bearing iterators. Fire RAISE/STOP_ITERATION for FOR_ITER StopIteration, and skip both END_FOR and POP_ITER for the exhausted-branch destination. Assisted-by: Grok:grok-4.6 * Skip FOR_ITER RAISE for sequence iterators List/range/tuple exhaustion matches FOR_ITER_LIST/RANGE/TUPLE and must not emit a monitoring RAISE. That event stopped pdb's monitoring backend on for-over-list with Internal StopIteration. Assisted-by: Grok:grok-4.6 * Fix generator FOR_ITER events and review issues Restore settrace StopIteration for generator for-loops so test_bdb next/return cases pass, while sequence iterators still skip RAISE. Honor review notes: addop oparg bounds, linetable locations, AST instance errors, MRO subclass lock, builtin-dict notify, direct-base layout flags, and leftover _testinternalcapi args. Assisted-by: Grok:grok-4.6 |
||
|
Jeong, YunWon
|
850fadd388 |
Classify continuable unclosed strings as incomplete (#8694)
* Classify continuable unclosed strings as incomplete Mark EOF-open strings for PyCF_ALLOW_INCOMPLETE_INPUT, treat blank eval input as incomplete, detect unparenthesized lambda after a comma in f-strings, and advance parameter scans past default values. Assisted-by: Grok:4.6 * Diagnose incompatible prefixes and nested f-strings Report incompatible string prefixes, malformed \\N escapes, and f/t-string nesting beyond 150. Omit the extra newline on unterminated triple-quoted f-string SyntaxError.text. Assisted-by: Grok:4.6 * Fix empty defaults and longer incompatible prefixes Report an empty parameter default before later parameters, collect the full prefix before a quote, and prefer the u/b pair. Assisted-by: Grok:4.6 |
||
|
Jeong, YunWon
|
699913d8cd |
Align codec, call, exception, and Unix module behavior with CPython (#8647)
* Collect consecutive charmap encoding failures Pass one contiguous run of unmappable characters to the codec error handler, and use the full run when a text replacement is itself unmappable. Assisted-by: Codex:gpt-5 * Leave conversions out of standalone format specs A conversion belongs to a replacement field rather than the format specification consumed by format(), so stop accepting and storing it in FormatSpec. Assisted-by: Codex:gpt-5 * Decode empty byte strings without looking up a codec Match PyUnicode_FromEncodedObject by returning the shared empty string before resolving the requested encoding for empty bytes and bytearray inputs. Assisted-by: Codex:gpt-5 * Validate call shape before converting builtin arguments Reject unknown keywords and excess arguments before conversions can raise a different error. Align Python-function positional-only handling, use declared list arguments in heapq, keep delegated codecs positional-only, and preserve CPython parameter names and validation order. Assisted-by: Codex:gpt-5 * Align attribute and exception initialization errors Give __dict__ and type __qualname__ their CPython diagnostics, preserve OSError filename fields correctly, accept seven-item SyntaxError details, and report Unicode error constructor arity before converting values. Assisted-by: Codex:gpt-5 * Expose missing Unix and Darwin module APIs Add the platform constants exported by fcntl, mmap, select, socket, syslog, termios, stat, errno, time, and _sre. Implement posix.killpg and resource.getpagesize, and expose resource.error as OSError. Assisted-by: Codex:gpt-5 * Honor the configured Python CPU count Parse -X cpu_count and PYTHON_CPU_COUNT, expose the stored value through sys._get_cpu_count_config, and let os.cpu_count prefer it over the host count. Also publish getsizeof with its CPython parameter name. Assisted-by: Codex:gpt-5 * Fix CPU count and WASI CI coverage Remove the obsolete expected-failure marker now that CPU count overrides pass, and avoid exporting FASYNC where WASI libc has no O_ASYNC constant. Assisted-by: Codex:gpt-5 * self review |
||
|
OkJa
|
93f151691c |
interpolated strings: align f-string and t-string diagnostics with CPython (#8601)
* builtins: name the other operand with tp_name in str and Template __add__
`str.__add__` and `Template.__add__` built their TypeError with
`PyType::name()`, which drops the module, where CPython formats `tp_name`:
>>> t"a" + "b"
TypeError: can only concatenate Template (not 'str') to Template
CPython 3.14 reports the qualified name and quotes the operand with double
quotes, as `Objects/unicodeobject.c` and `Objects/templateobject.c` do:
TypeError: can only concatenate string.templatelib.Template
(not "str") to string.templatelib.Template
`Template.__add__` also spelled its own name literally rather than taking it
from `PyClassDef::TP_NAME`, so the two halves of the message could drift apart.
Known gap: `slot_name()` still does not qualify a class defined in a module, so
`"a" + collections.OrderedDict()` names `OrderedDict` where CPython names
`collections.OrderedDict`. This affects both methods.
Unblocks test_template_concatenation in test_tstring.
Assisted-by: Claude Code:claude-opus-5
* interpolated strings: align f-string and t-string diagnostics with CPython
RustPython re-derives CPython's syntax diagnostics by scanning the source rather
than by translating ruff's parse errors, and the scanner shared by f-strings and
t-strings misread several replacement field states:
- A field that ran off the end of the literal (`f'{'`) was reported as an
expression that failed to start, where CPython asks for the closing brace.
- A field whose expression cannot start (`f'{;'`) and one whose expression runs
into a stray character (`f'{a;'`) both fell through to the missing brace
message instead of CPython's two distinct ones.
- A conversion or format spec following `=` was never validated, and a format
spec following a valid conversion was skipped entirely.
- Format specs were scanned with the `{{` escape rule that only applies to
literal text, so nested replacement fields inside them went unchecked.
- Brackets opened inside a field were never matched, so `f'{a[4)}'` and
`f'{3)+(4}'` fell back to a bare "invalid syntax".
- Comments inside a field were not recognised, so a `#` that swallows the
closing brace was reported as an unterminated literal, and a comment in a
multi-line field hid the expression behind it.
- Unterminated literals were named "string" whatever their prefix, and an
interpolated literal left with a replacement field open reported the missing
quote rather than the brace.
A field is only "never closed" while the tokenizer is still reading its
expression. Past the field's own `:` it is emitting literal text again, so
`f'{a:>5` runs out of input as an ordinary unterminated literal while `f'{a`
reports the brace. The field's own `:` is also not the one in a slice, a display
or a lambda, and an unclosed bracket inside the expression is what CPython names
rather than the field around it, so this walk tracks the whole delimiter stack.
Two messages were worded from the wrong branch of CPython's tokenizer. The hint
"perhaps you escaped the end quote?" is raised only where lexer.c handles a
literal without an interpolation prefix; its `%c-string` branch has just the
triple-quoted and plain forms, so pairing the hint with a prefix produced
`unterminated f-string literal (...); perhaps you escaped the end quote?`, which
CPython never emits. And a bracket mismatch inside a field dropped the
` on line %d` clause that lexer.c adds whenever `parenlinenostack[level]` differs
from the current `lineno`; the general bracket scanner in this file already
computed that suffix, so `f"""{a[\n4)}"""` now names line 1 as CPython does.
The rules these now follow are the ones CPython spells out in
Grammar/python.gram (`invalid_fstring_replacement_field` and its t-string twin)
and in Parser/lexer/lexer.c, which likewise parameterises the literal's prefix
rather than duplicating the messages.
Only a field's expression is code. A format spec is text, so a `(` there opens
nothing and a `#` selects the alternate form instead of starting a comment;
inside the expression a `#` does start one. Scanning the whole field for either
therefore reported diagnostics against valid literals, and because these
scanners only run once the source has already failed to parse, that let a good
literal take the blame for an error further down the file. Expression-level
checks now stop at the field's top-level separator and skip comments, and the
literal is walked field by field rather than byte by byte so that a `{` inside a
comment no longer opens one. A pre-existing instance of the same bug in the
line-continuation check is fixed along with them.
Ruff reports a mixed literal concatenation only as a bytes/non-bytes mix, so
`t"x" b"y"` arrived as a bytes error where CPython names the t-string. CPython's
`invalid_string_tstring_concat` is an `invalid_` rule, reached only on the error
pass, and `strings` tries `(fstring|string)+` ahead of it: that alternative
consumes the concatenation's leading run of non-t-string literals, so a mix among
those raises from `_PyPegen_concatenate_strings` on the first pass and sets
`error_indicator`, which suppresses the t-string rule. The t-string message
therefore wins for `t"x" b"y"` and the bytes message keeps precedence for
`"a" b"b" t"c"`. Both are now settled here rather than left to whichever scanner
runs next, which had been answering the second case with an unrelated
"Is this intended to be part of the string?".
A leading doubled `=` or `!` was read as a marker with an empty expression before it,
so `f"{==a}"` reported `valid expression required before '='` where CPython has no
expression to report at all. The same lookahead now guards that branch, and a
doubled `=` or `!` counts as an expression that cannot start.
A top-level `=` or `!` was read as a debug or conversion marker without checking
whether it belonged to a longer operator, so `f"{a==b}"` and `f"{a!=b}"` were also
read as fields ending early. CPython tokenises `==`, `!=`, `<=` and the rest as
single tokens before it ever considers the debug marker, so the separator scan now
skips a `=` that follows one of `= ! < > + - * / % & | ^ @ :` or precedes another
`=`, and a `!` that precedes one.
An expression cannot end on an operator that still wants an operand, and CPython
points at that operator rather than at the brace. `f"{a==}"`, `f"{a and}"` and
`f"{a.b.}"` fell through to a plain `invalid syntax`; they now carry the same
message and column CPython gives, reusing the scan that already handled `;` and
`$` for exactly this shape. `is not` and `not in` are single operators, so the
first word is what gets pointed at, and `...` is consumed in whole triples so
that `f"{....}"` blames the fourth dot rather than the first. The region has to
be walked forwards: a comment's terminating newline is whitespace, so trimming
backwards from the end would step into the comment body and read a triple-quoted
field whose comment ends in `+` as an expression ending in `+`.
That check runs only once the field is known to close, which is where a stray
character and a dangling operator part ways. A stray character is a finished
token, so the parser rejects it on the lookahead in
`annotated_rhs !('='|'!'|':'|'}')` and `f"{a;"` gets the separator message even
with no closing brace. A dangling operator instead makes the parser ask for one
more token, and producing it runs into the literal's closing quote, where
lexer.c answers from its `INSIDE_FSTRING(tok)` branch with
`%c-string: expecting '}'` before any `invalid_` rule is reached. So `f"{a and"`
is a missing brace while `f"{a and}"` names the operator.
A leading `.` was read as a character that cannot start an expression, so the
Ellipsis literal `f"{...}"`, the float `f"{.5}"` and its signed form `f"{-.5}"`
were reported as broken whenever the file failed to parse somewhere else.
`pegen` joins the cpython spelling dictionary, for the `_PyPegen_*` names these
comments cite.
Known gaps, none covered by either test file. Needing the longest valid expression
prefix, which a character scanner cannot compute: a starred tuple (`f"{*a,}"`) and
a dict-unpacking display are read as unable to start an expression, and
`f"{a===b}"`, `f"{a b}"` and `f"{a,,}"` fall through to a bare `invalid syntax`.
The caret still differs from CPython's on `expecting a valid expression after '{'`,
`expecting '}'`, `expecting '}', or format specs` and `unterminated ... literal`,
and on the messages CPython points at a whole token for, such as
`f"{lambda x: x}"` and `f"{x! r}"`. A quote inside a format spec is still treated
as a string delimiter.
Two more are reachable only through a t-string concatenation that ruff does not
report as a bytes mix, so `mixed_tstring_literal_error` never runs: three or more
literals with no bytes literal among them (`t"a" t"b" "c"`, `"a" "b" t"c"`) answer
with `invalid syntax. Is this intended to be part of the string?`, and so does a
tokenizer-level nesting overflow where CPython has
`too many nested f-strings or t-strings`. Two-literal mixes are correct because
ruff's own error carries the wording.
Two pre-existing bugs outside this change are worth naming, since it touches
their neighbourhood. `unterminated triple-quoted ... literal` reports
`detected at line 2` for a one-line source where CPython reports line 1, for every
prefix including none, so it is in the shared line arithmetic rather than the
interpolated path. And `str.__add__`'s message is unreachable for an operand that
defines `__radd__`: `"a" + 1` falls through to the generic
`unsupported operand type(s)` from the binop dispatch, where CPython has
`can only concatenate str (not "int") to str`.
Unblocks test_syntax_errors and test_literal_concatenation in test_tstring, and
test_comments,
test_conversions, test_invalid_syntax_error_message, test_mismatched_braces,
test_mismatched_parens, test_parens_in_expressions and
test_syntax_error_after_debug in test_fstring.
Assisted-by: Claude Code:claude-opus-5
* diagnostics: give the source scanners a diagnostic type
Every scanner in `cpython_parse_diagnostic_override` returned a bare
`Option<(String, usize, usize)>` — 69 signatures and 114 construction sites of
message, start offset, end offset — which the consumer then reassembled into a
message and a range one call later. They return a `CpythonDiagnostic` now, and
the two things that consume one, `NormalizedParseDiagnostic::other` and
`CompileError::from_source_error`, take it whole.
The type earns its name. These scanners run *after* ruff's parse has failed, so
what they produce is not a parse error but a reconstruction of what CPython
would have said about the same source; nothing here ever hands one back to ruff.
Reusing `ruff_python_parser::ParseError` would have fit the shape — it is
`{ ParseErrorType, TextRange }` — but every one of the 114 sites would have
wrapped its message in the single `OtherError` variant of a ninety-variant enum,
widened `other` and `from_source_error` to accept variants no caller
constructs, and, being a foreign type, ruled out the constructor that now
carries the `u32` cast and its justification. The `OtherError` wrapping happens
once per consumer instead, at the boundary where this does cross into ruff's
vocabulary.
Two nameless tuples that carried more than a diagnostic get names, following
`CallArgFrame` and `AssignmentContext` in this file:
- `bracket_syntax_error` returned the message alongside a bare `bool` for
whether the bracket was left open, which the caller needs apart from the
message because ruff reports that case as an EOF error. It returns a
`BracketError` now.
- `unclosed_replacement_field_error` walked its delimiter stack as
`Vec<(usize, u8, bool)>`, where the third element decided whether the field
had reached its format spec — `.2 = true` and `Some((_, b'{', false))` at the
use sites. `OpenDelimiter` names all three.
The `source_error!` macro is untouched at the 34 call sites it already had and
drops to five lines. Two copies of its body had been written out by hand
(`invalid_number_literal_error`, `unterminated_string_error`) and fold into it,
bringing it to 36.
One thing does change beyond the packaging. `TextRange::new` asserts that its
start does not exceed its end, and it asserts unconditionally. Building the
range at the scanner rather than at the consumer therefore puts that assert in
front of the hundred-odd scanners that reach `NormalizedParseDiagnostic::other`,
which never built a range before — they handed their two offsets straight to
`source_locations`. A scanner that emitted a reversed span used to produce a
garbled location; now it aborts. Every site was read for this and each clamp is
anchored to a bound at or after its start, and neither two hundred thousand
generated sources nor eleven thousand mutations of stdlib files reached it, so
this is a latent invariant made loud rather than a new failure — but it is not
packaging, so it is written down here.
Otherwise a pure refactor: no diagnostic changes message, column or line.
Verified by building the parent commit and diffing its output against this one
over a malformed-source corpus and over every `Lib/**/*.py` that parses clean
(1728 files, each recompiled with a trailing `$` so a scanner that misreads
valid code shows up as a moved line) — byte-identical, and the count of files
where the reported line moves is 1149 on both. test_fstring, test_tstring,
test_string_literals, test_syntax, test_exceptions, test_grammar and
test_compile all pass.
Addresses the review on #8601.
Assisted-by: Claude Code:claude-opus-5
* diagnostics: name the operator-character set behind the debug marker
`is_replacement_field_marker` carried its list of operator characters inline, a
sixteen-line `matches!` wrapped around three lines of logic, and it looked very
much like `is_dangling_operator_byte` a few functions down — enough that a
reviewer asked whether they were the same set.
They are not, and neither contains the other. Written in the same order the
difference is three characters:
precedes_equals_in_one_operator ! % & * + - / : < = > @ ^ |
is_dangling_operator_byte ! % & * + - . / < = > @ ^ | ~
The two are derived from different things. One asks which bytes pair with a
following `=` to make a single token, so that the `=` is part of that operator
rather than the start of a debug specifier — read off Python's token table, and
`.=` and `~=` are not on it, while `:=` is. The other asks which bytes make up
an operator that still wants an operand, which is why `~a` and `a.b.` put `.`
and `~` there and why the separator `:` stays out.
So the list moves next to the one it resembles, under a name, both sorted the
same way, and each says what the other has that it does not. Neither is defined
in terms of the other: the thirteen characters they share are two answers that
happen to coincide, not a set with a meaning of its own, and a common list would
tie each to the other's reasons to change.
No behavior change: the extracted list is character for character what was
inline. Verified against a build of the parent over a malformed-source corpus
and every `Lib/**/*.py` that parses clean — byte-identical — and with
test_fstring, test_tstring, test_string_literals, test_syntax, test_exceptions,
test_grammar and test_compile.
Addresses the review on #8601.
Assisted-by: Claude Code:claude-opus-5
* diagnostics: give the unterminated-literal scan one exit
A single-quoted literal that meets a newline and one that runs off the end of
the source are the same failure, and CPython's lexer says so in one condition:
if (c == EOF || (quote_size == 1 && c == '\n')) {
Everything after that — the check for a replacement field left open, and the
choice of message — is written once there. This scan had grown a second exit
for the newline case with its own copy of both, so the guard added for
interpolated literals had to be added twice and the message assembled twice.
The newline case now breaks out of the scan and falls into the exit that was
already handling end of source. `line` cannot have moved from `start_line` on
that path, and `quote_size` is 1, so the surviving exit computes the same
`detected_line` and the same `triple` the deleted one did.
No behavior change: verified against a build of the parent over a
malformed-source corpus and every `Lib/**/*.py` that parses clean —
byte-identical — and with test_fstring, test_tstring, test_string_literals,
test_syntax, test_exceptions, test_grammar and test_compile.
Assisted-by: Claude Code:claude-opus-5
* diagnostics: classify an unterminated triple-quoted t-string as incomplete input
`analyze_compile_error` pairs the f-string and t-string error variants
everywhere it names a message, but the match that decides whether a failure is
a `SyntaxError` or an `IncompleteInputError` listed only the f-string one. So a
`t'''` typed at the prompt would not have asked for another line where an
`f'''` would.
Would not have, rather than does not: the arm is unreachable today. The
compiler's `unterminated_string_error` scanner claims these first and hands
back an `OtherError`, so `f'''` and a bare `'''` get a `SyntaxError` at the
prompt too, on `main` as much as here — which is why
`test_codeop.test_incomplete` carries an `expectedFailure`. Whoever untangles
that precedence should not then find that f-strings work and t-strings
silently do not.
CPython does not separate the two either: `lexer.c` emits one message
parameterised with `%c` for the prefix and sets `E_EOFS` the same way for both,
and `_is_end_of_source` (`Parser/pegen.c`) looks only at that code, never at
which kind of literal produced it.
No behavior change, and none is testable while the arm cannot be reached.
Assisted-by: Claude Code:claude-opus-5
|
||
|
Jeong, YunWon
|
322f294a21 |
common: add VM-independent codec and text engines (#8622)
* common: add PyPy CJK codec engines
Assisted-by: Codex:GPT-5
* common: mark CJK mapping tables generated
Assisted-by: Codex:GPT-5
* common: add reusable byte and text engines
Assisted-by: Codex:GPT-5
* common: record opt-in engine dependencies
Assisted-by: Codex:GPT-5
* stdlib: add `_multibytecodec` and the six `_codecs_*` modules
Wire the CJK codec engines in `rustpython-common` up to the VM. `_multibytecodec`
holds `MultibyteCodec`, `MultibyteIncrementalEncoder`, `MultibyteIncrementalDecoder`,
`MultibyteStreamReader` and `MultibyteStreamWriter`, the encode/decode drivers and
the error-handler protocol; each `_codecs_*` module exposes `getcodec` for the
encodings of its region.
`rustpython-stdlib` now enables the `cjk-codecs` feature of `rustpython-common`.
Add `cjk::decode_reset`, which the decoder `reset()` methods need. It differs from
building a fresh decoder state: ISO 2022 keeps its G1..G3 designations and the
escape-throughout flag, and resets only G0 and the shifted flag.
Drop the markers from the CJK codec tests and from the `test_codecs`, `test_io`,
`test_email`, `test_zipfile` and `test_traceback` cases that needed a CJK codec.
`test_codecs.BasicUnicodeTest.{test_basics,test_decoder_state}` stay expected
failures for an unrelated reason: `codecs.charmap_encode` rejects a `None` mapping.
Assisted-by: Claude Code:claude-opus-5
* stdlib: use the byte and text engines from `rustpython-common`
`binascii`, `_json` and `_socket`'s address converters now call the engines in
`rustpython-common` instead of carrying their own copies, and `rustpython-stdlib`
enables the `binascii`, `inet` and `json` features for them. That leaves
`base64` and `crc32fast` with no caller in `rustpython-stdlib`, and
`crates/stdlib/src/json/machinery.rs`, the last of the json_in_type-derived code,
goes away with its last caller.
The address converters change behaviour. `inet_aton` reads the lenient forms
(`127.1`, `16909060`, `0x7f.1`, `0177.0.0.1`), `inet_pton` rejects an octet with a
redundant leading zero and a group of five hex digits, `inet_ntop` writes the
IPv4-compatible form as `::c000:201`, and an address family with no converter
raises `EAFNOSUPPORT` instead of a bare `OSError`.
Give the JSON engine the closing-quote scan and the allocation-free `\uXXXX`
escape it needs to carry `_json`'s workload. Against the code it replaces,
`json.dumps` runs 2.3x faster with `ensure_ascii=True` and 1.7x without it, and
`json.loads` 1.2x.
`binascii.a2b_base64` now reports the messages `binascii.c` does: no
`error decoding base64: ` prefix, and the data-character count in parentheses.
Assisted-by: Claude Code:claude-opus-5
* test: enable the Windows cp932 codec case
The shared CJK engine now satisfies the Windows code-page adapter, so keeping this test marked as an expected failure turns its success into a CI failure.
Assisted-by: Codex:GPT-5
|
||
|
Jeong, YunWon
|
7c6c78f955 |
Implement CPython 3.14 subinterpreter APIs (#8605)
* Add _interpreters and _interpchannels modules
Apply the patch baseline for the low-level subinterpreter modules:
- crates/vm/src/stdlib/_interpreters.rs: create/destroy/list_all/get_main/
get_current/is_running/exec/call/run_string/run_func/set___main___attrs/
is_shareable/whence and the Interpreter*Error exceptions
- crates/vm/src/stdlib/_interpchannels.rs: channel create/destroy/send/recv/
list_all/list_interpreters/release/close and the Channel*Error exceptions
- crates/vm/src/vm/crossinterp.rs: cross-interpreter data protocol
- crates/vm/src/vm/{interpreter,mod,runtime,vm_new}.rs: interpreter registry
and lifecycle plumbing the modules need
- Lib/concurrent/interpreters/: high-level PEP 734 package
Assisted-by: Claude:claude-opus-5
* Correct _interpreters and _interpchannels against CPython 3.14
Argument matching:
- Add `function::ArgSpec`, a `PyArg_ParseTupleAndKeywords` equivalent that
applies the `|`, `$` and `:` markers of a format string to a `kwlist`, and
route every module function of both modules through it. This enforces
keyword-only parameters, rejects unexpected keywords, accepts keyword forms
that were previously positional-only (`whence(id=)`, `get_config(id=)`,
`set___main___attrs(id=, updates=)`, `capture_exception(exc=)`,
`_register_end_types(send=, recv=)`), and reproduces the arity messages.
- `O!` slots (`shared`, `updates`, `call`'s `args`/`kwargs`) now reject None,
and `_PyArg_BadArgument` renders None as "None" rather than "NoneType".
- `new_config` takes at most one positional `str`; `list_all` of
`_interpchannels` is argument-less.
- Convert arguments in `kwlist` order, so an argument's own error is raised
ahead of the checks that follow it.
_interpreters:
- Add the `CrossInterpreterBufferView` type and build the received memoryview
on it, keeping the sending interpreter's exporter out of the destination.
- Build `excinfo.errdisplay` from `traceback.TracebackException`.
Feature flags:
- `os.fork` checks finalization before `allow_fork`, matching `os_fork_impl`.
- `_thread.start_new_thread` and `start_joinable_thread` reject an interpreter
without `allow_threads`.
Lib/concurrent/interpreters/_crossinterp.py: restore the stripped docstrings
and comments.
Assisted-by: Claude:claude-opus-5
* Correct the stateless checks and error shapes of _interpreters against CPython 3.14
verify_stateless_function follows _PyFunction_VerifyStateless: it rejects
non-dict builtins, a non-empty __defaults__, __kwdefaults__ or __closure__,
and code whose LOAD_GLOBAL names are held by the function's globals or
absent from its builtins.
code_returns_only_none follows _PyCode_ReturnsOnlyNone: generator,
coroutine and async-generator code is rejected up front, the instruction
walk skips inline caches and maps specialized and instrumented opcodes
back, and the LOAD_CONST preceding each RETURN_VALUE is compared against
the index of None in co_consts.
ExcInfo::capture keeps an empty exception message instead of dropping it,
and builds `formatted` as `module.qualname: msg`, leaving out the builtins
and __main__ modules.
_interpreters.call packs the callable through _PyFunction_GetXIData before
pickle and re-raises the stateless-check failure when both fail. A func,
args or kwargs that cannot be rebuilt in the target, and a result without
cross-interpreter data, now raise NotShareableError with the snapshot as
the cause rather than being returned as excinfo.
_interpreters.capture_exception() without an argument returns None.
pickle_loads and _PyFunction_GetXIData attach the underlying failure as
__cause__.
channel_send converts the object to cross-interpreter data after the
channel's `closing` check.
ChannelID rich comparison returns the result of comparing its id with the
other number instead of coercing that result to bool.
PyMemoryView_FromObjectAndFlags raises "memoryview: a bytes-like object is
required, not 'X'" for an object that is not a buffer.
Assisted-by: Claude:claude-opus-5
* Correct cross-interpreter data conversion and channel argument parsing
Bound shareable ints by isize, raising OverflowError("try sending as
bytes") outside that range and keeping it as the NotShareableError cause.
Report an unshareable object by its repr, and attach a failing conversion
as __cause__ of the NotShareableError raised for the object it was called
for, so each level of a nested tuple appears in the chain.
Guard each tuple item conversion with with_recursion("while sharing a
tuple").
Gate the memoryview getdata function on `_interpreters` having been
imported, and route channel_send_buffer through a memoryview so it uses
that function and the resolved fallback.
Create the cross-interpreter exception types from the module_exec of
either module that raises them, instead of only `_interpreters`.
parse_cid raises OverflowError("int too big to convert"); int_arg
reproduces the `i` converter's three overflow messages and is also used
while parsing channel_send arguments.
Add BASETYPE to _queue.Empty, and gate nt.execv/nt.execve on allow_exec.
Assisted-by: Claude:claude-opus-5
* Correct interpreter ID parsing and the channel_send defaults lookup
_PyInterpreterState_ObjectToID accepts any object with __index__, raises
OverflowError("int too big to convert") outside the int64 range, and
reports a negative ID with the repr of the original object.
channelsmod_send reads the channel's default unboundop and fallback only
when one of the two arguments is negative, so an explicit pair is
validated before the channel is looked up.
Assisted-by: Claude:claude-opus-5
* Add the _interpqueues module and concurrent.interpreters._queues
_interpqueues holds a process-wide queue table and exposes create,
destroy, list_all, put, get, bind, release, get_maxsize,
get_queue_defaults, is_full, get_count and _register_heap_types, raising
QueueError and QueueNotFoundError. QueueEmpty and QueueFull, which
subclass queue.Empty and queue.Full, are registered per interpreter by
concurrent.interpreters._queues.
Cross-interpreter data carries a queue as a refcounted queue ID that
binds when the data is captured and releases when it is dropped, and
is_shareable reports a registered Queue as shareable.
Queue and channel items owned by an interpreter are now cleared from
Interpreter::finalize, after module finalization, rather than from
destroy_owned_interpreter, so values sent by an atexit callback also
become unbound.
is_shareable gates memoryview on the same registration flag as the
getdata lookup.
Assisted-by: Claude:claude-opus-5
* Add the high-level concurrent.interpreters API
Interpreter, ExecutionFailed and the queue aliases, copied verbatim from
CPython 3.14.
Assisted-by: Claude:claude-opus-5
* Destroy leftover subinterpreters when the main interpreter finalizes
Interpreter::finalize now runs finalize_subinterpreters between the atexit
handlers and the finalizing flag: when the main interpreter still owns
subinterpreters it emits the RuntimeWarning "remaining subinterpreters;
close them with Interpreter.close()" and then finalizes each of them.
runtime::owned_interpreter_ids lists the runtime-owned interpreters.
Assisted-by: Claude:claude-opus-5
* Absolutize the script path in run_file
_Py_abspath and _PyPathConfig_ComputeSysPath0 arrive as abs_path and
script_sys_path0. run_file hands the absolutized path to get_importer
and to the runner, so __file__ and co_filename no longer stay relative,
and inserts the symlink-resolved script directory as sys.path[0]
instead of the argument's parent.
test_cmd_line_script.test_script_abspath no longer expects a failure.
Assisted-by: Claude:claude-opus-5
* Name types in error messages the way tp_name does
The object, number, sequence and mapping protocols, the
unsupported-operand helpers, and the type-specific messages fed from
them now read PyType::slot_name() where they read PyType::name(),
so a type declared with a module keeps it.
Message texts corrected along the way:
- PySequence_Size, PyMapping_Size, PySequence_GetItem and
PySequence_SetItem/DelItem split their merged wording in two, and
PyObject_SetItem/DelItem say "object".
- PyObject_GenericSetAttr reports a read-only attribute and the
missing __dict__.
- check_class drops the type name it appended.
- __int__ and __index__ drop the class prefix; __float__ keeps it.
- float(), complex(), round(), next(), str subscripting and list
subscripting take their own wording.
- str.join and bytes.join number the item they turn down and name what
they wanted, and report a non-iterable as "can only join an
iterable".
test_class.testObjectAttributeAccessErrorMessages,
string_tests.test_subscript and the test_descrtut doctest no longer
expect a failure.
Assisted-by: Claude:claude-opus-5
* Gate script_code's source compilation on the compiler feature
crossinterp::script_code called vm.compile unconditionally, which only
exists with the rustpython-compiler feature. Builds without it, such as
example_projects/barebone, now raise a TypeError instead of failing to
compile.
Assisted-by: Claude:claude-opus-5
* Skip test_pyinit_function_raises_exception
The test imports _testsinglephase in its body without a decorator
guarding it, and now that _interpreters exists requires_subinterpreters
no longer skips it.
Assisted-by: Claude:claude-opus-5
|
||
|
Jeong, YunWon
|
4fab5c966e |
Raise where the fuzzer found aborts and stack overflows, and collect with the count in the object header (#8551)
* Correct what the non-unix frame stack is for
The comment said non-unix threading builds have no stop-the-world.
CollectStopTheWorld is gated on `feature = "threading"` alone, and
sys._current_frames stops the world on both paths; the field is the
fallback a reader uses when there is no `top_iframe` to materialize from.
Assisted-by: Claude
* Seek by the whole file position on Windows
`SetFilePointer` answers with the low half of the new position and signals
failure with INVALID_SET_FILE_POINTER, which is also that half of a
position four gigabytes in; telling them apart takes the error code, which
this did not read, so such a seek was reported as an error. Deciding
seekability from it also called such a file unseekable.
`SetFilePointerEx` returns the whole position and a success flag of its
own, which also removes the transmute of the position into halves.
Assisted-by: Claude
* memoryview: tell a value of the wrong kind from one that does not fit
Assigning an item reported every packing failure as a TypeError, so
m[0] = 300 on a 'B' view said the value was the wrong type rather than out
of range. Packing now says which of the two it was, and whether the value's
own code raised, in which case that error is the answer as it is:
m[0] = 300 ValueError: invalid value for format 'B'
m[0] = "x" TypeError: invalid type for format 'B'
m[0] = <__index__ that raises> the raised error
`struct` reports both as `struct.error` and is unchanged; the kind travels
beside the exception for the caller that tells them apart.
Also: None was read as a deletion, so m[0] = None answered "cannot delete
memory" instead of packing it; and deleting through the mapping protocol
never reached the read-only check, which comes first.
Assisted-by: Claude
* Raise MemoryError for allocations sized by Python input
os.read, _RawIOBase.read, int.to_bytes, struct.pack, ctypes array
creation and the _ssl RAND functions sized a Vec from a Python-supplied
length with vec![], which calls handle_alloc_error and, under
panic = "abort", ends the process. They now allocate through
vm.new_zeroed_bytes and raise MemoryError.
itertools.product built its pools without checking that
len(iterables) * repeat is representable; it now raises OverflowError
"repeat argument too large" and reserves the pool and index vectors
fallibly. repeat is read as isize, so a negative one raises ValueError
"repeat argument cannot be negative" instead of the conversion's message.
_ssl.RAND_bytes and RAND_pseudo_bytes read n as i32, matching the int
they are declared with.
Assisted-by: Claude
* Collect with the count in the object header
A collection kept the count it was working with in a table keyed by the
object's address, and the objects it had proved reachable in a second one.
Between them they were hashed once per candidate and twice per edge in the
heap, which is where most of a collection over a live heap went.
The count now lives in `PyInner::gc_refs`, with `GcBits::COLLECTING` saying
it is meaningful, and reachability is `gc_refs == GC_REACHABLE` rather than
membership in a set. Step 5 splits the candidates and clears the bit in one
pass. gcbench, five interleaved pairs, median: a live heap of 423k objects
goes from 0.101s to 0.055s and a dead one from 0.402s to 0.383s.
The bits, generation, owner and count take eight bytes between them. A
64-bit header had those eight as the padding its alignment forces, so it is
unchanged at 48 bytes; a 32-bit header grows from 24 to 28.
Assisted-by: Claude
* Take an explicit thread stack size as a floor in debug builds
A debug build already started Python threads on 8 MiB rather than Rust's
2 MiB default, but an explicit threading.stack_size(N) went through
verbatim. test_threading asks for 256 KiB, and starting a thread on that
walked off the end of the stack: the guard page fault landed in the
prologue of ExecutingFrame::run.
Unoptimized, that prologue reserves 80,848 bytes where the optimized one
reserves 656 -- execute_instruction is #[inline(always)] and LLVM only
colors stack slots from opt-level 1, so the frame is the sum of all 200
instruction arms' temporaries rather than the largest. A Python call costs
88,672 bytes of native stack there, and threading's bootstrap is six
frames deep, so 256 KiB holds less than half of what starting a thread
takes.
The floor reaches thread::Builder only; threading.stack_size() still
answers with what was asked for, and release builds are unchanged.
Assisted-by: Claude
* Check the native stack on every frame entry
The C-stack guard ran on one frame entry in eight. That asks the margin to
cover eight frames rather than one, and it does not: an unoptimized frame
entered through native code takes 88,672 bytes against a debug margin of
262,144. A recursion whose steps re-enter that way -- `__add__` calling
itself, a sort key that sorts -- ran off the end of the stack instead of
raising RecursionError. On a debug build `class Add: __add__ = lambda s, o:
s + o; Add() + 1` segfaulted on the main thread; it now raises, as it does
under CPython and in release builds.
enter_iframe checked and then called enter_iframe_unchecked, which checked
again; it now leaves the check to the one call.
Measured on a call-dominated benchmark, five interleaved pairs: instructions
retired go up 0.17%, about four per call, which is the stack pointer read
and the compare.
Assisted-by: Claude
* Take an iterable's length hint when a list is filled from it
`map_py_iter` read `__length_hint__` only to pass it to `PyIterIter`, and
returned an empty vector when the hint was `isize::MAX` or more. Collecting
through `PyResult` dropped the iterator's lower bound, so nothing reserved
the room the hint asked for.
`list()`, `list.extend()` and `list.__iadd__()` now reserve it and report a
hint they cannot honour as `MemoryError`; a hint that leaves no room for the
elements the list already holds is passed over, as `list_extend()` does.
`tuple()` and the other callers keep filling up without reserving.
`length_hint_opt` errors other than the `TypeError` it already turns into
`None` now reach the caller instead of being dropped.
`__iadd__` and `inplace_concat` went through `extract_cloned`, which reads
`__len__` and not `__length_hint__`; both call `PyList::extend` now, the way
`list_inplace_concat()` calls `list_extend()`.
The tuple, list and dict fast paths of `extract_elements_inner` reserve their
known length, which the `collect()` they used dropped.
Assisted-by: Claude
* Drop map.__length_hint__
`map_methods` has no `__length_hint__`, so `operator.length_hint()` on a map
answers 0, not the length of what it draws from.
The method walked into the length hint of every iterator it holds, and a
chain of maps 10000 long overflowed the native stack answering for the
outermost one. It also took the longest of its iterators, where a map stops
at the shortest.
Assisted-by: Claude
* Ask for a length hint where each caller asks for it
`map_py_iter` asked the iterable it was handed, for every caller, and reported
what asking raised. Only some callers ask it: `list_extend()` and
`_PyBytes_FromIterator()` ask the iterable, `PySequence_Tuple()` asks the
iterator, and the bytearray constructor asks nothing. `tuple()`, `min()`,
`max()`, `collections.deque()` and `f(*x)` raised for an iterable whose
`__len__` or `__length_hint__` does, where they answer.
Which object is asked is now the caller's to say. `sorted()` asks the iterable,
being `PySequence_List()`.
`bytes_from_object()` stood in for `PyBytes_FromObject()`, for
`bytearray_extend()` and for the bytearray constructor, which do not agree on
this: the first two ask, the last does not. It is split, and assigning to a
bytearray slice takes the constructor's side with `PyByteArray_FromObject()`.
`list.extend()` counted what it held before the iterable had been asked, where
`list_extend()` reads `Py_SIZE(self)` after. A `__length_hint__` that adds to
the list made the overflow guard read a count too small and raise `MemoryError`
where nothing is wrong; one that empties it made the guard skip a reservation
that cannot be served.
Assisted-by: Claude
* Settle product's pool count before it reads its arguments
`product_new()` checks `repeat` and works out `npools` before it calls
`PySequence_Tuple()` on any argument, and fills the pools `npools` times.
The pools were filled by repeating the arguments `repeat` times instead, which
walks that many steps even with no arguments to repeat: `product(repeat=2**62)`
counted up to it rather than answering `[()]`. The count was also worked out
after the arguments had been read, so a repeat too large to serve ran their
code first.
Assisted-by: Claude
* Let the bool format answer with the error its value raised
`pack_single()` leaves `'?'` to `PyObject_IsTrue()` and returns what that
raised. Packing classified the error instead, so a `ValueError` from a
`__bool__` came back as "memoryview: invalid value for format '?'".
Assisted-by: Claude
* Release a cell's old value after the lock
`PyCell::set` dropped what it replaced while still holding the mutex guarding
the cell contents. A `__del__` running from that drop and reading the same cell
waited on a lock its own caller held, so `del it` on a closure variable whose
value has such a `__del__` deadlocked.
The replaced value is now released once the guard is gone, as `Py_XSETREF`
stores before it decrefs.
Assisted-by: Claude
* Have a set iterator hold the set it iterates
The iterator kept only a reference to the inner hash table, so in
`it = iter(A(*args))` the `A()` temporary was the last owner and died as the
call returned, before `it` was bound. A `__del__` reading `it` there saw an
unbound name and its error was printed and ignored.
The iterator now holds the set object, as `si_set` does, and releases it once
exhausted. That release happens after the lock is dropped, where
`setiter_iternext()` puts its `Py_DECREF(so)` past `Py_END_CRITICAL_SECTION()`,
so a `__del__` that iterates again does not wait on a lock the call holds.
Assisted-by: Claude
* Release an exhausted iterator's container after the lock
`PositionIterInternal::_next` overwrote its `IterStatus::Active` while the
caller still held the mutex around it. Dropping the container there ran any
`__del__` under that lock, and a `__del__` that iterated the same object again
blocked on it.
`exhaust()` now hands the container back instead of dropping it, and
`locked_step()` releases it after the guard. list, list_reverseiterator, tuple,
str, dict and its views and reverse views, bytes, bytearray, memoryview,
array, deque, and the enumerate and sequence iterators all go through it.
Assisted-by: Claude
* Unskip test_free_after_iterating
Assisted-by: Claude
* Stop listing test_set as an environment polluter
`check_free_after_iterating` no longer leaves an ignored exception behind, and
the job that reruns the listed tests ten times fails once one of them stops
polluting.
Assisted-by: Claude
* Drop winsound imports left unused
`TryFromBorrowedObject`, `exceptions`, and `ToWideString` have no reference in
the module, which fails the Windows clippy line under `-Dwarnings`.
Assisted-by: Claude
* Ask nothing where the caller takes no room
`map_py_iter` asked the iterator for a length hint on behalf of every caller
that does not reserve, and reported what asking raised. Those callers ask
nothing at all: `tuple()`, `f(*x)`, `bytearray(x)`, `min()`, `max()` and
`collections.deque()` answer for an iterator whose `__length_hint__` raises,
where they had been raising it.
The answer was also never spent. It reached `PyIterIter` for a `size_hint()`
the push loop does not read, so the lookup and any call it made were work
thrown away: `tuple()` over a generator drops 22% of its instructions, and 17%
over an iterator with a `__length_hint__` written in Python.
Assisted-by: Claude
* Stop asking an iterator how long it is to walk it
`PyIter::iter` and `PyIter::into_iter` asked for a length hint and reported
what asking raised. Nothing spent the answer: it reached `PyIterIter` for a
`size_hint()` that every caller either loops past or drops, since collecting
into a `Result` reports no lower bound.
23 operations answered for an iterator whose `__length_hint__` raises, where
they had been raising it: `set`, `frozenset` and the nine `set` methods that
take an iterable, `dict.fromkeys` and the dict view operators, `array` and
`array.extend`, `all`, `any`, `sum`, `io.writelines`, `math.fsum`,
`math.prod`, and `csv.writerow` and `writerows`.
Over a generator, `set()` drops 16% of its instructions and `all()` 23%.
`str.join` and `bytes.join` do ask, reaching their elements through
`PySequence_Fast()`, which fills a list from the iterator. They take
`iter_sized()`, which is now the only way to ask. `iter_without_hint` is gone,
its callers being what `iter` already does.
Assisted-by: Claude
* Name the matrix jobs rather than let the matrix name them
A generated job name lists every value in the matrix entry, so `cargo check`
carried the booleans its dependencies and `skip_ssl` keys expand to, and the
snippets job carried its test arguments and timeout. Adding or removing a key
renames the check, which drops it from the required list until that list is
edited to match. Emptying `env_polluting_tests` renamed three checks this way.
`Run rust tests` and `clippy` keep the names they had. `cargo check` drops the
booleans from six of its nine, and the snippets job drops its arguments and
timeout from all three.
Assisted-by: Claude
* Keep a sequence iterator active when an element raises
`PositionIterInternal::_next` exhausted the iterator for any non-`Return`
result, so an error from `__getitem__` ended the walk. `iter_iternext()`
lets go of its sequence for `IndexError` and `StopIteration` alone, and
`PyIterReturn::from_getitem_result` has already turned the first of those
into the second, so only `StopIteration` exhausts now.
Both deque iterators keep exhausting on their own mutation guard, which
`deque_iternext()` does by zeroing the counter before it raises; they
share one step function for it, and the message it raises is lowercased
to match the three other sites in the module.
Assisted-by: Claude
* Name the caller in the bytes conversion TypeError
An object with no iteration protocol reached `PyObject_GetIter`'s "not
iterable" message, and `bytearray.extend()` reported bytes. Each entry
point now checks for the protocol first and names itself:
bytes(object()) cannot convert 'object' object to bytes
bytearray(object()) cannot convert 'object' object to bytearray
bytearray().extend(object()) can't extend bytearray with object
Assisted-by: Claude
* Take a strong count too large to hold as reachable
`start_gc_refs` clipped a count at `GC_REACHABLE - 1`, a number the
per-reference subtraction could still walk down to zero and collect a
live object. It now stores `GC_REACHABLE`, and `subtract_gc_ref` leaves
that value alone.
Assisted-by: Claude
* Raise AssertionError where a snippet asserts False
`assert False` is removed under `-O`, which the snippet suite may run.
Assisted-by: Claude
* Keep raising for a collection that moved under its iterator
A deque, set or dict iterator raised once and then read as spent. The
guard is sticky: `deque_iternext()` looks at the deque's state before the
count it keeps, and `dictiter_iternextkey()` and `setiter_iternext()`
write a size no collection can have, so every later call finds the same
thing and raises again. `dequereviter_next()` is the one exception,
looking at its count first, so it runs out after the first raise.
Both deque iterators now carry `dequeiterobject.counter` rather than
reading a length back from the deque, and the dict and set iterators
compare the size they captured against the collection's own every time
they are asked how much is left, which is what makes that answer nothing
from the moment the collection changes rather than only once the
iterator has raised.
The set's message is capitalized to match `setiter_iternext()`.
Measured against 3.14.6, for each of deque, reversed deque, set, dict
and a reversed dict view: the hint after the change, the error, the hint
after the error, and what a later call answers.
Assisted-by: Claude
* Give apt-get update a deadline to fail on
The step waits on `apt-get update`, which has no deadline of its own, so
a source that takes the connection and then stops answering holds the
job until the workflow's own timeout. The retry that disables the
Microsoft and azure-cli sources runs only when the update exits
non-zero, which a held connection never does; three jobs on this branch
sat on this step for 20 minutes to five and a half hours.
Each attempt is now bounded, and the transports are given a timeout, so
a source that stops answering reaches the retry.
Assisted-by: Claude
|
||
|
Jeong, YunWon
|
cdf006cfa1 |
Split InterpreterFrame from FrameObject for stack-allocated execution (#8354)
* Add LightFrame for unobserved Python-to-Python calls
Allocate a lightweight LightFrame header on the DataStack instead of a
full Frame PyObject for specialized exact-args call paths. The frame is
materialized lazily only when observed (sys._getframe, traceback, tracing).
Key changes:
- LightFrame struct with borrowed pointers and no refcount overhead
- FrameSource enum (Heavy/Light) replacing the object field on ExecutingFrame
- CURRENT_LIGHT_FRAME TLS for the light frame chain
- Interleaved heavy/light frame walking in frame_at_offset_vm and
current_thread_frame_vm
- invoke_light_slots on PyFunction for the fast call path
Call overhead reduced from ~92ns to ~58ns (~37% reduction).
fib(28) improved from ~152ms to ~111ms (~27% reduction).
Assisted-by: Claude
* Streamline with_frame: inline recursion check, skip tracing dispatch
- Inline check_recursive_call instead of wrapping in with_recursion closure
- Amortize C stack overflow check to every 64th call depth
- Skip dispatch_traced_frame when use_tracing is false (hot path)
- Add recursion_depth_increment/decrement helpers
Assisted-by: Claude
* Address review findings for LightFrame correctness
- Fix materialize_light_frame previous pointer: use payload pointer
(&**frame) instead of Py<Frame> pointer, matching the chain convention
- Fix predecessor ref management: use retained_back instead of
mem::forget to keep materialized predecessors alive
- Fix materialized ref reclaim: use FrameRef::from_raw to drop the
leaked keeper ref without double-incrementing
- Use PyAtomic<u32> for LightFrame.lasti instead of AtomicU32, removing
the unsound cast on non-threading targets
- Add amortized C stack overflow check to light frame path
- Add scopeguard to light frame execution for panic safety
- Remove unnecessary Send/Sync impls from LightFrame
- Make check_c_stack_overflow pub(crate) for light frame path access
Assisted-by: Claude
* Fix CI failures: cspell, stale materialized state, f_back chain gaps
- Fix cspell: 'amortised' → 'amortized'
- Sync lasti, prev_line, and fastlocals when re-observing a materialized
light frame, fixing stale f_locals/f_lineno in test_inspect and others
- In with_frame, materialize any active light frame as the heavy frame's
predecessor and store it in retained_back, so f_back and inspect.stack
see the correct interleaved order (H_new → L_mat → H_old)
- Guard invoke_light_slots on NEWLOCALS | OPTIMIZED code flags
Assisted-by: Claude
* Unify frame chain: replace dual-chain with single FrameChainPtr
Replace the dual-chain frame management (CURRENT_FRAME AtomicPtr<Frame>
+ CURRENT_LIGHT_FRAME Cell<*const LightFrame>) with a single unified
chain using FrameChainPtr, a tagged pointer where bit 0 distinguishes
heavy (*const Frame) from light (*const LightFrame) entries.
Changes:
- Add FrameChainPtr type with tagged pointer encoding
- Change LightFrame: replace previous_light + saved_current_frame with
single previous: FrameChainPtr field
- Change InterpreterFrame.previous from AtomicPtr<Frame> to PyAtomic<usize>
- Change CURRENT_FRAME TLS from AtomicPtr<Frame> to AtomicUsize
- Remove CURRENT_LIGHT_FRAME TLS entirely
- Simplify all chain walkers (frame_at_offset, frame_at_offset_vm,
find_owned_chain_frame, for_each_current_frame, current_thread_frame)
- Remove light-frame materialization block from with_frame
- Update f_back to dispatch on FrameChainPtr tags
- Update faulthandler and gc_state to walk unified chain
- Adaptive C stack check: every call in debug, every 16th in release
Assisted-by: Claude
* Fix light frame materialization: current_frame, f_back, and GC tracking
- current_frame() now uses current_thread_frame_vm() to materialize
light frames, fixing super()/compile()/exec() when called from light
frame context
- materialize_light_frame stores retained_back for heavy predecessors
too, so f_back resolves after the caller returns
- Escaped materialized frames (traceback, sys._getframe) are GC-tracked
at cleanup so reference cycles are collectible
- Restore pre-existing @expectedFailure markers for test_frame proxy tests
Assisted-by: Claude
* Limit light frame materialization to super() only
current_frame() returns heavy-only again to avoid unnecessary
materialization that extends object lifetimes. super() init uses
current_thread_frame_vm() directly since it must inspect the
light frame's code and freevars to find __class__.
Assisted-by: Claude
* Fix remaining CI failures: C stack check, GC tracking threshold, expectedFailure
- Check C stack overflow every call in light frame path (not every
16th) to prevent stack overflow on CI workers with smaller stacks
- Fix GC tracking threshold: escaped materialized frame detection
uses strong_count > 1 (was > 2, missed frames referenced only by
traceback in the result)
- current_frame() materializes light frames so warnings, compile(),
and other callers see the correct frame
- Mark test_futures2 PyFutureTests.test_task_exc_handler_correct_context
as expectedFailure (pre-existing on main: PyTask __del__ timing)
Assisted-by: Claude
* Avoid materialization in hot paths: add light-frame-aware accessors
current_frame() stays heavy-only to avoid costly materialization on
every call. Instead, add current_globals(), current_code(), and
current_builtins() that read directly from light frame raw pointers
without creating frame PyObjects.
Callers migrated:
- PyFunction:🆕 current_builtins() for fallback builtins lookup
- compile()/eval(): current_code() for future feature flags
- type.__new__: current_globals() for __module__ detection
- typevar/typing: current_globals() for caller module name
- _io: current_code() for source path check
- super(): current_thread_frame_vm() for __class__ cell access
- warnings.warn: current_thread_frame_vm() for stack level walk
- import: current_globals() for module resolution
Assisted-by: Claude
* Fix current_locals() to include light frames
current_locals() must materialize light frames so locals() builtin
returns the correct scope. This fixes test_zipimport doctest and
other tests that rely on locals() in light frame context.
Assisted-by: Claude
* Fix clippy, profile tracing, resume_gen_frame, and faulthandler issues
- Fix clippy: collapsible_if in materialize_light_frame, map_unwrap_or in import
- Fix profile breakage: use current_thread_frame_vm in trace_event_inner to
materialize light frames, preventing frame identity mismatch when profiling
is enabled inside a light frame (e.g. profile.Profile.runctx)
- Disable light frames inside trace/profile callbacks (tracing_is_suppressed)
- Fix resume_gen_frame: skip light frames in previous chain to avoid dangling
pointers after generator suspend
- Fix faulthandler: emit '<no Python frame>' when chain has only light frames
Assisted-by: Claude
* Fix materialized frame refcount: transfer localsplus ownership on exit
Remove @expectedFailure from test_futures2: the underlying issue was that
materialized light frames cloned localsplus values (refcount +1 each),
keeping objects alive until the next GC cycle collection.
Fix: after run_light_frame returns, transfer ownership of localsplus
values from the light frame to the materialized frame, replacing the
clones. This eliminates the extra refcount, allowing objects to be
GC'd immediately when their last reference is dropped.
Also stabilize materialized frame's localsplus on the heap in
sync_materialized_on_exit before GC tracking — the data-stack backing
will be popped imminently.
Assisted-by: Claude
* Fix rustfmt, increase stack margin, address review comments
- Fix rustfmt formatting issues (lint CI failure)
- Increase STACK_MARGIN_BYTES from 2048 to 3072 words to prevent C stack
overflow in deep recursion tests on macOS CI (test_functools SIGSEGV)
- Skip dispatch_traced_frame when use_tracing is false (review comment)
- Fix post-fork top_frame initialization: walk past light frames to find
nearest heavy frame (review comment)
Assisted-by: Claude
* Remove redundant test override in test_futures2
The override only called super() without any marker, making it a
no-op that prek flags as a redundant test patch.
Assisted-by: Claude
* Fix test_pdb and test_functools CI failures
- Revert with_frame dispatch_traced_frame skip: tracing can be enabled
mid-execution (pdb.set_trace), so Return events must always be checked.
This fixes 3 test_pdb doctest failures across all platforms.
- Check C stack overflow on every with_frame call instead of every 64th:
the previous sampling approach missed overflows between checks when
native stack frames are large (invoke_light_slots + lru_cache recursion).
- Add #[inline(never)] to invoke_light_slots to prevent the optimizer
from merging its stack frame into callers.
- Add early C stack check before DataStack allocation in invoke_light_slots.
Assisted-by: Claude
* Rename Frame to FrameObject, FrameRef to FrameObjectRef
Prepare for _PyInterpreterFrame/PyFrameObject separation by
renaming the PyObject-backed frame type to FrameObject.
Python-visible name ("frame") is unchanged.
Assisted-by: Claude
* Remove Deref<Target = InterpreterFrame> from FrameObject
Replace implicit field access through Deref with explicit
iframe()/iframe_ref()/iframe_mut() calls. This decouples
FrameObject from InterpreterFrame field layout, enabling
independent modification of InterpreterFrame in future commits.
Assisted-by: Claude
* Move InterpreterFrame identity fields to raw pointers
InterpreterFrame's code/globals/builtins/func_obj fields are now
borrowed raw pointers instead of owned PyRef/PyObjectRef. Ownership
of these references is anchored in FrameObject's new owned_code,
owned_globals, owned_builtins, owned_func_obj fields.
This aligns InterpreterFrame's layout with LightFrame's existing
raw-pointer pattern, enabling future DataStack-based allocation
without PyObject heap allocation.
- Add code()/globals()/builtins()/func_obj() accessor methods
- Add FrameObject::new_ref() for combined create+allocate+patch
- Update ExecutingFrame to use &Py<PyCode> / &Py<PyDict> / &PyObject
- Update monitoring.rs functions to take &Py<PyCode> instead of &PyRef<PyCode>
- Add Send+Sync impls for InterpreterFrame (raw pointers are !Send by default)
Assisted-by: Claude
* Remove LightFrame, FrameChainPtr, and dual-frame-type infrastructure
Delete LightFrame struct, FrameChainPtr tagged pointer, FrameSource enum,
and all supporting functions: materialize_light_frame, sync_light_to_materialized,
transfer_localsplus_to_materialized, sync_materialized_on_exit,
run_light_frame, invoke_light_slots body, LocalsPlus::from_datastack_raw.
All call paths now use a single frame type (FrameObject wrapping
InterpreterFrame). The frame chain is a simple *const FrameObject linked
list through InterpreterFrame.previous.
invoke_light_slots delegates to invoke_exact_args_slots.
ExecutingFrame.frame_source replaced with ExecutingFrame.frame.
Frame chain walking simplified throughout (no more light/heavy dispatch).
Net: -724 lines of dual-frame-type complexity.
Assisted-by: Claude
* Clean up invoke_light_slots and _vm function variants
- Replace invoke_light_slots calls with invoke_exact_args_slots
- Delete the invoke_light_slots delegate method
- Replace current_thread_frame_vm/frame_at_offset_vm with their
non-_vm equivalents (no more light frame materialization needed)
- Remove stale light-frame comment in callable.rs
Assisted-by: Claude
* Fix CI lint issues
- Move github context expressions from run blocks to env blocks
(zizmor/template-injection)
- Add workflow-level permissions: {} to lib-deps-check.yaml
- Suppress zizmor excessive-permissions for lib-deps-check.yaml
(pull_request_target is required for PR comments)
- Apply rustfmt formatting
Assisted-by: Claude
* Stack-allocate InterpreterFrame for non-generator function calls
- Extract InterpreterFrame::new() from FrameObject::new()
- Change frame chain from *const FrameObject to *const InterpreterFrame
- Add InterpreterFrame.materialized field for lazy FrameObject creation
- Add vm.with_iframe()/run_frame_fast() for stack-allocated frame execution
- Move ExecutingFrame.frame to iframe: *const InterpreterFrame
- Implement lazy materialize() for on-demand FrameObject creation
- Update invoke_with_locals/invoke_prepared_exact_args to skip heap allocation
- Update frame chain walking in faulthandler, gc_state, builtins/frame
- super() reads InterpreterFrame directly without materialization
Assisted-by: Claude
* Remove unnecessary refcount and atomic ops from fast call path
- Use raw pointer for func_obj instead of cloning PyObjectRef (saves 2 atomic RMW)
- Skip owner AcqRel swap for stack-allocated frames (always Thread-owned)
Assisted-by: Claude
* Change trace field to Option<PyObjectRef>, remove vm from InterpreterFrame::new
- trace: PyMutex<PyObjectRef> -> PyMutex<Option<PyObjectRef>>
- Eliminates vm.ctx.none() refcount inc/dec per frame init
- Remove vm parameter from InterpreterFrame::new() (no longer needed)
- f_trace getter returns None when trace is unset
- f_trace setter stores None instead of the None singleton
Assisted-by: Claude
* Store current frame pointer on VirtualMachine for fast lookup
- Add current_frame_ptr Cell<usize> field to VirtualMachine
- Hot path (with_iframe) reads/writes vm.current_frame_ptr instead of TLS
- TLS CURRENT_FRAME still maintained for signal-safe traceback walking
- Use set_current_frame_nosave() to skip cross-thread top_frame update
- super(), frame_at_offset, current_thread_frame_materialize use vm Cell
Assisted-by: Claude
* Apply rustfmt
Assisted-by: Claude
* Fix clippy use_self warnings, add inline hints for hot path
Assisted-by: Claude
* Remove code refcount clone from fast call path
Defer code.to_owned() to the generator/coroutine path only. On the fast
path, use &Py<PyCode> directly — the code object is alive via the
PyFunction on the caller's stack.
Assisted-by: Claude
* Remove VM current_frame_ptr Cell (fix SIGSEGV in threading builds)
The Cell<usize> on VirtualMachine could desynchronize from the TLS
CURRENT_FRAME in nested VM scenarios (enter_vm called with different
VMs on the same thread). Revert to TLS-only frame chain management.
Keep set_current_frame_nosave() for the stack-frame fast path: it
skips cross-thread top_frame publication but still does the TLS swap.
Fix clippy not_unsafe_ptr_arg_deref in set_current_frame.
Fix unnecessary Result wrapper in make_generator_or_coro.
Assisted-by: Claude
* Fix CI failures: GC frame assertion, faulthandler Radium import, dead code warning
- Fix SIGSEGV in threading builds: GC debug assertion walked frame
chain via top_frame, which is not updated by set_current_frame_nosave
used by stack-allocated iframes. Walk CURRENT_FRAME TLS chain instead.
- Add InterpreterFrame::get_lasti() accessor, remove direct Radium
import from faulthandler.rs (fixes unused import warning).
- Add cfg_attr allow(dead_code) for from_payload_ptr without threading
(fixes WASM build warning).
Assisted-by: Claude
* Fix materialized frame: copy localsplus, clear stale previous pointers
- materialize_slow: copy fastlocals snapshot to heap instead of empty
localsplus (fixes index-out-of-bounds panic in locals()/f_locals)
- materialize_slow: set previous to null (stack iframes become dangling)
- with_frame/resume_gen_frame: clear iframe.previous on pop (prevents
dangling pointers to freed stack iframes)
- f_back: walk TLS CURRENT_FRAME chain instead of top_frame for
cross-thread lookup (top_frame is stale for stack-allocated iframes)
- check_locals_access: match materialized frames by comparing the
Py<FrameObject> address stored in the stack iframe's materialized field
Assisted-by: Claude
* Fix Windows clippy: gate unix-only imports and variables
FrameObject, Py imports and top_iframe variable are only used in
unix+threading cfg blocks. Gate them properly for non-unix builds.
Assisted-by: Claude
* Fall back to FrameObject when tracing; fix cross-thread frame access
- invoke_with_locals: fall back to heap-allocated FrameObject path
when use_tracing is active, so trace/profile callbacks fire correctly.
This fixes test_trace, test_bdb, test_monitoring regressions.
- set_current_frame: publish top_iframe alongside top_frame in ThreadSlot
so cross-thread readers (sys._current_frames) can materialize
stack-allocated frames from other threads under stop-the-world.
- get_all_current_frames: use TLS CURRENT_FRAME for current thread,
fall back to top_iframe materialization for other threads.
- Use set_current_frame (not nosave) in with_iframe for proper
cross-thread visibility.
Assisted-by: Claude
* Apply rustfmt
Assisted-by: Claude
* Fix Windows: gate unix-only top_iframe_ptr variable
Assisted-by: Claude
* Fix frame chain: f_back, retained_back, owner, faulthandler
- Set materialized frame owner to FrameObject (not Thread) so
frame.clear() works on traceback frames
- Fix f_back for materialized frames: walk TLS chain to find source
iframe and materialize its previous frame
- Fix faulthandler dump_traceback to use dump_live_frames (reads
live lasti from stack iframes instead of stale materialized copies)
- Add retained_back propagation in with_frame cleanup: when a frame
escapes, materialize caller and mark it escaped for chain propagation
- Fix release_datastack_frame: don't overwrite retained_back if
already set by with_frame cleanup
Assisted-by: Claude
* Apply rustfmt
Assisted-by: Claude
* Refine with_frame retained_back: use frame_obj instead of materialize
Avoid materializing caller frames in with_frame cleanup to prevent
creating localsplus snapshots that keep extra refcounts. Only use
existing FrameObjects (frame_obj) for retained_back. Also keep
previous pointer alive in release_datastack_frame since the caller
is still executing at that point.
Assisted-by: Claude
* Fix f_lineno: use Cell<u32> for prev_line, update on every instruction
prev_line was only updated when tracing was active, causing f_lineno
to return stale values when observed mid-CALL (e.g. from warnings.warn
or sys._getframe). Changed prev_line from u32 to Cell<u32> for
interior mutability so it can be read safely while ExecutingFrame
holds a shared reference, and update it on every instruction so
f_lineno always returns the correct line.
Assisted-by: Claude
* Fix f_lineno for materialized frames: read live prev_line from TLS chain
Materialized FrameObjects have a snapshot of prev_line from materialize
time. When f_lineno is called on a materialized frame that is still
executing (its source iframe is on the TLS chain), walk the TLS chain
to find the source iframe and read its live prev_line. This fixes
lineno tracking in warnings, gettext, and other modules that read
frame line numbers during function calls.
Also fixed the pointer comparison: use from_payload_ptr to convert
FrameObject payload address to Py<FrameObject> address for matching
against the materialized field.
Assisted-by: Claude
* Sync live frame state for materialized frames
- Add find_live_source_iframe() to walk TLS chain and find the live
source InterpreterFrame for a materialized FrameObject
- f_lasti: read live lasti from source iframe when available
- f_lineno: use read_volatile for live prev_line from source iframe
- sync_visible_locals_to_mapping: read live localsplus from source iframe
- framelocalsproxy_getval: read live localsplus from source iframe
- with_iframe cleanup: sync localsplus, prev_line, lasti to materialized
FrameObject when the frame returns
- sync_fastlocals_from: new LocalsPlus method to update fastlocals
Fixes test_inspect (stale f_locals, stale lineno in positions),
test_listcomps (stale iteration variable in locals()).
Assisted-by: Claude
* Fix retained_back: use materialize_chain and strong_count check
- Add materialize_chain() that creates a lightweight FrameObject with
empty localsplus for f_back chain building only. This avoids cloning
local variables which would create extra refcounts (fixes test_memoryview).
- with_frame cleanup: use strong_count > 1 instead of escaped flag to
determine if retained_back is needed, ensuring chain propagation.
- with_iframe cleanup: use materialize_chain for retained_back to avoid
extra refcounts from localsplus snapshots.
Fixes test_memoryview (refcount from retained_back chain) and
test_traceback.test_extract_stack (f_back chain depth).
Assisted-by: Claude
* Fix cross-thread f_back chain for sys._current_frames/exceptions
- get_all_current_frames: materialize entire frame chain and link
retained_back during stop-the-world for other threads
- f_back: use stop-the-world to safely materialize cross-thread frame
chains on unix when prev iframe is on another thread
- f_back: check frame_obj() before STW for fast path
Fixes test_sys.test_current_frames and test_sys.test_current_exceptions.
Assisted-by: Claude
* Fix monitoring LINE events and faulthandler thread dump
- Skip all instrumented opcodes in bytecode loop prev_line update to
avoid defeating InstrumentedLine de-duplication
- Add prev_line update in execute_instrumented for non-RESUME/non-LINE
instrumented opcodes
- update_events_mask: walk TLS iframe chain to re-instrument all frames
including stack-allocated ones (fixes missing LINE events for inline
frames already past RESUME)
- faulthandler: use top_iframe fallback for stack-allocated frames in
dump_all_threads and watchdog_thread
- faulthandler: use dump_live_frames for current thread dump
Fixes test_monitoring LINE/CALL tests, test_faulthandler thread dumps.
Assisted-by: Claude
* Fix clippy: remove duplicate Radium import, use previous() helper
Assisted-by: Claude
* Fix materialized frame issues: GC tracking, live locals, f_trace propagation
- Track materialized FrameObjects in GC so cycle collection works
(fixes __del__ not called at exit for exception traceback cycles)
- materialize_chain returns owned PyRef without temporary_refs to
avoid defeating GC cycle detection from non-GC-tracked storage
- framelocalsproxy_setval writes to live source iframe when available
(fixes f_locals proxy writes not affecting executing frame)
- f_lineno uses lasti-based line for non-executing frames instead of
prev_line (fixes PEP 626 wrong line after exception unwind)
- f_trace/f_trace_lines/f_trace_opcodes setters propagate to live
source iframe (fixes pdb set_trace not working through materialized
frames)
- frame.clear() checks find_live_source_iframe to reject clearing a
frame that is backed by a live stack-allocated iframe
- Fix clippy warnings and unused imports in faulthandler
- Remove unsafe top_iframe access from faulthandler watchdog thread
Assisted-by: Claude
* Fix faulthandler crashes, GC tracking timing, and test_generators
- Use top_iframe instead of top_frame for frame chain walking in
faulthandler (dump_all_threads and watchdog), avoiding stale or
GC-cleared FrameObject pointer dereference
- Defer GC tracking of materialized FrameObjects to with_iframe
cleanup (after set_current_frame restores old chain), preventing
premature collection while temporary_refs still holds the only ref
- Add FrameObject::try_iframe() for safe access to potentially
GC-cleared frames
- Remove expectedFailure for test_exhausted_generator_frame_cycle
(now passes with GC tracking fix)
- Add 'noalias' to cspell dictionary
Assisted-by: Claude
* Fix Windows frame dump and remove dead code
- Publish top_iframe on all platforms (not just unix) so Windows
faulthandler and sys._current_frames can see stack-allocated frames
- Use top_iframe instead of frames Mutex in Windows faulthandler
dump and watchdog paths
- Use top_iframe in non-unix get_all_current_frames for sys._current_frames
- Remove unused dump_traceback_thread_chain (replaced by top_iframe walk)
- Mark test_pdb_await_support as expected failure
Assisted-by: Claude
* Fix ThreadSlot init on all platforms, skip test_pdb_await_support
- Add top_iframe to ThreadSlot initializer in init_thread_slot_if_needed
(fixes wasm32, Windows compile errors)
- Skip test_pdb_await_support on RustPython (async pdb exception
callback receives None exc argument)
Assisted-by: Claude
* Remove dead faulthandler code (dump_frame_from_ref, dump_traceback_thread_frames, try_iframe)
These functions are no longer called after switching all frame chain
walking to use top_iframe directly.
Assisted-by: Claude
* Fix Windows sys._current_frames: materialize full frame chain
The non-unix get_all_current_frames path was only materializing the
top iframe without linking retained_back, so f_back chain walking
could not find deeper frames like f123() in test_sys tests.
Replicate the full chain materialization from the unix path.
Assisted-by: Claude
* Use STW-based cross-thread f_back on all platforms
Replace the non-unix frames-mutex fallback with the same
stop-the-world iframe chain materialization used on unix.
Fixes test_current_exceptions on Windows where stack-allocated
iframes were not found in the FrameObject-only frames list.
Assisted-by: Claude
* Fix set_f_lineno to write to live iframe, use STW for cross-thread access
- set_f_lineno: write lasti/pending_stack_pops to the live source
iframe instead of the materialized copy so debugger jumps take
effect on stack-allocated frames
- f_back cross-thread: enter STW before dereferencing the prev
pointer to prevent use-after-free if the owning thread returns
- get_all_current_frames (non-unix): add STW protection when
walking cross-thread iframe chains, matching the unix path
Assisted-by: Claude
* Guard current_location() against lasti==0, always capture retained_back
- current_location(): return first_line_number instead of
panicking when lasti is 0 (before first instruction executes)
- with_frame retained_back: materialize the caller iframe when
needed so f_back always resolves for escaped FrameObjects
Assisted-by: Claude
* Fix format, revert eager retained_back materialization
Remove double blank line in frame.rs (lint failure).
Revert with_frame retained_back to only use already-materialized
caller FrameObjects — eagerly materializing the caller added
refcounts on local variables, preventing timely deallocation and
causing test_io, test_memoryview, and test_futures failures.
Assisted-by: Claude
* Remove rustpython-unicode-isolation-issue.md from gitignore
Assisted-by: Claude
* Remove test_pdb_await_support skip: async pdb now works correctly
Assisted-by: Claude
* Fix STOP_ITERATION monitoring: wrap value in StopIteration instance
fire_stop_iteration was passing the raw iterator return value to
callbacks, but the STOP_ITERATION event callback signature expects
a StopIteration exception instance. Wrap non-StopIteration values
in a new StopIteration(value), matching PyMonitoring_FireStopIterationEvent.
This fixes test_pdb_await_support where bdb's exception_callback
received None instead of a StopIteration instance.
Assisted-by: Claude
|
||
|
Bas Schoenmaeckers
|
57b630a6c7 | Add str compare functions to c-api (#8268) | ||
|
Jeong, YunWon
|
210ccc889c |
Thread-safe type operations: type lock, QSBR type-cache reclamation, GC stop-the-world, and interpreter optimizations (#7416)
* type lock * Drop old PyObjectRef outside type lock to prevent deadlock Dropping values inside with_type_lock can trigger weakref callbacks, which may access attributes (LOAD_ATTR specialization) and re-acquire the non-reentrant type mutex, causing deadlock. Return old values from lock closures so they drop after lock release. * Align type lock behavior with CPython * Add PUBLISHED flag bit to RefCount state word Assisted-by: Claude * Add QSBR module for deferred memory reclamation Assisted-by: Claude * Defer memory free of cache-published objects via QSBR Assisted-by: Claude * Wire QSBR checkpoints into thread lifecycle and eval breaker Assisted-by: Claude * Publish type-cache values to QSBR and bypass cache without a VM Mark cached type-method values as QSBR-published before storing their pointer in TYPE_CACHE, so racing readers never try-incref freed memory. Debug-assert that the current thread is ATTACHED whenever a lock-free cache read happens. Delete find_name_in_mro_without_vm and its unsound direct SeqLock read outside any VM/thread registration; find_name_in_mro now falls back to the locked, uncached MRO walk when no VM is current. This also removes assign_version_tag()'s last caller, so its unlocked fallback is deleted along with it (version_for_specialization keeps the locked path). VirtualMachine::initialize() runs Python bytecode (e.g. importing codecs/encodings) before any enter_vm scope exists, which left the bootstrap thread not ATTACHED for cache reads during startup. Add thread::VmBootstrapGuard, an RAII counterpart to enter_vm usable across statements that need &mut VirtualMachine, and wrap initialize() with it. Assisted-by: Claude * Use try-incref reads and QSBR-backed swaps in specialization cache Assisted-by: Claude * Reset QSBR thread registry after fork in the child Add Qsbr::reset_after_fork, which clears the registered thread slots before draining the retire queue. Call it in py_os_after_fork_child, right after type_cache_after_fork() and before _thread::after_fork_child() re-registers the surviving thread. Dead parent threads' Arc<QsbrSlot> handles live on in the child's copied memory with no destructor ever running, so without this the registry keeps them 'online' forever and future grace periods never complete. Assisted-by: Claude * Add threaded stress test for type cache mutation races Assisted-by: Claude * Restrict qsbr internals to pub(crate) and fix clippy lints Downgrade pub items in the private qsbr::threading module (QsbrSlot, Qsbr, QSBR, and its methods) to pub(crate), since they were never reachable outside the crate. Downgrade ThreadSlot::qsbr accordingly to avoid a private-interfaces warning. Import Arc/Weak from alloc instead of std to match clippy::std_instead_of_alloc. Rewrite the process() comment: goals are not strictly ordered by push order since advance() and the queue push aren't atomic together, so concurrent free_delayed calls can interleave; the drained prefix stays sound because each item individually passed poll, with reordering only affecting reclamation latency. Assisted-by: Claude * Gate per-instruction QSBR check behind a global pending flag eval_breaker_tripped() and check_signals() called thread::qsbr_break_requested() on every bytecode instruction, which does a thread-local lookup, RefCell borrow, and atomic load even when no QSBR retirement is pending. This caused a measured 10-18% slowdown on tight interpreter loops. Add Qsbr::pending, a global AtomicBool set under the queue lock while free_delayed() holds a retired allocation, and cleared under the same lock by process()/drain_all() once the queue empties. The hot path now checks Qsbr::break_pending() (a single relaxed static load) before touching thread-local state, and only pays the TLS cost while a retirement is actually in flight. Assisted-by: Claude * Merge signal and QSBR eval-breaker flags into one atomic word Replace the separate ANY_TRIGGERED AtomicBool and QSBR pending AtomicBool checks in the per-instruction eval-breaker path with a single AtomicU8 (EVAL_BREAKER) holding a bit per source. The global QSBR instance mirrors its pending state into the QSBR bit under the same queue-lock critical sections that already toggle pending, while local QSBR instances used by unit tests are left untouched. eval_breaker_tripped() and check_signals() now read/act on the merged word (eval_breaker_pending(), qsbr_bit_set()) instead of issuing two separate atomic loads per instruction. set_triggered() switches from store to fetch_or so a signal handler never clobbers the QSBR bit. Removed the now-unreferenced is_triggered() helper; break_pending() is kept for unit tests only and marked allow(dead_code) outside test builds. Assisted-by: Claude * Skip freelist reuse for published objects in default_dealloc Freelist-eligible payloads (tuple, int, list, dict, float, ...) were routed to T::freelist_push before ever reaching PyInner::dealloc, so a published object (e.g. a tuple cached as a type attribute) bypassed the is_published -> free_delayed QSBR hook entirely. That let PyRef::new_ref reuse the slot and rewrite the refcount word with a non-atomic core::ptr::write racing a reader's atomic try-incref, and let FreeList::drop call alloc::dealloc directly while a reader could still be mid-read. Gate the freelist branch in default_dealloc on RefCount::is_published(), checked before any teardown. A published object now always falls through to PyInner::dealloc, whose existing hook defers the memory free via QSBR. Document the non-unix (e.g. Windows) reclamation-latency limitation in the QSBR design doc's Lifecycle integration section (previously only noted in a code comment in vm/thread.rs). Extend the type-cache stress test so the mutator also churns a freelist-eligible published tuple class attribute (C.shape), read by readers each inner loop and deleted on the same cadence as C.m, to exercise the fixed bypass. Assisted-by: Claude * Use itoa for PyInt::to_str_radix_10 i64 fast path Format small integers with itoa::Buffer instead of i64::to_string() in PyInt::to_str_radix_10(), which backs both str(int)/repr(int) for exact int and PyObject::str()'s exact-int fast path. Adds the itoa crate as a workspace dependency. Assisted-by: Claude * Make object attribute dunders wrapper descriptors Remove the __getattribute__, __setattr__ and __delattr__ #[pymethod]s from PyBaseObject so add_operators installs PyWrapper slot wrappers for them from the #[pyslot] functions instead of method descriptors. lookup_slot_in_mro now classifies these entries as NativeSlot, so heap types without Python-level overrides keep the native getattro/setattro slot functions, which lets LOAD_ATTR/STORE_ATTR specialization apply to instances of Python-defined classes. Since __setattr__ and __delattr__ share the setattro slot, resolve both names together in update_one_slot: overriding only one of them no longer lets the other name's native resolution overwrite the dispatching wrapper. Assisted-by: Claude * Resolve TpSetattro pair on attribute deletion, not just addition update_one_slot's TpSetattro branch only ran the combined __setattr__/__delattr__ resolution (added in the previous commit) when ADD was true. On deletion (e.g. del C.__setattr__), it fell back to inherit_from_mro, which copies the base class's slot and discards the class's own remaining Python-level override of the other name. lookup_slot_in_mro reads the current attribute dicts, so the same resolution is correct for both addition and deletion; the ADD-only guard is removed and the logic now always runs. Assisted-by: Claude * Generate Opcode::cache_entries/deopt as table lookups generate_rs_opcode_metadata.py now emits a 256-entry const array for each function instead of a chained match over opcode names. Opcode::deopt() indexes an [Option<Opcode>; 256] table built from the existing specialization-to-base mapping, and Opcode::cache_entries() indexes a [u8; 256] table that bakes in the same deopt/to_base composition deoptimize() previously performed before its own match. Regenerated opcode_metadata.rs from the updated script; PseudoOpcode's bodies are unchanged since it has no cache/deopt entries to table. Added an exhaustive test in instruction.rs that checks every valid u8 opcode against a frozen copy of the old chained-match bodies. Assisted-by: Claude * Make Opcode/Instruction numeric conversions O(1) Opcode::as_numeric, Opcode::as_instruction and Instruction::as_opcode were each a per-variant match over ~230 arms in the define_opcodes! macro. Opcode now carries the same explicit $op_id discriminants and #[repr($typ)] as Instruction, which makes the conversions sound as: - Opcode::as_numeric: a plain `self as $typ` identity cast. - Opcode::as_instruction / Instruction::as_opcode: a mem::transmute, relying on both enums sharing one-$typ-wide layout (Instruction's payload fields are all the zero-sized Arg<T> marker, checked by a new per-instantiation size_of assertion). try_from_numeric is left as a match: $op_id values have gaps (specialized/instrumented opcode ranges), so a range check can't replace it. Adds #[inline] to the rewritten conversions plus deoptimize, and to the small generated wrapper/table-lookup functions from the previous opcode-metadata change (as_u8/as_u16, cache_entries, deopt, to_base) via generate_rs_opcode_metadata.py, then regenerates opcode_metadata.rs. Extends the instruction.rs equivalence-test pattern with two exhaustive tests (u8 and u16 instantiations) checking the new identity-cast/transmute conversions against the untouched try_from_numeric/TryFrom paths. Assisted-by: Claude * Pop CallIsinstance arguments directly instead of collecting into Vecs The specialized handler built two heap-allocated Vecs per call (pop_multiple().collect() plus a with_capacity(2) buffer) before invoking is_instance. Pop the class and instance straight off the stack for the effective two-argument case, removing the per-call allocations. Assisted-by: Claude * Inherit tp_new instead of always installing new_wrapper update_one_slot's TpNew branch stored new_wrapper for every ADD, so all heap types had slots.new == new_wrapper and the CallAllocAndEnterInit specialization never fired. Now __new__ is resolved through the MRO dicts: an own or Python-level definition installs new_wrapper, while a builtin __new__ entry (or none) inherits slots.new from the solid base. The now-reachable CallAllocAndEnterInit path gains the missing guards: the specializer requires co_argcount == oparg + 1 (via can_specialize_call) and rejects generator-like __init__, and the handler re-checks the argcount against the class-level init cache. Without the argcount check, __init__ defaults broke re._parser (UnboundLocalError in SubPattern.__init__). Remove the now-unused is_simple_for_call_specialization. Assisted-by: Claude * Skip locals dict allocation for the init-cleanup shim frame Mark the shim code object NEWLOCALS and create the shim frame with no locals mapping so FrameLocals::lazy() is used. The shim only executes ExitInitCheck/ReturnValue and never touches locals, so this removes one dict allocation per specialized instantiation. Assisted-by: Claude * Guard CALL_ALLOC_AND_ENTER_INIT with cached function version The init specialization cache stored only the type version, so swapping __init__.__code__ (which zeroes func_version) did not deopt the warmed call site and stale code assumptions (argcount shape, no varargs/kwonly, not a generator) were applied to the new code object. Store get_version_for_current_state() in the specialization cache when caching __init__ and re-check func_version() in the handler before using the cached function, deopting to the generic call path on mismatch, the same way the getitem specialization cache does. Assisted-by: Claude * Apply rustfmt and ruff-format fixes to files from earlier commits Assisted-by: Claude * Add frame object freelist Frame.iframe becomes FrameUnsafeCell<Option<InterpreterFrame>> so a dead frame can be left as an empty husk. Traverse::clear extracts all owned child references (including localsplus values via LocalsPlus::clear_into) and empties the cell. Dead frame allocations are cached in a thread-local FreeList<Frame> (up to 200 entries) and reused by PyRef::new_ref. clear_generator returns early when the frame was already cleared by cycle collection; generator drops can run after tp_clear of their frame. gc: skip tp_clear for objects saved to gc.garbage by DEBUG_SAVEALL, since they remain reachable from Python. Assisted-by: Claude * Drop frame children directly in tp_clear instead of extracting Pushing the frame's ~10 child references into the clear buffer grew a Vec on every frame deallocation. Take the interpreter frame out of the cell and drop it in place; LocalsPlus::clear_into is removed. Assisted-by: Claude * Untrack objects before tp_clear and guard cross-thread f_locals The cycle collector called tp_clear on unreachable objects while they were still linked into the GC generation lists, so another thread could obtain a strong reference to an already-cleared object (e.g. a frame husk with iframe == None) via gc.get_objects() and access the cleared payload. Untrack the dead set before the clear phase, mirroring the untrack-then-clear ordering of the refcount dealloc path. Objects that gained an external reference before untracking are found by comparing each object's strong count against the references coming from within the dead set; such late-resurrected objects and everything reachable from them are re-tracked and skipped, letting a later collection retry once the external references are released. Also reject f_locals access for a frame currently executing on another thread. Reading fastlocals of such a frame races with the executing thread overwriting the slots (torn reads of dropped values). Access from the executing thread itself (locals(), trace callbacks) is still allowed via the current-frame chain check. Assisted-by: Claude * Skip localsplus heap copy for uniquely referenced dying frames release_datastack_frame now checks the frame's strong count after untracking it from the GC generation lists. When the caller holds the only reference, localsplus values are dropped in place and the data stack storage is released without the heap copy; escaped frames (traceback, sys._getframe, trace callbacks) keep the copy path. The three inline materialize+pop call sites in function.rs are routed through release_datastack_frame. Assisted-by: Claude * Stage exact-args call arguments in stack slot buffers CallPyExactArgs, CallBoundMethodExactArgs and CallAllocAndEnterInit no longer collect popped arguments into a per-call Vec (plus a second Vec for self-prepending). Arguments are popped into a fixed-size Option<PyObjectRef> slot buffer (CallArgBuffer, 8 inline slots, heap fallback for larger arities) and moved into fastlocals via the new invoke_exact_args_slots / a take() iterator. - prepare_exact_args_frame now accepts an ExactSizeIterator of args - specialization_run_init_cleanup_shim takes the slot buffer and fills slot 0 with new_obj, dropping one redundant clone - LoadAttrGetattributeOverridden, LoadAttrProperty and BinaryOpSubscrGetitem pass inline slot arrays instead of vec![] Assisted-by: Claude * Reduce allocations in generic call paths - FuncArgs::prepend_arg: use reserve instead of reserve_exact so exact-capacity vectors do not realloc on every prepend. - IntoFuncArgs::into_method_args: build the final args vec once with capacity len + 1 instead of prepending into a full vector. - PyType::call: skip cloning FuncArgs when no init slot can run after slot_new (no init slot, not `type`, and slot_new is not new_wrapper). Assisted-by: Claude * Skip redundant exc_info restore on frame exit with_frame_impl saves the current exc_info slot value and restores it when the frame exits. When the slot still holds the same value, the restore rewrites the slot and recomputes the thread-exception mirror for no effect. Add restore_exception, which compares the slot against the saved value by pointer identity and skips the store when they match. The saved value is a strong reference held for the whole frame scope, so the pointed-to object cannot be freed and its address reused while the frame runs, making the pointer comparison free of ABA. Assisted-by: Claude * Apply rustfmt to gc_state.rs Assisted-by: Claude * Avoid empty FuncArgs clone in PyType::call Cloning args for the init call clones the kwargs map even when args is empty, which shows up in no-argument instantiation profiles. Prepare the init-call args before slot_new: a default FuncArgs when args is empty (indistinguishable from a clone of empty args), the existing clone otherwise. The empty check also keeps the init-slot load of the clone-elision path off the no-argument call path. Assisted-by: Claude * Push freelist husks only after tp_clear and child drops default_dealloc pushed the object onto the thread-local freelist before running clear_fn. Payloads that drop children inside clear (Frame) can run __del__ there, and a reentrant allocation could pop the husk and overwrite its payload while clear_fn still held a &mut borrow of it. Tuples had the same window through the extracted-edges drop, which ran after the push. Reorder default_dealloc to clear, drop the extracted children, and only then attempt the freelist push, so the husk becomes reusable only when no borrows into the payload remain. The tuple freelist bucketed husks by element count, which required reading the payload during push; after this reordering the elements are already cleared. PyInner<PyTuple> is a fixed-size allocation (the elements box is dropped and replaced on reuse), so replace the per-size buckets with the shared single-list FreeList. Remove the now-unused pyinner_layout helper. Assisted-by: Claude * Resolve __set__ and __delete__ together for the descr_set slot The descr_set slot serves both __set__ and __delete__, but the TpDescrSet accessor resolved only the single modified name and fell back to MRO inheritance on delete. Deleting one of the pair could disable the surviving operation: after `del D.__set__`, an instance delete stopped calling the remaining __delete__. Resolve both names together like the TpSetattro accessor does: any Python-level definition selects the dispatching wrapper, matching native functions are stored directly, a single native function with the other name absent is stored directly, and the slot is inherited from the MRO only when neither name resolves. Assisted-by: Claude * Update type base on __bases__ assignment set_bases replaced bases and mro but never updated the stored base, so __base__ kept reporting the old value after reassignment and slot resolution (tp_new inheritance, set_new/set_alloc during init_slots) read the stale solid base. Change the base field to PyAtomicRef<Option<PyType>> so it can be swapped under the type lock while remaining lock-free for readers, and recompute it in set_bases with the same best_base validation type creation uses (BASETYPE flag, instance layout conflict among bases). The swapped-out base is parked in the frame's temporary refs; other references released inside the critical section are dropped after the lock is released. Also in set_bases: - remove this class from the old bases' subclass lists (pruning dead entries), so repeated assignment no longer accumulates duplicates - roll back bases, base, and all updated mros when the recursive mro update fails, instead of leaving the type half-reparented - fix the misformatted empty-tuple error message Remove the expectedFailure marker from test_unsubclassable_types, which now passes. Assisted-by: Claude * Fix rollback order, dead weakrefs, and lock-held drops in set_bases - Restore recorded mros in reverse on rollback so a class visited multiple times through diamond inheritance ends with its original mro instead of an intermediate one. - Skip dead weakrefs in the subclass list during the recursive mro update instead of panicking on upgrade. - Retire the replaced mros on the success path instead of dropping them while the type lock is held. Assisted-by: Claude * Reify number sub-slot wrapper once in update_one_slot The update_sub_slot! macro expanded the Python-method wrapper closure at both the own and inherited store sites. Each expansion is a distinct fn item, so a base and a non-overriding subclass stored wrappers with distinct addresses in unmerged debug builds. binary_op1 compares slot fn addresses to detect whether a subclass overrides the operator, so the inherited slot was misread as an override and C() // E() dispatched to __rfloordiv__ instead of __floordiv__. Bind the wrapper store in a single closure and call it from both branches so the fn item is reified once and the slot value stays identical across a base and its subclasses. Document the conservative-on-mismatch nature of the new_wrapper address guard in call_wrapped. Assisted-by: Claude * Restrict type_cache_after_fork visibility and guard type-cache reads type_cache_after_fork is only called from within the crate, so mark it pub(crate) to resolve the unreachable-pub warning. Add debug_assert_current_thread_attached at the has_name_in_mro lock-free type-cache read site, matching lookup_ref_and_version_interned, and gate the function on debug_assertions so it is not compiled unused in release builds where the call sites are elided. Assisted-by: Claude * Guard BinaryOpSubscrGetitem with the cached type version The specialized BINARY_OP_SUBSCR_GETITEM handler used the cached __getitem__ after checking only the function version, unlike the sibling specialized handlers which revalidate the type version tag first. Store the type version in the inline cache at specialization time and revalidate owner.class().tp_version_tag against it before using the cached function, deopting to the generic subscript path on mismatch. Assisted-by: Claude * Call __abstractmethods__ __len__ once in object.__new__ The abstract-method check invoked the user-visible __len__ twice: once via length_opt for the count and again while materializing the method names. Derive the count from the materialized list instead, so __len__ runs once. Assisted-by: Claude * Use i64 fast paths for specialized int add/sub/mul Rewrite execute_binary_op_int to box results through new_int via i64 checked arithmetic instead of raw BigInt ops with new_bigint. Add an int_mul helper and a shared int_fast_op that computes the i64 result and falls back to the BigInt operation on to_i64 or checked-op failure. Wire int_mul into both the specialized BinaryOpMultiplyInt handler and the generic execute_bin_op Multiply/InplaceMultiply path, gated on exact int operands so subclasses keep dispatching through _mul/_imul. Assisted-by: Claude * Use i64 fast paths for exact int floordiv and remainder Add floordiv_i64/mod_i64 computing i64 floor-division and divisor-signed remainder, guarding zero divisor and i64::MIN overflow by returning None. Wire them through int_floordiv/int_mod (shared int_div_fast_op boxes via new_int) into the generic execute_bin_op FloorDivide/Remainder and their Inplace variants, gated on exact int operands so subclasses and zero divisors fall through to the existing _floordiv/_mod/_ifloordiv/_imod slow path. Assisted-by: Claude * Skip trashcan and untrack for non-GC-tracked objects in dealloc default_dealloc read is_gc_tracked() only to decide untracking, and entered the trashcan recursion guard unconditionally. Non-GC objects (int, float, str, ...) own no child references that recurse during deallocation, so they need neither the trashcan nor untracking. Read is_gc_tracked() once and gate both trashcan begin/end and untrack on it, removing three thread-local accesses per non-GC object deallocation. Assisted-by: Claude * Merge trashcan depth and defer queue into one thread-local struct trashcan::end accessed two separate thread-locals (DEALLOC_DEPTH and DEALLOC_QUEUE) at the outermost deallocation. Combine both into a single `Trashcan` thread-local holding Cell-based depth and queue fields, so begin and end each reach their state through one thread-local access. The queue is set back before each deferred dealloc call so reentrant begin/end during draining never holds an outstanding borrow. Assisted-by: Claude * Check object layout compatibility on __bases__ and __class__ assignment Add a shared compatible_for_assignment helper that walks each type to the base that fixes its instance layout and rejects the assignment when the old and new layouts differ (basicsize, itemsize, member count, __dict__, __weakref__, and __slots__ names). Wire it into set_bases, which previously performed no layout check, and replace the inline check in __class__ assignment, which compared the two types directly and over-rejected a subclass that adds no layout. Enable test_descr.test_builtin_bases, which the new set_bases check passes. Assisted-by: Claude * Stop the world around GC pointer-reading phases The cycle collector reads each tracked object's interpreter state during reference subtraction, the reachability walk and the strong-reference snapshot, including the localsplus of frames other threads are executing. Those slots are written without synchronization, so the reads are a data race under threading. collect_inner now stops the world before taking the generation read locks and restarts it after the strong-reference snapshot, before the finalizer/weakref/tp_clear phases. A debug assertion checks that no frame on any thread's call stack was classified unreachable. Automatic collections from maybe_collect are deferred to the next bytecode safepoint via a new eval-breaker GC bit instead of running synchronously: a synchronous collection can hold an internal lock (e.g. the lazy frame locals cell) that another thread is blocked on with no way to reach a safepoint, deadlocking the stop. Hardens the stop-the-world machinery for these frequent, concurrent GC stops: attach_thread now honors a pending stop after re-attaching so a thread doing rapid allow_threads calls cannot run past the requester; stop_the_world completion is level-triggered on all-threads-suspended rather than an edge-triggered countdown; and the thread-start started/ready handshakes detach while waiting so the waiter is parkable. Updates the Frame and PyRwLock/PyMutex traversal SAFETY comments to state the actual invariant. Assisted-by: Claude * Add threaded GC vs executing-frame stress snippet Workers churn frame state (recursion, generators, frame cycles) while a collector loops gc.collect() and an introspector walks live frame objects. Exercises the stop-the-world barrier around GC traversal. Assisted-by: Claude * Serialize fork and GC stop-the-world requesters fork() (posix before/after-fork) and the cycle collector both drive the single StopTheWorldState. With no mutual exclusion their requester word and suspension countdown could interleave and be clobbered, so the completion check never converged and a requester waited on itself forever (reproducible parent-side hang when forking with GC enabled). Add one exclusion held for the whole stop->start span of either requester: stop_the_world acquires it before any stop bookkeeping and start_the_world/reset_after_fork release it. The acquire is park-friendly (poll try-lock and honor a pending suspend between tries) so a requester blocked behind an active stop can still be force-parked instead of deadlocking the active requester. The acquirer holds no other lock while spinning. Also correct the PyRwLock traverse SAFETY note: a failed try_read may mean a force-parked thread holds the write lock; skipping is safe because under-traversal only over-approximates liveness. Document the residual gc.collect()-under-a-non-generation-lock exposure at the collect_inner barrier. Assisted-by: Claude * Add fork under concurrent GC stop-the-world snippet Worker threads allocate cyclic garbage with GC enabled while the main thread forks repeatedly and each child collects. Exercises the fork/GC stop-the-world exclusion; the allocation rate is throttled so a collection stays cheap in unoptimized builds. Assisted-by: Claude * Detach while acquiring the import lock The global import lock is held across bytecode by the importlib bootstrap, so its holder can be parked at a safepoint mid-hold. Acquiring it while attached let another thread block attached on the lock, so a stop-the-world requester could wait forever for that attached thread to suspend while the holder stayed parked. Wrap IMP_LOCK acquisition in allow_threads in both _imp.acquire_lock and the pre-fork acquire_imp_lock_for_fork so the wait honors stop-the-world requests. Correct the acquire_exclusion comment: a spinning requester may hold IMP_LOCK (fork) or the collecting mutex (GC); safety relies on those never being acquired attached-blocking by another thread. Assisted-by: Claude * Add concurrent-import vs GC deadlock snippet Two threads re-import modules (contending the import lock) while a third storms the cycle collector and a fourth allocates cyclic garbage. Regressed as a hang before the import lock acquisition was made park-friendly. Assisted-by: Claude * Create call frames untracked and track generators explicitly Add PyPayload::NEW_REF_UNTRACKED (default false, true for Frame) so PyRef::new_ref skips auto-tracking ordinary call frames in the GC. Generator/coroutine/async-generator frames are tracked explicitly in invoke_with_locals before their generator back-reference is installed. run_frame debug-asserts that a datastack frame is untracked on entry. Assisted-by: Claude * Track escaped call frames lazily at datastack release Rewrite release_datastack_frame around the invariant that a datastack frame is never GC-tracked while it runs: strong_count() == 1 means the frame never escaped, so drop localsplus in place without touching the GC; strong_count() > 1 means it escaped, so materialize localsplus onto the heap and then track the frame. This removes the untrack and the untrack-recheck dance from the common non-escaping path. Assisted-by: Claude * Guard the tracked-frame localsplus invariant Document at Frame::traverse that references to a frame are always recorded as graph edges, so the collector reads a frame's localsplus only when the frame is itself a tracked candidate. Add debug assertions at both frame track sites (escaped datastack frames and generator frames) that a frame is heap-backed before it is tracked, so a collector never reads data-stack-resident, still-mutating storage. Assisted-by: Claude * Reword frame/dealloc invariant comments and drop needless borrow - frame.rs: replace the task codename in the tracked-frame localsplus invariant comment with a description of the collector-vs-executing-frame behavior. - object.rs: remove needless borrow on current_cls in the __class__ assignment compatibility check. - core.rs: rewrite the default_dealloc trashcan/untrack-skip comment to state the actual invariant, which now covers untracked non-escaped frames releasing at interpreter depth with bounded recursion. Assisted-by: Claude * Move type SetAttr slot rewrite inside the type lock and tighten cache reads Run update_slot inside the same with_type_lock transaction as modified_inner and the attributes dict mutation, so the version invalidation, dict change, and slot-table rewrite are published together. Load init_version and getitem_version with Acquire in the specialization cache readers to pair with the Release stores in the writers. Assisted-by: Claude * Capture the type version before reading mutable slots in specializers specialize_load_attr, specialize_store_attr, specialize_to_bool, and the CallAllocAndEnterInit path of specialize_call read the version tag with version_for_specialization before inspecting getattro, setattro, the bool/len slots, and tp_new/tp_alloc, so a concurrent install of the corresponding dunder invalidates the version the specialization is cached against. In the BinaryOpSubscr __getitem__ path, check the HEAPTYPE and eval-frame gates before lookup_ref_and_version_interned, which takes the global type lock and may allocate a version tag. Assisted-by: Claude * Use _get_method_dict for the test_type check in the test runner Replace the direct __func__.__dict__ access with the _get_method_dict helper so plain functions without __func__ do not raise AttributeError. Assisted-by: Claude * Fix lint hooks: cspell dictionary entries, spelling fix, formatting Add qsbr to the rustpython dictionary and reborrows/reparenting to the top-level word list. Rename oldto/newto locals to old_to/new_to. Fix Stabilise -> Stabilize typo in a frame.rs comment. Apply cargo fmt to object.rs, type.rs, and frame.rs, and ruff format/check fixes to two extra_tests snippet files. Assisted-by: Claude * Rebuild all slots for type and descendants on __bases__ reassignment Add PyType::update_all_slots, which invalidates version tags and iterates the full SLOT_DEFS name table calling update_slot::<true> for each distinct name. Unlike init_slots, which is additive and driven only by dunder names present in the current MRO, this resets a slot whose method left the MRO instead of leaving a stale dispatcher pointer. Call it from set_bases in place of the previous modified_inner + init_slots pair, so reassigning __bases__ rebuilds slots for the type and every descendant. Add extra_tests/snippets/type_bases_slot_rebuild.py covering removed-method resets on zelf and deep descendants, the wrong-target switch, __getattr__, the added-method mirror, and a swap-away-and-back round trip. Assisted-by: Claude * Derive the thread-local frame stack from the frame chain Remove the per-VM `frames` Vec. current_frame, sys._getframe, sys._getframemodulename, frame.f_back, sys.monitoring re-instrumentation, gc.get_referrers, faulthandler stack dumps and the post-fork slot rebuild now walk the signal-safe CURRENT_FRAME/`previous` chain instead. Add `Py::from_payload_ptr` to recover a frame object from a chain pointer. The cross-thread `ThreadSlot::frames` registry (sys._current_frames, cross-thread f_back, GC stop-the-world assertion) is unchanged. Assisted-by: Claude * Inherit sub-slot fallback into the field being resolved update_one_slot's number/sequence/mapping fallback inherited via the accessor's default field. Left and right binary ops (add/right_add) share one accessor but occupy distinct fields, so resolving an absent right op or deleting it reset the left op's field, dropping a still-defined __add__ dispatcher. Inherit the exact field under resolution instead. Assisted-by: Claude * Gate type_cache_after_fork to its fork caller and use Self in downcast type_cache_after_fork is only called from the unix fork path, so gate it with all(feature = "host_env", unix) to match the caller and avoid a dead-code error on non-unix targets. Replace the explicit PyType with Self in the modified_inner downcast. Assisted-by: Claude * Unmark test_attr and test_method_call_error in test_monitoring Both TestLoadSuperAttr tests now pass; remove their expectedFailure markers. Assisted-by: Claude * Run tp_new specialization __init__ without a trampoline frame CallAllocAndEnterInit ran __init__ inside a synthetic init-cleanup shim frame whose code carried the __init__ name. Since the thread frame stack is derived from the frame chain, that shim frame was visible to sys._getframe, f_back walks, traceback construction and inspect.stack / inspect.trace. Its code object has empty co_positions, so a stack walk that reached it raised StopIteration inside inspect._get_code_position, cascading into unrelated failures (test_inspect trace/stack/frame, asyncio source traceback). Call __init__ directly via run_frame, enforce the __init__() should return None contract inline, and drop the shim: the init-cleanup code object, its builder, with_frame_untraced, monitoring_disabled_for_code, and the extra-frame datastack/recursion budget. Removing the second frame per construction also cuts the specialization's per-call cost. Assisted-by: Claude * Replace per-call thread-frame mutex with an atomic top-of-stack (unix) On unix threading builds, publish each thread's top Python frame in a single relaxed AtomicPtr store from set_current_frame instead of pushing onto a parking_lot::Mutex<Vec<FramePtr>> per call. Cross-thread readers (sys._current_frames, cross-thread f_back, faulthandler.dump_traceback, the GC unreachable debug-assert) run under stop-the-world and walk the published top frame down the Frame::previous chain; the owning thread is then parked at a safepoint, so the pointer and the frames it reaches are quiescent and alive. The faulthandler watchdog is a plain OS thread that cannot stop-the-world, so it walks the chain lock-free and best-effort. Non-unix threading builds have no stop-the-world and keep the existing mutex-guarded frame stack unchanged. Assisted-by: Claude * Skip exc_info save/restore for callees that never touch the slot with_frame saves and restores the shared exc_info slot around every Python call to contain frames that leave it unbalanced. Cache a has_exc_handling bit on PyCode at creation, set when the bytecode contains any opcode that calls vm.set_exception (PushExcInfo, PopExcept, CheckEgMatch, EndAsyncFor, InstrumentedEndAsyncFor). A callee whose code has none of these cannot mutate the slot, so the save and restore are skipped for it. Generators go through resume_gen_frame and are unaffected. Assisted-by: Claude * Return a write-through FrameLocalsProxy from frame.f_locals Optimized (function) frames now expose `f_locals` as a `FrameLocalsProxy` implementing PEP 667 semantics instead of a cached snapshot dict: - reads go live through the fast-local slots; each access mints a fresh proxy; keys that do not name a fast local are stored in a per-frame `f_extra_locals` side dict and folded into `locals()`. - writes to a fast-local key store into the slot (or its cell) in place; deleting a fast local raises ValueError; extra keys delete normally. - full mapping protocol: keys/values/items (lists), get/pop/setdefault, update (dict or FrameLocalsProxy only), __or__/__ior__/__ror__ (dict result), copy (plain dict), __reduce__ blocks pickling/copy, repr with recursion guard, mapping-pattern and Mapping ABC support. Class/module/exec frames keep returning their namespace mapping directly. Cross-thread access to a frame running on another thread still raises RuntimeError. A closed generator now keeps its frame locals when a durable frame reference escaped (f_locals proxy, sys._getframe, f_back), matching take_ownership; the escape is tracked with a per-frame flag. The snapshot-then-fold locals_to_fast/locals_dirty write-back is retired since proxy writes reach the slots directly. Assisted-by: Claude * Retain the caller frame so f_back resolves after it returns When a frame escapes its execution (referenced through a traceback, `sys._getframe`, `f_locals`, ...), capture a strong reference to its caller at release time. `f_back` consults it once the caller has left the live frame chain, so the Python-visible frame chain survives return. The retained reference is a GC-traversed edge and is cleared by `frame.clear()`, so ancestor chains stay collectable. Assisted-by: Claude * Fix debug-build native stack overflow on deep recursion Make check_c_stack_overflow one-sided (trip whenever the stack pointer is below the soft limit) so a single native frame larger than the margin cannot step past the danger band undetected. Raise the debug STACK_MARGIN_BYTES from 4096 to 16384 words so the margin exceeds a single debug interpreter frame, leaving headroom to raise RecursionError. Release margin unchanged. Clamp the soft-limit margin to half the stack so small explicit thread stacks do not get a soft limit above their stack top. Assisted-by: Claude * Allocate exception instance __dict__ lazily Exception construction went through into_ref_with_type, which eagerly allocated an empty instance dict for every HAS_DICT type. Add into_ref_with_type_lazy_dict, which builds the instance with an unallocated dict slot, and route the four exception construction sites (PyBaseException, PyOSError, OSErrorBuilder, PyBaseExceptionGroup) through it. The dict now materializes on first attribute write or __dict__ access via the existing get_or_insert path. add_note and PyImportError::slot_init now obtain the dict through object_get_dict so they materialize it instead of assuming it exists. A freshly constructed exception no longer reports an empty dict in gc.get_referents, matching the reference interpreter. Assisted-by: Claude * Allocate exception __dict__ lazily in vm.new_exception Route vm.new_exception() through into_ref_with_type_lazy_dict so internally raised exceptions (new_type_error, new_value_error, etc.) start without an instance dict, matching the slot_new path. The dict is materialized on the first attribute write or __dict__ access. Assisted-by: Claude * Validate FrameLocalsProxy.update() arguments Reject keyword arguments and require exactly one positional argument, raising TypeError with the "takes no keyword arguments" and "takes exactly one argument (N given)" messages. Assisted-by: Claude * Address review nits in frame and faulthandler - Drop stale vm.frames reference from the release_datastack_frame uniqueness argument. - Assert has_exc_handling when unwinding an Except-typed stack slot, documenting the invariant that guards the shared exc_info write. - Truncate the specialized __init__ return-type name to 200 chars, matching the unspecialized wrapper. - Reword two comments to describe behavior without prose references to CPython. Assisted-by: Claude * Use scopeguard for start_the_world in faulthandler dump_all_threads Wrap the unix stop-the-world registry walk in a scope with scopeguard::defer! so start_the_world runs on panic, matching the f_back and get_all_current_frames sites. Add scopeguard to the stdlib dependencies. Also correct the restore_exception doc comment to name with_frame after the rename. Assisted-by: Claude * Gitignore docs/superpowers Assisted-by: Claude * Rename type_bases_slot_rebuild.py to builtin_type_bases.py Match the builtin_* naming convention of extra_tests/snippets. Assisted-by: Claude * Apply rustfmt, ruff, and cspell lint fixes Reformat with rustfmt, fix import spacing in builtin_type_bases.py with ruff, and add "pointee" to the cspell word list. Assisted-by: Claude * Gate unix-only QSBR methods to their call sites `online`, `drain_all`, and `reset_after_fork` are called only from unix code (thread attach/detach, post-fork reset), and `offline` from unix code plus a unit test. Gate them with matching cfg so `-D dead_code` does not fire on non-unix targets. Assisted-by: Claude * Visit the function closure tuple as a GC edge PyFunction::traverse visited the cells inside the closure tuple instead of the tuple object itself, so the tuple's reference from the function was never subtracted during cycle collection. A closure tuple that reached back to its function (or, through a frame retained by f_back, to a Thread) was stranded as a false GC root and never collected, leaking the whole cycle. Visit the tuple itself, matching clear(). Assisted-by: Claude * Apply formatting hook fix to builtin_type_bases.py Assisted-by: Claude * Unmark asyncgen finalization-by-gc tests in test_base_events test_asyncgen_finalization_by_gc and test_asyncgen_finalization_by_gc_in_other_thread now pass; GC finalizes the async generators. Assisted-by: Claude * Unmark test_sni_callback_refcycle in test_ssl The servername-callback reference cycle is now collected by GC. Assisted-by: Claude * Widen GC stop-the-world gates from unix-only to all threading builds Change `cfg(all(unix, feature = "threading"))` to `cfg(feature = "threading")` on the stop-the-world machinery so it also compiles and runs on non-unix threading builds: - StopTheWorldState, its stats, stw_trace, and the stop_the_world field - ThreadSlot state/stop_requested/thread fields and their initializers - wait_while_suspended/attach_thread/detach_thread/suspend_if_needed/do_suspend, allow_threads, stop_requested_for_current_thread, and the enter_vm / VmBootstrapGuard / attach_current_thread / release_current_thread / cleanup attach-state wiring - eval_breaker_tripped, check_signals, run_scheduled_gc, signal GC_BIT / schedule_gc / take_gc_scheduled, and the frame.rs safepoint call - CollectStopTheWorld and its use in collect_inner - QSBR::online/offline, now called from attach/detach on all threading builds - debug_assert_current_thread_attached and its type-cache call sites maybe_collect defers auto-collection to the bytecode safepoint on every threading build instead of only unix; non-threading builds keep the inline collect. stw_trace writes to std stderr on non-unix. top_frame publishing (CURRENT_TOP_FRAME_SLOT, set_current_frame), the frame-walk debug assert in collect_inner, and the fork reinit helpers remain unix-only; non-unix keeps ThreadSlot::frames for introspection. Assisted-by: Claude * Detach current thread around blocking _winapi/_overlapped waits Wrap the blocking Windows wait calls in `vm.allow_threads` so the calling thread transitions ATTACHED -> DETACHED for the duration of the wait: - _winapi: WaitForSingleObject, WaitForMultipleObjects, BatchedWaitForMultipleObjects, ConnectNamedPipe, ReadFile, Overlapped.GetOverlappedResult - _overlapped: Overlapped.getresult These previously blocked while ATTACHED, so a stop-the-world requester could never suspend the thread and spun in its wait loop indefinitely. * Format WaitForMultipleObjects allow_threads closure * Treat concurrent sni_callback removal as no-op in invoke_sni_callback * Assert capi refcount on a fresh mortal list instead of the int type object The refcount test asserted exact incref/decref deltas on PyInt's shared type object. That object's reference count is perturbed by the other capi tests running in parallel, and is immortal under some interpreter configurations, so the deltas were not reliably +1/-1. Assert them on a freshly created, uniquely owned list whose reference count is private to the test and mortal. Assisted-by: Claude * Traverse and GC-track PyOSError instances The `#[pyexception]` struct macro now forwards a `traverse` option to the generated `#[pyclass]`, and `ExceptionItemMeta` accepts the `traverse` key. `PyOSError` is marked `traverse = "manual"`, so `HAS_TRAVERSE` is true and OSError-family instances are tracked at creation and traversed by the collector. `PyOSError::traverse` now visits the underlying `PyBaseException` (traceback, cause, context, args) instead of `PyException::try_traverse`, which was a no-op because `PyException` has `HAS_TRAVERSE = false`. * Unmark test_blockingioerror in test_io The BlockingIOError reference cycle is now collected by GC. Assisted-by: Claude |
||
|
Bas Schoenmaeckers
|
1e0f95ab3c | Add pyframe functions to c-api (#8215) | ||
|
Bas Schoenmaeckers
|
4caf959e58 |
Use PyO3 main & enable c-api tests (#8000)
Enable `abi3t` support in PyO3 Add left over c-api functions Fix PyLong_AsUnsignedLongLongMask Update pyo3 to `0.29` Add `PyInt::as_u64_mask` Fix `PyInt::as_u32_mask` |
||
|
Bas Schoenmaeckers
|
d1792fd2ce |
Add more int functions to the c-api (#8159)
* Add more int functions to the c-api * Simplify as_mask logic * Add `PyLong_FromNativeBytes` & `PyLong_FromUnsignedNativeBytes` |
||
|
Bas Schoenmaeckers
|
ab3d050485 | Add allocator functions to c-api (#8168) | ||
|
Jeong, YunWon
|
5add3d6a69 |
Align codegen passes and opcode metadata with CPython (#7987)
* Share marshal ref table between code object and its internals
read_marshal_bytes, _str, _str_vec, _name_tuple, and _const_tuple now
take a shared ref table and resolve TYPE_REF / register FLAG_REF
entries. deserialize_code is split into a public wrapper and an inner
function that receives the ref table; deserialize_value_depth opens a
fresh inner ref space when it hits Type::Code, mirroring CPython's
behaviour of putting the code object itself at ref slot 0. Nested code
objects inside const tuples reuse the surrounding code's ref space via
the new read_const_value helper.
* Align PYC magic number, FORMAT_VERSION, and header check with CPython 3.14
PYC_MAGIC_NUMBER changes from 2994 to 3627, matching CPython 3.14's
pyc_magic_number_token (0x0a0d0e2b). marshal FORMAT_VERSION drops from
5 to 4 (the encoder/marshal.version value; the decoder already accepts
both). check_pyc_magic_number_bytes now compares all four magic bytes
instead of the first two.
* Accept CPython-tagged .pyc as read-only bytecode source
SourceFileLoader.get_code now also looks for .pyc files using
_RP_FALLBACK_CACHE_TAGS (currently ('cpython-314',)) in addition to
sys.implementation.cache_tag. The matched .pyc is only used for
reading; recompilation still writes to the RustPython-tagged path, so
CPython's .pyc is never overwritten. Source-stat / hash / timestamp
validation logic is unchanged.
* Apply rustfmt to marshal helpers
* Marshal PySlice from format version 4 instead of 5
CPython's marshal supports TYPE_SLICE from format version 4 onwards
and that is the default version. Rejecting slice dumps below version
5 made marshal.dumps(slice(...)) fail with the default version and
broke test.test_marshal.SliceTestCase.test_slice.
* Revert "Accept CPython-tagged .pyc as read-only bytecode source"
Lib/importlib/_bootstrap_external.py is CPython's own code copied
verbatim; local patches here defeat compatibility tracking. The
cpython-XX cache_tag fallback needs to live on the RustPython side
(Rust code or sys.implementation.cache_tag policy), not as edits to
the imported standard library.
This reverts commit 1fc426d0fb5fcdb50d35cad13bbb43e8f6ce1c7f.
* Set marshal FORMAT_VERSION to 5 to match CPython 3.14.5
Py_MARSHAL_VERSION is 5 in CPython 3.14.5 (Include/marshal.h:16) and
TYPE_SLICE serialization rejects version < 5 (Python/marshal.c:720).
Restore the same threshold and constant so marshal.version and the
slice-marshal gate match CPython.
* Thread marshal recursion depth through nested code objects
Code objects embedded in const-tuples reset the depth budget on each
recursion, so a hostile or pathological marshal stream of code-in-tuple-
in-code can blow the stack despite MAX_MARSHAL_STACK_DEPTH. Pass the
current depth through deserialize_code_inner and read_marshal_const_tuple
and decrement at each code-object/tuple boundary.
Also route dict keys through deserialize_value_after_header so TYPE_CODE
keys decode instead of failing with BadType.
* align compiler to CPython
* Align codegen with CPython compile.c
Rename CFG helpers and accessors to the names used in CPython's
compile.c (basicblock_next_instr, basicblock_last_instr,
basicblock_append_instructions, bb_has_fallthrough, is_jump,
make_cfg_traversal_stack, mark_warm/mark_cold, etc.). Drop the unused
boolop-folding gate, mark_cpython_cfg_label_block helper, and
ComprehensionLoopControl::iter_range field.
Track an is_coroutine flag on SymbolTable, set in async def, await, and
async comprehensions, and propagate it through non-generator
comprehensions per symtable_handle_comprehension().
Mark SetupCleanup/SetupFinally/SetupWith as has_arg pseudo-ops, mark
ForIter as a terminator, and add has_arg/has_const on AnyInstruction.
Fix Instruction::stack_effect_jump to delegate to the opcode's
stack_effect_jump rather than stack_effect.
* Align codegen IR with CPython CFG structures
* Match CPython CFG annotation offset arithmetic
* Propagate CPython CFG label translation errors
* Align CPython exception target labeling flow
* Propagate CPython CFG traversal stack allocation errors
* Match CPython optimize_load_fast allocation flow
* Propagate CPython basicblock allocation errors
* Propagate CPython redundant NOP cleanup errors
* Propagate CPython unused const cleanup errors
* Propagate CPython const folding errors
* Propagate CPython swaptimize allocation errors
* Match CPython list-to-tuple fold allocation
* Skip const folding on CPython allocation failures
* Skip subscript folding on CPython allocation failures
* Propagate CPython assembler allocation errors
* Propagate CPython localsplus allocation errors
* Propagate CPython localsplus setup allocation errors
* Propagate CPython jump label map allocation errors
* Propagate CPython instruction sequence allocation errors
* Propagate CPython instruction label allocation errors
* Guard CPython codegen block allocation
* Guard CPython label shadow allocation
* Propagate CPython label shadow allocation errors
* Align CPython c-array allocation updates
* Align CPython ref stack growth
* Match CPython CFG builder debug check
* Avoid Rust-only CFG append clone
* Reuse CPython cleared block slots
* Use CPython block append in copy_basicblock
* Match CPython cfg builder creation order
* Clear label map after CPython apply pass
* Match CPython cfg builder allocation check
* Propagate CPython c-array size errors
* Drop Rust-only label uniqueness check
* Drop Rust-only label shadow debug checks
* Propagate CFG block index overflow
* Propagate CFG label oparg overflow
* Model CPython basicblock instruction storage
* Drop Rust-only recorded CFG precheck
* Model CPython instruction sequence storage
* Propagate instruction sequence offset overflow
* Model CPython instruction sequence labels
* Match CPython jump offset arithmetic
* Match CPython exception table arithmetic
* Match CPython label index arithmetic
* Match CPython instruction offset casts
* Match CPython jump offset indexing
* Match CPython oparg locals casts
* Match CPython localsplus offset arithmetic
* Match CPython cell prefix indexing
* Match CPython C array growth arithmetic
* Match CPython label map allocation arithmetic
* Match CPython label map size tracking
* Use CPython label map size in sequence passes
* Assert CPython label map clearing invariants
* Match CPython label oparg assignment
* Match CPython compiler direct arithmetic
* Match CPython load fast local casts
* Match CPython load fast depth assert
* Match CPython resume depth flagging
* Match CPython stack depth arithmetic
* Drop Rust-only stack overflow error
* Return CPython stack depth directly
* Match CPython C array growth errors
* Match CPython instruction insert asserts
* Match CPython unreachable pseudo jump
* Match CPython CFG size guard
* Match CPython superinstruction assert
* Match CPython redundant jump assert
* Match CPython stackdepth errors
* Match CPython jump offset flow
* Match CPython assembler buffer defaults
* Match CPython bytecode emit growth
* Match CPython assembler entry growth
* Match CPython assembler growth overflow check
* Match CPython remove_unreachable structure
* Match CPython static swap flow
* Inline CPython code unit preprocessing
* Match CPython C array growth checks
* Match CPython label map size guard
* Match CPython load-fast flow
* Simplify CPython CFG condition flow
* Align exception fallthrough propagation
* Match CPython pseudo target table
* Match CPython annotations CFG assert
* Match CPython inverted op assert
* Match CPython many-locals guard
* Reject deopt opcodes in CFG stack effects
* Match CPython invalid stack effect error
* Test CPython deopt stack effect guard
* Match CPython load-fast extended-arg assert
* Match CPython instruction allocation asserts
* Match CPython basicblock last-instr asserts
* Match CPython opcode range asserts
* Assert CPython fallthrough line propagation invariant
* Assert CPython CFG target offset sign
* Assert CPython exception fallthrough invariant
* Assert CPython exception stack bounds
* Assert CPython traversal stack allocation
* Match CPython label-map allocation in shadow
* Mirror CPython label-map sentinel fill
* Match CPython CFG builder allocation asserts
* Match CPython exception stack structure
* Match CPython ref stack structure
* Match CPython CFG traversal stack structure
* Mirror CPython CFG traversal stack pointer
* Use CPython fixed exception handler stack
* Mirror CPython ref stack capacity field
* Match CPython swap optimizer scratch stack
* Align static swap helpers with CPython blocks
* Align swaptimize signature with CPython
* Match CPython redundant pair pass result
* Match CPython inline pass result
* Match CPython redundant NOP pass results
* Fix bytecode metadata after upstream rebase
* Match CPython opcode stack metadata
* Add CPython identifiers to cspell dictionary
Add CNOTAB, LNOTAB, ialloc, ioffset, iused, nblocks, ncellsused,
ncellvars, nextop, noffsets, nvars, swaptimize, untargeted to
.cspell.dict/cpython.txt for the new CFG/assembler code in
crates/codegen/src/ir.rs.
* Fix CI failures from bytecode-parity work
- clippy: drop redundant `test_` prefix on three test functions and
remove an unnecessary `u32` cast in basicblock_clear_reuses_cpython_spare_slots_in_offset_order
- insta: regenerate nested_double_async_with snapshot to match the new
CFG output that drops unreferenced labels after the redundant-NOP pass
- regrtest: drop `@expectedFailure` markers from test_func_args,
test_meth_args (test_compile), test_disassemble_with,
test_disassemble_try_finally (test_dis), and test_except_star
(test_monitoring) which now pass
* Resync generated opcode metadata
Empty conf.toml since WithExceptStart and Setup{Cleanup,Finally,With}
stack effects already match CPython, so the TODO override entries are
stale and only cause CI hook diffs.
Regenerate opcode_metadata.rs and drop the matching SetupCleanup/
SetupFinally/SetupWith assertions on PseudoOpcode::has_arg(); their
`HAS_ARG` flag comes from pseudo definitions in bytecodes.c that the
upstream analyzer does not propagate through PseudoInstruction.properties,
so the generated has_arg() excludes them. has_target() still covers
these block-push pseudos via is_block_push().
* Drop is_block_push has_arg invariant
The CPython invariant `assert(OPCODE_HAS_ARG(op) || !IS_BLOCK_PUSH(op))`
relies on SETUP_{FINALLY,CLEANUP,WITH} carrying `HAS_ARG_FLAG` in
CPython's metadata. The autogen tool reads pseudo-opcode properties from
target instructions and does not propagate the pseudo's own
HAS_ARG flag, so PseudoOpcode::has_arg() omits these three opcodes.
Drop the debug_assert that fired inside py_freeze proc-macro expansion.
* Auto-generate has_eval_break and route AnyInstruction has_arg/has_const via macro
Add fn_has_eval_break to generate_rs_opcode_metadata.py using CPython's
Properties.eval_breaker, removing the hand-written matches! body for
Opcode::has_eval_break and PseudoOpcode::has_eval_break.
Forward has_arg/has_const from Instruction and PseudoInstruction to
their opcode, so AnyInstruction can use either_real_pseudo! like the
other has_* accessors instead of an open-coded match.
|
||
|
Bas Schoenmaeckers
|
1128bb8853 | Add capsule support to c-api (#7940) | ||
|
Jeong, YunWon
|
6440e64355 | Align codegen metadata with CPython (#7952) | ||
|
Bas Schoenmaeckers
|
6d0fde896c |
Add call functions to c-api (#7937)
* Add call functions to c-api * Review * Move conversion impl to c-api |
||
|
Jeong, YunWon
|
d438b2ef16 |
Bytecode parity - CFG reorders and LOAD_FAST_BORROW chain (#7870)
* Align LOAD_FAST_BORROW analysis with CPython chain shape Three changes that bring optimize_load_fast_borrow closer to CPython's optimize_load_fast in flowgraph.c: * ir.rs: split mark_cold into the CPython-style two passes. Phase 1 propagates "warm" from the entry block, phase 2 propagates "cold" from except_handler blocks. Blocks reached by neither phase keep cold=false and stay in their original b_next position, matching CPython's handling of empty placeholders left by remove_unreachable (e.g. the inner_end of a nested try/except whose incoming jumps were re-routed by optimize_cfg). * ir.rs: in optimize_load_fast_borrow, push the fall-through successor only when the current block has a last instruction (is_some_and). Empty blocks now terminate fall-through propagation, matching the `term != NULL` check in optimize_load_fast. * compile.rs: add switch_to_new_or_reuse_empty() helper and use it in compile_while. The helper reuses the current block when it is empty and unlinked, mirroring USE_LABEL absorption in cfg_builder_maybe_start_new_block. This stops a stray empty block from appearing between e.g. a try/except end_block and the following while loop header. Four codegen tests that depended on the previous fall-through-through- empty behavior are marked #[ignore] with TODO comments. Also includes a handful of dictionary entries in .cspell.dict picked up during the work. * Interleave const fold passes per-block to match CPython Mirror CPython's optimize_basic_block() (flowgraph.c) by walking each block once in instruction order and trying tuple, list, set, unary, and binop folding at each position before advancing. This replaces the previous global-pass sequence where every fold_unary_constants pattern in the whole CFG was registered before any tuple constant, leaving negated literals like `-1` at co_consts positions earlier than CPython produces (e.g. snippets.py: -1 at idx 280 vs CPython idx 726). Changes: - Extract `fold_unary_constant_at` and `fold_binop_constant_at` per- position helpers from the existing global passes; the global passes now call the helpers in a loop. - Add `fold_constants_per_block` that walks each block to a fixed point, trying all five folds at each instruction position. - Call the new walker before the legacy global passes in optimize_finalize so co_consts insertion order matches CPython's. Measured on the full Lib tree: differing files 270 → 269; the only newly matching file is `test/test_ast/snippets.py`, the case raised in youknowone/RustPython#28. * Inline small fast-return blocks only through unconditional jumps `inline_small_fast_return_blocks` previously appended the target `LOAD_FAST(_BORROW)/RETURN_VALUE` block's instructions onto any predecessor whose fall-through eventually reached it, in addition to the unconditional-jump case CPython handles in `inline_small_or_no_lineno_blocks` (flowgraph.c:1210). CPython only inlines through unconditional jumps, leaving fall-through predecessors to reach the shared return block via the natural CFG layout. The extra fall-through branch duplicated the return tail (e.g. `if/elif/return` emitted two adjacent `LOAD_FAST_BORROW x; RETURN_VALUE` sequences). Remove the fall-through inlining branch and keep only the unconditional-jump path. Measured on the full Lib tree: differing files 270 → 239 (-31), no new regressions. Files newly matching include copy.py, argparse.py, dataclasses.py, logging/__init__.py, pathlib/__init__.py, etc. * Allow scope-exit/jump-back reorder within a shared except handler `reorder_conditional_scope_exit_and_jump_back_blocks` previously skipped any reorder where the conditional, scope-exit, or jump-back block had an `except_handler` attached, even when all three shared the same handler. CPython reorders these regardless of try/except context, as long as the blocks stay within the same protected region. The over- conservative guard left patterns like `try: for: if cond: return` with the loop body's scope-exit ahead of the backedge, while CPython places the backedge first and inverts the conditional. Replace the `block_is_protected` triple-check with a single `mismatched_protection` test: skip only when the three blocks do not share the same `except_handler`. Same-handler reorders preserve the protected range because every instruction's `except_handler` field stays attached as `.next` pointers shift. Measured on the full Lib tree: differing files 239 → 237; no new regressions. * Skip jump-over-cleanup reorder when target restarts an exception scope reorder_jump_over_exception_cleanup_blocks was swapping a small scope-exit target with a preceding cold cleanup chain even when the target block began a fresh try (SETUP_FINALLY/SETUP_CLEANUP/SETUP_WITH). The swap moved the next try's setup ahead of the prior handler's cleanup_end/next_handler/cleanup_block, making the cleanup_body's JUMP_FORWARD fall through directly to the cleanup_end and get elided as redundant. The bytecode then lacked the JUMP_FORWARD that skips the cleanup blocks and matched the prior handler's borrow tail incorrectly. Skip the reorder when the target block contains any block-push pseudo op so a new try's setup stays in source order. Re-enables the four named/typed except-cleanup borrow tests that were marked #[ignore] in commit 7481459ea. * fix if block_idx == BlockIdx::NULL |
||
|
Bas Schoenmaeckers
|
4ec52c2e94 |
Add basic capi error support (#7787)
* Add basic capi error support * Add missing symbols to make tests compile again * Add `pyerrors` to dictionary * Return exception in `Py_GetConstantBorrowed` * Remove `allow(dead_code)` * Fix windows * Load stdlib when calling `Py_InitializeEx` * Debug tests * Revert "Load stdlib when calling `Py_InitializeEx`" This reverts commit |
||
|
Jeong, YunWon
|
edd7b4d45d | Align bytecode CFG cleanup with CPython (#7781) | ||
|
Bas SchoenmaeckersandJeong, YunWon
|
f122e3e12a |
Add minimal capi lifecycle support (#7648)
* Add minimal capi lifecycle support * Force enable `threading` on `stdlib` --------- Co-authored-by: Jeong, YunWon <jeong@youknowone.org> |
||
|
Jeong, YunWon
|
fb9cafa8c9 |
Enable PEP 709 inlined comprehensions (#7412)
* Enable PEP 709 inlined comprehensions for function-like scopes Activate the existing compile_inlined_comprehension() implementation by fixing 6 bugs that prevented it from working: - LoadFastAndClear: push NULL (not None) when slot is empty so StoreFast can restore empty state after comprehension - StoreFast: accept NULL from stack for the restore path - sub_tables.remove(0) replaced with next_sub_table cursor to match the pattern used elsewhere in the compiler - in_inlined_comp flag moved from non-inlined to inlined path - is_inlined_comprehension_context() now checks comp_inlined flag and restricts inlining to function-like scopes - comp_inlined set only when parent scope uses fastlocals Symbol table analysis handles conflict detection: - Nested scopes in comprehension → skip inlining - Bound name conflicts with parent symbol → skip inlining - Cross-comprehension reference conflicts → skip inlining - Splice comprehension sub_tables into parent for nested scope tracking * Add localspluskinds, unify DEREF to localsplus index - Add CO_FAST_LOCAL/CELL/FREE/HIDDEN constants and localspluskinds field to CodeObject for per-slot metadata - Change DEREF instruction opargs from cell-relative indices (NameIdx) to localsplus absolute indices (oparg::VarNum) - Add fixup_deref_opargs pass in ir.rs to convert cell-relative indices to localsplus indices after finalization - Replace get_cell_name with get_localsplus_name in InstrDisplayContext trait - Update VM cell_ref/get_cell_contents/set_cell_contents to use localsplus indices directly (no nlocals offset) - Update function.rs cell2arg, super.rs __class__ lookup with explicit nlocals offsets * Fix clippy warnings, formatting, restore _opcode_metadata.py Fix cast_possible_truncation, nonminimal_bool, collapsible_if, manual_contains clippy lints. Restore _opcode_metadata.py to upstream/main version (3.14 aligned). Pre-copy closure cells in Frame::new for coroutine locals(). Handle raw values in merged cell slots during inlined comps. Exclude async comprehensions from inlining path. * Exclude async/await comprehensions from PEP 709 inlining in symboltable Async comprehensions and comprehensions with await in the element expression need their own coroutine scope and cannot be inlined. The symboltable builder was not checking these conditions, causing incorrect symbol scope resolution when an async comprehension was nested inside an inlined comprehension (e.g. [[x async for x in g] for j in items]). |
||
|
Jeong, YunWon
|
60c1f77288 |
update cspell config (#7452)
* add allowCompoundWords * cspell dict |
||
|
Jeong, YunWon
|
13a0623eb1 | Adapt bitflagset to use enum (#7419) | ||
|
Jeong, YunWon
|
8bc9a47e68 |
Add per-size tuple freelist (20 buckets × 2000 each) (#7361)
* Add per-size tuple freelist (20 buckets × 2000 each) Implement PyTuple freelist matching tuples[PyTuple_MAXSAVESIZE]: - TupleFreeList with 20 per-size buckets (sizes 1..=20, 2000 capacity each) - freelist_push uses pre-clear size hint for correct bucket selection - freelist_pop takes &Self payload to select bucket by size - Type guard in new_ref handles structseq types sharing PyTuple vtable - Add pyinner_layout<T>() helper for custom freelist Drop impls - Update freelist_pop/push signatures across all freelist types * freelist: exact type check at pop call-site Move exact-type filtering from freelist_pop implementations to the single call-site in new_ref. This prevents structseq and other subtypes from popping tuple freelist entries entirely, rather than popping and then deallocating on type mismatch. Add Context::try_genesis() that returns None during bootstrap to avoid deadlock when genesis() is called during Context initialization. * Move exact type check from freelist_push to call-site in default_dealloc Remove typ parameter from freelist_push trait signature. The exact type check is now done once at the call-site alongside the heaptype check, simplifying all freelist_push implementations. * Remove freelist_hint; call freelist_push before tp_clear By calling freelist_push before tp_clear, the payload is still intact and can be read directly (e.g. tuple element count for bucket selection). This eliminates freelist_hint and the hint parameter entirely. |
||
|
Jeong, YunWon
|
7333192406 |
Suspend Python threads before fork() (#7364)
* Suspend Python threads before fork()
Add stop-the-world thread suspension around fork() to prevent
deadlocks from locks held by dead parent threads in the child.
- Thread states: DETACHED / ATTACHED / SUSPENDED with atomic CAS
transitions matching _PyThreadState_{Attach,Detach,Suspend}
- stop_the_world / start_the_world: park all non-requester threads
before fork, resume after (parent) or reset (child)
- allow_threads (Py_BEGIN/END_ALLOW_THREADS): detach around blocking
syscalls (os.read/write, waitpid, Lock.acquire, time.sleep) so
stop_the_world can force-park via CAS
- Acquire/release import lock around fork lifecycle
- zero_reinit_after_fork: generic lock reset for parking_lot types
- gc_clear_raw: detach dict instead of clearing entries
- Lock-free double-check for descriptor cache reads (no read-side
seqlock); write-side seqlock retained for writer serialization
- fork() returns PyResult, checks PythonFinalizationError, calls
sys.audit
|
||
|
Jeong, YunWon
|
99060c1dcd |
Object header slimming: prefix allocation for ObjExt (#7334)
* Object header slimming: prefix allocation for ObjExt Extract dict, weak_list, and slots fields from PyInner<T> into a separate ObjExt struct allocated as a prefix before PyInner using Layout::extend(). Objects that don't need these fields (int, str, float, list, tuple, dict, etc.) skip the prefix entirely. - Add HAS_WEAKREF flag to PyTypeFlags for per-type weakref control - Add HAS_EXT bit to GcBits indicating prefix presence - Define ObjExt struct with dict, weak_list, slots - Shrink PyInner header from ~80-88 bytes to ~32 bytes for lightweight objects - Update all accessor methods to go through ext_ref() - Update bootstrap type hierarchy to use prefix allocation - Add __weakref__ getset descriptor for heap types - Set HAS_WEAKREF on builtin types that support weak references - Remove test_weak_keyed_bad_delitem expectedFailure (now passes) * Add HAS_WEAKREF to _asyncio Future/Task, rename weakref helpers - Add HAS_WEAKREF flag to PyFuture and PyTask (matches CPython) - Rename subtype_getweakref/setweakref to subtype_get_weakref/set_weakref to fix cspell unknown word lint * Add HAS_WEAKREF to array, deque, _grouper; remove expectedFailure markers - Add HAS_WEAKREF to PyArray and PyDeque (matches CPython) - Add HAS_WEAKREF to PyItertoolsGrouper (internal use by groupby) - Remove 6 expectedFailure markers from test_dataclasses for weakref/slots tests that now pass * Add HAS_WEAKREF to code, union, partial, lock, IO, mmap, sre, sqlite3, typevar types Add HAS_WEAKREF flag to built-in types that support weakref: - PyCode, PyUnion, PyPartial, Lock, RLock - All IO base/concrete classes (_IOBase, _RawIOBase, _BufferedIOBase, _TextIOBase, BufferedReader, BufferedWriter, BufferedRandom, BufferedRWPair, TextIOWrapper, StringIO, BytesIO, FileIO, WindowsConsoleIO) - PyMmap, sre Pattern, sqlite3 Connection/Cursor - TypeVar, ParamSpec, ParamSpecArgs, ParamSpecKwargs, TypeVarTuple Remove 3 expectedFailure markers from test_descr for now-passing tests. * Add HAS_DICT to type flags and handle non-METHOD/CLASS in descr_get - Add HAS_DICT flag to PyType (type metaclass) alongside existing HAS_WEAKREF. All type objects are instances of type and need dict support, matching CPython's PyType_Type. - Replace unimplemented!() in PyMethodDescriptor::descr_get with fallback to bind obj directly, matching CPython's method_get which uses PyCFunction_NewEx for non-METH_METHOD methods. * Fix ext detection, HeapMethodDef ownership, WASM error - Remove HAS_EXT gc_bits flag; detect ext from type flags using raw pointer reads to avoid Stacked Borrows violations - Store HeapMethodDef owner in payload instead of dict hack - Clear dict entries in gc_clear_raw to break cycles - Add WASM error fallback when exception serialization fails |
||
|
Jeong, YunWon
|
cf26f85230 |
Remove Frame mutex and use DataStack bump allocator for LocalsPlus (#7333)
* Remove PyMutex<FrameState> from Frame, use UnsafeCell fields directly Move stack, cells_frees, prev_line out of the mutex-protected FrameState into Frame as FrameUnsafeCell fields. This eliminates mutex lock/unlock overhead on every frame execution (with_exec). Safety relies on the same single-threaded execution guarantee that FastLocals already uses. * Add thread-local DataStack for bump-allocating frame data Introduce DataStack with linked chunks (16KB initial, doubling) and push/pop bump allocation. Add datastack field to VirtualMachine. Not yet wired to frame creation. * Unify FastLocals and BoxVec stack into LocalsPlus Replace separate FastLocals (Box<[Option<PyObjectRef>]>) and BoxVec<Option<PyStackRef>> with a single LocalsPlus struct that stores both in a contiguous Box<[usize]> array. The first nlocalsplus slots are fastlocals and the rest is the evaluation stack. Typed access is provided through transmute-based methods. Remove BoxVec import from frame.rs. * Use DataStack for LocalsPlus in non-generator function calls Normal function calls now bump-allocate LocalsPlus data from the per-thread DataStack instead of a separate heap allocation. Generator/coroutine frames continue using heap allocation since they outlive the call. On frame exit, data is copied to the heap (materialize_to_heap) to preserve locals for tracebacks, then the DataStack is popped. VirtualMachine.datastack is wrapped in UnsafeCell for interior mutability (safe because frame allocation is single-threaded LIFO). * Fix clippy: import Layout from core::alloc instead of alloc::alloc * Fix vectorcall compatibility with LocalsPlus API Update vectorcall dispatch functions to use localsplus stack accessors instead of direct stack field access. Add stack_truncate method to LocalsPlus. Update vectorcall fast path in function.rs to use datastack and fastlocals_mut(). * Add datastack, nlocalsplus, ncells, tstate to cspell dictionary * Fix DataStack pop() for non-monotonic allocation addresses Check both bounds of the current chunk when determining if a pop base is in the current chunk. The previous check (base >= chunk_start) fails on Windows where newer chunks may be allocated at lower addresses than older ones. * Fix stale comments: release_datastack -> materialize_localsplus * Fix non-threading mode for parallel test execution Two fixes for Cell-based types used in static items under non-threading mode, which cause data races when Rust test runner uses parallel threads: 1. LazyLock: use std::sync::LazyLock when std is available instead of wrapping core::cell::LazyCell with a false `unsafe impl Sync`. The LazyCell wrapper is kept only for no-std (truly single-threaded). 2. gc_state: use static_cell! (thread-local in non-threading mode) instead of OnceLock, so each thread gets its own GcState with Cell-based PyRwLock/PyMutex that are not accessed concurrently. * Fix CallAllocAndEnterInit to use LocalsPlus stack API * Use checked arithmetic in LocalsPlus and DataStack allocators * Address code review: checked arithmetic, threading feature deps, Send gate - Use checked arithmetic for nlocalsplus in Frame::new - Add "std" to threading feature dependencies in rustpython-common - Gate GcState Send impl with #[cfg(feature = "threading")] * Clean up comments: remove redundant/stale remarks, fix CPython references |
||
|
Jeong, YunWon
|
af8f41ded1 |
freelist optimization (#7337)
* Add object freelist infrastructure and PyFloat freelist Add freelist_push/freelist_pop hooks to PyPayload trait with default no-op implementations. Modify default_dealloc to offer dead objects to the freelist before freeing memory. Modify PyRef::new_ref to try popping from the freelist before allocating. Implement thread-local freelist for PyFloat (max 100 entries). Float objects are ideal candidates: fixed-size payload (f64), no GC tracking, no instance dict, trivial Drop. Benchmark improvement on float-heavy workloads: float_arith: ~12% faster float_list: ~17% faster * Add PyList freelist and GC-safe freelist infrastructure Add HAS_FREELIST const to PyPayload trait. For freelist types, GC untracking is done immediately (not deferred) during dealloc to prevent race conditions when the object is reused. Restructure new_ref so freelist-popped objects also get GC tracked, enabling freelist support for GC-tracked types like PyList. Add drop_in_place for old payload before writing new one, required for types with non-trivial Drop (e.g. PyList's RwLock<Vec>). Implement thread-local freelist for PyList (max 80 entries). * Add PyDict freelist and exact-type guard for freelist push Add thread-local freelist for PyDict (max 80 entries). Add heaptype check in default_dealloc to prevent subclass instances from entering the freelist. Subclass objects have a different Python type than the base type, so reusing them would return objects with the wrong __class__. Skip PyTuple freelist: structseq types (stat_result, struct_time) are static subtypes sharing the same Rust payload, making type-safe reuse impractical with heaptype check alone. * Add PySlice freelist (max 1, matching PySlice_MAXFREELIST) * Add FreeList<T> wrapper to drain cached objects on thread teardown Replace Cell<Vec<*mut PyObject>> with Cell<FreeList<T>> which implements Drop to properly deallocate PyInner<T> when threads exit. Also fix freelist_pop safety doc to match actual contract. * Add manual Traverse impl for PySlice with clear() and fix comment * Deduplicate allocation fallback in PyRef::new_ref |
||
|
Shahar Naveh
|
80e0403729 |
Bytecode enum named oparg (#7294)
* Align real opcodes * Align pseudoes * Add `consti` to known words |
||
|
Jeong, YunWon
|
f95be1c82b |
Implement LOAD_ATTR inline caching with adaptive specialization (#7292)
* Implement LOAD_ATTR inline caching with adaptive specialization Add type version counter (tp_version_tag) to PyType with subclass invalidation cascade. Add cache read/write methods (u16/u32/u64) to CodeUnits. Implement adaptive specialization in load_attr that replaces the opcode with specialized variants on first execution: - LoadAttrMethodNoDict: cached method lookup for slotted types - LoadAttrMethodWithValues: cached method with dict shadow check - LoadAttrInstanceValue: direct dict lookup skipping descriptors Specialized opcodes guard on type_version_tag and deoptimize back to generic LOAD_ATTR with backoff counter on cache miss. * Add BINARY_OP and CALL adaptive specialization BINARY_OP: Specialize int add/subtract/multiply and float add/subtract/multiply with type guards and deoptimization. CALL: Add func_version to PyFunction, specialize simple function calls (CallPyExactArgs, CallBoundMethodExactArgs) with invoke_exact_args fast path that skips FuncArgs allocation and fill_locals_from_args. * Lazy quickening for adaptive specialization counters Move counter initialization from compile-time to RESUME execution, matching CPython's _PyCode_Quicken pattern. Store counter in CACHE entry's arg byte to preserve op=Instruction::Cache for dis/JIT. Add PyCode.quickened flag for one-time initialization. * Add Instruction::deoptimize() and CodeUnits::original_bytes() - deoptimize() maps specialized opcodes back to their base adaptive variant - original_bytes() produces deoptimized bytecode with zeroed CACHE entries - co_code now returns deoptimized bytes, _co_code_adaptive returns current bytes - Marshal serialization uses original_bytes() instead of raw transmute * Fix monitoring and specialization interaction - cache_entries() returns correct count for instrumented opcodes - deoptimize() maps instrumented opcodes back to base - quicken() skips adaptive counter for instrumented opcodes - instrument_code Phase 3 deoptimizes specialized opcodes and clears CACHE entries to prevent stale pointer dereferences * Address review: bounds checks, UB fix, version overflow, error handling - Add bounds checks to read_cache_u16/u32/u64 - Fix quicken() aliasing UB by using &mut directly - Add JumpBackwardJit/JumpBackwardNoJit to deoptimize() - Guard can_specialize_call with NEWLOCALS flag check - Use compare_exchange_weak for version tag to prevent wraparound - Propagate dict lookup errors in LoadAttrMethodWithValues - Apply adaptive backoff on version tag assignment failure - Remove duplicate imports in frame.rs |
||
|
Jeong, YunWon
|
0868e256d8 |
Implement PyStackRef borrowed stack references for LOAD_FAST_BORROW (#7290)
* Implement PyStackRef borrowed stack references for LOAD_FAST_BORROW * Fix PyStackRef: use NonZeroUsize for niche optimization, revert borrowed refs - Change PyStackRef.bits from usize to NonZeroUsize so Option<PyStackRef> has the same size as Option<PyObjectRef> via niche optimization - Revert LoadFastBorrow to use clone instead of actual borrowed refs to avoid borrowed refs remaining on stack at yield points - Add static size assertions for Option<PyStackRef> - Add stackref, fastlocal to cspell dictionary - Remove debug eprintln statements - Fix clippy warning for unused push_borrowed * Add borrowed ref debug_assert to InstrumentedYieldValue, clean up comments |
||
|
Jeong, YunWon
|
6015e15c16 |
Implement trace/profile events and opcode tracing for generators (#7268)
- Add TraceEvent::Exception and Opcode variants with profile filtering - Extract dispatch_traced_frame helper for Call/Return trace events - Fire exception trace on new raises, SEND StopIteration, FOR_ITER StopIteration - Fire opcode trace events gated by f_trace_opcodes - Move prev_line to FrameState for persistence across generator suspend/resume - Reset prev_line in gen_throw for correct LINE monitoring after yield - Add per-code event mask (events_for_code) to prevent unrelated code instrumentation - Remove expectedFailure markers from test_bdb (5) and test_sys_setprofile (14) |
||
|
Jeong, YunWon
|
e123b9c073 |
set_f_lineno, set_f_lasti, PyAtomic refactor
- Implement set_f_lineno with stack analysis and deferred unwinding - Add Frame::set_lasti() for trace callback line jumps - Implement co_branches() on code objects - Clear _cache_format in opcode.py (no inline caches) - Fix getattro slot inheritance: preserve native slot from inherit_slots - Fix BRANCH_RIGHT src_offset in InstrumentedPopJumpIf* - Move lasti increment before line event for correct f_lineno - Skip RESUME instruction from generating line events - Defer stack pops via pending_stack_pops/pending_unwind_from_stack to avoid deadlock with state mutex - Fix ForIter exhaust target in mark_stacks to skip END_FOR - Prevent exception handler paths from overwriting normal-flow stacks - Replace #[cfg(feature = "threading")] duplication with PyAtomic<T> from rustpython_common::atomic (Radium-based unified API) - Remove expectedFailure from 31 now-passing jump tests |
||
|
Jeong, YunWon
|
8a74956590 |
more algorithm-independent GC infra (#7194)
* mark poluting tests * GC-infra independent to EBR * trashcan * add overflow guard to inc(), #[must_use] on dec()/safe_inc(), trashcan debug_assert, weakref generic re-check |
||
|
Jeong, YunWon
|
88242301ff |
Use _print_exception_bltin in excepthook, register source in linecache (#7177)
- excepthook: call traceback._print_exception_bltin instead of traceback.print_exception to match PyErr_Display behavior - run_string: register compiled code in linecache._interactive_cache so traceback can display source lines and caret indicators - Remove test_sys_tracebacklimit expectedFailure |
||
|
CPython Developers
|
0b0450e7da | Update test_winconsoleio from v3.14.3 | ||
|
Jeong, YunWon
|
42740f7ce6 | Rename constevaluator_alloc to const_evaluator_alloc, add to cspell dict | ||
|
Jeong, YunWon
|
654a3eab8c | Implement more warnings | ||
|
Jeong, YunWon
|
48e4e7c661 |
Implement more ast features (#6986)
Assisted-by: Codex:gpt-6 |
||
|
Jeong, YunWon
|
bededc2c01 | Add GC infrastructure: object tracking, tp_clear, and helper methods (#6994) | ||
|
Jeong, YunWonandClaude Sonnet 4.5
|
3814348943 |
dealloc and finalize_modules (#6934)
* rewrite finalize_modules with phased algorithm
Replace the absence of module finalization during interpreter shutdown
with a 5-phase algorithm matching pylifecycle.c finalize_modules():
1. Set special sys attributes to None, restore stdio
2. Set all sys.modules values to None, collect module dicts
3. Clear sys.modules dict
4. Clear module dicts in reverse import order (2-pass _PyModule_ClearDict)
5. Clear sys and builtins dicts last
This ensures __del__ methods are called during shutdown and modules are
cleaned up in reverse import order without hardcoded module names.
* dealloc the rigth way
* fix finalize_modules: only clear __main__ dict, mark daemon thread tests as expected failure
Without GC, clearing all module dicts during finalization causes __del__
handlers to fail (globals are None). Restrict Phase 4 to only clear
__main__ dict — other modules' globals stay intact for their __del__
handlers.
Mark test_daemon_threads_shutdown_{stdout,stderr}_deadlock as expected
failures — without GC+GIL, finalize_modules clears __main__ globals
while daemon threads are still running.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
|
||
|
Jeong, YunWonandgithub-actions[bot]
|
60bec8a561 |
rework weakref (#6916)
* Replace WeakListInner with inline atomic weakref list and stripe locks Remove heap-allocated WeakListInner (OncePtr<PyMutex<WeakListInner>>). WeakRefList now holds two inline atomic pointers (head, generic). PyWeak.parent replaced with wr_object pointing directly to referent. Add weakref_lock module with AtomicU8 spinlock array for thread safety. Rewrite upgrade/clear/drop_inner/count/get_weak_references with stripe lock. Make Pointers methods public in linked_list.rs. Remove expectedFailure from test_subclass_refs_dont_replace_standard_refs. * Auto-format: cargo fmt --all --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
||
|
Jeong, YunWon
|
714d1ce58b |
gc module internal structure and API (#6910)
* gc module internal structure and API Add gc_state module with GcState, GcGeneration, GcDebugFlags, GcStats. Replace gc module stubs with working API backed by gc_state. Add gc_callbacks and gc_garbage to Context. Add is_gc_tracked, gc_finalized, gc_get_referents to PyObject. Collection is stubbed (returns 0) — actual algorithm to follow. * fix dict/weakref/generators * unmark test_asyncio * apply review |
||
|
Jeong, YunWon
|
95abec4c21 |
More Load instructions including LoadLocals, LoadFastBorrow (#6886)
* fix stdlib_io * more scpell dict * LoadLocals * LoadFastBorrow * more ops |