* Fix implementation of bytes.isspace to match the implementation in CPython.
* Fix implementation of bytes.istitle so that test_bigmem now passes.
Also added some extra_test to capture this.
* Fix some clippy issues
Signed-off-by: Hanif Ariffin <hanif.ariffin.4326@gmail.com>
* bytes.istitle returns false when the bytes is empty.
* Fix some edges of istitle with non-alphanumeric characters
* Disable a line because python 3.12 errors on bad escape sequence
* Match CPython's istitle a little closer
* Fix clippy warning
* Rewrite bytes.istitle along _Py_bytes_istitle
The lookahead-based check accepted b"Not--a Titlecase String", which
test_bytes covers. Track whether the previous byte was cased instead.
isspace compares bytes directly rather than converting to char.
Restore the b"omkmok\Xaa" assert: 3.14 reports an invalid escape
sequence as a SyntaxWarning, not an error.
Assisted-by: Claude
---------
Signed-off-by: Hanif Ariffin <hanif.ariffin.4326@gmail.com>
Co-authored-by: Jeong YunWon <jeong@youknowone.org>
`method_descriptor.__get__(obj)` raised a TypeError when the owner (the
optional second argument) was omitted, because the METHOD-flag branch
required the owner to be a type. Match CPython: a missing owner binds to
`obj`, while a non-type owner still raises "needs a type, not ...".
Assisted-by: Claude Code:claude-opus-4-8
* csv: port CPython reader state machine
Replace the csv-core reader and per-item quote scanner with one Rust
implementation of CPython's nine-state reader parser. Keep parser state
across iterator items and distinguish virtual item boundaries from true
iterator exhaustion.
Centralize field completion so quote provenance, empty-field None conversion,
float conversion, strict parsing, field limits, blank rows, and escaped or
quoted newlines share one path. Apply the existing reentrant-iterator
generation check to every iterator item consumed by a record.
Also accept empty ASCII line terminators like CPython, report a missing
csv-core writer sentinel as _csv.Error instead of panicking, and name the
FSM entry point process_parser_input.
Remove the expected-failure markers from the eleven reader tests that now
pass.
Assisted-by: Codex:gpt-5.6-sol
* csv: test CRLF boundaries with custom lineterminator
* Add LightFrame for unobserved Python-to-Python calls
Allocate a lightweight LightFrame header on the DataStack instead of a
full Frame PyObject for specialized exact-args call paths. The frame is
materialized lazily only when observed (sys._getframe, traceback, tracing).
Key changes:
- LightFrame struct with borrowed pointers and no refcount overhead
- FrameSource enum (Heavy/Light) replacing the object field on ExecutingFrame
- CURRENT_LIGHT_FRAME TLS for the light frame chain
- Interleaved heavy/light frame walking in frame_at_offset_vm and
current_thread_frame_vm
- invoke_light_slots on PyFunction for the fast call path
Call overhead reduced from ~92ns to ~58ns (~37% reduction).
fib(28) improved from ~152ms to ~111ms (~27% reduction).
Assisted-by: Claude
* Streamline with_frame: inline recursion check, skip tracing dispatch
- Inline check_recursive_call instead of wrapping in with_recursion closure
- Amortize C stack overflow check to every 64th call depth
- Skip dispatch_traced_frame when use_tracing is false (hot path)
- Add recursion_depth_increment/decrement helpers
Assisted-by: Claude
* Address review findings for LightFrame correctness
- Fix materialize_light_frame previous pointer: use payload pointer
(&**frame) instead of Py<Frame> pointer, matching the chain convention
- Fix predecessor ref management: use retained_back instead of
mem::forget to keep materialized predecessors alive
- Fix materialized ref reclaim: use FrameRef::from_raw to drop the
leaked keeper ref without double-incrementing
- Use PyAtomic<u32> for LightFrame.lasti instead of AtomicU32, removing
the unsound cast on non-threading targets
- Add amortized C stack overflow check to light frame path
- Add scopeguard to light frame execution for panic safety
- Remove unnecessary Send/Sync impls from LightFrame
- Make check_c_stack_overflow pub(crate) for light frame path access
Assisted-by: Claude
* Fix CI failures: cspell, stale materialized state, f_back chain gaps
- Fix cspell: 'amortised' → 'amortized'
- Sync lasti, prev_line, and fastlocals when re-observing a materialized
light frame, fixing stale f_locals/f_lineno in test_inspect and others
- In with_frame, materialize any active light frame as the heavy frame's
predecessor and store it in retained_back, so f_back and inspect.stack
see the correct interleaved order (H_new → L_mat → H_old)
- Guard invoke_light_slots on NEWLOCALS | OPTIMIZED code flags
Assisted-by: Claude
* Unify frame chain: replace dual-chain with single FrameChainPtr
Replace the dual-chain frame management (CURRENT_FRAME AtomicPtr<Frame>
+ CURRENT_LIGHT_FRAME Cell<*const LightFrame>) with a single unified
chain using FrameChainPtr, a tagged pointer where bit 0 distinguishes
heavy (*const Frame) from light (*const LightFrame) entries.
Changes:
- Add FrameChainPtr type with tagged pointer encoding
- Change LightFrame: replace previous_light + saved_current_frame with
single previous: FrameChainPtr field
- Change InterpreterFrame.previous from AtomicPtr<Frame> to PyAtomic<usize>
- Change CURRENT_FRAME TLS from AtomicPtr<Frame> to AtomicUsize
- Remove CURRENT_LIGHT_FRAME TLS entirely
- Simplify all chain walkers (frame_at_offset, frame_at_offset_vm,
find_owned_chain_frame, for_each_current_frame, current_thread_frame)
- Remove light-frame materialization block from with_frame
- Update f_back to dispatch on FrameChainPtr tags
- Update faulthandler and gc_state to walk unified chain
- Adaptive C stack check: every call in debug, every 16th in release
Assisted-by: Claude
* Fix light frame materialization: current_frame, f_back, and GC tracking
- current_frame() now uses current_thread_frame_vm() to materialize
light frames, fixing super()/compile()/exec() when called from light
frame context
- materialize_light_frame stores retained_back for heavy predecessors
too, so f_back resolves after the caller returns
- Escaped materialized frames (traceback, sys._getframe) are GC-tracked
at cleanup so reference cycles are collectible
- Restore pre-existing @expectedFailure markers for test_frame proxy tests
Assisted-by: Claude
* Limit light frame materialization to super() only
current_frame() returns heavy-only again to avoid unnecessary
materialization that extends object lifetimes. super() init uses
current_thread_frame_vm() directly since it must inspect the
light frame's code and freevars to find __class__.
Assisted-by: Claude
* Fix remaining CI failures: C stack check, GC tracking threshold, expectedFailure
- Check C stack overflow every call in light frame path (not every
16th) to prevent stack overflow on CI workers with smaller stacks
- Fix GC tracking threshold: escaped materialized frame detection
uses strong_count > 1 (was > 2, missed frames referenced only by
traceback in the result)
- current_frame() materializes light frames so warnings, compile(),
and other callers see the correct frame
- Mark test_futures2 PyFutureTests.test_task_exc_handler_correct_context
as expectedFailure (pre-existing on main: PyTask __del__ timing)
Assisted-by: Claude
* Avoid materialization in hot paths: add light-frame-aware accessors
current_frame() stays heavy-only to avoid costly materialization on
every call. Instead, add current_globals(), current_code(), and
current_builtins() that read directly from light frame raw pointers
without creating frame PyObjects.
Callers migrated:
- PyFunction:🆕 current_builtins() for fallback builtins lookup
- compile()/eval(): current_code() for future feature flags
- type.__new__: current_globals() for __module__ detection
- typevar/typing: current_globals() for caller module name
- _io: current_code() for source path check
- super(): current_thread_frame_vm() for __class__ cell access
- warnings.warn: current_thread_frame_vm() for stack level walk
- import: current_globals() for module resolution
Assisted-by: Claude
* Fix current_locals() to include light frames
current_locals() must materialize light frames so locals() builtin
returns the correct scope. This fixes test_zipimport doctest and
other tests that rely on locals() in light frame context.
Assisted-by: Claude
* Fix clippy, profile tracing, resume_gen_frame, and faulthandler issues
- Fix clippy: collapsible_if in materialize_light_frame, map_unwrap_or in import
- Fix profile breakage: use current_thread_frame_vm in trace_event_inner to
materialize light frames, preventing frame identity mismatch when profiling
is enabled inside a light frame (e.g. profile.Profile.runctx)
- Disable light frames inside trace/profile callbacks (tracing_is_suppressed)
- Fix resume_gen_frame: skip light frames in previous chain to avoid dangling
pointers after generator suspend
- Fix faulthandler: emit '<no Python frame>' when chain has only light frames
Assisted-by: Claude
* Fix materialized frame refcount: transfer localsplus ownership on exit
Remove @expectedFailure from test_futures2: the underlying issue was that
materialized light frames cloned localsplus values (refcount +1 each),
keeping objects alive until the next GC cycle collection.
Fix: after run_light_frame returns, transfer ownership of localsplus
values from the light frame to the materialized frame, replacing the
clones. This eliminates the extra refcount, allowing objects to be
GC'd immediately when their last reference is dropped.
Also stabilize materialized frame's localsplus on the heap in
sync_materialized_on_exit before GC tracking — the data-stack backing
will be popped imminently.
Assisted-by: Claude
* Fix rustfmt, increase stack margin, address review comments
- Fix rustfmt formatting issues (lint CI failure)
- Increase STACK_MARGIN_BYTES from 2048 to 3072 words to prevent C stack
overflow in deep recursion tests on macOS CI (test_functools SIGSEGV)
- Skip dispatch_traced_frame when use_tracing is false (review comment)
- Fix post-fork top_frame initialization: walk past light frames to find
nearest heavy frame (review comment)
Assisted-by: Claude
* Remove redundant test override in test_futures2
The override only called super() without any marker, making it a
no-op that prek flags as a redundant test patch.
Assisted-by: Claude
* Fix test_pdb and test_functools CI failures
- Revert with_frame dispatch_traced_frame skip: tracing can be enabled
mid-execution (pdb.set_trace), so Return events must always be checked.
This fixes 3 test_pdb doctest failures across all platforms.
- Check C stack overflow on every with_frame call instead of every 64th:
the previous sampling approach missed overflows between checks when
native stack frames are large (invoke_light_slots + lru_cache recursion).
- Add #[inline(never)] to invoke_light_slots to prevent the optimizer
from merging its stack frame into callers.
- Add early C stack check before DataStack allocation in invoke_light_slots.
Assisted-by: Claude
* Rename Frame to FrameObject, FrameRef to FrameObjectRef
Prepare for _PyInterpreterFrame/PyFrameObject separation by
renaming the PyObject-backed frame type to FrameObject.
Python-visible name ("frame") is unchanged.
Assisted-by: Claude
* Remove Deref<Target = InterpreterFrame> from FrameObject
Replace implicit field access through Deref with explicit
iframe()/iframe_ref()/iframe_mut() calls. This decouples
FrameObject from InterpreterFrame field layout, enabling
independent modification of InterpreterFrame in future commits.
Assisted-by: Claude
* Move InterpreterFrame identity fields to raw pointers
InterpreterFrame's code/globals/builtins/func_obj fields are now
borrowed raw pointers instead of owned PyRef/PyObjectRef. Ownership
of these references is anchored in FrameObject's new owned_code,
owned_globals, owned_builtins, owned_func_obj fields.
This aligns InterpreterFrame's layout with LightFrame's existing
raw-pointer pattern, enabling future DataStack-based allocation
without PyObject heap allocation.
- Add code()/globals()/builtins()/func_obj() accessor methods
- Add FrameObject::new_ref() for combined create+allocate+patch
- Update ExecutingFrame to use &Py<PyCode> / &Py<PyDict> / &PyObject
- Update monitoring.rs functions to take &Py<PyCode> instead of &PyRef<PyCode>
- Add Send+Sync impls for InterpreterFrame (raw pointers are !Send by default)
Assisted-by: Claude
* Remove LightFrame, FrameChainPtr, and dual-frame-type infrastructure
Delete LightFrame struct, FrameChainPtr tagged pointer, FrameSource enum,
and all supporting functions: materialize_light_frame, sync_light_to_materialized,
transfer_localsplus_to_materialized, sync_materialized_on_exit,
run_light_frame, invoke_light_slots body, LocalsPlus::from_datastack_raw.
All call paths now use a single frame type (FrameObject wrapping
InterpreterFrame). The frame chain is a simple *const FrameObject linked
list through InterpreterFrame.previous.
invoke_light_slots delegates to invoke_exact_args_slots.
ExecutingFrame.frame_source replaced with ExecutingFrame.frame.
Frame chain walking simplified throughout (no more light/heavy dispatch).
Net: -724 lines of dual-frame-type complexity.
Assisted-by: Claude
* Clean up invoke_light_slots and _vm function variants
- Replace invoke_light_slots calls with invoke_exact_args_slots
- Delete the invoke_light_slots delegate method
- Replace current_thread_frame_vm/frame_at_offset_vm with their
non-_vm equivalents (no more light frame materialization needed)
- Remove stale light-frame comment in callable.rs
Assisted-by: Claude
* Fix CI lint issues
- Move github context expressions from run blocks to env blocks
(zizmor/template-injection)
- Add workflow-level permissions: {} to lib-deps-check.yaml
- Suppress zizmor excessive-permissions for lib-deps-check.yaml
(pull_request_target is required for PR comments)
- Apply rustfmt formatting
Assisted-by: Claude
* Stack-allocate InterpreterFrame for non-generator function calls
- Extract InterpreterFrame::new() from FrameObject::new()
- Change frame chain from *const FrameObject to *const InterpreterFrame
- Add InterpreterFrame.materialized field for lazy FrameObject creation
- Add vm.with_iframe()/run_frame_fast() for stack-allocated frame execution
- Move ExecutingFrame.frame to iframe: *const InterpreterFrame
- Implement lazy materialize() for on-demand FrameObject creation
- Update invoke_with_locals/invoke_prepared_exact_args to skip heap allocation
- Update frame chain walking in faulthandler, gc_state, builtins/frame
- super() reads InterpreterFrame directly without materialization
Assisted-by: Claude
* Remove unnecessary refcount and atomic ops from fast call path
- Use raw pointer for func_obj instead of cloning PyObjectRef (saves 2 atomic RMW)
- Skip owner AcqRel swap for stack-allocated frames (always Thread-owned)
Assisted-by: Claude
* Change trace field to Option<PyObjectRef>, remove vm from InterpreterFrame::new
- trace: PyMutex<PyObjectRef> -> PyMutex<Option<PyObjectRef>>
- Eliminates vm.ctx.none() refcount inc/dec per frame init
- Remove vm parameter from InterpreterFrame::new() (no longer needed)
- f_trace getter returns None when trace is unset
- f_trace setter stores None instead of the None singleton
Assisted-by: Claude
* Store current frame pointer on VirtualMachine for fast lookup
- Add current_frame_ptr Cell<usize> field to VirtualMachine
- Hot path (with_iframe) reads/writes vm.current_frame_ptr instead of TLS
- TLS CURRENT_FRAME still maintained for signal-safe traceback walking
- Use set_current_frame_nosave() to skip cross-thread top_frame update
- super(), frame_at_offset, current_thread_frame_materialize use vm Cell
Assisted-by: Claude
* Apply rustfmt
Assisted-by: Claude
* Fix clippy use_self warnings, add inline hints for hot path
Assisted-by: Claude
* Remove code refcount clone from fast call path
Defer code.to_owned() to the generator/coroutine path only. On the fast
path, use &Py<PyCode> directly — the code object is alive via the
PyFunction on the caller's stack.
Assisted-by: Claude
* Remove VM current_frame_ptr Cell (fix SIGSEGV in threading builds)
The Cell<usize> on VirtualMachine could desynchronize from the TLS
CURRENT_FRAME in nested VM scenarios (enter_vm called with different
VMs on the same thread). Revert to TLS-only frame chain management.
Keep set_current_frame_nosave() for the stack-frame fast path: it
skips cross-thread top_frame publication but still does the TLS swap.
Fix clippy not_unsafe_ptr_arg_deref in set_current_frame.
Fix unnecessary Result wrapper in make_generator_or_coro.
Assisted-by: Claude
* Fix CI failures: GC frame assertion, faulthandler Radium import, dead code warning
- Fix SIGSEGV in threading builds: GC debug assertion walked frame
chain via top_frame, which is not updated by set_current_frame_nosave
used by stack-allocated iframes. Walk CURRENT_FRAME TLS chain instead.
- Add InterpreterFrame::get_lasti() accessor, remove direct Radium
import from faulthandler.rs (fixes unused import warning).
- Add cfg_attr allow(dead_code) for from_payload_ptr without threading
(fixes WASM build warning).
Assisted-by: Claude
* Fix materialized frame: copy localsplus, clear stale previous pointers
- materialize_slow: copy fastlocals snapshot to heap instead of empty
localsplus (fixes index-out-of-bounds panic in locals()/f_locals)
- materialize_slow: set previous to null (stack iframes become dangling)
- with_frame/resume_gen_frame: clear iframe.previous on pop (prevents
dangling pointers to freed stack iframes)
- f_back: walk TLS CURRENT_FRAME chain instead of top_frame for
cross-thread lookup (top_frame is stale for stack-allocated iframes)
- check_locals_access: match materialized frames by comparing the
Py<FrameObject> address stored in the stack iframe's materialized field
Assisted-by: Claude
* Fix Windows clippy: gate unix-only imports and variables
FrameObject, Py imports and top_iframe variable are only used in
unix+threading cfg blocks. Gate them properly for non-unix builds.
Assisted-by: Claude
* Fall back to FrameObject when tracing; fix cross-thread frame access
- invoke_with_locals: fall back to heap-allocated FrameObject path
when use_tracing is active, so trace/profile callbacks fire correctly.
This fixes test_trace, test_bdb, test_monitoring regressions.
- set_current_frame: publish top_iframe alongside top_frame in ThreadSlot
so cross-thread readers (sys._current_frames) can materialize
stack-allocated frames from other threads under stop-the-world.
- get_all_current_frames: use TLS CURRENT_FRAME for current thread,
fall back to top_iframe materialization for other threads.
- Use set_current_frame (not nosave) in with_iframe for proper
cross-thread visibility.
Assisted-by: Claude
* Apply rustfmt
Assisted-by: Claude
* Fix Windows: gate unix-only top_iframe_ptr variable
Assisted-by: Claude
* Fix frame chain: f_back, retained_back, owner, faulthandler
- Set materialized frame owner to FrameObject (not Thread) so
frame.clear() works on traceback frames
- Fix f_back for materialized frames: walk TLS chain to find source
iframe and materialize its previous frame
- Fix faulthandler dump_traceback to use dump_live_frames (reads
live lasti from stack iframes instead of stale materialized copies)
- Add retained_back propagation in with_frame cleanup: when a frame
escapes, materialize caller and mark it escaped for chain propagation
- Fix release_datastack_frame: don't overwrite retained_back if
already set by with_frame cleanup
Assisted-by: Claude
* Apply rustfmt
Assisted-by: Claude
* Refine with_frame retained_back: use frame_obj instead of materialize
Avoid materializing caller frames in with_frame cleanup to prevent
creating localsplus snapshots that keep extra refcounts. Only use
existing FrameObjects (frame_obj) for retained_back. Also keep
previous pointer alive in release_datastack_frame since the caller
is still executing at that point.
Assisted-by: Claude
* Fix f_lineno: use Cell<u32> for prev_line, update on every instruction
prev_line was only updated when tracing was active, causing f_lineno
to return stale values when observed mid-CALL (e.g. from warnings.warn
or sys._getframe). Changed prev_line from u32 to Cell<u32> for
interior mutability so it can be read safely while ExecutingFrame
holds a shared reference, and update it on every instruction so
f_lineno always returns the correct line.
Assisted-by: Claude
* Fix f_lineno for materialized frames: read live prev_line from TLS chain
Materialized FrameObjects have a snapshot of prev_line from materialize
time. When f_lineno is called on a materialized frame that is still
executing (its source iframe is on the TLS chain), walk the TLS chain
to find the source iframe and read its live prev_line. This fixes
lineno tracking in warnings, gettext, and other modules that read
frame line numbers during function calls.
Also fixed the pointer comparison: use from_payload_ptr to convert
FrameObject payload address to Py<FrameObject> address for matching
against the materialized field.
Assisted-by: Claude
* Sync live frame state for materialized frames
- Add find_live_source_iframe() to walk TLS chain and find the live
source InterpreterFrame for a materialized FrameObject
- f_lasti: read live lasti from source iframe when available
- f_lineno: use read_volatile for live prev_line from source iframe
- sync_visible_locals_to_mapping: read live localsplus from source iframe
- framelocalsproxy_getval: read live localsplus from source iframe
- with_iframe cleanup: sync localsplus, prev_line, lasti to materialized
FrameObject when the frame returns
- sync_fastlocals_from: new LocalsPlus method to update fastlocals
Fixes test_inspect (stale f_locals, stale lineno in positions),
test_listcomps (stale iteration variable in locals()).
Assisted-by: Claude
* Fix retained_back: use materialize_chain and strong_count check
- Add materialize_chain() that creates a lightweight FrameObject with
empty localsplus for f_back chain building only. This avoids cloning
local variables which would create extra refcounts (fixes test_memoryview).
- with_frame cleanup: use strong_count > 1 instead of escaped flag to
determine if retained_back is needed, ensuring chain propagation.
- with_iframe cleanup: use materialize_chain for retained_back to avoid
extra refcounts from localsplus snapshots.
Fixes test_memoryview (refcount from retained_back chain) and
test_traceback.test_extract_stack (f_back chain depth).
Assisted-by: Claude
* Fix cross-thread f_back chain for sys._current_frames/exceptions
- get_all_current_frames: materialize entire frame chain and link
retained_back during stop-the-world for other threads
- f_back: use stop-the-world to safely materialize cross-thread frame
chains on unix when prev iframe is on another thread
- f_back: check frame_obj() before STW for fast path
Fixes test_sys.test_current_frames and test_sys.test_current_exceptions.
Assisted-by: Claude
* Fix monitoring LINE events and faulthandler thread dump
- Skip all instrumented opcodes in bytecode loop prev_line update to
avoid defeating InstrumentedLine de-duplication
- Add prev_line update in execute_instrumented for non-RESUME/non-LINE
instrumented opcodes
- update_events_mask: walk TLS iframe chain to re-instrument all frames
including stack-allocated ones (fixes missing LINE events for inline
frames already past RESUME)
- faulthandler: use top_iframe fallback for stack-allocated frames in
dump_all_threads and watchdog_thread
- faulthandler: use dump_live_frames for current thread dump
Fixes test_monitoring LINE/CALL tests, test_faulthandler thread dumps.
Assisted-by: Claude
* Fix clippy: remove duplicate Radium import, use previous() helper
Assisted-by: Claude
* Fix materialized frame issues: GC tracking, live locals, f_trace propagation
- Track materialized FrameObjects in GC so cycle collection works
(fixes __del__ not called at exit for exception traceback cycles)
- materialize_chain returns owned PyRef without temporary_refs to
avoid defeating GC cycle detection from non-GC-tracked storage
- framelocalsproxy_setval writes to live source iframe when available
(fixes f_locals proxy writes not affecting executing frame)
- f_lineno uses lasti-based line for non-executing frames instead of
prev_line (fixes PEP 626 wrong line after exception unwind)
- f_trace/f_trace_lines/f_trace_opcodes setters propagate to live
source iframe (fixes pdb set_trace not working through materialized
frames)
- frame.clear() checks find_live_source_iframe to reject clearing a
frame that is backed by a live stack-allocated iframe
- Fix clippy warnings and unused imports in faulthandler
- Remove unsafe top_iframe access from faulthandler watchdog thread
Assisted-by: Claude
* Fix faulthandler crashes, GC tracking timing, and test_generators
- Use top_iframe instead of top_frame for frame chain walking in
faulthandler (dump_all_threads and watchdog), avoiding stale or
GC-cleared FrameObject pointer dereference
- Defer GC tracking of materialized FrameObjects to with_iframe
cleanup (after set_current_frame restores old chain), preventing
premature collection while temporary_refs still holds the only ref
- Add FrameObject::try_iframe() for safe access to potentially
GC-cleared frames
- Remove expectedFailure for test_exhausted_generator_frame_cycle
(now passes with GC tracking fix)
- Add 'noalias' to cspell dictionary
Assisted-by: Claude
* Fix Windows frame dump and remove dead code
- Publish top_iframe on all platforms (not just unix) so Windows
faulthandler and sys._current_frames can see stack-allocated frames
- Use top_iframe instead of frames Mutex in Windows faulthandler
dump and watchdog paths
- Use top_iframe in non-unix get_all_current_frames for sys._current_frames
- Remove unused dump_traceback_thread_chain (replaced by top_iframe walk)
- Mark test_pdb_await_support as expected failure
Assisted-by: Claude
* Fix ThreadSlot init on all platforms, skip test_pdb_await_support
- Add top_iframe to ThreadSlot initializer in init_thread_slot_if_needed
(fixes wasm32, Windows compile errors)
- Skip test_pdb_await_support on RustPython (async pdb exception
callback receives None exc argument)
Assisted-by: Claude
* Remove dead faulthandler code (dump_frame_from_ref, dump_traceback_thread_frames, try_iframe)
These functions are no longer called after switching all frame chain
walking to use top_iframe directly.
Assisted-by: Claude
* Fix Windows sys._current_frames: materialize full frame chain
The non-unix get_all_current_frames path was only materializing the
top iframe without linking retained_back, so f_back chain walking
could not find deeper frames like f123() in test_sys tests.
Replicate the full chain materialization from the unix path.
Assisted-by: Claude
* Use STW-based cross-thread f_back on all platforms
Replace the non-unix frames-mutex fallback with the same
stop-the-world iframe chain materialization used on unix.
Fixes test_current_exceptions on Windows where stack-allocated
iframes were not found in the FrameObject-only frames list.
Assisted-by: Claude
* Fix set_f_lineno to write to live iframe, use STW for cross-thread access
- set_f_lineno: write lasti/pending_stack_pops to the live source
iframe instead of the materialized copy so debugger jumps take
effect on stack-allocated frames
- f_back cross-thread: enter STW before dereferencing the prev
pointer to prevent use-after-free if the owning thread returns
- get_all_current_frames (non-unix): add STW protection when
walking cross-thread iframe chains, matching the unix path
Assisted-by: Claude
* Guard current_location() against lasti==0, always capture retained_back
- current_location(): return first_line_number instead of
panicking when lasti is 0 (before first instruction executes)
- with_frame retained_back: materialize the caller iframe when
needed so f_back always resolves for escaped FrameObjects
Assisted-by: Claude
* Fix format, revert eager retained_back materialization
Remove double blank line in frame.rs (lint failure).
Revert with_frame retained_back to only use already-materialized
caller FrameObjects — eagerly materializing the caller added
refcounts on local variables, preventing timely deallocation and
causing test_io, test_memoryview, and test_futures failures.
Assisted-by: Claude
* Remove rustpython-unicode-isolation-issue.md from gitignore
Assisted-by: Claude
* Remove test_pdb_await_support skip: async pdb now works correctly
Assisted-by: Claude
* Fix STOP_ITERATION monitoring: wrap value in StopIteration instance
fire_stop_iteration was passing the raw iterator return value to
callbacks, but the STOP_ITERATION event callback signature expects
a StopIteration exception instance. Wrap non-StopIteration values
in a new StopIteration(value), matching PyMonitoring_FireStopIterationEvent.
This fixes test_pdb_await_support where bdb's exception_callback
received None instead of a StopIteration instance.
Assisted-by: Claude
Validate parameter annotations while converting ast.arguments, before
merging positional and keyword-only defaults. This gives invalid annotation
contexts precedence over default-list shape errors, as in CPython.
Merge positional defaults before keyword-only defaults so an excess
positional-default error also takes precedence over a keyword-only default
length mismatch.
Remove the expected-failure markers from the FunctionDef and Lambda AST
validator tests that now pass.
Assisted-by: Codex:gpt-5.6-sol
* fix dict update type error
* Use tuple for dictionary update sequence pairs
* Propagate errors while adding dict update sequence notes
* Match BaseException.add_note error message with CPython
* csv: support multi-character lineterminator in writer
Store the dialect line terminator as an owned String instead of the
single-byte csv_core::Terminator, dropping PyDialect's Copy derive. The
manual writer paths emit the full terminator; the csv-core-backed
QUOTE_ALL/QUOTE_NONNUMERIC paths emit a sentinel byte (preserving
csv-core's quote/empty-record bookkeeping) and append the real
terminator. field_needs_quotes/escape now quote a field containing any
terminator byte. The reader ignores lineterminator and always uses CRLF,
matching CPython and avoiding mid-UTF-8 record splits.
* csv: unmark now-passing test_write_lineterminator
* csv: address review feedback
- Remove the redundant per-branch sentinel terminator setup in to_writer;
the unconditional terminator call after the match is the single source.
- Reword the empty-lineterminator errors to "must not be empty" (the
constraint is non-empty, not single-character) on both entry points.
- Promote the sentinel invariant check in writerow from debug_assert_eq!
to assert_eq! so it also guards release builds.
- Add a snippet case for a field containing a line-break byte to ensure
the csv-core path drops only the trailing terminator.
* csv: reject non-ASCII lineterminator
The writer decides what to quote and escape by comparing raw bytes, so a
non-ASCII terminator quoted a field that merely shared a UTF-8 lead byte,
and QUOTE_NONE escaped individual bytes of the terminator and then failed to
decode the record back to a string.
Reject non-ASCII terminators, including lone surrogates, as csv.Error when
the dialect is parsed, and leave code-point-wise handling to a follow-up
(#8310). The non-string error message now matches CPython as well.
* Fix SymbolUsge::Iter
* fix os.rename
* fix sre
* Re-export rename from crate::posix on Windows
posix_windows.rs is mounted as the `posix` module on Windows, so
`crate::posix_windows` does not resolve.
Assisted-by: Claude
* Apply rustfmt to sre_engine tests
Assisted-by: Claude
* Update test expectations for the sre and tokenize fixes
- test_re: drop expectedFailure from test_word_boundaries,
test_possessive_quantifiers and test_bug_gh101955
- test_inspect, test_pydoc: drop expectedFailure from the signature
tests that now pass
- test_pdb: drop expectedFailure from the two file-modification tests
and +EXPECTED_FAILURE from the test_post_mortem_chained and
test_pdb_asynctask doctests
- test_unittest: mark test_autospec_on_bound_builtin_function as
expectedFailure; inspect.signature() now succeeds on time.ctime
because of its auto-generated __text_signature__
Assisted-by: Claude
* Fix dict iterator __reduce__ to resume from current position
Pickling a partially-consumed dict / dict-view iterator restarted from the
beginning because __reduce__ materialized every entry and ignored the
iterator's position. Walk from the current position (mirroring next) so
only the not-yet-yielded entries are captured, matching CPython, which
reduces both directions to iter(remaining).
Fixing the reverse iterator also uncovered two pre-existing bugs in
reverse iteration itself, both from prev_entry saturating at 0: a hole
just above index 0 made the entry at 0 yield twice, and deleting the
first-inserted key made reversed() loop forever. prev_entry now returns
the found entry's actual index and stops cleanly at index 0, and the
reverse iterator/reduce detect exhaustion from that index.
Verified against CPython 3.14 across all pickle protocols, dict states
(including deletions), iterator kinds, and consumption counts. No
regressions in test_dict, test_dictviews, test_ordered_dict,
test_collections, test_userdict, test_iter, or test_copy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Apply cargo fmt to dict.rs imports
Collapse the builtins import block left multi-line after removing the
now-unused builtins_reversed import.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Unmark TestSyncManagerTypes.test_dict as expectedFailure
The dict iterator __reduce__ fix makes SyncManager dict proxies pickle
correctly, so this test now passes under spawn and forkserver. Remove the
stale expectedFailure marker that was causing an UNEXPECTED SUCCESS CI
failure.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Support the CPython 3.13+ format spec syntax that puts a grouping option
after the precision, so `format(1234.56789, '.6,f')` gives
'1234.567,890'. The integer and fractional parts group independently and
may use different separators (`,.6_f` -> '1,234.567_890'), so the spec
carries a separate `frac_grouping_option`.
`parse_precision` now consumes the separator that follows the precision
digits and rejects a `,`/`_` mix. A repeated separator is deliberately
left in the spec so the trailing-text check reports it, which is how
CPython arrives at a different message there. A dot followed by neither
digits nor a separator now raises "Format specifier missing precision"
rather than an unrelated error.
Fraction digits group away from the decimal point, so the last group may
be short, and any exponent or percent tail is left intact. The
separators count toward the field width, so zero padding of the integer
part reserves room for them.
'n' takes its separators from the locale and so cannot carry one. The
complex locale path rewrites 'n' to 'g' before delegating, so it has to
validate first; that also makes `format(1+2j, ',n')` fail as it does in
CPython, which it previously did not.
Reference (CPython 3.14), Python/formatter_unicode.c:
- parsing: https://github.com/python/cpython/blob/3.14/Python/formatter_unicode.c#L257-L299
- 'n' rejection: https://github.com/python/cpython/blob/3.14/Python/formatter_unicode.c#L361-L367
- number split: https://github.com/python/cpython/blob/3.14/Python/formatter_unicode.c#L488-L516
- zero-pad width: https://github.com/python/cpython/blob/3.14/Python/formatter_unicode.c#L604-L606
Remove `@unittest.expectedFailure` from tests that now pass
(2 in test_format, 1 in test_float).
Assisted-by: Claude:claude-opus-5
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* Implement termios.tcgetwinsize/tcsetwinsize
Adds ioctl(TIOCGWINSZ/TIOCSWINSZ) wrappers in host_env and the
corresponding Python-facing functions in the termios stdlib module,
removing the associated expectedFailure markers in test_termios.py.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT
* Use ret < 0 convention for ioctl error checks in termios winsize
POSIX ioctl only guarantees -1 on failure, not exactly 0 on success;
matches the existing check_libc_neg convention used elsewhere in
host_env (e.g. fcntl.rs, posix.rs::get_terminal_size).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT
* Reject non-sequence size in tcsetwinsize
extract_elements_with also accepted dicts (treating keys as elements).
Switch to try_sequence(), matching CPython's PySequence_Check, so only
real sequences (tuple/list/etc.) are accepted.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT
* Use `core::mem::zeroed` instead of `std::mem::zeroed`
`std::mem` just re-exports `core::mem`, but clippy prefers importing
from core when there's no OS dependency. This was breaking the wasm CI build.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT
* Use try_index for tcsetwinsize row/col conversion
CPython's PyLong_AsLong calls __index__ on non-int objects before
converting, so downcast_ref::<PyInt> was stricter than CPython.
try_index matches that behavior and is more concise.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT
* Mark test_fork/test_spawn_doesnt_hang as expected failures
Both were hidden by a skipIf on tty.tcgetwinsize, which now exists.
Root cause: pty.fork() calls os.login_tty(), which isn't implemented,
so the forked child crashes before the test body runs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT
* Use unittest.skip instead of expectedFailure for pty.fork() tests
expectedFailure still runs the test body, so pty.fork()'s real fork()
still happens and the child crashes on missing os.login_tty inside the
parallel test runner's worker process, corrupting its JSON reporting
channel ("worker bug"). skip prevents the method from running at all,
avoiding that. Verified locally with -j 2.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT
* Clarify skip reason for pty.fork() tests
* Simplify tcgetwinsize/tcsetwinsize fd argument with Fildes destructuring
* Explain why pty.fork() tests use skip instead of expectedFailure
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
CPython raises ProgrammingError with "Binding N is a named parameter"
when a query uses named placeholders (, , ) but
the caller passes a sequence instead of a mapping.
RustPython's bind_parameters_sequence() did not check whether each
binding slot is a named parameter, so no error was raised.
Fix: call sqlite3_bind_parameter_name() for each slot in
bind_parameters_sequence(). If the first byte of the returned name is
not '?' (i.e. it is a named placeholder), raise ProgrammingError before
attempting to bind.
Assisted-by: GitHub Copilot:claude-sonnet-4-6
Raise OverflowError when a finite f64 value overflows while being converted for the struct f format. Remove the expected-failure marker from the corresponding regression test.
Assisted-by: Codex:gpt-5
Row(cursor, data) raised ValueError when cursor.description was None.
CPython allows this case and returns an empty key list.
- Use empty tuple when description is None instead of raising
- Include the key name in the IndexError when a string key is not found
Assisted-by: GitHub Copilot:claude-sonnet-4-6
When a Connection is explicitly closed via con.close(), subsequent
operations (cursor(), commit(), rollback(), create_function(), etc.)
should raise ProgrammingError with 'Cannot operate on a closed database.'
to match CPython behaviour.
Previously, _db_lock() always returned 'Base Connection.__init__ not
called.' when self.db was None, without distinguishing between a
connection that was never initialised (subclass before __init__) and
one that was initialised and then explicitly closed.
Fix: inspect the initialized atomic flag — if True but db is None,
the connection was closed; if False, it was never initialised.
Assisted-by: GitHub Copilot:claude-sonnet-4-6
* Keep StringIO text operations CPython-compatible
Constraint: email.feedparser depends on StringIO(newline='') recognizing CR, LF, and CRLF boundaries.
Rejected: an email-specific parser workaround | StringIO is the shared root cause.
Confidence: high
Scope-risk: moderate
Directive: Keep internal storage byte-based; convert only at StringIO API boundaries.
Tested: prek run --all-files; test_memoryio; test_email; full workspace verification by contributor
Assisted-by: Codex:gpt-5.6-sol
* Run restored StringIO conformance tests
Constraint: upstream StringIO tests already cover the repaired behavior.
Rejected: new RustPython-only regression tests | existing CPython coverage is sufficient.
Confidence: high
Scope-risk: narrow
Directive: Remove expected-failure markers only while the inherited tests pass.
Tested: prek run --all-files; test_memoryio; test_email
Assisted-by: Codex:gpt-5.6-sol
* Keep StringIO newline handling consistent
Constraint: StringIO must apply its newline mode consistently when constructing, writing, and reading text.\nRejected: a readline-only fix | it leaves CR and CRLF modes internally inconsistent.\nConfidence: high\nScope-risk: moderate\nDirective: Keep StringIO buffer contents valid WTF-8 before using unchecked views.\nTested: prek run --all-files; cargo clippy -p rustpython-vm -- -D warnings; test_memoryio; test_shlex; test_email; workspace excluding rustpython-capi; full workspace manually verified by contributor\nNot-tested: local rustpython-capi workspace test crashes with a pre-existing macOS SIGSEGV\nAssisted-by: Codex:gpt-5.6-sol
* bz2: return streaming output from BZ2Compressor.compress()
Return output produced during compression instead of waiting for flush().
Constraint: Preserve encoder state without flushing the BZ2 stream
Confidence: high
Scope-risk: narrow
Directive: Do not flush merely to retrieve partial output
Tested: test_bz2; stdlib_bz2.py; prek run --all-files
Assisted-by: Codex:gpt-5.6-sol
* bz2: remove redundant streaming-output snippet
* csv: handle empty fields with skipinitialspace
skipinitialspace preprocessed records by trimming both ends of every split
field. Fields containing only spaces could therefore produce an invalid
slice, while trailing spaces were removed even though CPython preserves them.
Trim only field prefixes in the csv-core path and make the remaining
all-space trimming helper return an empty slice safely.
QUOTE_NOTNULL and QUOTE_STRINGS add another distinction: an unquoted empty
field becomes None, but a quoted empty field remains an empty string. Both
forms have identical decoded bytes, so extend the existing QUOTE_NONE custom
reader into one shared path that records whether each field started quoted.
Use that metadata only for the null conversion while retaining QUOTE_NONE
escape behavior.
Unmark Test_Csv.test_read_skipinitialspace now that its standard,
QUOTE_NOTNULL, and QUOTE_STRINGS cases pass.
This keeps the existing one-item reader lifecycle and does not add the larger
strict or multiline parser state machine.
Assisted-by: Codex:gpt-5-sol
* csv: make skipinitialspace quote-aware
The csv-core preprocessing path split each raw iterator item on every
delimiter before trimming field prefixes. Delimiters inside quoted fields were
therefore treated as separators, so spaces after them could be removed before
csv-core parsed the record.
Extract the quote, escape, delimiter, and field-start transitions from
read_quote_record() into a small per-item scanner. Reuse its events in the
skipinitialspace preprocessor and discard only InitialSpace events while
copying all other source bytes unchanged.
This keeps the custom quote-mode reader and preprocessing logic aligned
without introducing persistent multiline or strict parser state. Remove the
delimiter cache that was only needed by the old split-and-join path and add a
regression for spaces following a delimiter inside a quoted field.
Assisted-by: Codex:gpt-5-sol
* csv: reject reentrant reader advancement
Snapshot the reader generation before calling the input iterator. If a nested
call completes a record and advances the same reader, reject the outer call
with csv.Error instead of continuing with stale parser state.
Increment the generation on each successful record path, matching CPython's
handling of the existing reentrant-reader regression. Enable that test by
removing its expectedFailure marker.
Assisted-by: Codex:gpt-5.6-sol
* csv: invalidate outer reads after reentry
Advance the reader generation immediately after the reentrancy check, before
validating or parsing the returned input item. This invalidates the outer call
even when a nested read later fails validation or parsing.
Remove the per-return generation updates now that the common path advances it
exactly once.
Assisted-by: Codex:gpt-5.6-sol
* csv: generation type conversion(usize to u64)
On 32-bit targets (like WebAssembly, which RustPython supports), usize is 32-bit.
Csv with over 32-bit rows can raise panic on these platform.
type conversion from usize to u64.
* csv: quote embedded CR/LF under QUOTE_MINIMAL with custom lineterminator
QUOTE_MINIMAL rows were serialized by csv-core, which only quotes bytes
registered for the configured terminator, so fields containing '\r' or
'\n' were left unquoted when the lineterminator was not CR/LF (e.g. '!'
or '\0'). Route QUOTE_MINIMAL through a hand-written path that uses
field_needs_quotes, which always treats '\r'/'\n' as needing quotes.
A single empty field is quoted and an empty row emits only the
terminator, matching CPython. QUOTE_ALL / QUOTE_NONNUMERIC still use
csv-core.
Assisted-by: Claude Code:claude-opus-4-8
* csv: unmark now-passing writer tests
test_write_iterable and test_write_empty_fields pass with the
QUOTE_MINIMAL writer fix (empty rows no longer emit an empty field).
Assisted-by: Claude Code:claude-opus-4-8
* Store StopIteration.value in a struct field like SystemExit
* Unmark test_pydoc's test_member_descriptor
* Use expect instead of unwrap in new_stop_iteration
Dual-Root Mismatch in RustPython:
Unlike CPython, RustPython manages the standard library
in two separate locations in the workspace:
- Workspace 'Lib/': Contains the test suite ('Lib/test/...') and
standard library files.
- crates/pylib/Lib/: A helper directory used for packaging.
'crates/pylib/Lib' is a symlink pointing to the real workspace '../../Lib/'.
Runtime Directory Discrepancy:
- 'sys.path' contains 'crates/pylib/Lib' (the path to the symlink).
- Thus, when 'test.support' is imported, its '__file__' is resolved relative
to the symlink: '/opt/RustPython/crates/pylib/Lib/test/support/__init__.py'.
- Consequently, 'STDLIB_DIR' resolves to the symlink root:
'/opt/RustPython/crates/pylib/Lib'.
- However, since 'test_program.py' is executed directly from the workspace,
its 'pkg_dir' is: '/opt/RustPython/Lib/test/test_unittest/testmock'.
- When 'os.path.relpath(pkg_dir, top_dir)' is called:
---
os.path.relpath(
"/opt/RustPython/Lib/test/test_unittest/testmock",
"/opt/RustPython/crates/pylib/Lib"
)
# Result: "../../../Lib/test/test_unittest/testmock"
---
Because the relative path starts with '..', the assertion fails.
Assisted-by: Gemini
* weakref: reuse callback-less proxy objects like CPython
Make PyWeakProxy a #[repr(transparent)] view sharing PyWeak's payload
(manual PyPayload with shared PAYLOAD_TYPE_ID + class check), so a proxy
is the weakref-list node itself and add()'s reuse applies to it
directly. Port CPython's insert_weakref/get_basic_refs slot discipline:
genericref at head, generic proxy right after it, with callback/class
re-verificationbefore reuse. Constructor moves to a slot_new override
since reuse returns an existing object. Drops the hidden __weakproxy
marker subclass and the proxy branch in PyWeakref_GetRef.
Assisted-by: Claude Code:claude-opus-4-8
* weakref: always validate class in basic-proxy slot lookup
* weakref: rustfmt find_generic_proxy_ptr
* weakref: unify reuse check shape
When `ordered_attributes` is set, report attributes to the
StartElementHandler as a flat `[name, value, ...]` list matching
CPython's my_StartElementHandler, instead of always passing a dict.
`xml.etree.ElementTree` and `xml.dom.minidom` (via expatbuilder) both
enable this flag and rely on the flat-list contract, so parsing any
element with attributes previously failed: the dict made `_start` do
integer indexing, raising KeyError which was silently swallowed by the
handler invoker, leaving the tree empty ("missing toplevel element").
Reference (CPython 3.14):
https://github.com/python/cpython/blob/3.14/Modules/pyexpat.c#L439-L443
Remove `@unittest.expectedFailure` from tests that now pass
(23 in test_xml_etree, 15 in test_minidom, 1 in test_regrtest).
Assisted-by: Claude:claude-opus-4-8
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* csv: map QUOTE_MINIMAL to Necessary quote style
QuoteStyle::Minimal was mapped to csv_core's Always, which quoted every
field even under QUOTE_MINIMAL. Map it to Necessary so only fields that
require quoting are quoted.
Unmarks Test_Csv.test_write_quoting.
Assisted-by: Claude Code:claude-opus-4-8
* csv: apply dialect quoting and lineterminator in writer
csv.writer ignored the quoting and lineterminator from a dialect (whether
passed by name or object), always using the defaults. Resolve both from the
dialect via get_quoting()/get_lineterminator(), mirroring get_delimiter(), so
an explicit keyword argument still overrides the dialect.
Unmarks TestDialectUnix.test_simple_writer.
Assisted-by: Claude Code:claude-opus-4-8
* module: honor module-level __dir__ (PEP 562)
dir(module) ignored a module-level __dir__ function and returned the raw
__dict__ keys. Look up __dir__ in the module dict and, when present, call
it and return its result — matching CPython's module___dir___impl. The
existing expectedFailure markers on test_module_dir/test_module_dir_errors
are removed since both now pass.
Fixes#8231
Assisted-by: Claude Code:claude-opus-4-8
* test: unmark zoneinfo test_dir_contains_all (PEP 562 __dir__)
Honoring a module-level __dir__ makes dir(zoneinfo) contain everything
in __all__, so test_dir_contains_all now passes. Remove its stale
expectedFailure marker to avoid an unexpected-success CI failure.
Assisted-by: Claude Code:claude-opus-4-8
* test: unmark test_support.test_check__all__ (PEP 562 __dir__)
check__all__ iterates dir(module); honoring a module-level __dir__ fixes
dir(unittest), so test_check__all__ now passes. Remove its stale
expectedFailure marker to avoid an unexpected-success CI failure.
Assisted-by: Claude Code:claude-opus-4-8
* Support array 'w' (Py_UCS4) typecode
Python 3.13 added the 'w' typecode holding fixed 4-byte unicode code
points, unlike 'u' which is platform-dependent wchar_t. Add a Ucs4Char
element type and route it through the existing unicode-array paths:
constructor from str, fromunicode()/tounicode(), __reduce_ex__, repr,
byteswap, and the machine-format code (Utf32). Also register 'w' as a
struct format char in vm::buffer.
Remove the now-passing expectedFailure markers for 'w' tests in
test_array.py.
Assisted-by: Claude
* Reject non-native byte order for the 'w' format char
FormatType::parse accepted 'w' after '<', '>', '!', or '=', but the
non-native branch of FormatType::info has no Ucs4Char arm and hits
unreachable!(), so struct.calcsize('>w') panicked the interpreter.
Add Ucs4Char to the existing native-only filter in FormatCode::parse
(alongside SSizeT/SizeT/VoidP) so those formats fail with the regular
"bad char in struct format" error instead. Native 'w' and
memoryview(array('w', ...)) are unaffected.
Assisted-by: Claude
* Unmark 'w'-typecode tests in test_re and test_csv
test_re's test_empty_array and test_csv's test_char_write construct
array('w', ...) and were marked expectedFailure while the typecode was
unsupported. Now that 'w' works they pass, and regrtest counts the
unexpected successes as failures in CI. Remove the two markers, which
restores both tests to their CPython v3.14.6 form; test_re, test_csv,
and test_array all pass.
Assisted-by: Claude
* Restore pyexpat lexical callback compatibility
Forward processing instructions, comments, and CDATA boundaries already emitted by xml-rs so registered Python handlers observe CPython-compatible events.
Constraint: Reuse existing xml-rs events without changing the dependency
Confidence: high
Scope-risk: narrow
Tested: prek; test_pyexpat; test_sax; 402 extra tests; workspace tests; clippy
Assisted-by: Codex:gpt-5.6-sol
* Recognize XML compatibility restored by lexical events
Remove stale expected-failure markers for CPython XML tests now satisfied by pyexpat lexical-event forwarding, and simplify parser event error propagation.
Constraint: CPython test edits are limited to removing stale expectedFailure markers
Confidence: high
Scope-risk: narrow
Directive: Keep parser event dispatch aligned with pyexpat handler behavior
Tested: user-verified full suite; prek run --all-files; cargo run --release -- -m test test_pyexpat test_xml_etree test_xml_etree_c test_minidom test_sax
Assisted-by: Codex:gpt-5.6-sol
* csv: apply dialect escapechar when reading
ReaderBuilder only received escapechar when it was passed directly as a
keyword argument. A dialect object's or registered dialect's escapechar was
visible through reader.dialect but not used by csv_core.
Pass escapechar through for named, object, and default dialect paths. This
makes escaped delimiters inside quoted fields follow the configured dialect,
for example '"abc\,def"' reads as 'abc,def'.
Unmark TestQuotedEscapedExcel.test_read_escape_fieldsep.
Assisted-by: Codex
* csv: handle escaped delimiters with QUOTE_NONE
csv_core applies escapechar while reading quoted fields, but its unquoted
field state treats an escaped delimiter as a delimiter. Consequently,
QUOTE_NONE with escapechar split 'abc\,def' into 'abc\' and 'def'.
Add a small QUOTE_NONE record parser for dialects with escapechar. It keeps
the byte following an escape character as field data, so escaped delimiters
do not end a field while ordinary delimiters and record terminators retain
their usual meaning.
Unmark TestEscapedExcel.test_read_escape_fieldsep.
Assisted-by: Codex
* csv: respect skipinitialspace with QUOTE_NONE
The QUOTE_NONE parser returned before the normal reader's whitespace
preprocessing, so it ignored skipinitialspace when escapechar was set.
Track whether parsing has just followed a delimiter and skip only ordinary
spaces in that position. Escaped spaces remain field data.
Add a stdlib_csv snippet regression test for QUOTE_NONE with escapechar and
skipinitialspace.
Assisted-by: Codex:gpt-5.6-terra
* csv: deduplicate reader dialect configuration
to_reader configured ReaderBuilder separately for named, object, and default
dialects even though the configuration was identical.
Resolve the PyDialect once, then apply delimiter, double_quote, escapechar,
and quotechar in one place. Preserve the existing unregistered-name fallback
and required excel dialect lookup behavior.
Assisted-by: Codex:gpt-5.6-terra
Match.expand delegated to re._expand, which CPython removed in 3.12
during the template-expansion refactor. With the 3.14 stdlib that
helper no longer exists, so expand raised AttributeError.
Route expand through the compiled-template path
(Template::compile -> re._compile_template) and share the
template-filling logic with Pattern.sub via a Match::expand_template
helper. Each caller still compiles the template once, so sub keeps
its no-recompile-per-match fast path.
Remove the expectedFailure marker on test_re.test_expand.
Assisted-by: Claude Code:claude-opus-4-8
