mirror of
https://github.com/RustPython/RustPython.git
synced 2026-10-28 18:59:04 +08:00
358f6a875afa154f9eb04411c0bc8288b2f1082a
17342
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jeong, YunWonandLee Dogeon
|
358f6a875a |
native _json, _pickle, pyexpat, and _elementtree (#8741)
* _json: add native make_encoder (c_make_encoder)
json.encoder fell back to the pure-Python _make_iterencode because _json
had no make_encoder, making json.dumps ~25x slower than CPython. Implement
the C encoder semantics natively: str/None/bool/int/float/list/tuple/dict
ordering, key coercion and skipkeys, sort_keys via the VM's fallible sort
(comparison errors propagate instead of panicking), circular reference
detection, indent handling, allow_nan, default() fallback with recursion
guard, and CPython's add_note annotations. Removes the now-passing
expectedFailure markers in test_json.
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: add UseForeignDTD, fix silently swallowed handler exceptions
genshi's XML template loader (bm_genshi) called xmlparser.UseForeignDTD(),
which pyexpat.rs didn't implement, raising AttributeError. Add it as an
accepted no-op: the pyexpat module here wraps the pure-Rust xml-rs crate
(not real libexpat), which has no hook for splicing in a foreign DTD.
Root-causing the resulting bug also uncovered why genshi's template
generate()/render() produced empty output instead of erroring: do_parse's
invoke_handler swallowed every exception raised by a Python callback
(handler.call(...).ok()), and genshi's expat callbacks fail immediately
by reading `parser.CurrentLineNumber`/`CurrentColumnNumber`, which
pyexpat.rs never exposed. The AttributeError was silently discarded,
so no start/end/text events ever reached genshi's queue.
Fix both: propagate handler exceptions out of Parse()/ParseFile() (matching
libexpat/CPython semantics), and add CurrentLineNumber/CurrentColumnNumber/
CurrentByteIndex as read-only attributes backed by xml-rs's per-event
TextPosition (byte index is derived from it and is exact for ASCII
documents). This also fixes test_minidom's
testExceptionOnSpacesInXMLNSValue, previously an expected failure because
the ValueError from an invalid xmlns value was swallowed the same way.
PositionTest in test_pyexpat.py checks byte-exact expat semantics (token
*start* position) that xml-rs's *current parse cursor* position can't
reproduce exactly, so that one test is now marked an expected failure.
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: support multi-chunk Parse()/ParseFile() feeds
bm_xml_etree failed with AssertionError: missing end tags from
TreeBuilder.close(), because xml.etree.ElementTree reads and feeds large
documents to XMLParser.feed() in ~64KB chunks (and xml.sax's
IncrementalParser and ElementTree's XMLPullParser do the same). Each
Parse()/ParseFile() call built a brand-new xml-rs EventReader from only
the bytes passed in that call, discarding all parser state between calls.
When a chunk boundary fell in the middle of an element, the fragment
parsed by itself was invalid/incomplete XML, so xml-rs silently errored
out (matching this module's existing "ignore stricter xml-rs errors"
behavior) and the corresponding start/end events were dropped entirely
-- not just delayed.
xml-rs has no API to resume a parser across separate Read sources, so
instead each call now appends the new chunk to a persistent buffer and
re-parses the buffer as a whole, replaying only the events at or beyond
the count already dispatched in previous calls (dispatched_events). This
also makes position tracking (CurrentLineNumber/Column/ByteIndex) correct
regardless of how the input was chunked. Trade-off: each Parse() call is
now O(size fed so far), so a document delivered in many small chunks is
parsed several times over; acceptable for correctness but noticeably
slower than a true incremental parser for pathological chunk sizes.
This also fixes several previously-xfailed tests that depend on
incremental feeding: test_xml_etree.py's XMLPullParserTest (test_events,
test_feed_while_iterating, test_simple_xml, test_simple_xml_chunk_22,
test_simple_xml_with_ns) and ElementTreeTest.test_parseliteral, plus
test_sax.py's ExpatReaderTest (test_expat_entityresolver_default,
test_expat_incremental, test_expat_incremental_reset) -- their
expectedFailure markers are removed. test_simple_xml_chunk_1/_chunk_5
(1- and 5-byte chunks) still fail and remain xfail: at that granularity
xml-rs's stricter-than-expat tokenizer errors on some intermediate
prefixes that libexpat tolerates.
Assisted-by: Claude Code:claude-fable-5-1
* test: drop stale expectedFailure markers fixed by the pyexpat changes
test_plistlib's four invalid-document tests and test_xmlrpc's
test_loads_unsupported pass now that pyexpat no longer swallows
handler exceptions and keeps parser state across chunked feeds.
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: drive a single background EventReader for true incremental Parse()
8802451f3 fixed multi-chunk Parse()/ParseFile() feeds by accumulating every
fed byte and re-parsing the whole buffer from scratch on each call, which is
O(total bytes) per chunk -- quadratic in document size. xml-rs's EventReader
has no API to pause/resume across separate Read sources, so instead one
EventReader is now driven to completion on a dedicated background thread fed
through a condvar-backed queue (FeedBuffer/FeedReader); Parse()/ParseFile()
push their chunk, wake the thread, and drain (on the calling thread only)
whatever events that unblocks, stopping once the thread reports it is
blocked waiting for more input. Handlers always run on the calling thread.
A handler exception stops dispatch without tearing down the stream, so the
next Parse() resumes where it left off, mirroring libexpat. CurrentByteIndex
is now computed incrementally per line (with an amortized per-query cursor,
needed since real documents are often a single very long line) instead of
scanning the whole buffer. ParserCreate() never spawns a thread; one is
spawned lazily by the first Parse()/ParseFile() call, and dropping the
parser always stops and joins it.
Fixes test_xml_etree{,_c}'s test_events_comment, previously xfailed because
the old whole-buffer replay lost top-level comment events across feeds.
Measured (M4 Pro, release build):
- Scaling (ElementTree.parse, newline-separated doc): new backend 1/4/16 MB
= 0.15/0.61/2.35s (linear); CPython 3.14 same file = 0.015/0.055/0.23s.
- pyperformance's bm_xml_etree doc (156KB, effectively one line): old
backend single etree.parse() = 1.99s, single iterparse() = 3.53s; new
backend = 0.025s / 0.031s (~80-110x). Full bench_parse/iterparse/generate/
process (30/10/10/1 reps) now finish in ~1.5s/0.6s/0.4s/0.3s instead of
timing out past pyperformance's 400s limit.
- 10k ParserCreate() (never fed / partially fed / fully fed and dropped):
0.32s, no residual native threads, no RSS growth.
test -j4 test_pyexpat test_xml_etree test_xml_etree_c test_sax test_minidom
test_xml_dom_minicompat test_pulldom test_plistlib test_xmlrpc test_html
test_htmlparser test_threading: all pass (1350 tests, 89 skipped).
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: add a Sync fallback backend for targets without threads
2f58e9531 made pyexpat always drive its EventReader on a background
thread (ParserStream::spawn), panicking via .expect() if the thread
couldn't be spawned. pyexpat is registered unconditionally, including
on wasm32, where std::thread::spawn always fails at runtime, so the
first Parse() call panicked there.
Introduce a Backend enum: Backend::Threaded (the existing ParserStream)
on targets that can spawn threads, and a new Backend::Sync, the
accumulate-and-reparse implementation from 8802451f3, used on wasm32
and as a runtime fallback if spawning a thread ever fails. Both share
LineTracker for identical CurrentLineNumber/CurrentColumnNumber/
CurrentByteIndex and handler/error semantics.
Fixed two correctness bugs while porting the Sync backend forward:
marking the parser permanently "finished" as soon as a from-scratch
reparse happened to reach EndDocument (which can be a false positive
on a not-yet-complete buffer, e.g. right before epilog content
arrives) rather than only on the caller's real isfinal signal; and
counting that same EndDocument towards the dispatched-event index,
which could make a real event land on an index a later, longer reparse
had already skipped past. Verified by force-selecting Backend::Sync
under a test-only env var (removed) and diffing its output against
Backend::Threaded for the xml/sax/etree/plistlib/xmlrpc test suites
and a 2MB document fed in 1KB chunks.
Assisted-by: Claude Code:claude-fable-5-1
* _pickle: native Pickler, Unpickler and PickleBuffer
Adds `crates/stdlib/src/pickle.rs`, a native `_pickle` implementing
Pickler, Unpickler, PickleBuffer, PicklerMemoProxy/UnpicklerMemoProxy,
dump/dumps/load/loads and the PickleError family, so `Lib/pickle.py`
stops falling back to `_Pickler`/`_Unpickler`.
Protocols 0-5 are supported and produce byte-identical output to the
pure-Python pickler, including framing for protocol >= 4, memoization,
`fast` mode, dispatch_table, reducer_override, persistent_id /
persistent_load, find_class with `_compat_pickle` fix_imports, buffer
callbacks and out-of-band protocol 5 buffers. This also turns on
`pickle.PickleBuffer`, which RustPython did not have before.
Speedups over the pure-Python implementation (best of 5, us/op):
pickle 835 -> 21 (39x), pickle_list 525 -> 4.4 (119x),
pickle_dict 4018 -> 32 (125x), unpickle 630 -> 24 (26x),
unpickle_list 317 -> 5.0 (63x). CPython 3.14's C accelerator runs the
same workloads 1.5-2.9x faster than this.
test_pickle now runs 1008 tests (was 470) since the C-implementation
test classes are enabled; test_pickle, test_pickletools,
test_picklebuffer are green and 22 obsolete expectedFailure markers are
removed.
Assisted-by: Claude Code:claude-fable-5-1
* _pickle: raise the pickle.find_class audit event
Unpickler.find_class now raises `sys.audit('pickle.find_class', module,
name)` like CPython, so audit hooks can veto global lookups
(test_audit.test_pickle). Also drops four expectedFailure markers that
the native module makes pass: test_pydoc's builtin-method description
tests, test_memoryview.test_picklebuffer_reference_loop and
test_pyclbr.test_others.
Full suite: 456 test files / 42,648 tests green.
Assisted-by: Claude Code:claude-fable-5-1
* _pickle: avoid temporary buffers on the save fast paths
Opcode plus payload is now appended straight to the output buffer
(`write_pair`) instead of building a `Vec` per int, str, bytes and memo
opcode, `save_tuple` no longer copies the element slice, and the
`when serializing ...` note text is only formatted when an error is
actually raised.
bm_pickle "pickle" workload: 21.4 -> 17.0 us/op.
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: make the threaded backend fork-safe
fork() only keeps the calling thread. test_platform's
test_mac_ver_with_fork crashed the child with SIGTRAP: plistlib parses
a plist through pyexpat (spawning a background parser thread) before
os.fork(), then parses again in the child. Bisecting away every other
explanation (a plain threading.Thread survives the same fork fine,
since it goes through this VM's own fork-safe thread bookkeeping)
showed the crash is not from touching the *inherited* ParserStream
(already joined by refcounting before the fork in this repro) but from
spawning a *brand new* parser thread from the child, which reliably
crashed during the new thread's own startup.
Register a pthread_atfork child hook (lazily, the first time a parser
thread is spawned) that sets a permanent, process-wide flag the
instant this process is ever the child of a fork(). Once set:
- ParserStream::try_spawn refuses to spawn any further thread and
Backend::new falls back to Backend::Sync instead, exactly as on
targets that cannot spawn threads at all.
- Feeding an inherited Backend::Threaded raises a clear ExpatError
instead of touching its FeedBuffer/channel (whose mutex/condvar may
look held by a thread that no longer exists), and dropping it leaks
its JoinHandle/buffer/channel via ManuallyDrop rather than joining or
destroying them, since even destroying a POSIX mutex that looks held
is undefined behavior.
The parent is unaffected: the pthread_atfork hook only runs in the
child. Verified with a 200-iteration fork stress test (fresh parsers
and parsers forked mid-parse) and the full test_platform/xml/fork/
subprocess/multiprocessing_fork test modules.
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: implement name interning via self.intern, matching libexpat
self.intern (an always-present dict by default) was stored but never
consulted; every StartElement/EndElement/attribute name allocated a
fresh PyStr even when the same tag or attribute name repeated across
many sibling events. Wire it up the way libexpat does
(PyDict_SetDefault(self->intern, name, name)): element and attribute
*names* are now memoized so a repeated name reuses the same PyStr
object (and its cached hash) instead of reallocating and rehashing.
Attribute values are left out of the cache, same as libexpat.
This fixes test_pyexpat's previously-xfailed InterningTest.test, which
asserts object identity across repeated StartElementHandler/
EndElementHandler calls.
Measured on the pyperformance xml_etree parse/iterparse/generate/
process variants: no measurable wall-clock change (the dominant cost
there is Python-level TreeBuilder/Element bytecode execution and
thread hand-off, not name allocation), so this is a correctness/
feature-completion fix rather than a throughput win for that
benchmark; kept because it is a real, previously-inert compatibility
gap and is otherwise free.
Assisted-by: Claude Code:claude-fable-5-1
* _elementtree: native Element, SubElement and ParseError
Adds crates/stdlib/src/elementtree.rs providing the accelerator names
xml/etree/ElementTree.py picks up at import time. Element carries the
full API (tag/text/tail/attrib, indexing and slicing, find*/iter/
itertext, copy/deepcopy/pickle state) with CPython's lazily created
attrib dict and deferred text join, is subclassable, weak-referenceable
and GC-traversable, and drops every displaced object after releasing its
lock so a __del__ can re-enter safely.
xml_etree, best of 3 (s): parse 1.64 -> 1.14, iterparse 0.68 -> 0.54,
generate 0.44 -> 0.25, process 0.27 -> 0.16.
Assisted-by: Claude Code:claude-fable-5-1
* _elementtree: native TreeBuilder and XMLParser
TreeBuilder keeps CPython's element/data/tail bookkeeping in Rust,
including the pull-parser event hooks _setevents installs. XMLParser
drives a pyexpat parser whose handlers are this type's own native
methods, so a start or end tag reaches the builder without a Python
frame: the universal-name cache, attribute dict and TreeBuilder call all
happen in Rust when the target is our own builder, while foreign targets
still go through the documented start/end/data/comment/pi protocol.
ElementTree.parse() also picks up the _parse_whole fast path.
xml_etree, best of 3 (s): parse 1.14 -> 0.53, iterparse 0.54 -> 0.28,
generate 0.25 -> 0.24, process 0.16 -> 0.16. Against the pure-Python
baseline that is parse 3.1x, iterparse 2.4x, generate 1.8x, process
1.7x.
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: stop rescanning the whole line per position query
LineTracker converted a TextPosition column to a byte offset by
validating and decoding its entire buffered line tail on every query,
and appended fed bytes to that tail one at a time. On a document that is
one long line -- what xml.etree.tostring() writes, and what the
xml_etree benchmark parses -- the tail is the whole read-ahead, so a
per-event lookup became quadratic. Walk only the characters the query
actually asks for, over the deque's two halves in place, and record fed
bytes a line at a time. Byte indices are unchanged.
xml_etree, best of 3 (s): parse 0.52 -> 0.38, iterparse 0.28 -> 0.25.
Assisted-by: Claude Code:claude-fable-5-1
* _elementtree: charge Element.__deepcopy__ to the recursion budget
A chain of elements is copied by native recursion, so a deep enough tree
overflowed the real stack instead of raising. Wrap the copy in
with_recursion, as CPython does, and drop the skip on
test_deeply_nested_deepcopy for the accelerated run.
Assisted-by: Claude Code:claude-fable-5-1
* pyexpat: don't let a stale NeedMore end a Parse() early
The parser thread reports NeedMore when it finds the feed buffer empty
and parks. If it reached that point before the chunk Parse() was about
to push became visible -- which it does whenever the thread happens to
be scheduled first, so more often under load -- pump() consumed that
report as if it answered the new chunk and handed control back to Python
having dispatched nothing, silently losing the whole document.
Stamp each NeedMore with the push count observed when the buffer was
found empty and ignore reports older than the push being pumped. The
push notifies the condvar under the same mutex the report was sent
from, so the parser thread is guaranteed to wake and report again.
Fixes an intermittent failure of test_pyexpat's InterningTest under
parallel load (about 5 losses per 20000 parses before, none after).
Assisted-by: Claude Code:claude-fable-5-1
* Restore xml.etree test markers instead of skipTest in bodies
Assisted-by: Grok:4.6
* Clone state before try_to_bool in pickle load_build
try_to_bool takes the object by value, and load_build still needs the
state dict afterwards.
Assisted-by: Grok:4.6
* Drop decorative section-separator comments
Assisted-by: Grok:4.6
* Represent pickle opcodes as an enum
Assisted-by: Grok:4.6
* Use full paths for thread-only types in pyexpat
Assisted-by: Grok:4.6
---------
Co-authored-by: Lee Dogeon <dev.moreal@gmail.com>
|
||
|
Jeong, YunWon
|
6d39d132d8 |
Keep SemLock OS calls behind SemHandle methods (#8740)
Stop exporting libc semaphore pointers from host_env. Stdlib now waits, posts, and reads values through SemHandle instead of *mut sem_t / HANDLE. Assisted-by: Grok:4.6 |
||
|
Jeong, YunWon
|
0bdb61bfb6 |
Serialize SemLock bookkeeping for Condition notify (#8739)
* Serialize SemLock owner bookkeeping around waits Hold a mutex around SemLock acquire/release/_count so the per-object owner count cannot tear while threads and processes share a Condition. Drop the mutex only for the blocking wait. Mark Unix SemHandle Send/Sync. Add a spawn Condition.notify_all snippet. Assisted-by: Grok:4.6 * Repeat the Condition.notify_all snippet 20 times Give the snippets job more chances to see a lost wakeup. Assisted-by: Grok:4.6 * Cover one-by-one Condition.notify in the snippet Wake a process waiter and a thread waiter with two notify() calls, not only notify_all(), and keep the 20-repeat loop. Assisted-by: Grok:4.6 * address review: bound snippet waits with timeouts Assisted-by: Grok:4.6 |
||
|
Jeong, YunWon
|
a7c0500a42 |
host_env: export remaining Windows socket and fault constants (#8736)
* host_env: export remaining Windows socket and fault constants Publish AF_HYPERV/AF_BLUETOOTH, HV_* GUIDs, SIO_TCP_SET_ACK_FREQUENCY, EXCEPTION_CONTINUE_SEARCH, and winreg ERROR_FILE_NOT_FOUND/INVALID_DATA. os.socket pack/unpack Hyper-V and RFCOMM addresses through host_env. Assisted-by: Claude * socket: publish HV_GUID_* as string attributes Assisted-by: Claude * host_env: cast SIO_TCP_SET_ACK_FREQUENCY to i32 The WinSock constant is u32 and does not fit i32 without a wrapping cast. Socket module attributes are i32. Assisted-by: Claude * socket: use socklen_t width for Windows SockAddr::new Windows socklen_t is i32; a u32 cast does not match socket2. Assisted-by: Claude * socket: drop unused Windows AF_* imports from c AF_BTH, AF_IRDA, and AF_SNA are published via host_socket pyattr and are not read through the Windows c alias module. Assisted-by: Claude * host_env: export Windows faulthandler exception codes Route stdlib _EXCEPTION_* pyattrs and exception_description through named host_env constants instead of hex literals. Assisted-by: Claude |
||
|
Jeong, YunWon
|
0c5a47bae5 |
Unskip MP tests and shrink the flaky MP CI list (#8731)
* Unskip multiprocessing RLock repr test Assisted-by: Grok:4.6 * Drop stable modules from the flaky MP CI list Assisted-by: Grok:4.6 * Keep only multiprocessing fork on the flaky MP list Assisted-by: Grok:4.6 * Drop gc_collect bench and unbreak two CI jobs Remove benches/benchmarks/gc_collect.py so CodSpeed no longer includes that execution benchmark. On Windows, run test_multiprocessing_spawn serially; the parallel snippets job hit a regrtest worker temp-dir error while that suite was still running. Install cargo-shear from a pinned prebuilt binary so the job does not fall back to compiling 1.13.4 from source. Assisted-by: Grok:4.6 * Unskip hang tests that now pass under -m test Drop the RustPython hang skips on interpreter shutdown, submit-after-shutdown, and connection pickling after they passed with rustpython -m test. Remove the unused expectedFailureIf around killpg on the resource tracker deadlock test; os.killpg is present. Pass matrix flaky-MP extras through the job environment instead of interpolating them into the run script. Assisted-by: Grok:4.6 * Restore hang skips and isolate spawn on CI Put test_multiprocessing_spawn and test_multiprocessing_forkserver back on the serial flaky MP list. Ubuntu timed out in spawn test_notify_all under -j, and forkserver.test_processes needed a rerun. Restore the hang skips on interpreter shutdown, submit-after shutdown, and connection pickling. Assisted-by: Grok:4.6 |
||
|
Jeong, YunWon
|
73a3f45ee4 |
host_env: add Windows FindFirstFileW scandir walk (#8734)
* host_env: add Windows FindFirstFileW scandir walk nt::scandir walks with FindFirstFileW/FindNextFileW and yields WIN32_FIND_DATA fields. os.scandir uses it on Windows instead of std::fs::ReadDir. Assisted-by: Claude * host_env: store scandir find handle as isize HANDLE is a raw pointer and is not Sync, so Scandir could not sit in the os.scandir iterator lock. Store the handle as isize. Assisted-by: Claude * address review: is_dir skips directory symlinks Windows DirEntry.is_dir used FILE_ATTRIBUTE_DIRECTORY whenever follow_symlinks was false, so a directory symlink reported true. A directory symlink is not a directory unless it is followed. Junctions stay on the find-data bit; they are not symlinks. Assisted-by: Claude |
||
|
Jeong, YunWon
|
b2c48d3ab4 |
Keep specialized CALL under Return/Unwind events (#8735)
* Keep specialized CALL under Return/Unwind events Instrument only opcodes whose EVENT_FOR_OPCODE is active so CALL_ALLOC_AND_ENTER_INIT survives Return+Unwind. LINE and INSTRUCTION still restore every specialized opcode. FOR_ITER RAISE/STOP_ITERATION consult the global event mask. Assisted-by: Grok:grok-4.6 * Fix frame repr, f_lineno delete, and class cell timing frame.__repr__ matches the C format. Deleting f_lineno raises AttributeError. __classcell__ is filled before a custom mro() so methods that close over __class__ can run during type creation. Assisted-by: Grok:grok-4.6 |
||
|
Jeong, YunWon
|
faa5a60638 |
host_env: bound symlink probe; add Win32 times (#8733)
nt::symlink's directory probe now uses two WCHAR[MAX_PATH] buffers and treats an overflowing join as not a directory. winapi::version_ex_triple exposes GetVersionExW. nt::set_file_times opens with FILE_WRITE_ATTRIBUTES and writes wrapping FILETIMEs. Assisted-by: Claude |
||
|
Jeong, YunWon
|
300d8bdc02 |
Honor inline values, custom mro, and type lookup (#8729)
* Add inline values flags and special-method lookup lookup_maybe_method clears AttributeError from descr_get. slot_tp_richcompare treats a miss as NotImplemented. hackcheck rejects object.__setattr__ on types. Heap types with a managed dict and itemsize 0 get INLINE_VALUES; _testinternalcapi.has_inline_values reports the valid bit. super is a base type; subclass __get__ binds the instance. __classcell__/__qualname__ may share a name with a slot. Unset member access uses the type's fully qualified name. Assisted-by: Grok:grok-4.6 * Call custom mro and descr_get for type.__new__ type.mro() computes C3 from the current bases. A metaclass other than type supplies the MRO via mro(). super() compares obj.__class__ to the instance type. slot_tp_new looks up __new__ through descr_get. is_subtype walks the stored MRO without an identity shortcut. __slots__ names are sorted so __class__ assignment matches. Assisted-by: Grok:grok-4.6 * Expose _testinternalcapi recursion and rare events get_recursion_depth feeds test.support.infinite_recursion. SPECIALIZATION_* and SHARED_KEYS_MAX_SIZE match the adaptive counters. Rare-event hooks increment on __class__/__bases__/function/builtins writes. Assisted-by: Grok:grok-4.6 * Add remaining _testinternalcapi code and locale APIs Expose get_co_localskinds, get_code_var_counts, verify_stateless_code, code_returns_only_none, normalize_path, EncodeLocaleEx, DecodeLocaleEx, and run_in_subinterp_with_config. Mark unimplemented compiler IR and specialization monitoring tests as expectedFailure. Assisted-by: Grok:grok-4.6 * Fix lint for _testinternalcapi identifiers Add CPython identifier spellings to the cspell dictionary and apply the rustfmt signature that CI rewrote. Assisted-by: Grok:grok-4.6 * Expand _testinternalcapi with remaining test APIs Add constants, compiler_cleandoc, dict_getitem_knownhash, time helpers, interpreter/frame accessors, configs, static types, a persistent hamt map, and GC/size counters used by the test suite. Assisted-by: Grok:grok-4.6 * Expose compiler IR APIs in _testinternalcapi Add InstructionSequence and compiler_codegen, optimize_cfg, and assemble_code_object. Drop expectedFailure marks on tests that pass. Assisted-by: Grok:grok-4.6 * Match optimize_cfg consts and addop locations Write optimized consts back into the caller's list, mark SETUP_* pseudo-ops as having an argument, and store addop locations in instruction-sequence field order. Drop expectedFailure marks on tests that pass. Assisted-by: Grok:grok-4.6 * Fix hamt API, SETUP_* HAS_ARG, and FOR_ITER events Honor explicit pseudo flags when generating opcode metadata so SETUP_FINALLY/CLEANUP/WITH keep HAS_ARG after regeneration. Complete the _testinternalcapi hamt mapping: identity on set/delete, richcompare, weakrefs, subscript, contains, and length-bearing iterators. Fire RAISE/STOP_ITERATION for FOR_ITER StopIteration, and skip both END_FOR and POP_ITER for the exhausted-branch destination. Assisted-by: Grok:grok-4.6 * Skip FOR_ITER RAISE for sequence iterators List/range/tuple exhaustion matches FOR_ITER_LIST/RANGE/TUPLE and must not emit a monitoring RAISE. That event stopped pdb's monitoring backend on for-over-list with Internal StopIteration. Assisted-by: Grok:grok-4.6 * Fix generator FOR_ITER events and review issues Restore settrace StopIteration for generator for-loops so test_bdb next/return cases pass, while sequence iterators still skip RAISE. Honor review notes: addop oparg bounds, linetable locations, AST instance errors, MRO subclass lock, builtin-dict notify, direct-base layout flags, and leftover _testinternalcapi args. Assisted-by: Grok:grok-4.6 |
||
|
Jeong, YunWon
|
982cbd4d25 |
Add os.wait3 and os.wait4 with EINTR retry (#8728)
* Add os.wait3 and os.wait4 with EINTR retry Assisted-by: Grok:4.6 * Unskip multiprocessing Finalize thread-safety test Assisted-by: Grok:4.6 * Fix wait4 clippy lint and use libc wait4 Assisted-by: Grok:4.6 |
||
|
sijun-yang
|
ce470e3928 |
Implement os.pread and os.pwrite (#8730)
Assisted-by: Codex:gpt-6-astra |
||
|
Jeong, YunWon
|
c382a607bb |
host_env: keep rustls off the rustls-free ssl surface (#8727)
* host_env: keep rustls off the rustls-free ssl surface ssl is MemoryBIO, constants, OID, and ALPN. ssl-rustls is the rustls engine. Drop the wasm32 rustls-pki-types web dep; the browser crate already enables that feature. Assisted-by: Claude * host_env: drop rustls-pki-types from the lockfile Assisted-by: Claude * host_env: name the rustls engine feature rustls ssl is the rustls-free surface. rustls is the engine. stdlib's ssl-rustls product feature still selects this. Assisted-by: Claude |
||
|
Bas Schoenmaeckers
|
da694300b8 |
Add Py_IncRef, Py_DecRef, Py_XNewRef (#8726)
|
||
|
Jeong, YunWon
|
7da001a7c9 |
address review: kqueue lock and host_env select (#8724)
* address review: kqueue lock and timeout overflow Hold the kqueue fd lock through kevent and drop it before changelist parsing and signal handlers. Reject timeouts that overflow Instant or Duration. Round poll millisecond timeouts upward. Cast kevent defaults for NetBSD field widths. Provide login_tty on Solaris/illumos via setsid, TIOCSCTTY, and dup2. Assisted-by: Grok:grok-4.6 * Move select timeout conversion into host_env poll millisecond ceiling, kevent timespec rounding, and default filter/flag widths now live in host_env. stdlib select.rs keeps Python conversion and signal retry only. Assisted-by: Grok:grok-4.6 * Move socket wait and rusage timeval into host_env socket.sock_wait_deadline now calls host_env wait_fd. resource and process-time conversion no longer read timeval fields in stdlib or vm. Assisted-by: Grok:grok-4.6 * Deny libc select syscalls in vm and stdlib clippy Block libc poll/select/kqueue/epoll entry points and non-reexported ABI types. host_env keeps empty lists. Assisted-by: Grok:grok-4.6 * address review: portable timeval and wait_fd Widen rusage timeval math to i64 so 32-bit time_t compiles. Saturate duration_to_timeval when tv_sec is narrower than u64. Round wait_fd poll timeouts up and accept Windows RawSocket. Ignore EPERM from setsid in the Solaris login_tty fallback. Deny libc::epoll_pwait2 in vm and stdlib clippy. Assisted-by: Grok:grok-4.6 |
||
|
Hanjeong Lee
|
df653cea69 |
Name generated signature parameters the way CPython names them (#8725)
* Leave Rust-only name decoration out of a generated signature A leading underscore marks an argument Rust sees as unused and `r#` escapes a Rust keyword. Neither describes the parameter a Python caller passes, and `r#` is not valid Python, so `inspect.signature()` rejects any signature carrying one. Reporting the bare name fixes _sre.template, select.epoll.__exit__ and bytearray.__reduce_ex__ without touching them, and lets a method name a parameter after a Rust keyword. A parameter CPython itself names with a leading underscore, such as compile's `_feature_version`, reaches Python through a FromArgs field, which this never sees. Assisted-by: Claude Code:claude-opus-5 * Name builtin type parameters the way CPython names them A generated __text_signature__ reports the Rust parameter name, so a name chosen for Rust reaches anyone reading help() or inspect. Rename the ones CPython describes differently: list.pop's `i` becomes `index`, list.count's `needle` becomes `value`, __format__'s `spec` becomes `format_spec`, bytes.strip's `chars` becomes `bytes`, and object.__reduce_ex__'s `proto` becomes `protocol`. Left alone are the parameters CPython reports as `object` or `unused`. Those are synthesised from the METH_* flags rather than named: CPython reports slice.indices as ($self, object, /) while its own docstring calls the argument len. Assisted-by: Claude Code:claude-opus-5 * Name os parameters the way CPython names them os.read's `n` becomes `length`, os.lseek's `how` becomes `whence`, os.kill's `sig` becomes `signal`, os.stat's `file` becomes `path`, and os.putenv's `key` becomes `name`, among others. os.chmod is left alone. It reports (path, dir_fd, mode, follow_symlinks) while the call binds mode second and takes follow_symlinks by keyword, so the order and the kind are wrong rather than the names, which a rename cannot fix. os.fstatvfs is left alone too: it shares one Rust function with statvfs, which CPython names `path`, so the two aliases cannot report different names. Assisted-by: Claude Code:claude-opus-5 * Name sys parameters the way CPython names them sys.excepthook's (exc_type, exc_val, exc_tb) become (exctype, value, traceback), sys._getframe's `offset` becomes `depth`, sys.intern's `s` becomes `string`, and sys.settrace's `tracefunc` becomes `function`. sys._clear_type_descriptors takes the name CPython gives it, which Rust spells r#type. Assisted-by: Claude Code:claude-opus-5 * Name vm stdlib parameters the way CPython names them deque.rotate's `mid` becomes `n`, deque.extend's `iter` becomes `iterable`, _sre.ascii_tolower's `ch` becomes `character`, _operator.neg's `pos` becomes `a`, signal.alarm's `time` becomes `seconds`, and codecs.register_error's `name` becomes `errors`. gc.collect and charmap_build keep theirs. Their argument is a FromArgs struct or a PosArgs bundle, so the reported name is the binding rather than a parameter, and renaming the binding would only disguise that. Assisted-by: Claude Code:claude-opus-5 * Name stdlib crate parameters the way CPython names them unicodedata.category's `character` becomes `chr`, math.isqrt's `x` becomes `n`, cmath.polar's `x` becomes `z`, struct.calcsize's `fmt` becomes `format`, binascii.crc32's `init` becomes `crc`, zlib.adler32's `begin_state` becomes `value`, and array.append's `x` becomes `v`. array.fromunicode's parameter is `ustr`, which cspell has to be told about. array.__deepcopy__ keeps `_memo`, which the generator now reports as `memo`; CPython calls it `unused`, a name synthesised from METH_O rather than chosen. Assisted-by: Claude Code:claude-opus-5 * Name BaseExceptionGroup.split's parameter the way CPython names it CPython calls the argument of split and subgroup `matcher_value`. Assisted-by: Claude Code:claude-opus-5 * Strip a generated parameter name with unraw and one prefix check syn already removes r# through IdentExt::unraw, the way from_args.rs does for a field name. The empty-name guard never ran: a bare `_` is Pat::Wild, which func_sig has already refused. Assisted-by: Claude Code:claude-fable-5-1 * Take object and unused from CPython's Clinic signatures CPython also falls back to these names for METH_O, but list.append and insert, set.add, discard and remove, _weakref.getweakrefcount and getweakrefs, sys.displayhook and getrefcount, and array.__deepcopy__ get them from an Argument Clinic signature. The earlier commit that kept array.__deepcopy__'s `_memo` was wrong to call `unused` synthesised. test_pydoc's set.add summary-line tests now pass. Assisted-by: Claude Code:claude-opus-5 |
||
|
Jeong, YunWon
|
3191ecaf38 |
host_env: share thread, locale, loader and BSTR (#8702)
* host_env: share thread, locale, loader and BSTR
Add Windows thread name, locale-info/wcscoll, CompareStringOrdinal,
LoadLibraryEx/GetProcAddress/FreeLibrary, BSTR and COM helpers, and
AF_HYPERV UUID conversion. Wire _thread, locale, ctypes LoadLibrary
and WMI BSTRs through those wrappers. Ensure =X: before spawnve/execve.
Assisted-by: Claude
* host_env: expose rustls-free ssl core on wasm
Compile MemoryBIO, protocol constants, OID tables, hostname
checks, and ALPN parsing without the rustls feature so wasm
and other hosts can implement _ssl on the shared types.
Assisted-by: Claude
* stdlib: add wasm _ssl on rustls-free host_env
Install _ssl on wasm32 against rustpython_host_env::ssl.
MemoryBIO, constants, OID, hostname, and ALPN work. Context
settings are stored; wrap/handshake and rustls-only loaders
raise SSLError until a wasm TLS engine is wired.
Assisted-by: Claude
* host_env: address #8702 review comments
Cache-miss Windows ctypes lookup uses GetProcAddress. FreeLibrary
propagates OSError. start_new_thread returns GetThreadId. Export
_thread._get_name and drop pyfunction doc comments. Reject
embedded NULs in Windows locale.strcoll/strxfrm. Terminate
wcscoll/wcsxfrm and SetThreadDescription inputs.
wasm _ssl accepts wrap_bio owner and get_ca_certs(binary_form),
and MemoryBIO.read(-1) returns all pending bytes.
Assisted-by: Claude
* host_env: gate the whole ssl module on feature ssl
Compile rustpython_host_env::ssl only with the ssl feature.
Rustls crates stay native-only, so enabling ssl on wasm does
not pull rustls. Inside the module, rustls files follow
not(wasm32) rather than a second feature flag.
Assisted-by: Claude
* host_env: fix Windows types and thread names
Cast SysAllocStringLen to *mut u16 and RPC status to u32 so
windows-gnu and MSVC agree. Export _NAME_MAXLEN, truncate names
at NUL and the OS byte cap, and decode get_name with fsdecode.
Assisted-by: Claude
* host_env: compile rustls engine on WASI
Keep rustls-free MemoryBIO/constants/OID/ALPN on all wasm
targets. Compile the rustls engine on WASI, where UnixTime::now
exists. Allow the rustpython ssl feature on wasm without
ssl-rustls or ssl-openssl.
Assisted-by: Claude
* compiler: pick the earliest CPython parse override
A later incompatible-prefix scan must not hide an earlier
number or bracket diagnostic. Parser-driven tokenization
stops at the first failure.
Assisted-by: Claude
* host_env: address remaining #8702 review comments
Map a missing Windows export to a symbol-load error, keep
LoadLibrary's Win32 code on the OSError, drop incomplete
UTF-16 pairs when truncating thread names, hide set_name
and _get_name off linux/macOS/Windows, and let ssl-rustls
enable host_env.
Assisted-by: Claude
* stdlib: add wasm _socket and read-all MemoryBIO
Register a rustls-free-style _socket shim on wasm32 so
Lib/ssl.py can import. Treat MemoryBIO.read(-1) as read-all,
matching _ssl_MemoryBIO_read_impl.
Assisted-by: Claude
* host_env: fix clippy, WASI ssl cfg, and wasm rustls
Defer unclosed-bracket overrides until no token diagnostic
exists, keep WASI rustls and wasm ssl modules exclusive,
preserve lone Windows surrogates, compile the rustls engine
on browser wasm, and clear clippy denials.
Assisted-by: Claude
* host_env: fix Windows clippy and wasm UnixTime
Gate thread-name byte truncation to linux/macOS, allow
GetProcAddress/FreeLibrary pointer deref, and enable
rustls-pki-types web time on browser wasm.
Assisted-by: Claude
* host_env: lock rustls-pki-types web for wasm
Assisted-by: Claude
* host_env: silence Windows unused thread id and shear
Assisted-by: Claude
* stdlib: drop Windows locale clippy needless wraps
Windows clippy failed on locale.strcoll/strxfrm needless return
and _getdefaultlocale always wrapping Ok.
Assisted-by: Claude
* compiler: keep lexer errors over print hints
print x; 0x must report an invalid hex literal. Ranking by source
offset let the earlier print hint win. Treat a finished radix
literal as valid so print x; 0x1 still gets the print message.
Assisted-by: Claude
* compiler: keep decode errors over unclosed openers
Ranking by offset made ( '\\N' report an unclosed paren and
wiped the unclosed-bracket flag on f'{, so compile snippets
raised SyntaxError instead of incomplete input. Export wasm
socket._accept for Lib/socket.py.
Assisted-by: Claude
* wasm: drop duplicate _socket; split locale cfgs
_socket lives in rustpython-stdlib's socket_wasm. The wasm crate
only re-exported that file when freeze-stdlib was off. Split
strcoll, strxfrm, and getencoding into cfg'd function defs.
Assisted-by: Claude
* compiler: keep decode blockers over unclosed openers
(print x; '\\N' must keep the print hint. A later malformed
escape still suppresses the EOF opener even when print wins.
Assisted-by: Claude
* compiler: keep earlier f-string over later lexer errors
f'{x'; ' and f'{x'; 0x must not report the trailing quote or
hex literal. Decode and f-string diagnostics now compete with
lexer failures by offset.
Assisted-by: Claude
* compiler: rank decode vs lexer vs print by class
Unterminated quotes no longer hide format-spec newlines or
empty f-string fields. Print is considered only after the
final winner is not a lexer failure, so print x; f'{x'; 0x
keeps the print hint. Line-continuation uses CPython wording.
Assisted-by: Claude
* compiler: keep unclosed paren over line-cont at EOF
test_exceptions wants '(' was never closed at the opener when
a final backslash follows. Skip escaped quotes in the bracket
scan so {\'a\'} still reports the continuation error. Un-xfail
test_join_daemon_thread_in_finalization.
Assisted-by: Claude
* test_threading: xfail daemon join finalization off Linux
The test passes on Linux. macOS and Windows leave an ignored
deallocator exception on stderr.
Assisted-by: Claude
* vm: hang daemon threads at interpreter finalize
Forced-off daemons must stay not-done so join() during
finalization GC raises PythonFinalizationError and
Thread.is_alive() stays true. SystemExit made the handle
look finished on macOS/Windows. Drop the Lib/test xfail.
Assisted-by: Claude
(cherry picked from commit
|
||
|
Jeong, YunWon
|
7c4017b3d8 |
Retry select/poll/fcntl on EINTR and add kqueue (#8721)
* Retry select/poll/fcntl on EINTR
select() no longer shares one empty list for r/w/x and now
calls select(2) even with empty sets so the timeout elapses.
poll() rejects type instantiation, treats float timeout as
milliseconds, and raises RuntimeError on concurrent poll().
fcntl, ioctl, flock, and lockf retry after EINTR.
Assisted-by: Grok:grok-4.6
* Add select.kqueue and select.kevent
Expose kqueue(2)/kevent(2) on BSD and macOS, including
control() EINTR retry and marking open kqueue objects
closed after fork.
Assisted-by: Grok:grok-4.6
* Add login_tty and fix termios/syslog/mmap
termios.error is Exception, not OSError.
os.login_tty prepares a tty for a new session.
syslog.openlog encodes ident as UTF-8.
mmap.seek returns the new position.
Assisted-by: Grok:grok-4.6
* Honor mmap trackfd on Unix
When trackfd is false, do not keep a duplicated file
descriptor. size() then fails with EBADF and resize()
raises ValueError.
Assisted-by: Grok:grok-4.6
* address review: portable kevent and kqueue close
Build kevent values by field assignment so FreeBSD's
ext array compiles. Normalize timespec nanoseconds,
keep the kqueue fd live across control(), reinit the
open list lock after fork, make trackfd keyword-only,
and skip login_tty on Solaris/illumos.
Assisted-by: Grok:grok-4.6
(cherry picked from commit
|
||
|
Jeong, YunWon
|
cdad8e2ac9 |
Skip rust CI jobs when sources did not change (#8719)
* Skip rust CI jobs when sources did not change
Keep required rust-test and wasm-wasi checks green
without rebuilding. Run macOS/Windows clippy only
when the OS surface changed.
Assisted-by: Grok:4.6
* Cover root rust targets and honor cross-check clippy
Assisted-by: Grok:4.6
(cherry picked from commit
|
||
|
sijun-yang
|
6cc068474f |
Return distinct empty lists from select() (#8720)
Assisted-by: Codex:gpt-6-astra
(cherry picked from commit
|
||
|
dependabot[bot]
|
f569444f7d |
build(deps): bump the wasmtime group across 1 directory with 2 updates (#8710)
Bumps the wasmtime group with 1 update in the / directory: [cranelift-jit](https://github.com/bytecodealliance/wasmtime).
Updates `cranelift-jit` from 0.135.1 to 0.135.2
- [Release notes](https://github.com/bytecodealliance/wasmtime/releases)
- [Changelog](https://github.com/bytecodealliance/wasmtime/blob/main/RELEASES.md)
- [Commits](https://github.com/bytecodealliance/wasmtime/commits)
Updates `cranelift-module` from 0.135.1 to 0.135.2
- [Release notes](https://github.com/bytecodealliance/wasmtime/releases)
- [Changelog](https://github.com/bytecodealliance/wasmtime/blob/main/RELEASES.md)
- [Commits](https://github.com/bytecodealliance/wasmtime/commits)
---
updated-dependencies:
- dependency-name: cranelift-jit
dependency-version: 0.135.2
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: wasmtime
- dependency-name: cranelift-module
dependency-version: 0.135.2
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: wasmtime
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit
|
||
|
dependabot[bot]
|
c1c5136749 |
build(deps): bump taiki-e/install-action from 2.87.2 to 2.87.8 (#8709)
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.87.2 to 2.87.8.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/install-action/compare/1ed6d7be6168f6c9046541087ff549b6bc581fdf...d438492cf8a250514fa2d34b30bc3c0dc37c65ff)
---
updated-dependencies:
- dependency-name: taiki-e/install-action
dependency-version: 2.87.8
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(cherry picked from commit
|
||
|
Jeong, YunWon
|
6c1003fd03 |
Honor asyncgen ag_closed and wasm BIO EOF (#8718)
* address review: drain plaintext before BIO EOF
Skip feed_tls([]) while rustls still has application
data, matching the native SSL read EOF order.
Assisted-by: Grok:grok-4.6
* Honor asyncgen ag_closed and asend reuse
Keep ag_closed apart from a finished frame. aclose()
sets it before GeneratorExit. asend() already-running
closes the awaitable so it does not warn unawaited.
Assisted-by: Grok:grok-4.6
(cherry picked from commit
|
||
|
Jeong, YunWon
|
fc80382af2 |
Skip cross-target cargo check unless the OS surface changed (#8717)
* Split cron-ci cargo run and skip long jobs on PRs Assisted-by: Grok:4.6 * Run cross-target cargo check only on platform diffs Assisted-by: Grok:4.6 * Trigger cross-target checks on OS FFI and posix files Assisted-by: Grok:4.6 * Watch every OS-specific module for cross-target checks Assisted-by: Grok:4.6 * Restore cron-ci coverage and benchmarks on workflow PRs Assisted-by: Grok:4.6 |
||
|
dependabot[bot]
|
0bc7fe15d0 |
build(deps): bump uuid from 1.24.0 to 1.26.0 (#8715)
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.24.0 to 1.26.0. - [Release notes](https://github.com/uuid-rs/uuid/releases) - [Commits](https://github.com/uuid-rs/uuid/compare/v1.24.0...v1.26.0) --- updated-dependencies: - dependency-name: uuid dependency-version: 1.26.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
1b15903164 |
build(deps): bump der from 0.8.1 to 0.8.2 (#8714)
Bumps [der](https://github.com/RustCrypto/formats) from 0.8.1 to 0.8.2. - [Commits](https://github.com/RustCrypto/formats/compare/der/v0.8.1...der/v0.8.2) --- updated-dependencies: - dependency-name: der dependency-version: 0.8.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
cb8878485d |
build(deps): bump cargo-bins/cargo-binstall from 1.22.0 to 1.23.0 (#8711)
Bumps [cargo-bins/cargo-binstall](https://github.com/cargo-bins/cargo-binstall) from 1.22.0 to 1.23.0. - [Release notes](https://github.com/cargo-bins/cargo-binstall/releases) - [Changelog](https://github.com/cargo-bins/cargo-binstall/blob/main/release-plz.toml) - [Commits](https://github.com/cargo-bins/cargo-binstall/compare/75b4bfae1b2c753a6806bbce6e6cb89b602de33c...b874e25ea559687bec77e281e9b271aa1367b624) --- updated-dependencies: - dependency-name: cargo-bins/cargo-binstall dependency-version: 1.23.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
6c1d0ddbc9 |
build(deps): bump reviewdog/action-actionlint from 1.73.2 to 1.73.4 (#8713)
Bumps [reviewdog/action-actionlint](https://github.com/reviewdog/action-actionlint) from 1.73.2 to 1.73.4. - [Release notes](https://github.com/reviewdog/action-actionlint/releases) - [Commits](https://github.com/reviewdog/action-actionlint/compare/dbe5299849118fd6f099ba563d263d770955a64a...d290e336d5a743810aef4404f757dc862276d2ae) --- updated-dependencies: - dependency-name: reviewdog/action-actionlint dependency-version: 1.73.4 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Jeong, YunWon
|
98b245f987 |
address review: assignment scan and wasm TLS (#8716)
* address review: assignment scan and wasm TLS Skip comments and lambda defaults in the condition assignment scanner. Restrict the suite-header colon fallback to yield targets. Capture __del__ before calling it. Store browser socket timeouts as f64 and accept integers. Fail closed when wasm verify_mode is not CERT_NONE. Export getpeercert and feed MemoryBIO EOF. Assisted-by: Grok:grok-4.6 * address review: parenthesized yield suite header Walk through grouping parentheses before requiring yield after a suite-header colon. Assisted-by: Grok:grok-4.6 |
||
|
dependabot[bot]
|
a0bc6b2f5e |
build(deps): bump textwrap from 0.16.2 to 0.16.3 (#8712)
Bumps [textwrap](https://github.com/mgeisler/textwrap) from 0.16.2 to 0.16.3. - [Release notes](https://github.com/mgeisler/textwrap/releases) - [Changelog](https://github.com/mgeisler/textwrap/blob/main/CHANGELOG.md) - [Commits](https://github.com/mgeisler/textwrap/compare/0.16.2...0.16.3) --- updated-dependencies: - dependency-name: textwrap dependency-version: 0.16.3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
2f5ad0000c |
build(deps): bump github/gh-aw/actions/setup from 0.87.10 to 0.88.2 (#8708)
Bumps [github/gh-aw/actions/setup](https://github.com/github/gh-aw) from 0.87.10 to 0.88.2. - [Release notes](https://github.com/github/gh-aw/releases) - [Changelog](https://github.com/github/gh-aw/blob/main/CHANGELOG.md) - [Commits](https://github.com/github/gh-aw/compare/ff62cdbec36230acbae869ddb28806e8eca01ea1...8e30bcd8897f5047051fa3971188e1dd4cdb23cf) --- updated-dependencies: - dependency-name: github/gh-aw/actions/setup dependency-version: 0.88.2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
df14c3b337 |
build(deps): bump https://github.com/streetsidesoftware/cspell-cli (#8706)
Bumps [https://github.com/streetsidesoftware/cspell-cli](https://github.com/streetsidesoftware/cspell-cli) from v10.1.0 to 10.2.0. - [Release notes](https://github.com/streetsidesoftware/cspell-cli/releases) - [Changelog](https://github.com/streetsidesoftware/cspell-cli/blob/main/CHANGELOG.md) - [Commits](https://github.com/streetsidesoftware/cspell-cli/compare/v10.1.0...v10.2.0) --- updated-dependencies: - dependency-name: https://github.com/streetsidesoftware/cspell-cli dependency-version: 10.2.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Jeong, YunWon
|
73e547a9c3 |
Honor generator close, throw context, and docs (#8701)
* Honor close unraisable and deallocator messages A failed close lookup on a yield-from target is unraisable. Ignored GeneratorExit from finalize gets a traceback and the closing-generator err_msg. __del__ uses the deallocator message. Assisted-by: Grok:grok-4.6 * address review: claim, ag_running, frame steal Keep the running claim through close() cleanup. ag_running reads running_async. aclose ignore-yield does not mark the generator closed. Steal the frame slot before uniqueness and expose it with try_to_owned. Assisted-by: Grok:grok-4.6 * Honor wrapper docs, yield assign, throw context method-wrapper exposes the slot __doc__. Parenthesized yield assignment uses the invalid-target messages. throw() restores without overwriting __context__, then chains only from the generator's own exc_info slot. Assisted-by: Grok:grok-4.6 * Remove passing expectedFailure markers method-wrapper docs, deallocator messages, and help pager output now match, so drop the leftover markers. Assisted-by: Grok:grok-4.6 * Honor wrapper signatures and assignment diagnostics Slot docs include text signatures. method-wrapper compares and hashes by identity. Keyword genexp and if/while assignment use the invalid-target messages. Assisted-by: Grok:grok-4.6 * address review: aclose, unraisable, frame publish aclose ignore-yield does not close the generator. Format unraisable object is None. close() leaves traceback to the caller; finalize attaches the generator frame. Publish the iframe before the lock-free slot. Assisted-by: Grok:grok-4.6 * Honor member descriptors and wrapper annotations Assisted-by: Grok:grok-4.6 * address review: getitem, bool doc, if assign Assisted-by: Grok:grok-4.6 * Honor eval-mode assignment invalid syntax Assisted-by: Grok:grok-4.6 * Supply getset and member docs from rustpython-doc Assisted-by: Grok:grok-4.6 * Enable rustls SSL on WASI targets Compile the rustls _ssl module for WASI and stub socket waits plus the native certificate store. Assisted-by: Grok:grok-4.6 * Add browser TLS _ssl in rustpython-wasm Register rustls _ssl and a MemoryBIO-oriented _socket from rustpython-wasm. Use rustls-rustcrypto because ring cannot compile for wasm32-unknown-unknown. Assisted-by: Grok:grok-4.6 * Define SSL protocol and cert constants as enums Keep OpenSSL bitmask defines as integer constants. Assisted-by: Grok:grok-4.6 * Ignore rustls deps cargo-shear cannot see The rustpython-wasm _ssl sources are compiled through a symlink, so shear does not count their crate uses. Assisted-by: Grok:grok-4.6 * Replace wasm ssl symlink with a BIO module Drop the stdlib ssl.rs link and keep a rustpython-wasm _ssl that only covers MemoryBIO wrap_bio TLS. Assisted-by: Grok:grok-4.6 |
||
|
Jeong, YunWon
|
d7e41ebff5 |
Split CI cargo run and test into build then execute (#8705)
* Run gc_collect ten times per benchmark iteration Assisted-by: Grok:4.6 * Reuse the prebuilt rustpython for CI whats_left whats_left was invoking cargo build --release without threading, so the snippets job rebuilt the interpreter after the earlier release build. Assisted-by: Grok:4.6 * Split CI cargo run and test into build then execute Assisted-by: Grok:4.6 * Compile doctests in the CI build step cargo test --no-run skips doctests, so rustdoc compilation was still happening in the run step. Assisted-by: Grok:4.6 * Drop cron-ci.yaml from this PR A pull_request path filter matches the whole PR, so every push was starting the periodic benchmark and coverage jobs. Assisted-by: Grok:4.6 * Run doctests in their own CI step cargo test --doc --no-run is rejected, so doctests cannot be compiled without also executing them. Assisted-by: Grok:4.6 * Do not run benchmark targets in rust tests --all-targets includes Criterion benches, which cargo test does not run by default. Assisted-by: Grok:4.6 |
||
|
Jeong, YunWon
|
a7217a087b |
Classify incomplete input from tokenizer EOF flags (#8703)
* Classify incomplete input from tokenizer EOF flags Treat exec's implicit newline, single-quoted f/t-strings, blank single input, and unclosed field braces the way E_EOF, E_EOFS, and E_EOLS do. Assisted-by: Grok:4.6 * address review: lambda defaults, unicode prefixes, comments Keep lambda parameter defaults and Unicode identifiers continuable, and report comment-only single input as SyntaxError. Assisted-by: Grok:4.6 |
||
|
Bas Schoenmaeckers
|
dacf514210 |
Add long export/writer to c-api (#8161)
* Add long export/writer c-api * Add `PyLong_AsNativeBytes` * Fix PyLongWriter_Create errors * Set PyLongWriter_Create digits to null on err |
||
|
Jeong, YunWon
|
8de8f4bdb0 |
Bump version to 0.6.0 and use the crates.io ruff crates (#8700)
Point ruff_python_parser, ruff_python_ast, ruff_text_size and ruff_source_file at the published rustpython-ruff_* 0.16.5 crates instead of the git tag. Update chacha20 to 0.10.2 because 0.10.1 is yanked. Assisted-by: Claude Code:claude-fable-5-1 |
||
|
Jeong, YunWon
|
5ae21bee60 |
Keep escaped generator frames and wrap asyncgen throw (#8697)
* Keep escaped generator frames and wrap asyncgen throw Mark gi_frame/cr_frame/ag_frame as escaped so locals survive deallocation, and let frame.clear() drop those locals after owner finalize. Route throw() through finalize_send_result so StopAsyncIteration is wrapped like send(). Add ag_suspended and drop never-started frame locals on close(). Assisted-by: Grok:grok-4.6 * Store escaped on iframe; skip unused cold alloc Keep the escaped flag on InterpreterFrame so mark_escaped and has_escaped do not allocate FrameColdData. Clear optional cold fields only when they exist. Transfer frame owner in close() the same way send/throw already do. Assisted-by: Grok:grok-4.6 * Replace escaped flag with take_ownership gi_frame returns the frame object. Close uses a uniquely-referenced check to clear locals or take_ownership onto a husk. frame.clear() finalizes a generator-owned frame and only clears FRAME_OBJECT frames. close() leaves the generator suspended when it yields on GeneratorExit. Assisted-by: Grok:grok-4.6 * Drop frame mutex; steal frame_obj atomically take_ownership stores None in an atomic slot instead of locking and swapping a husk. gi_code reads the kept executable. Resume exclusivity stays on the running claim. Assisted-by: Grok:grok-4.6 |
||
|
Jeong, YunWon
|
a973fbf472 |
Remove the deprecated compiler-source crate and mark venvlauncher unpublishable (#8698)
* Remove the deprecated rustpython-compiler-source crate Nothing in the workspace depends on it, and its version was pinned to 0.4.1+deprecated, which is already on crates.io. Drop the crate, its Cargo.lock entry, and the mentions in host_env_proposal.md. Assisted-by: Claude Code:claude-fable-5-1 * ci: drop the rustpython-compiler-source exclusions The crate no longer exists in the workspace. Assisted-by: Claude Code:claude-fable-5-1 * venvlauncher: mark the crate as not publishable It is a Windows-only binary that venv copies out of the installation, not a crates.io dependency. Assisted-by: Claude Code:claude-fable-5-1 |
||
|
Lee DogeonandClaude Sonnet 5
|
44dcd1de2d |
ci: add pyperformance benchmarking scripts and workflow (#8691)
* Add script to run upstream pyperformance against RustPython pyperformance can't be installed as-is on RustPython since pyperf hard-depends on psutil, a C extension RustPython cannot build or load (no CPython C-API / extension-module support). Since pyperf already disables psutil usage on interpreters reporting Py_GIL_DISABLED=1 (which RustPython does, having no GIL), a functionality-free pure-Python psutil stub is enough to satisfy pip's dependency resolution and unblock any pure-Python benchmark. scripts/pyperformance/run_all.py drives pyperformance (under a host CPython venv) with RustPython as the --python target, installs the stub via PIP_FIND_LINKS + --inherit-environ, and catalogs pass/fail/timeout per benchmark with an automatic failure classifier (C-extension build blocked vs. a genuine exception raised by the benchmark itself). Assisted-by: Claude Code:claude-sonnet-5 * pyperformance: support targeting any Python and comparing catalogs Generalize run_all.py from a RustPython-only runner to a --python/--label pair, so the same script can catalog a real CPython (with --no-psutil-stub, since it has a genuine psutil) alongside RustPython. Each label gets its own results/<label>/ subdirectory. Add compare.py, which loads two labels' catalogs and writes a per-benchmark Markdown table with both targets' status/mean and the candidate/baseline time ratio, plus the median slowdown across benchmarks both targets passed. Assisted-by: Claude Code:claude-sonnet-5 * ci: compare base, head and CPython with pyperformance on one runner A ratio between two pyperformance runs measured on different machines can move further than the change under review did, so a PR's numbers are only worth reading when base, head and CPython were measured back to back on the same runner. This adds a job that does exactly that -- two release builds kept side by side with the stdlib of their own commit, then three runs with nothing in between -- and posts the table as a PR comment. `pr_diff.py` renders the three catalogs `run_all.py` writes: per benchmark the CPython time, both RustPython times, both ratios against CPython, and the head/base change, with anything under 3% shown as `~` because a shared runner is not quiet enough to say more. The comment is handed to a `workflow_run` companion the same way `codspeed-comment.yaml` does it, since a `pull_request` run from a fork gets a read-only token however the workflow is configured. The job is opt-in behind a `run:pyperformance` label: it runs 28 benchmarks three times over and costs well over an hour, which most PRs do not need. `run_all.py` now forwards RUSTPYTHONPATH through pyperf, which a binary copied away from its checkout needs in order to find the stdlib at all; the two names go in one comma-separated `--inherit-environ`, because repeating the flag keeps only the last. Assisted-by: Claude Code:claude-fable-5-1 * ci: comment from the pyperformance run itself when the PR allows it The `workflow_run` companion only starts firing once it is on the default branch, so a pull request that introduces it -- this one -- would upload the table and never show it. A pull request opened from this repo carries a token that may write comments, so post it from the run that measured it and leave the companion to the forks that genuinely cannot. Both paths upsert on the same marker, so a fork PR that reaches the companion after this step was skipped still lands one comment. Assisted-by: Claude Code:claude-fable-5-1 * ci: address zizmor findings and fall back to head vs. CPython - Pin `actions/github-script` to the commit the rest of the repo pins it to; the hash carried over from an in-flight branch pointed elsewhere than the `v9.0.0` comment claimed (ref-version-mismatch). - Read the PR number, the CPython path and version, and the two commit shas from `env` rather than expanding them into the shell (template-injection). - Record the `workflow_run` trigger in `.github/zizmor.yml` next to the existing `pull_request_target` entry, with the same justification: the workflow reads one artifact and writes a comment, and never checks out or runs pull request code (dangerous-triggers). Building or measuring the base commit is now non-fatal, and `pr_diff.py` renders head against CPython alone when the base column has nothing in it -- a base that will not build should still leave a usable table rather than failing the job. The time and ratio columns are also named apart, since `| base | head | base | head |` gave no clue which pair was which. Assisted-by: Claude Code:claude-fable-5-1 * ci: derive pyperformance comment target from trusted workflow_run metadata pr_number.txt came from the artifact a `pull_request` run uploads, which a fork could tamper with to make this privileged `workflow_run` comment on an arbitrary PR/issue. Resolve the PR from context.payload.workflow_run instead (falling back to a commit-associated open PR lookup for workflow_dispatch runs), and escape any HTML comments already present in diff.md so they can't be mistaken for the marker. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * ci: grant actions:read to the pyperformance comment workflow download-artifact needs actions:read to fetch another run's artifact by run-id. Specifying a permissions block sets every unlisted scope to none, so without this the cross-run download silently failed for fork PRs -- the scenario this workflow exists for -- continue-on-error masked it, and the comment step found no diff.md and returned without posting. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: kill the whole process group on a benchmark timeout subprocess.run's own timeout only terminates the direct pyperformance process; pyperf re-execs the target interpreter as a child, which can survive the timeout and keep burning CPU into the next benchmark's measurement. Run each benchmark in its own session/process group instead, and on timeout SIGKILL the whole group via os.killpg. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: compute the head-vs-base cell from head_vs_base directly The cell was derived from base_vs_cpython and head_vs_cpython, both of which are None when a benchmark has no CPython time -- so it rendered "-" even though base and head both had valid times and head_vs_base was already computed. Use that precomputed ratio instead. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: invalidate the resumed catalog when the target executable changes catalog.json records results without any fingerprint of target_python, and resume skips benchmarks by name alone, so a local rebuild (or swapping the target) under the same --label could silently reuse stale measurements. Store a sha256 of the executable's contents alongside the catalog and, when it no longer matches, ignore the existing catalog and re-run everything. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * ci: fall back to github.sha for the head checkout on workflow_dispatch github.event.pull_request.head.sha is empty on a workflow_dispatch run (no pull_request payload), so `git checkout --quiet "" --` failed with "fatal: invalid reference" on this non-optional step, aborting every manual run before it could measure or render results. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: resolve --out/--cache-dir to absolute paths Every benchmark subprocess runs with cwd=work_dir (under --cache-dir), so a relative --out or --cache-dir passed on the command line resolved against that directory instead of the one run_all.py was invoked from, breaking catalog output, the pyperformance binary path, and PIP_FIND_LINKS. Both default to absolute paths already; resolve them up front so a relative override behaves the same way. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: persist catalog invalidation before the new fingerprint The fingerprint file was rewritten before the invalidated catalog was ever written back to disk. An interruption between the two (e.g. a crash during the first benchmark of the re-run) left a fingerprint matching the new executable pointing at a catalog.json still full of results measured against the old one, so the next run would trust it as fresh. Write the reset (empty) catalog first, then record the fingerprint. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: show benchmarks that fail on both base and head Rows were dropped whenever neither side had a mean, which also hid a benchmark that failed or timed out identically on base and head -- e.g. a C-extension gap unrelated to the PR under review. Only skip a benchmark never attempted on either commit; keep rows with a fail/timeout status so they still show up in the table. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: write catalog.json atomically write_catalog runs after every single benchmark, and Path.write_text truncates the file before writing the replacement. An interruption mid-write left catalog.json both missing every earlier result and invalid JSON, so the next run's resume logic failed outright at json.loads(). Write to a temporary sibling file and rename it into place instead. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: fix passed count and median population in the summary table "Benchmarks passed" counted len(base_ratios)/len(head_ratios), which require a *CPython* ratio too -- a CPython-only failure hid an otherwise-passing benchmark from both sides' counts. Count actual "ok" status per side instead. The two medians were also computed over independently-filtered populations: whichever benchmarks had a valid ratio on that side alone. A benchmark that regressed to failure on only one side dropped out of only that side's median, which could shift it well beyond what any surviving benchmark's timing actually changed by. Restrict both medians to benchmarks with a valid ratio on both sides, and label the population size. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: include RUSTPYTHONPATH's stdlib in the executable fingerprint The catalog-invalidation fingerprint hashed only the executable's bytes, but a RustPython binary copied away from its checkout finds its stdlib through RUSTPYTHONPATH -- pyperf re-execs the target interpreter into a venv and forwards that variable so it can still find Lib/. Changing that path, or editing the files under it, without rebuilding the binary left resume trusting results measured against a now-different stdlib. Fold its contents into the fingerprint too. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: require an explicit ok/fail status for pass/fail transitions only_head/only_base flagged a benchmark as newly passing or no longer passing based only on whether the other side had a mean, so a benchmark never attempted on one side (e.g. a partial --benchmarks run) was reported as a regression or fix it never was. Require the other side's status to be an actual fail/timeout, not just missing. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: don't gate the head/base geometric mean on a CPython ratio The summary panel -- pass counts and the head/base geometric mean, neither of which need a CPython comparison -- only rendered when at least one benchmark also had a valid ratio against CPython on both sides. A build with a perfectly good head-vs-base comparison but no common CPython result lost the whole panel, including the one aggregate that didn't depend on CPython at all. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: normalize RUSTPYTHONPATH and extract main() into named steps Fix: a relative RUSTPYTHONPATH was fingerprinted relative to this script's invocation directory but resolved by pyperf's subprocess relative to work_dir (its cwd), so a relative value could be hashed against the wrong directory entirely, or fail to resolve at runtime. Resolve it to an absolute path up front, alongside --out and --cache-dir. Refactor: main() had accumulated argparse setup, executable resolution, and the fingerprint/resume block inline, each wrapped in an explanatory comment -- a sign the function was doing too much rather than that it needed more narration. Extracted parse_args(), resolve_target_python(), and load_resumable_catalog() so main() reads as a short sequence of named steps and each comment sits next to the one function it actually explains. No behavior change; verified with unit tests against the extracted functions (fingerprint reacting to RUSTPYTHONPATH, resume picking up a prior catalog, invalidation on an executable change). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: split pr_diff.py's main() and fix a status-transition gap Fix: only_head was still gated on have_base, which only asks whether *any* base benchmark produced a timing -- so a benchmark that flipped from failing to passing went unreported whenever every other base benchmark also happened to fail. Status transitions don't need that gate; they already require an explicit ok/fail/timeout status on both sides. Refactor: main() had grown into one long function mixing argument parsing, catalog loading, row computation, and three separate rendering concerns. Split it into parse_args(), load_base_catalog(), build_rows(), render_intro(), render_summary_panel(), render_table(), and find_transitions(), each documenting the one thing it decides. No behavior change beyond the fix above; re-verified against every scenario exercised in the last several fixes (dual-fail rows, geo mean without a CPython ratio, unattempted benchmarks, and the have_base gap) plus the existing manual smoke tests. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * pyperformance: don't resolve an empty RUSTPYTHONPATH to the cwd The normalization only checked "RUSTPYTHONPATH" in os.environ, so a set-but- empty value (RUSTPYTHONPATH="") resolved to the current directory via Path("").resolve(). executable_fingerprint() would then recursively hash every file under wherever run_all.py was invoked from -- including its own results directory while writing to it -- causing spurious catalog invalidation and a lot of pointless hashing. Check for a nonempty value instead, matching the truthiness check already used everywhere else this variable is read. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv * docs(pyperformance): distinguish the host runner from the benchmarked target A reviewer read "pyperformance cannot be installed as-is on RustPython" and reasonably asked why RustPython would need to support pyperformance's CLI at all, since it's normally run as a command under any Python (see https://github.com/RustPython/RustPython/pull/8691#discussion_r3997957942). It doesn't: the pyperformance CLI (the host process) always runs under a real CPython regardless of --python. What actually needs psutil/pyperf importable is the *benchmark* subprocess -- pyperf re-execs the --python target and that subprocess itself does `import pyperf`. Spell out that host/worker split up front instead of leading with "cannot be installed". Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Joshua Megnauth
|
54b6865de8 |
windows: Small clean ups, less allocs (#8688)
* Remove a couple of unneeded NUL checks since WideCString handles them already * Amortize a few allocs |
||
|
Jeong, YunWon
|
850fadd388 |
Classify continuable unclosed strings as incomplete (#8694)
* Classify continuable unclosed strings as incomplete Mark EOF-open strings for PyCF_ALLOW_INCOMPLETE_INPUT, treat blank eval input as incomplete, detect unparenthesized lambda after a comma in f-strings, and advance parameter scans past default values. Assisted-by: Grok:4.6 * Diagnose incompatible prefixes and nested f-strings Report incompatible string prefixes, malformed \\N escapes, and f/t-string nesting beyond 150. Omit the extra newline on unterminated triple-quoted f-string SyntaxError.text. Assisted-by: Grok:4.6 * Fix empty defaults and longer incompatible prefixes Report an empty parameter default before later parameters, collect the full prefix before a quote, and prefer the u/b pair. Assisted-by: Grok:4.6 |
||
|
Jiseok CHOI
|
08e6e697a1 |
sqlite3: fix iterdump import and filter support (#8692)
* sqlite3: fix iterdump import and filter support Fix Connection.iterdump() to import sqlite3.dump with a from-list and forward its keyword-only filter argument, matching CPython 3.14. Remove stale expected-failure markers from all passing iterdump tests. Assisted-by: GitHub Copilot:GPT-5.6 Luna (high) * sqlite3: fix clippy core iterator lint Use core::iter instead of std::iter in the iterdump filter kwargs path, as required by RustPython clippy. Add the sqlite3 expected-failure workflow and CI clippy feedback reports. Assisted-by: GitHub Copilot:GPT-5.6 Luna (high) |
||
|
Jeong, YunWon
|
a0dc7c2e13 |
host_env: share file identity, dll dirs, link and startfile (#8696)
* host_env: share file identity, dll dirs, link and startfile Add GetFileInformationByHandle, AddDllDirectory/RemoveDllDirectory, CreateHardLinkW, ShellExecuteW, and a wide ExpandEnvironmentStringsW reader that keeps unpaired surrogates. Assisted-by: Claude * vm: use host_env for startfile, dll dirs and link Wire nt.startfile, nt._add_dll_directory/_remove_dll_directory and Windows os.link through the host_env wrappers. ExpandEnvironmentStrings keeps unpaired surrogates via the wide reader. Assisted-by: Claude:grok-4.6 * host_env: publish Windows =X: drive cwd for spawn Extract publish/ensure helpers from set_current_dir and call ensure before _wspawnv/_wexecv so CRT env walks have a first =X: entry. Assisted-by: Claude:grok-4.6 |
||
|
Jeong, YunWon
|
7c6510334c |
Keep import module names as PyUtf8Str (#8690)
Add PyStr::as_utf8() for a non-erroring UTF-8 view. Thread PyUtf8Str through import resolution, module attribute errors, and IMPORT_FROM instead of converting names to owned String. Assisted-by: Grok:4.6 |
||
|
Jeong, YunWon
|
ce45fc24c7 |
Honor coroutine reuse, origin, and unawaited warnings (#8695)
Raise RuntimeError on send/throw of a closed coroutine, track cr_origin, emit never-awaited warnings through the warnings helper, and add cr_suspended. Stop treating coroutines as iterable. Warn on unawaited asyncgen asend/athrow/aclose, match async for TypeError messages, and attach coroutine method and name getset docs. Assisted-by: Grok:grok-4.6 |
||
|
Jiseok CHOI
|
407ba437cd |
sqlite3: fix row_factory deletion error (#8693)
Match CPython 3.14 by distinguishing row_factory assignment from deletion and raising AttributeError with the compatible message. Unmark the passing row_factory deletion test. Assisted-by: GitHub Copilot:GPT-5.6 Luna (high) |
||
|
Jeong, YunWon
|
a2e2bd299e |
asyncio: honor pending-task, unawaited, and get_referrers (#8689)
* asyncio: honor pending-task, unawaited, and get_referrers Report Task destructor as 'Task was destroyed but it is pending!'. Warn when a never-started coroutine is collected. Preserve signal() errno. Attribute generator-owned frame locals to the owner in gc.get_referrers. Remove the expectedFailure markers that now pass. Assisted-by: Grok:grok-4.6 * tests: drop expectedFailure after unawaited warning The never-started coroutine RuntimeWarning unblocks test_bpo_45813_1 and the AsyncMock assertNeverAwaited cases. Assisted-by: Grok:grok-4.6 |
||
|
Jeong, YunWon
|
66a3d9daa7 |
common: implement remaining builtin text codecs (#8687)
* common: implement remaining builtin text codecs Move utf-16/32/7, unicode-escape, raw-unicode-escape and the bytes escape transform into rustpython-common and call them from _codecs instead of _pycodecs. Assisted-by: Claude:grok-4.6 * host_env: share locale grouping, uuid and pipe state Add localeconv_numeric so grouping keeps a CHAR_MAX terminator, UuidCreateSequential for _uuid, and GetNamedPipeHandleStateW. Assisted-by: Claude:grok-4.6 * Fix utf-7 clippy and the utf-16-be snippet length Assisted-by: Claude:grok-4.6 * codecs: treat escape decode as final and accept str unicode_escape_decode and raw_unicode_escape_decode default final to true and accept a str as UTF-8. Fix clippy on wide_order, escape encode, and encode closures. Remove expectedFailure markers that now pass. Assisted-by: Grok:grok-4.6 |
||
|
Jeong, YunWon
|
43ed0a80b6 |
Warn on invalid octal escapes and parse-fail strings (#8686)
* Warn on invalid octal escapes and parse-fail strings Treat three-digit octal values above 0o377 as invalid escapes and scan quoted literals even when parse fails, so SyntaxWarning is emitted before a later SyntaxError. Keep warning and SyntaxError text on InvalidEscape. Enable the remaining test_string_literals invalid-escape cases. Assisted-by: Grok:4.6 * Remove expectedFailure from passing test_codeop cases test_invalid_warning and test_raw_raises_error now emit SyntaxWarning on parse-fail invalid escapes. Assisted-by: Grok:4.6 * Scan unparsed strings with token-boundary prefixes Require a complete prefix at an identifier boundary, and walk f-string interpolations separately after a parse failure. Assisted-by: Grok:4.6 |
||
|
Lee Dogeon
|
a582b938bd |
Exclude C-API tests from workspace coverage (#8619)
Assisted-by: Codex:gpt-5.6-sol |