* Fix SymbolUsge::Iter
* fix os.rename
* fix sre
* Re-export rename from crate::posix on Windows
posix_windows.rs is mounted as the `posix` module on Windows, so
`crate::posix_windows` does not resolve.
Assisted-by: Claude
* Apply rustfmt to sre_engine tests
Assisted-by: Claude
* Update test expectations for the sre and tokenize fixes
- test_re: drop expectedFailure from test_word_boundaries,
test_possessive_quantifiers and test_bug_gh101955
- test_inspect, test_pydoc: drop expectedFailure from the signature
tests that now pass
- test_pdb: drop expectedFailure from the two file-modification tests
and +EXPECTED_FAILURE from the test_post_mortem_chained and
test_pdb_asynctask doctests
- test_unittest: mark test_autospec_on_bound_builtin_function as
expectedFailure; inspect.signature() now succeeds on time.ctime
because of its auto-generated __text_signature__
Assisted-by: Claude
* Fix dict iterator __reduce__ to resume from current position
Pickling a partially-consumed dict / dict-view iterator restarted from the
beginning because __reduce__ materialized every entry and ignored the
iterator's position. Walk from the current position (mirroring next) so
only the not-yet-yielded entries are captured, matching CPython, which
reduces both directions to iter(remaining).
Fixing the reverse iterator also uncovered two pre-existing bugs in
reverse iteration itself, both from prev_entry saturating at 0: a hole
just above index 0 made the entry at 0 yield twice, and deleting the
first-inserted key made reversed() loop forever. prev_entry now returns
the found entry's actual index and stops cleanly at index 0, and the
reverse iterator/reduce detect exhaustion from that index.
Verified against CPython 3.14 across all pickle protocols, dict states
(including deletions), iterator kinds, and consumption counts. No
regressions in test_dict, test_dictviews, test_ordered_dict,
test_collections, test_userdict, test_iter, or test_copy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Apply cargo fmt to dict.rs imports
Collapse the builtins import block left multi-line after removing the
now-unused builtins_reversed import.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Unmark TestSyncManagerTypes.test_dict as expectedFailure
The dict iterator __reduce__ fix makes SyncManager dict proxies pickle
correctly, so this test now passes under spawn and forkserver. Remove the
stale expectedFailure marker that was causing an UNEXPECTED SUCCESS CI
failure.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`memchr` is used throughout RustPython for searching through bytes
expediently. However, for NUL checks, it's scantily used. Instead, our
NUL checks either use `contains` or `memchr`.
I switched all of the `contains(b'\0')` I could find to using memchr
instead. I marked the failure paths as cold to hint to LLVM that
interior NULs are truly exceptional. This should help branch prediction
a bit which is nice for string/bytes functions since they are likely
called a lot.
Support the CPython 3.13+ format spec syntax that puts a grouping option
after the precision, so `format(1234.56789, '.6,f')` gives
'1234.567,890'. The integer and fractional parts group independently and
may use different separators (`,.6_f` -> '1,234.567_890'), so the spec
carries a separate `frac_grouping_option`.
`parse_precision` now consumes the separator that follows the precision
digits and rejects a `,`/`_` mix. A repeated separator is deliberately
left in the spec so the trailing-text check reports it, which is how
CPython arrives at a different message there. A dot followed by neither
digits nor a separator now raises "Format specifier missing precision"
rather than an unrelated error.
Fraction digits group away from the decimal point, so the last group may
be short, and any exponent or percent tail is left intact. The
separators count toward the field width, so zero padding of the integer
part reserves room for them.
'n' takes its separators from the locale and so cannot carry one. The
complex locale path rewrites 'n' to 'g' before delegating, so it has to
validate first; that also makes `format(1+2j, ',n')` fail as it does in
CPython, which it previously did not.
Reference (CPython 3.14), Python/formatter_unicode.c:
- parsing: https://github.com/python/cpython/blob/3.14/Python/formatter_unicode.c#L257-L299
- 'n' rejection: https://github.com/python/cpython/blob/3.14/Python/formatter_unicode.c#L361-L367
- number split: https://github.com/python/cpython/blob/3.14/Python/formatter_unicode.c#L488-L516
- zero-pad width: https://github.com/python/cpython/blob/3.14/Python/formatter_unicode.c#L604-L606
Remove `@unittest.expectedFailure` from tests that now pass
(2 in test_format, 1 in test_float).
Assisted-by: Claude:claude-opus-5
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* Implement termios.tcgetwinsize/tcsetwinsize
Adds ioctl(TIOCGWINSZ/TIOCSWINSZ) wrappers in host_env and the
corresponding Python-facing functions in the termios stdlib module,
removing the associated expectedFailure markers in test_termios.py.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT
* Use ret < 0 convention for ioctl error checks in termios winsize
POSIX ioctl only guarantees -1 on failure, not exactly 0 on success;
matches the existing check_libc_neg convention used elsewhere in
host_env (e.g. fcntl.rs, posix.rs::get_terminal_size).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT
* Reject non-sequence size in tcsetwinsize
extract_elements_with also accepted dicts (treating keys as elements).
Switch to try_sequence(), matching CPython's PySequence_Check, so only
real sequences (tuple/list/etc.) are accepted.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT
* Use `core::mem::zeroed` instead of `std::mem::zeroed`
`std::mem` just re-exports `core::mem`, but clippy prefers importing
from core when there's no OS dependency. This was breaking the wasm CI build.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT
* Use try_index for tcsetwinsize row/col conversion
CPython's PyLong_AsLong calls __index__ on non-int objects before
converting, so downcast_ref::<PyInt> was stricter than CPython.
try_index matches that behavior and is more concise.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT
* Mark test_fork/test_spawn_doesnt_hang as expected failures
Both were hidden by a skipIf on tty.tcgetwinsize, which now exists.
Root cause: pty.fork() calls os.login_tty(), which isn't implemented,
so the forked child crashes before the test body runs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT
* Use unittest.skip instead of expectedFailure for pty.fork() tests
expectedFailure still runs the test body, so pty.fork()'s real fork()
still happens and the child crashes on missing os.login_tty inside the
parallel test runner's worker process, corrupting its JSON reporting
channel ("worker bug"). skip prevents the method from running at all,
avoiding that. Verified locally with -j 2.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbzcYLnX5tMfTM9BP7FDTT
* Clarify skip reason for pty.fork() tests
* Simplify tcgetwinsize/tcsetwinsize fd argument with Fildes destructuring
* Explain why pty.fork() tests use skip instead of expectedFailure
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Replace malformed unicode error helper calls with the _real variants
new_unicode_decode_error / new_unicode_encode_error build the exception
from a bare message without running the initializer, so the result has
none of the five attributes a unicode error must carry and str() renders
as an empty string. Convert the non-Windows call sites in csv, socket,
getlogin and the fs-path decoders to the _real constructors, passing the
source bytes/str and the failing offset from the captured Utf8Error.
The Windows-gated sites (nt, mbcs/oem codecs) and the sites whose source
object is not reachable (uname, array) are left for follow-ups.
Assisted-by: Claude Code:claude-fable-5
* Report the full invalid UTF-8 span in the converted decode errors
The conversions used valid_up_to() + 1 for the decode-error end offset,
which under-reports multi-byte invalid sequences. Take the span from the
Utf8Error instead: valid_up_to() + error_len(), or the input length for a
truncated sequence (error_len() == None), matching CPython.
Assisted-by: Claude Code:claude-opus-4-8
CPython raises ProgrammingError with "Binding N is a named parameter"
when a query uses named placeholders (, , ) but
the caller passes a sequence instead of a mapping.
RustPython's bind_parameters_sequence() did not check whether each
binding slot is a named parameter, so no error was raised.
Fix: call sqlite3_bind_parameter_name() for each slot in
bind_parameters_sequence(). If the first byte of the returned name is
not '?' (i.e. it is a named placeholder), raise ProgrammingError before
attempting to bind.
Assisted-by: GitHub Copilot:claude-sonnet-4-6
Raise OverflowError when a finite f64 value overflows while being converted for the struct f format. Remove the expected-failure marker from the corresponding regression test.
Assisted-by: Codex:gpt-5
Row(cursor, data) raised ValueError when cursor.description was None.
CPython allows this case and returns an empty key list.
- Use empty tuple when description is None instead of raising
- Include the key name in the IndexError when a string key is not found
Assisted-by: GitHub Copilot:claude-sonnet-4-6
When a Connection is explicitly closed via con.close(), subsequent
operations (cursor(), commit(), rollback(), create_function(), etc.)
should raise ProgrammingError with 'Cannot operate on a closed database.'
to match CPython behaviour.
Previously, _db_lock() always returned 'Base Connection.__init__ not
called.' when self.db was None, without distinguishing between a
connection that was never initialised (subclass before __init__) and
one that was initialised and then explicitly closed.
Fix: inspect the initialized atomic flag — if True but db is None,
the connection was closed; if False, it was never initialised.
Assisted-by: GitHub Copilot:claude-sonnet-4-6
* Keep StringIO text operations CPython-compatible
Constraint: email.feedparser depends on StringIO(newline='') recognizing CR, LF, and CRLF boundaries.
Rejected: an email-specific parser workaround | StringIO is the shared root cause.
Confidence: high
Scope-risk: moderate
Directive: Keep internal storage byte-based; convert only at StringIO API boundaries.
Tested: prek run --all-files; test_memoryio; test_email; full workspace verification by contributor
Assisted-by: Codex:gpt-5.6-sol
* Run restored StringIO conformance tests
Constraint: upstream StringIO tests already cover the repaired behavior.
Rejected: new RustPython-only regression tests | existing CPython coverage is sufficient.
Confidence: high
Scope-risk: narrow
Directive: Remove expected-failure markers only while the inherited tests pass.
Tested: prek run --all-files; test_memoryio; test_email
Assisted-by: Codex:gpt-5.6-sol
* Keep StringIO newline handling consistent
Constraint: StringIO must apply its newline mode consistently when constructing, writing, and reading text.\nRejected: a readline-only fix | it leaves CR and CRLF modes internally inconsistent.\nConfidence: high\nScope-risk: moderate\nDirective: Keep StringIO buffer contents valid WTF-8 before using unchecked views.\nTested: prek run --all-files; cargo clippy -p rustpython-vm -- -D warnings; test_memoryio; test_shlex; test_email; workspace excluding rustpython-capi; full workspace manually verified by contributor\nNot-tested: local rustpython-capi workspace test crashes with a pre-existing macOS SIGSEGV\nAssisted-by: Codex:gpt-5.6-sol
Part of #8245 to reduce the amount of work needed to review. I
simplified the embedded nul errors by forwarding to the implementations
in `vm::exceptions`.
* Use dict keys-version stamps and entry-index hints in attr specializations
Add a keys-version stamp to Dict: a globally unique u32 assigned lazily
and reset on any key-set change (new key, deletion, clear). Value-only
updates keep the stamp.
- LoadAttrMethodWithValues / LoadAttrNondescriptorWithValues: cache the
instance dict's stamp in the pointer cache and skip the shadow probe
while the stamp matches.
- LoadAttrWithHint: cache the entry index at specialization time; a hit
is an identity check on the entry key instead of a hash probe, with
self-refresh on miss.
- StoreAttrWithHint / StoreAttrInstanceValue: replace values through the
cached entry index via set_item_with_hint.
Fixes youknowone#42
Assisted-by: Claude
* Share keys-version stamps between dicts with identical layouts
Derive the keys-version stamp from a shape — the ordered interned-key
sequence of a hole-free dict — instead of always allocating a unique
stamp. Dicts with identical layouts (instances of the same class built
by the same __init__) now carry equal stamps, so a LOAD_ATTR cache
entry populated by one instance skips the shadow probe for every
instance sharing the layout.
Shapes are held in a fixed-size lock-free table keyed by interned key
addresses; dicts with holes, non-interned keys, or more than 32 keys
fall back to dict-unique stamps.
Assisted-by: Claude
* Address clippy and review feedback for keys-version stamps
- Use BuildHasher::hash_one for shape hashing (clippy manual_hash_one)
- Fix set_item_with_hint doc: a refreshed hint is returned on any hint
miss, not only when the hinted slot was vacant
- Route both holey-dict instances through a single LOAD_ATTR cache site
in the snippet test
Assisted-by: Claude
* Restore try_read_cached_descriptor doc comment to its function
The doc block and #[inline] were left attached to store_attr_dict_hinted
when it was inserted above try_read_cached_descriptor.
Assisted-by: Claude
* Specialize LoadAttrWithHint even when the entry index exceeds u16
hint_for_key returns None both for an absent key and for a present key
whose entry index does not fit in u16, so very large instance dicts
stopped specializing entirely. Use get_item_opt_refresh_hint to decide
presence, degrading an unrepresentable hint to 0: the handler then
keeps taking its full-probe fallback path.
Assisted-by: Claude
* bz2: return streaming output from BZ2Compressor.compress()
Return output produced during compression instead of waiting for flush().
Constraint: Preserve encoder state without flushing the BZ2 stream
Confidence: high
Scope-risk: narrow
Directive: Do not flush merely to retrieve partial output
Tested: test_bz2; stdlib_bz2.py; prek run --all-files
Assisted-by: Codex:gpt-5.6-sol
* bz2: remove redundant streaming-output snippet
* csv: handle empty fields with skipinitialspace
skipinitialspace preprocessed records by trimming both ends of every split
field. Fields containing only spaces could therefore produce an invalid
slice, while trailing spaces were removed even though CPython preserves them.
Trim only field prefixes in the csv-core path and make the remaining
all-space trimming helper return an empty slice safely.
QUOTE_NOTNULL and QUOTE_STRINGS add another distinction: an unquoted empty
field becomes None, but a quoted empty field remains an empty string. Both
forms have identical decoded bytes, so extend the existing QUOTE_NONE custom
reader into one shared path that records whether each field started quoted.
Use that metadata only for the null conversion while retaining QUOTE_NONE
escape behavior.
Unmark Test_Csv.test_read_skipinitialspace now that its standard,
QUOTE_NOTNULL, and QUOTE_STRINGS cases pass.
This keeps the existing one-item reader lifecycle and does not add the larger
strict or multiline parser state machine.
Assisted-by: Codex:gpt-5-sol
* csv: make skipinitialspace quote-aware
The csv-core preprocessing path split each raw iterator item on every
delimiter before trimming field prefixes. Delimiters inside quoted fields were
therefore treated as separators, so spaces after them could be removed before
csv-core parsed the record.
Extract the quote, escape, delimiter, and field-start transitions from
read_quote_record() into a small per-item scanner. Reuse its events in the
skipinitialspace preprocessor and discard only InitialSpace events while
copying all other source bytes unchanged.
This keeps the custom quote-mode reader and preprocessing logic aligned
without introducing persistent multiline or strict parser state. Remove the
delimiter cache that was only needed by the old split-and-join path and add a
regression for spaces following a delimiter inside a quoted field.
Assisted-by: Codex:gpt-5-sol
Jeong, YunWon