* Remove unused rust impl for formatting dis output
* remove `examples/dis.rs`
* Added tests
* Update lock
* Try to set snapshot dir
* Remove verbose flag
* Regenerate snapshots after #7711
* Revert "Bump insta from 1.46.3 to 1.47.2 (#7706)"
This reverts commit e6d9ea6bfe.
* Debug info
* Show diff as well
* Show debug faster
* CI: true env
* Recert CI
* Add `CI: true` in ci emv
* Reapply "Bump insta from 1.46.3 to 1.47.2 (#7706)"
This reverts commit 693ca8cbe4d7885a81162a9be31e8bb567db885a.
* simplify macro
* trim on function side
* Force insta workspace root
* fix merge
* Report invalid \uXXXX escape position at the u character
CPython's json decoder reports the position of the `u` specifier
when a \uXXXX escape fails to parse, but RustPython was reporting
the preceding `\`. For surrogate-pair cases (\uXXXX\uYYYY) the
second call was passing char_offset + next_char_i + 1, which
lands on the first hex digit of the first escape -- unrelated to
the actual failure site.
Pass next_char_i (position of the primary `u`) to the primary
decode_unicode call, and capture the second `u`'s char index from
the next_tuple peek to pass to the surrogate-pair decode_unicode
call.
Verified: 13 targeted probes across invalid-hex, short, and pair
cases now all match CPython positions. test.test_json 214 tests
pass with no regressions.
* Add regression test for invalid \uXXXX escape position
* Use raise AssertionError instead of assert False (B011)
* Match CPython error type for non-ASCII struct format arguments
Struct() raised the wrong exception type when the format argument
contained non-ASCII characters:
- str input with non-ASCII char: RustPython raised UnicodeDecodeError
with an empty message; CPython raises UnicodeEncodeError as if
format.encode('ascii') had been called directly.
- bytes input with non-ASCII byte: same wrong UnicodeDecodeError;
CPython passes the bytes through to the format parser, which then
errors with struct.error("bad char in struct format").
Restructure IntoStructFormatBytes::try_from_object to:
- raise UnicodeEncodeError("ascii", s, start, start+1, "ordinal not
in range(128)") for non-ASCII str, with start computed as the
first non-ASCII code point position (matching CPython's natural
encoding-error format);
- raise struct.error("bad char in struct format") for non-ASCII bytes,
produced via the existing new_struct_error helper.
Probed byte-identical with CPython 3.14.4 for both cases. Full
test.test_struct (43 tests) passes with no regressions. Sanity-tested
all standard format/pack/unpack/calcsize call shapes remain unchanged.
* Add regression test for non-ASCII format string error types
* Use raise AssertionError instead of assert False (B011)
The _json decoder had two failure modes when a Python str value would
contain a lone surrogate (legal per the Python 3 str model):
1. Boundary UnicodeEncodeError: JsonScanner::Callable::call rejected
any input str with surrogates via try_into_utf8 before scanning
began.
2. Silent U+FFFD corruption: call_scan_once and parse_object's key
path called .to_string() on scanstring's Wtf8Buf output, which
routes through Wtf8::Display (lossy). Array values and dict keys
decoded from JSON \uXXXX escapes silently became U+FFFD.
Switch JsonScanner's five PyUtf8StrRef signatures to PyStrRef, drop
the entry-point try_into_utf8 call, and feed Wtf8Buf directly to
new_str instead of going through .to_string(). Key memoization now
uses HashMap<Wtf8Buf, PyStrRef> so surrogate-bearing keys survive
interning. parse_number takes &[u8] since JSON numbers are ASCII.
Extends the WTF-8 refactor pattern established in #7673 to the
decoder. machinery::scanstring already returns Wtf8Buf and is
unchanged.
Unmasks test_single_surrogate_decode. 214 tests in test.test_json
pass with no regressions. Decoder output verified byte-identical to
CPython 3.13.4 over 10,000 random fuzz cases (JSON docs containing
random surrogate escapes at root/list/dict positions, compared via
json.dumps(..., ensure_ascii=True, sort_keys=True)).
encode_basestring/encode_basestring_ascii took PyUtf8StrRef, so
json.dumps(str_with_lone_surrogate) raised UnicodeEncodeError at the
Python/Rust boundary before write_json_string ran. CPython's encoder
emits \uXXXX under ensure_ascii=True and passes raw WTF-8 otherwise.
Switch to PyStrRef + s.as_wtf8(), matching scanstring in the same file.
Rewrite write_json_string to accept &Wtf8 and iterate
code_point_indices, emitting \uXXXX for surrogates in ascii mode and
passing their bytes through otherwise. Stop escaping 0x7F in the
ensure_ascii=False path (matches py_encode_basestring). Return Wtf8Buf
via the checked from_bytes so invariant breaks panic instead of UB.
Fuzzing also exposed two pre-existing ESCAPE_CHARS typos: 0x0B was
"\u000" and 0x1B was "\u001" (both missing trailing 'b'). Fixed here.
Verified byte-identical with CPython 3.13.4 over 16 manual + 10,000
random fuzz cases. Full test.test_json: 214 tests, 0 failures, 0
unexpected successes. Unmasks test_ascii_non_printable_encode and
test_single_surrogate_encode. Decoder path is a follow-up.
* Fix stack overflow on deeply-nested JSON in json.loads()
json.loads() on a deeply-nested array or object payload (e.g.
'[' * 50000 + ']' * 50000) overflowed the native Rust stack and
crashed the interpreter process with SIGSEGV. CPython raises
RecursionError on the same input via _Py_EnterRecursiveCall in
Modules/_json.c.
The recursion lives in the mutual call chain:
JsonScanner::parse_object / parse_array
-> JsonScanner::call_scan_once
-> JsonScanner::parse_object / parse_array
Every descent funnels through call_scan_once, so wrapping its body
with vm.with_recursion covers both '{' and '[' paths (and their
mixed nesting) with a single guard.
Before:
./rustpython -c "import json; json.loads('[' * 50000 + ']' * 50000)"
-> SIGSEGV (exit 139)
After:
-> RecursionError: maximum recursion depth exceeded while
decoding a JSON object from a string
Verified:
- extra_tests/snippets/stdlib_json.py: all assertions pass
(includes 3 new regression cases: array, object, alternating
nesting at depth 100000)
- cargo run -- -m test test_json: 214 passed, 0 regressed
(9 skipped, 13 expected failures, all pre-existing)
- depth 500000 no longer crashes (RecursionError)
- shallow parsing unchanged
* Enable test_highly_nested_objects_decoding
Per @ShaharNaveh's review on #7632: this test was previously marked
`@unittest.skip("TODO: RUSTPYTHON; crashes")` because json.loads
would SIGSEGV on the 500_000-deep input. The recursion-guard added
in this PR makes it raise RecursionError like CPython, so the skip
decorator can be removed.
$ cargo run -- -m unittest \
test.test_json.test_recursion.TestCRecursion.test_highly_nested_objects_decoding \
test.test_json.test_recursion.TestPyRecursion.test_highly_nested_objects_decoding
...
Ran 2 tests in 0.825s
OK
$ cargo run -- -m test test_json
Ran 214 tests (7 skipped, 13 expected failures) — all pass.
* Add InstructionMetadata::stack_effect_jump for branch stack effects
CPython's compile.c provides stack_effect(opcode, oparg, jump) where the
jump parameter selects between fallthrough and branch effects. The existing
stack_effect() only returns the fallthrough effect.
Add stack_effect_jump() that returns the branch effect. Most instructions
have identical fallthrough/branch effects; ForIter and Send are the
exceptions (ForIter: fallthrough=+1, branch=-1; Send: fallthrough=0,
branch=-1).
* apply review
* Disallow instantiation of sys.getwindowsversion type
Add slot_new to PyWindowsVersion that raises TypeError,
matching sys.flags behavior.
* Remove incorrect WSAHOS errno constant
WSAHOS was hardcoded as an alias for WSAHOST_NOT_FOUND, but
CPython guards it with #ifdef WSAHOS which doesn't exist in
modern Windows SDK headers.
* Fix mmap resize to raise OSError instead of SystemError
* Fix CreateProcess with empty environment on Windows
Empty env dict produced a single null terminator, but
CreateProcessW requires a double null for a valid empty
environment block.
* Revert mmap resize error to SystemError and fix errno.rs formatting
mmap resize raises SystemError (not OSError) when mremap is unavailable,
matching CPython behavior. test_mmap catches SystemError to skip unsupported
resize operations.
* Fix named mmap resize to raise OSError and unmark test_sleep expectedFailure
Named mmap resize on Windows should raise OSError (not SystemError).
Remove expectedFailure mark from TimeEINTRTest.test_sleep as it now passes.
* Use expectedFailureIf for TimeEINTRTest.test_sleep on Linux
test_sleep passes on macOS but fails on Linux due to timing.
* Remove expectedFailure for TimeEINTRTest.test_sleep
test_sleep now passes on all platforms.
* Fix allow_threads and EINTR handling
- Wrap Windows SemLock acquire wait with allow_threads
- Retry nanosleep on EINTR with remaining time instead of
returning early
- Remove expectedFailure for test_sleep in _test_eintr.py
* Remove expectedFailureIfWindows for testHashComparisonOfMethods
* Migrate direct __repr__ definitions to Representable trait
Move legacy #[pymethod] __repr__ to impl Representable for:
- PySSLContext, PySSLSocket (ssl.rs)
- BufferedReader, BufferedWriter, BufferedRandom (_io.rs)
Enable __repr__ guard in derive to prevent future direct definitions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Migrate direct __del__ definition to Destructor trait for PySocket
Move #[pymethod] __del__ to impl Destructor for PySocket.
Preserves ResourceWarning emission and close behavior.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Remove redundant __iter__ pymethods from PyFuture and PyTask
Both types already have impl Iterable, so the direct #[pymethod]
__iter__ definitions were duplicates. Enable __iter__/__next__
guards in derive to prevent future direct definitions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Enable derive guards for __get__/__set__/__delete__ descriptor methods
All concrete types already use GetDescriptor/SetDescriptor traits.
Activate the compile-time guard to prevent future direct definitions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Enable derive guards for AsMapping/AsSequence slot methods
Remove redundant #[pymethod(name = "__len__")] from PyStr
(already provided via AsMapping/AsSequence trait impls).
Enable compile-time guards for __len__, __contains__, __getitem__,
__setitem__, __delitem__ to prevent future direct definitions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use single-record reading (recv_one_tls_record) for all SSL
reads, not just handshake. This prevents rustls from eagerly
consuming close_notify alongside application data, which left
the TCP buffer empty and caused select()-based servers to miss
readability and time out.
Also fix recv_one_tls_record to return Eof (not WantRead) when
peek returns empty bytes, since empty peek means the peer has
closed the TCP connection.
During STARTTLS handshake, sock_recv(16KB) could consume
application data that arrived alongside handshake records.
The consumed data ended up in rustls's internal buffer where
select() could not detect it, causing asyncore-based servers
to miss readable events and the peer to time out.
Use MSG_PEEK to find the TLS record boundary, then recv()
only one complete record. Remaining data stays in the kernel
TCP buffer, visible to select(). This matches OpenSSL's
default no-read-ahead behaviour.
Fixes flaky test_poplib (TestPOP3_TLSClass) failures.
* gc: add CollectResult, stats fields, get_referrers, and fix count reset
- Add CollectResult struct with collected/uncollectable/candidates/duration
- Add candidates and duration fields to GcStats and gc.get_stats()
- Pass CollectResult to gc.callbacks info dict
- Reset generation counts for all collected generations (0..=N)
- Return 0 for third value in gc.get_threshold() (3.13+)
- Implement gc.get_referrers() by scanning all tracked objects
- Add DEBUG_COLLECTABLE output for collectable objects
- Update test_gc.py to expect candidates/duration in stats
* Update test_gc from v3.14.3
* Update test_gc.py from CPython v3.15.0a5
Taken from v3.15 (not v3.14.3) because get_stats() candidates/duration
fields were added in 3.13+ and the corresponding test assertions only
exist in 3.15.
* Fix gc_state build on wasm32: skip Instant timing
* Add candidates/duration to gc callback info, mark v3.15 test failures
* Fix gc.get_referrers to exclude executing frames, fix Future cancelled exc leak
- get_referrers: skip frame objects on the execution stack, since
they are not GC-tracked in CPython (_PyInterpreterFrame)
- _asyncio Future/Task make_cancelled_error_impl: clear the stored
cancelled exception after returning it, matching the Python
_make_cancelled_error behavior
* Fix gc.get_threshold to return actual gen2 threshold value
* Fix inconsistent GC count reset in early-return paths
Use the same reset_end formula in unreachable-empty early returns
as in the main collection path and collecting-empty path.
* Accept keyword arguments in socket.__init__
Use a FromArgs struct instead of a positional-only tuple so that
family, type, proto, and fileno can be passed as keyword arguments.
* Disable comp_inlined in symbol table to match compiler
The compiler does not yet implement PEP 709 inlined comprehensions
(is_inlined_comprehension_context always returns false), but the
symbol table was marking comprehensions as inlined. This mismatch
could cause comprehension-local symbols to be merged into the parent
scope while the compiler still looks them up in a separate scope.
---------
Co-authored-by: CPython Developers <>
Shahar Naveh