* Add round-half-even hex float parser to common float_ops
Add float_ops::from_hex returning Result<f64, HexFloatError>, which parses
the coefficient and exponent, rounds round-half-even over the full exponent
range, and detects overflow. Route float.fromhex through it: raise
OverflowError for values too large to represent and ValueError for invalid
input. Remove the expectedFailure on test_from_hex.
Assisted-by: Claude
* Address review: Option-returning byte helpers, const fns, drop test_ prefix
Assisted-by: Claude
* Whitelist fdigits and inity in cspell
Assisted-by: Claude
* Home is_py_ascii_whitespace in rustpython-wtf8; share with common and sre_engine
Assisted-by: Claude
* Use shared wtf8 is_py_ascii_whitespace in bytes_inner
Remove the private copy of is_py_ascii_whitespace in bytes_inner.rs and
import the shared rustpython_common::wtf8::is_py_ascii_whitespace.
Assisted-by: Claude
* Move which crate dependency from vm to host_env
Add host_env::fs::which wrapper and update getpath.rs call site.
* Move readline module from vm to host_env
Move crates/vm/src/readline.rs to crates/host_env/src/readline.rs and
re-export it from rustpython_vm::readline. Move rustyline dependency
from vm to host_env. Drop the host_env feature gates on history I/O
inside the moved module.
* Move gethostname and mac_address dependencies from stdlib to host_env
Add host_env::socket::hostname and host_env::socket::mac_address
wrappers. Update _socket.gethostname and _uuid.get_node_id call sites.
* Re-export libc constants and types through host_env
Move direct libc constant and type references in stdlib (faulthandler,
fcntl, multiprocessing, posixshmem, select, syslog) to use host_env
re-exports. Replace libc::c_int and libc::c_uint in stdlib with
core::ffi equivalents.
* Re-export mmap libc constants through host_env
Move direct libc MADV_*, MAP_*, PROT_*, EOVERFLOW references in
stdlib::mmap to host_env::mmap re-exports. Replace libc::c_int with
core::ffi::c_int.
* Re-export locale and termios libc constants through host_env
Move direct libc LC_*, ABDAY_*, ABMON_*, TIOC*, FIO* and related
constants in stdlib::locale and stdlib::termios to host_env
re-exports. Replace libc::c_char in stdlib::locale and
stdlib::openssl::cert with core::ffi::c_char.
* Replace libc c-types and FILE wrappers in openssl with host_env
Replace libc::c_int, c_long, c_uchar, c_uint, c_ulong, c_void,
c_char in stdlib::openssl with core::ffi equivalents. Replace
libc::ENOENT with host_env::errno::errors::ENOENT. Add
host_env::fileutils::{CFile, fclose} and route the load_dh_params
fclose call and PEM_read_DHparams FILE pointer type through them.
* Fix Windows build: expose EOVERFLOW, SIGSEGV, SIGFPE on Windows
host_mmap::EOVERFLOW and host_faulthandler::{SIGSEGV, SIGFPE}
re-exports were gated to cfg(unix) but stdlib uses them from Windows
and platform-agnostic code paths.
* Re-export remaining libc types and constants through host_env
Move direct libc references in stdlib (resource, posixsubprocess,
grp, socket) to host_env re-exports. Replace libc::c_long,
libc::c_longlong, libc::c_char in stdlib::socket with core::ffi
equivalents. Add resource::{RLIMIT_*, RUSAGE_*, c_long, rlim_t,
rlimit, timeval}, posix::{c_char, pid_t}, grp::gid_t, and
socket::{AF_*, SOCK_STREAM, sa_family_t, socklen_t, sockaddr_*}
re-exports.
* Move socket2 and dns-lookup dependencies from stdlib to host_env
Re-export socket2 as host_env::socket::raw and dns-lookup as
host_env::socket::dns. Update _socket call sites to route through
host_env aliases.
* Move system-configuration dependency from stdlib to host_env
Re-export the macOS system-configuration crate as
host_env::system_configuration and update _scproxy call sites.
* Restrict CAN/ALG sockaddr structs to target_os linux
libc on Android (bionic) does not define sockaddr_can, so importing it
for target_os = "android" fails to compile. Keep AF_ALG/AF_CAN for both
linux and android, and gate sockaddr_alg/sockaddr_can to linux only.
Assisted-by: Claude
* common: fix decompose_float for subnormal inputs
Subnormals carry a biased exponent of 0 and no implicit leading mantissa
bit, so the bit-twiddling frexp misread them: hash(5e-324) returned
8404992 instead of CPython's 16777216. Scale subnormals into the normal
range by 2**54 (exact) before decomposing and fold the shift back into
the returned exponent. Adds a frexp-contract unit test over normals,
subnormals, and boundary values.
Assisted-by: Claude
* common: add HashSecret::from_keys
HashSecret's fields are private and the only constructor derives k0/k1
from a u32 seed, so an embedder cannot reproduce a fixed keying (e.g. a
deterministic PYTHONHASHSEED=0-style run). Add a const constructor
taking explicit SipHash keys; the seeded path is unchanged.
Assisted-by: Claude
* common: fix float format grouping and empty presentation type
Thousands grouping split the magnitude only on '.', so exponent and
percent tails received separators (format(1e20, ',e') gave
'1e+,20'-class corruption). Group only the leading integer digits for
interval-3 output; hex/octal/binary keep whole-magnitude grouping.
Empty presentation type: with a precision, clamp it to at least 1 and
restore a trailing '.0' on integer-looking results (Py_DTSF_ADD_DOT_0);
without one, alternate form now inserts the forced decimal point before
an exponent instead of appending it.
Verified byte-identical to CPython 3.14 over a hash/format differential
battery; test_format, test_float, test_fstring, test_hash pass.
Assisted-by: Claude
* common: satisfy rustfmt and clippy in new hash/float tests
- float_ops: use core::f64::consts::PI in the frexp-contract battery
(clippy::approx_constant)
- hash: rewrite the keyed_hash test via BuildHasher::hash_one
(clippy::manual_hash_one)
- rustfmt the new float_ops/hash/format test modules
Assisted-by: Claude
* common: add DTSF to format.rs spell-checker ignores
Assisted-by: Claude
* Extract unicodedata core into rustpython-unicode crate
Move the vm-free Unicode character database access — the generated UCD 3.2.0 /
latest tables, their build-time generation, and the icu4x/unicode_names2 lookups
for category/bidirectional/combining/east_asian_width/mirrored/decomposition/
normalize/is_normalized/digit/decimal/numeric/name/lookup — into a new leaf crate
over char/CodePoint/&Wtf8. stdlib/unicodedata.rs keeps the UCD pyclass binding: it
extracts the code point, boxes results, and maps errors. The unicode data files,
build.rs table generation, and the icu_properties/icu_normalizer/unicode_names2
dependencies move out of stdlib.
Assisted-by: Claude
* Route str predicates, casefold, and printable through rustpython-unicode
Add classify/case/identifier modules to the shared crate over char/&str/&Wtf8:
isalpha/isalnum/isdecimal/isdigit/isnumeric/isspace/isprintable classification,
XID identifier predicates, and full-mapping casefold. vm/builtins/str.rs and
literal/char.rs now call these instead of icu_properties/icu_casemap directly.
String-level iteration with final-sigma handling (lower/upper/title/capitalize,
islower/isupper) stays in the runtime.
Assisted-by: Claude
* Route sre_engine char classes through rustpython-unicode
Add the regex module with the SRE character-class and case predicates
(is_word/is_space/is_digit, is_uni_* Unicode variants, ascii/locale/unicode case
folding). sre_engine/string.rs now forwards to it, keeping its public API and
behavior identical — including the ASCII-only is_uni_digit and the hardcoded
is_uni_space table.
Assisted-by: Claude
* Route \N{} name lookups through rustpython-unicode
codegen/string_parser.rs (\N{...} escapes) and common/encodings.rs (the
namereplace error handler) now resolve character names via rustpython-unicode
instead of depending on unicode_names2 directly, leaving the crate as the sole
owner of the name database.
Assisted-by: Claude
* Add differential Unicode sweep and shared-crate snippet test
tests/differential.rs sweeps the full 0..0x110000 range and compares each str
classification predicate against a committed CPython 3.14 reference dataset
(tests/data/cpython3.14_predicates.txt, produced by generate_reference.py). Code
points that differ only because the Rust std / icu4x build ships a later Unicode
release than CPython 3.14's 16.0.0 are recorded in
tests/data/version_skew_cpython3.14.txt, regenerable via
RUSTPYTHON_UNICODE_REGEN_SKEW=1; the regen refuses to record any
cpython=true/crate=false divergence, so only newly-assigned code points are
allowed. Any other divergence fails. Both data files use a run-length
`predicate start:end,...` encoding.
extra_tests/snippets/stdlib_unicode_shared.py exercises the routed surface end to
end (str predicates, casefold, identifiers, unicodedata, normalize, \N{}, and re
character classes) and passes identically on CPython 3.14 and RustPython.
Assisted-by: Claude
* Match regex \d to Unicode decimal digits
is_uni_digit previously matched only ASCII 0-9, so re's \d in Unicode mode
missed decimal digits like ٥ and ५. SRE_UNI_IS_DIGIT matches Py_UNICODE_ISDECIMAL
(category Nd), so route it through classify::is_decimal. Unmasks test_bug_6561 in
test_re and extends the shared-crate snippet with the Nd/Nl/No cases.
Assisted-by: Claude
* Replace shallow Unicode wrappers with re-exports
Address review feedback on the extraction.
Functions that only forwarded to the shared crate become `use ... as`
re-exports rather than hand-written wrappers:
- literal: is_printable is dropped; escape.rs calls
rustpython_unicode::classify::is_repr_printable directly.
- unicode::identifier::is_continue re-exports is_xid_continue.
- unicode::data::lookup_character re-exports unicode_names2::character.
- Drop the unused repr(u8) on DecompositionType.
The SRE character-class and case predicates move back into
sre_engine::string (their pre-extraction home) instead of living in a
unicode::regex module that only sre_engine and the vm _sre binding used;
is_uni_digit/is_uni_alnum still delegate to rustpython_unicode::classify.
engine.rs and _sre consume them from sre_engine::string again.
The unicode crate's tests move from a single root tests module in lib.rs
into per-module test submodules (case, classify, data, identifier,
normalize).
Assisted-by: Claude
* no-std
* Address review feedback: test module, dead variant, WTF-8 normalization
- differential.rs: move the sweep tests into a `mod tests` block and drop
the file-level allow of clippy::tests_outside_test_module and
std_instead_of_alloc; the test uses alloc collections instead.
- data.rs: remove the never-constructed DecompositionType::Canonical variant
and its allow(unused); compatibility decomposition never produces it and
canonical decomposition is handled through icu4x.
- normalize.rs: is_normalized now takes &Wtf8 and checks each UTF-8 run,
skipping lone surrogates, matching normalize's run-wise behavior.
- unicodedata.rs: pass as_wtf8() to is_normalized.
Assisted-by: Claude
* Re-export mmap libc constants through host_env
Move direct libc MADV_*, MAP_*, PROT_*, EOVERFLOW references in
stdlib::mmap to host_env::mmap re-exports. Replace libc::c_int with
core::ffi::c_int.
* host_env/mmap: gate EOVERFLOW re-export on cfg(windows)
Its only consumer is the cfg(windows) named-mapping overflow check in
stdlib::mmap; the re-export was gated cfg(unix), so it was configured
out on Windows and the reference failed to resolve.
Assisted-by: Claude
* Move tuple/frozenset hash algorithms into common::hash
Move the pure tuplehash fold (xxHash-based) into `hash::hash_tuple`, taking
a fallible iterator of element hashes, and the frozenset XOR-fold into a
`hash::FrozenSetHash` streaming accumulator. `builtins::tuple` and
`builtins::set` keep only the element-hash loop and delegate the arithmetic.
Assisted-by: Claude
* common/hash: rename acc_pyhash to acc_py_hash
Drop the now-unused `// cspell:ignore pyhash` from tuple.rs left behind
by the tuple hash move; the only remaining occurrence was the
acc_pyhash local. Renaming it lets cspell pass without an ignore.
Assisted-by: Claude
* Prevent csv QUOTE_NONE writer panics without quotechar
Handle csv.writer rows with QUOTE_NONE through a RustPython-owned unquoted writer path so quotechar=None no longer reaches the unfinished csv_core configuration branch. Escape delimiter, newline, quotechar, and escapechar bytes according to the active dialect, and preserve CPython's single-empty-field error behavior.
Constraint: Match CPython csv.writer behavior without changing Lib/ copied stdlib files.
Rejected: Relying on csv_core QuoteStyle::Never | it does not model CPython QUOTE_NONE escaping with quotechar=None.
Confidence: high
Scope-risk: narrow
Directive: Keep QUOTE_NONE writer behavior separate unless csv_core gains matching CPython semantics.
Tested: prek run --all-files; cargo test --workspace --exclude rustpython_wasm --exclude rustpython-venvlauncher; cargo build --release --features sqlite; pytest -v in extra_tests
Assisted-by: Codex:gpt-5.5
* Keep csv parity tests aligned with QUOTE_NONE support
Remove the stale RustPython expected-failure marker for the CPython escaped-field writer test that now passes, and cover CR/LF escaping in the snippet regression requested during review.
Constraint: RustPython test policy allows removing expectedFailure markers only when the upstream test passes.
Rejected: Refactoring writer row boilerplate in this follow-up | review marked it as a heavy-lift nitpick and it would broaden the CI fix.
Confidence: high
Scope-risk: narrow
Tested: cargo run --release --features sqlite -- -m test test_csv; cargo run -- extra_tests/snippets/stdlib_csv.py; PATH=/tmp/pyshim:$PATH prek run --all-files; git diff --check; cargo test --workspace --exclude rustpython_wasm --exclude rustpython-venvlauncher; pytest -v in extra_tests
Assisted-by: Codex:gpt-5.5
* Keep csv quote-style dispatch coherent
Review feedback pointed out that adjacent branches selected behavior from the same quoting discriminator. Use one match so future quote-style additions are routed in a single place.
Constraint: RustPython review requested merging adjacent quote-style checks.
Confidence: high
Scope-risk: narrow
Tested: cargo fmt --check
Tested: target/release/rustpython extra_tests/snippets/stdlib_csv.py
Tested: target/release/rustpython -m test test_csv
Tested: PATH=/tmp/pyshim:$PATH prek run --all-files
Tested: RUST_TEST_THREADS=1 cargo test --workspace --exclude rustpython_wasm --exclude rustpython-venvlauncher -- --test-threads=1
Tested: cargo clippy --workspace --exclude rustpython_wasm --exclude rustpython-venvlauncher --all-targets
Assisted-by: Codex:gpt-5.5
Python supports `mkdir` on Windows. I deferred to calling Rust's
implementation for now. However, like `rename`, Python's implementation
supports additional features that are currently unsupported on
RustPython. I added a note for future reference.
Redox supports `mkdirat` which significantly cleans up the
implementation.
Enable `abi3t` support in PyO3
Add left over c-api functions
Fix PyLong_AsUnsignedLongLongMask
Update pyo3 to `0.29`
Add `PyInt::as_u64_mask`
Fix `PyInt::as_u32_mask`
Jeong, YunWon