7 Commits
Author SHA1 Message Date
youknowdot fb76135ec8 Align SSL APIs and transport behavior with Python 3.15 (#9167)
Refresh ssl.py and test_ssl.py through update_lib from CPython d220409a13b77de7a6c30abe16ea90db01528edc without local test changes. Implement rustls cipher-suite, group and signature policy, negotiated-group reporting, EOF and session-policy behavior.

Keep OpenSSL and wasm API imports compatible, with genuine version-gated queries and honest unsupported capabilities. Repair OpenSSL callback ownership and traversal, callback exceptions, EOF and shutdown, PHA, session destruction, and system-error decoding exposed by canonical coverage. Preserve existing I/O locking; the inherited optional-backend full-duplex hang remains a separate driver prerequisite.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-17 11:34:23 +09:00
Jeong, YunWon 2fc2479f72 openssl: use host_env for CA paths and file checks (#8680)
Replace openssl-probe and local std::fs checks with host_env
native_certs and fs. Vendored OpenSSL loads system CA paths
from host_env; OpenSSL's own defaults stay for a system lib.

Assisted-by: Grok:grok-4.6
2026-09-09 20:18:00 +09:00
Jeong, YunWon 55c012cffd unicode crate (#8211)
* Extract unicodedata core into rustpython-unicode crate

Move the vm-free Unicode character database access — the generated UCD 3.2.0 /
latest tables, their build-time generation, and the icu4x/unicode_names2 lookups
for category/bidirectional/combining/east_asian_width/mirrored/decomposition/
normalize/is_normalized/digit/decimal/numeric/name/lookup — into a new leaf crate
over char/CodePoint/&Wtf8. stdlib/unicodedata.rs keeps the UCD pyclass binding: it
extracts the code point, boxes results, and maps errors. The unicode data files,
build.rs table generation, and the icu_properties/icu_normalizer/unicode_names2
dependencies move out of stdlib.

Assisted-by: Claude

* Route str predicates, casefold, and printable through rustpython-unicode

Add classify/case/identifier modules to the shared crate over char/&str/&Wtf8:
isalpha/isalnum/isdecimal/isdigit/isnumeric/isspace/isprintable classification,
XID identifier predicates, and full-mapping casefold. vm/builtins/str.rs and
literal/char.rs now call these instead of icu_properties/icu_casemap directly.
String-level iteration with final-sigma handling (lower/upper/title/capitalize,
islower/isupper) stays in the runtime.

Assisted-by: Claude

* Route sre_engine char classes through rustpython-unicode

Add the regex module with the SRE character-class and case predicates
(is_word/is_space/is_digit, is_uni_* Unicode variants, ascii/locale/unicode case
folding). sre_engine/string.rs now forwards to it, keeping its public API and
behavior identical — including the ASCII-only is_uni_digit and the hardcoded
is_uni_space table.

Assisted-by: Claude

* Route \N{} name lookups through rustpython-unicode

codegen/string_parser.rs (\N{...} escapes) and common/encodings.rs (the
namereplace error handler) now resolve character names via rustpython-unicode
instead of depending on unicode_names2 directly, leaving the crate as the sole
owner of the name database.

Assisted-by: Claude

* Add differential Unicode sweep and shared-crate snippet test

tests/differential.rs sweeps the full 0..0x110000 range and compares each str
classification predicate against a committed CPython 3.14 reference dataset
(tests/data/cpython3.14_predicates.txt, produced by generate_reference.py). Code
points that differ only because the Rust std / icu4x build ships a later Unicode
release than CPython 3.14's 16.0.0 are recorded in
tests/data/version_skew_cpython3.14.txt, regenerable via
RUSTPYTHON_UNICODE_REGEN_SKEW=1; the regen refuses to record any
cpython=true/crate=false divergence, so only newly-assigned code points are
allowed. Any other divergence fails. Both data files use a run-length
`predicate start:end,...` encoding.

extra_tests/snippets/stdlib_unicode_shared.py exercises the routed surface end to
end (str predicates, casefold, identifiers, unicodedata, normalize, \N{}, and re
character classes) and passes identically on CPython 3.14 and RustPython.

Assisted-by: Claude

* Match regex \d to Unicode decimal digits

is_uni_digit previously matched only ASCII 0-9, so re's \d in Unicode mode
missed decimal digits like ٥ and ५. SRE_UNI_IS_DIGIT matches Py_UNICODE_ISDECIMAL
(category Nd), so route it through classify::is_decimal. Unmasks test_bug_6561 in
test_re and extends the shared-crate snippet with the Nd/Nl/No cases.

Assisted-by: Claude

* Replace shallow Unicode wrappers with re-exports

Address review feedback on the extraction.

Functions that only forwarded to the shared crate become `use ... as`
re-exports rather than hand-written wrappers:

- literal: is_printable is dropped; escape.rs calls
  rustpython_unicode::classify::is_repr_printable directly.
- unicode::identifier::is_continue re-exports is_xid_continue.
- unicode::data::lookup_character re-exports unicode_names2::character.
- Drop the unused repr(u8) on DecompositionType.

The SRE character-class and case predicates move back into
sre_engine::string (their pre-extraction home) instead of living in a
unicode::regex module that only sre_engine and the vm _sre binding used;
is_uni_digit/is_uni_alnum still delegate to rustpython_unicode::classify.
engine.rs and _sre consume them from sre_engine::string again.

The unicode crate's tests move from a single root tests module in lib.rs
into per-module test submodules (case, classify, data, identifier,
normalize).

Assisted-by: Claude

* no-std

* Address review feedback: test module, dead variant, WTF-8 normalization

- differential.rs: move the sweep tests into a `mod tests` block and drop
  the file-level allow of clippy::tests_outside_test_module and
  std_instead_of_alloc; the test uses alloc collections instead.
- data.rs: remove the never-constructed DecompositionType::Canonical variant
  and its allow(unused); compatibility decomposition never produces it and
  canonical decomposition is handled through icu4x.
- normalize.rs: is_normalized now takes &Wtf8 and checks each UTF-8 run,
  skipping lone surrogates, matching normalize's run-wise behavior.
- unicodedata.rs: pass as_wtf8() to is_normalized.

Assisted-by: Claude
2026-07-07 09:19:25 +09:00
Joshua Megnauth 5c1803e019 Fix most of unicodedata; unmask multiple tests (#7947) 2026-06-14 17:47:42 +09:00
f51f27c949 Create rustpython-host-env (#7582)
* Create rustpython-host-env crate; move host abstractions out of common

Move os, crt_fd, fileutils, windows, macros modules from
rustpython-common into the new rustpython-host-env crate.
This isolates host OS API wrappers behind a crate boundary
with zero Python runtime dependency.

- Add crates/host_env to workspace
- Drop nix, windows-sys, widestring deps from common
- Wire vm and stdlib to depend on rustpython-host-env
- Migrate all imports from common::{os,crt_fd,fileutils,windows}
  to rustpython_host_env::

* refactor: extract host helpers

Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/48d1e64d-37ce-409f-b511-8e61a349665c

Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>

* lint: enforce direct host API boundaries

Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/97225fb7-7b3d-4197-a77c-eb44aead5b13

Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>

* refactor: extract remaining host env helpers

Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/d96f57e1-b196-4460-9983-97d5ff118835

Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>

* fix: clean extracted host env follow-up

Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/d96f57e1-b196-4460-9983-97d5ff118835

Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>

* fix: document env mutation safety

Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/d96f57e1-b196-4460-9983-97d5ff118835

Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>

* refactor: split host fs helpers from fileutils

Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/c57424c5-0e1d-490a-82b3-2d2f6c8cf2cd

Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>

* fix: resolve latest host env ci regressions

Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/899eb717-ebc6-4a4a-870c-2a15c5f33e02

Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>

* fix: resolve remaining windows clippy host fs calls

Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/12f32740-8173-4b10-a1d6-00b29e90a8ec

Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>

* host_env

---------

Co-authored-by: Jeong, YunWon <jeong@youknowone.org>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
2026-04-21 00:13:25 +09:00
Jeong, YunWon a5fc3a82de reason inside #[allow] (#7049) 2026-02-08 13:11:50 +00:00
Shahar Naveh 88eca9693a Move stdlib -> crates/stdlib (#6268) 2025-11-16 23:25:45 +09:00