mirror of
https://github.com/RustPython/RustPython.git
synced 2026-10-29 03:04:38 +08:00
stack/python-3.15-8950
23
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 1ndahous3
|
5b578797e7 |
Link libffi only when the vm's host_env feature is on (#8912)
Assisted-by: Claude Code:claude-opus-5-5 |
||
|
Jeong, YunWonandLee Dogeon
|
afad92bf1b |
Add OrderedDict, marshal, inspect, and chflags bitflagset (#8780)
* Implement _collections.OrderedDict Extracted from #6799: native OrderedDict as a dict subclass, iterator mutation tracking, and mapping-view set operations. * Add OrderedDict, warnings source lines, and socket sendmsg Implement _collections.OrderedDict as a dict subclass with a linked list of insertion-order nodes. Pack struct F/D as native float/double complex pairs, and report unexpected __replace__ field names as a list repr. Register encodings.latin_1 at startup. Reject NUL in input() prompts. Copy non-exact type dicts in mapping order. Fetch warning source lines through _bless_my_loader and unicode splitlines. Keep the last warnings.filters list after deletion. Snapshot sendmsg sequences before converting buffers, parse ancillary ints with __index__, and dispatch AF_CAN by protocol. Store sys.trace and sys.profile on each ThreadSlot so _settraceallthreads/_setprofileallthreads are visible to sys.gettrace/getprofile on other threads. Use <frozen importlib._bootstrap> as frozen importlib co_filename. Keep HEAPTYPE socket objects GC-tracked. Assisted-by: Grok:grok-4.6 * Load marshal via readinto and dump sets in dumps order marshal.load fills a scratch buffer with readinto on a contiguous writable memoryview, and rejects a returned length other than the requested size. set and frozenset dumps write elements in sorted(v, key=marshal.dumps) order. Assisted-by: Grok:grok-4.6 * Add classmethod_descriptor and ABC IS_ABSTRACT flags Give METH_CLASS methods a distinct classmethod_descriptor type. Store Py_TPFLAGS_IS_ABSTRACT from __abstractmethods__ and look up that attribute only in the type's own dict. Expose defaultdict.default_factory as a member descriptor. Inherit bool's unused number slots from int. Mark set and frozenset __contains__ as METH_COEXIST. Parse classmethod, staticmethod, object.__new__, and heap type text signatures. Register interactive source in linecache. Assisted-by: Grok:grok-4.6 * address review: odict reuse, descr call, new wrapper Reuse odict node slots and walk the live list in next_after. Require both | operands to be dicts. Drop the odict iterator on GC clear. Bind classmethod descriptors through descr_get and update IS_ABSTRACT under the type lock. Give odict keys and items sq_contains. Walk tp_new_wrapper past new_wrapper so HEAPTYPE socket subclasses can call socket.__new__. Split FreeBSD Bluetooth protocol constants from Linux values. Assisted-by: Grok:grok-4.6 * Represent chflags bits with bitflagset Assisted-by: Grok:grok-4.6 * Represent access and mode bits with bitflagset Assisted-by: Grok:grok-4.6 * Continue unfinished REPL suites as incomplete input Assisted-by: Grok:grok-4.6 * Add pickle and os.stat inspect signatures Give _pickle.Pickler the clinic constructor text signature and accept fix_imports and buffer_callback as positional-or-keyword. Allow #[pyfunction(text_signature)] and use it so os.stat reports (path, *, dir_fd=None, follow_symlinks=True). Uncomment import _pickle and drop the pickle inspect expectedFailures. Assisted-by: Grok:grok-4.6 * Add Unpickler, dump/load, and lstat signatures Give _pickle Unpickler, dump, dumps, load, and loads clinic text signatures, and os.lstat (path, *, dir_fd=None). Assisted-by: Grok:grok-4.6 * Complete OrderedDict with insertion-order links Move the native OrderedDict into ordered_dict.rs and track order with a linked list of nodes, keeping mapping-view set operations. Assisted-by: Grok:grok-4.6 * address CI: stat casts, HCI_DATA_DIR, passing xfails Compare stat permission bits as Mode so Linux clippy does not flag a u32-to-u32 cast. Export HCI_DATA_DIR with the Bluetooth socket option values. Drop expectedFailure from tests that now pass. Assisted-by: Grok:grok-4.6 * Fix wasm teardown deadlock and bluetooth constants Assisted-by: Grok:grok-4.6 * Clear process-wide eval breaker bits in STW test Assisted-by: Grok:grok-4.6 * Drop exclusive-held panic that tripped wasm clippy Assisted-by: Grok:grok-4.6 * Fix OrderedDict contains and export Windows BDADDR Assisted-by: Grok:grok-4.6 * Add cmp_to_key and hash-once LRU cache _functools.cmp_to_key now returns KeyWrapper. LRU lookups store the key hash on _lru_list_elem so __hash__ is not called again on hits or eviction. Assisted-by: Grok:grok-4.6 * Allow subclass keywords on list, tuple, and float list.__init__ rejects keywords only when tp_new is still list's. tuple/float.__new__ reject them only when tp_init is still the base's. Assisted-by: Grok:grok-4.6 * Fix CI lint, test re-export, and structseq members Drop the duplicate clear_eval_breaker_for_test re-export. Format structseq imports. Expose struct sequence fields as readonly members so assignment raises "readonly attribute". Assisted-by: Grok:grok-4.6 * Allow subclass keywords on itertools and filter Positional-only constructors ignore extra keywords when a subclass has replaced tp_init, matching list/tuple/float. Assisted-by: Grok:grok-4.6 * Remove text_signature derive parameter Drop the string override on #[pyfunction] and #[pyclass]. Signatures are generated from the Rust parameter list only. Restore inspect expectedFailures that depended on the override. Assisted-by: Grok:grok-4.6 * Rename set contains_method to __contains__ Keep a borrowed contains helper for slots and C-API. The coexist pymethod is named __contains__. Assisted-by: Grok:grok-4.6 * Use MonitoringEvent instead of EVENT_* aliases Drop the unused MonitoringEvents bitflags and the EVENT_PY_START style mask constants. Call sites use the enum and .mask(). Assisted-by: Grok:grok-4.6 * Store warnings filters as PyListRef The list already serializes its elements. Drop the extra mutex around the cached list pointer. Assisted-by: Grok:grok-4.6 * Re-export clear_eval_breaker_for_test after rebase Keep the STW test helper visible without bringing back stop_bit_set. Assisted-by: Grok:grok-4.6 * Fix groupby reentrancy and islice arguments Give _grouper its own tgtkey and take currvalue after the key comparison. Accept __index__ on islice bounds, reject step 0, and drain the source up to stop. Assisted-by: Grok:grok-4.6 * SETREF warnings filters, onceregistry, and defaultaction Keep the last list, dict, or str after the module attributes are deleted by swapping the interpreter cache pointer. Do not store a None fallback for _warnings_context. Assisted-by: Grok:grok-4.6 * Iterate OrderedDict inner dict via next_entry &PyDict is not an iterator after the borrowed-ref signature change. Assisted-by: Grok:grok-4.6 * Remove expectedFailure from testBluetoothConstants Linux now exports BTPROTO_HCI and the rest of the set. Assisted-by: Grok:grok-4.6 * Borrow get_filter arguments as PyObject Match the warn_explicit call site that passes borrowed handles after the internal-helper borrow change. Assisted-by: Grok:grok-4.6 * Fix bool slots and SemHandle safety docs Leave unused bool number slots unset so inspect reports __add__ on int. Document SemHandle::from_raw safety. Assisted-by: Grok:grok-4.6 * Keep sendmsg errors and drop duplicate linecache Replace sendmsg iterable TypeError only. Register REPL source in linecache once per compiled input. Assisted-by: Grok:grok-4.6 * Nest REPL parse or-patterns for clippy Assisted-by: Grok:grok-4.6 * Mark pydoc os.stat signature test expectedFailure Assisted-by: Grok:grok-4.6 --------- Co-authored-by: Lee Dogeon <dev.moreal@gmail.com> |
||
|
Joshua Megnauth
|
86c7fa20c4 |
Fix missing workspace lints & fix lints (#8781)
Some of the workspace packages were missing the `[lints]` table. I enabled the lints then fixed the resulting errors. |
||
|
Jeong, YunWon
|
7b018871a3 |
host_env: export remaining stdlib constants (#8760)
* host_env: export winsound, spawn, and remaining SSL_OP flags Assisted-by: Claude * stdlib: take winsound, nt spawn, and SSL_OP names from host_env Assisted-by: Claude * host_env: export ssl alerts, sysexits, socket extras Assisted-by: Claude * stdlib: take remaining host_env and common constants Assisted-by: Claude * Remove unused workspace exitcode dependency Assisted-by: Claude * host_env: export PROTO, access, reparse, wait flags Assisted-by: Claude * stdlib: take PROTO, access, reparse, wait from host_env Assisted-by: Claude |
||
|
Jeong, YunWon
|
a7c0500a42 |
host_env: export remaining Windows socket and fault constants (#8736)
* host_env: export remaining Windows socket and fault constants Publish AF_HYPERV/AF_BLUETOOTH, HV_* GUIDs, SIO_TCP_SET_ACK_FREQUENCY, EXCEPTION_CONTINUE_SEARCH, and winreg ERROR_FILE_NOT_FOUND/INVALID_DATA. os.socket pack/unpack Hyper-V and RFCOMM addresses through host_env. Assisted-by: Claude * socket: publish HV_GUID_* as string attributes Assisted-by: Claude * host_env: cast SIO_TCP_SET_ACK_FREQUENCY to i32 The WinSock constant is u32 and does not fit i32 without a wrapping cast. Socket module attributes are i32. Assisted-by: Claude * socket: use socklen_t width for Windows SockAddr::new Windows socklen_t is i32; a u32 cast does not match socket2. Assisted-by: Claude * socket: drop unused Windows AF_* imports from c AF_BTH, AF_IRDA, and AF_SNA are published via host_socket pyattr and are not read through the Windows c alias module. Assisted-by: Claude * host_env: export Windows faulthandler exception codes Route stdlib _EXCEPTION_* pyattrs and exception_description through named host_env constants instead of hex literals. Assisted-by: Claude |
||
|
Jeong, YunWon
|
c382a607bb |
host_env: keep rustls off the rustls-free ssl surface (#8727)
* host_env: keep rustls off the rustls-free ssl surface ssl is MemoryBIO, constants, OID, and ALPN. ssl-rustls is the rustls engine. Drop the wasm32 rustls-pki-types web dep; the browser crate already enables that feature. Assisted-by: Claude * host_env: drop rustls-pki-types from the lockfile Assisted-by: Claude * host_env: name the rustls engine feature rustls ssl is the rustls-free surface. rustls is the engine. stdlib's ssl-rustls product feature still selects this. Assisted-by: Claude |
||
|
Jeong, YunWon
|
3191ecaf38 |
host_env: share thread, locale, loader and BSTR (#8702)
* host_env: share thread, locale, loader and BSTR
Add Windows thread name, locale-info/wcscoll, CompareStringOrdinal,
LoadLibraryEx/GetProcAddress/FreeLibrary, BSTR and COM helpers, and
AF_HYPERV UUID conversion. Wire _thread, locale, ctypes LoadLibrary
and WMI BSTRs through those wrappers. Ensure =X: before spawnve/execve.
Assisted-by: Claude
* host_env: expose rustls-free ssl core on wasm
Compile MemoryBIO, protocol constants, OID tables, hostname
checks, and ALPN parsing without the rustls feature so wasm
and other hosts can implement _ssl on the shared types.
Assisted-by: Claude
* stdlib: add wasm _ssl on rustls-free host_env
Install _ssl on wasm32 against rustpython_host_env::ssl.
MemoryBIO, constants, OID, hostname, and ALPN work. Context
settings are stored; wrap/handshake and rustls-only loaders
raise SSLError until a wasm TLS engine is wired.
Assisted-by: Claude
* host_env: address #8702 review comments
Cache-miss Windows ctypes lookup uses GetProcAddress. FreeLibrary
propagates OSError. start_new_thread returns GetThreadId. Export
_thread._get_name and drop pyfunction doc comments. Reject
embedded NULs in Windows locale.strcoll/strxfrm. Terminate
wcscoll/wcsxfrm and SetThreadDescription inputs.
wasm _ssl accepts wrap_bio owner and get_ca_certs(binary_form),
and MemoryBIO.read(-1) returns all pending bytes.
Assisted-by: Claude
* host_env: gate the whole ssl module on feature ssl
Compile rustpython_host_env::ssl only with the ssl feature.
Rustls crates stay native-only, so enabling ssl on wasm does
not pull rustls. Inside the module, rustls files follow
not(wasm32) rather than a second feature flag.
Assisted-by: Claude
* host_env: fix Windows types and thread names
Cast SysAllocStringLen to *mut u16 and RPC status to u32 so
windows-gnu and MSVC agree. Export _NAME_MAXLEN, truncate names
at NUL and the OS byte cap, and decode get_name with fsdecode.
Assisted-by: Claude
* host_env: compile rustls engine on WASI
Keep rustls-free MemoryBIO/constants/OID/ALPN on all wasm
targets. Compile the rustls engine on WASI, where UnixTime::now
exists. Allow the rustpython ssl feature on wasm without
ssl-rustls or ssl-openssl.
Assisted-by: Claude
* compiler: pick the earliest CPython parse override
A later incompatible-prefix scan must not hide an earlier
number or bracket diagnostic. Parser-driven tokenization
stops at the first failure.
Assisted-by: Claude
* host_env: address remaining #8702 review comments
Map a missing Windows export to a symbol-load error, keep
LoadLibrary's Win32 code on the OSError, drop incomplete
UTF-16 pairs when truncating thread names, hide set_name
and _get_name off linux/macOS/Windows, and let ssl-rustls
enable host_env.
Assisted-by: Claude
* stdlib: add wasm _socket and read-all MemoryBIO
Register a rustls-free-style _socket shim on wasm32 so
Lib/ssl.py can import. Treat MemoryBIO.read(-1) as read-all,
matching _ssl_MemoryBIO_read_impl.
Assisted-by: Claude
* host_env: fix clippy, WASI ssl cfg, and wasm rustls
Defer unclosed-bracket overrides until no token diagnostic
exists, keep WASI rustls and wasm ssl modules exclusive,
preserve lone Windows surrogates, compile the rustls engine
on browser wasm, and clear clippy denials.
Assisted-by: Claude
* host_env: fix Windows clippy and wasm UnixTime
Gate thread-name byte truncation to linux/macOS, allow
GetProcAddress/FreeLibrary pointer deref, and enable
rustls-pki-types web time on browser wasm.
Assisted-by: Claude
* host_env: lock rustls-pki-types web for wasm
Assisted-by: Claude
* host_env: silence Windows unused thread id and shear
Assisted-by: Claude
* stdlib: drop Windows locale clippy needless wraps
Windows clippy failed on locale.strcoll/strxfrm needless return
and _getdefaultlocale always wrapping Ok.
Assisted-by: Claude
* compiler: keep lexer errors over print hints
print x; 0x must report an invalid hex literal. Ranking by source
offset let the earlier print hint win. Treat a finished radix
literal as valid so print x; 0x1 still gets the print message.
Assisted-by: Claude
* compiler: keep decode errors over unclosed openers
Ranking by offset made ( '\\N' report an unclosed paren and
wiped the unclosed-bracket flag on f'{, so compile snippets
raised SyntaxError instead of incomplete input. Export wasm
socket._accept for Lib/socket.py.
Assisted-by: Claude
* wasm: drop duplicate _socket; split locale cfgs
_socket lives in rustpython-stdlib's socket_wasm. The wasm crate
only re-exported that file when freeze-stdlib was off. Split
strcoll, strxfrm, and getencoding into cfg'd function defs.
Assisted-by: Claude
* compiler: keep decode blockers over unclosed openers
(print x; '\\N' must keep the print hint. A later malformed
escape still suppresses the EOF opener even when print wins.
Assisted-by: Claude
* compiler: keep earlier f-string over later lexer errors
f'{x'; ' and f'{x'; 0x must not report the trailing quote or
hex literal. Decode and f-string diagnostics now compete with
lexer failures by offset.
Assisted-by: Claude
* compiler: rank decode vs lexer vs print by class
Unterminated quotes no longer hide format-spec newlines or
empty f-string fields. Print is considered only after the
final winner is not a lexer failure, so print x; f'{x'; 0x
keeps the print hint. Line-continuation uses CPython wording.
Assisted-by: Claude
* compiler: keep unclosed paren over line-cont at EOF
test_exceptions wants '(' was never closed at the opener when
a final backslash follows. Skip escaped quotes in the bracket
scan so {\'a\'} still reports the continuation error. Un-xfail
test_join_daemon_thread_in_finalization.
Assisted-by: Claude
* test_threading: xfail daemon join finalization off Linux
The test passes on Linux. macOS and Windows leave an ignored
deallocator exception on stderr.
Assisted-by: Claude
* vm: hang daemon threads at interpreter finalize
Forced-off daemons must stay not-done so join() during
finalization GC raises PythonFinalizationError and
Thread.is_alive() stays true. SystemExit made the handle
look finished on macOS/Windows. Drop the Lib/test xfail.
Assisted-by: Claude
(cherry picked from commit
|
||
|
Jeong, YunWon
|
66a3d9daa7 |
common: implement remaining builtin text codecs (#8687)
* common: implement remaining builtin text codecs Move utf-16/32/7, unicode-escape, raw-unicode-escape and the bytes escape transform into rustpython-common and call them from _codecs instead of _pycodecs. Assisted-by: Claude:grok-4.6 * host_env: share locale grouping, uuid and pipe state Add localeconv_numeric so grouping keeps a CHAR_MAX terminator, UuidCreateSequential for _uuid, and GetNamedPipeHandleStateW. Assisted-by: Claude:grok-4.6 * Fix utf-7 clippy and the utf-16-be snippet length Assisted-by: Claude:grok-4.6 * codecs: treat escape decode as final and accept str unicode_escape_decode and raw_unicode_escape_decode default final to true and accept a str as UTF-8. Fix clippy on wide_order, escape encode, and encode closures. Remove expectedFailure markers that now pass. Assisted-by: Grok:grok-4.6 |
||
|
Jeong, YunWon
|
979576e2e4 |
ssl: move TlsConnection and session store to host_env (#8678)
TlsConnection, RecordCursor, rustls error mapping, hostname/ALPN/version helpers, and the capturing client session store now live in host_env. stdlib _ssl keeps socket/MemoryBIO I/O and Python objects. Remove unused stdlib sha2 and leftover workspace TLS crates. Assisted-by: Grok:grok-4.6 |
||
|
Jeong, YunWon
|
bfd52eda61 |
ssl: move remaining rustls engine into host_env (#8675)
* ssl: move remaining rustls engine into host_env Cert loading, rustls verifiers, ClientConfig/ServerConfig construction, key logging, acceptor helpers, and X509 error mapping now live next to the trust store. stdlib _ssl keeps the Python object and socket adapters. Assisted-by: Claude * Remove unused workspace SSL dependencies Drop oid-registry, rustls-platform-verifier, and x509-cert from workspace.dependencies. Assisted-by: Grok 4.6 |
||
|
Jeong, YunWon
|
3a8a196996 |
ssl: share rustls engine in host_env and fire msg callbacks (#8665)
* ssl: fire msg callbacks and compute tls-unique Observe TLS records to invoke _msg_callback, and derive TLS 1.2 tls-unique from the KeyLog master secret plus rustls PRF. Restore CHANNEL_BINDING_TYPES and drop the remaining test_ssl RUSTPYTHON skips. Assisted-by: Claude * lzma: build the xz-core engine on wasm32 Expose `_lzma` on wasm32 and Android by compiling the xz-core engine on those targets instead of gating it out. Assisted-by: Claude * ssl: move the rustls engine into rustpython-common Extract the VM-independent rustls helpers (MemoryBio, certificate decode, cipher strings, OID table, chain builder, providers, and TLS record observation) so stdlib _ssl wraps a shared engine. Assisted-by: Claude * ssl: keep the rustls engine in host_env TLS helpers share the host trust store and default verify paths, so they belong next to native_certs rather than in rustpython-common. Assisted-by: Claude * stdlib: drop unused jiff after moving cert decode Assisted-by: Claude |
||
|
Jeong, YunWonandIvan Mironov
|
de6053099a |
ssl: take the OID table and the OpenSSL cipher-string parser from #8007 (#8646)
* ssl: read the OID table out of OpenSSL's object database
`txt2obj` and `nid2obj` have to answer with OpenSSL's own numbers, and the
table behind them was 39 entries transcribed by hand, so every object
outside that set was reported unknown.
Read OpenSSL's own files instead: `obj_mac.num` carries each object's NID
and `objects.txt` carries its OID, short name and description, and the
identifier `objects.pl` derives per object -- the `!Cname` override, else
the description, else the short name, each qualified by the enclosing
`!module` with `-` spelled `_` -- joins the two. `OidEntry`, `OidTable`
and the `find_by_*` functions keep their shape, so `ssl.rs` only changes
where an object has no OID to report.
Three details the object database settles that a curated table did not
have to:
- an object written with one name carries it as both, so
`nid2obj(48)` is `('emailAddress', 'emailAddress', ...)`;
- an object need not have an OID at all -- `DES-EDE3` and 139 other
algorithm names have a NID and nothing to encode -- so the fourth
field of the tuple is None for those, as it is on OpenSSL;
- one arc does not encode, since DER packs the first two into a byte,
so the bare roots `iso` and `itu-t` are OID-less too, while two arcs
such as X500's `2.5` are a real object.
Checked against `_ssl.nid2obj` on CPython built with OpenSSL, over every
NID it answers: 1387 of its 1390 present, with no disagreement on any of
them, up from 39. The three absent are OpenSSL's unnamed placeholder
slots, which it reports as ('NULL', 'NULL', None).
The data files come from openssl/openssl `crypto/objects/`, via the
rustls integration rewrite in #8007.
Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
Assisted-by: Claude
* ssl: parse OpenSSL cipher strings in set_ciphers
Replaces the pattern matcher `set_ciphers` used with the grammar
`man openssl-ciphers` documents, in a new `ssl/cipher.rs` taken from
#8007: `!`, `-`, `+`, `@STRENGTH`, `@SECLEVEL=n`, `+`-joined
conjunctions, the DEFAULT/ALL/COMPLEMENTOF* groups, protocol versions,
SUITEB*, and the aXXX/kXXX/eXXX prefixes.
`set_ciphers` raises "No cipher can be selected." for a string that
names nothing the provider offers, which is the one failure
SSL_CTX_set_cipher_list reports, and keeps the TLS 1.3 suites selected
in provider order because that setter does not reach them. A SUITEB*
string pins its own key exchange groups, which `prepare_kx_groups`
now reads.
Against CPython's OpenSSL over 78 cipher strings, the selections agree
except under `@SECLEVEL=n`, which OpenSSL applies at handshake time and
this applies to the selection.
`CryptoExt` gains `default_cipher_suites` so that `DEFAULT` and `ALL`
are distinguishable.
Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
Assisted-by: Claude
* ssl: finish cipher reporting and provider handling
Use the provider-ordered cipher metadata table as the single source for parsing, reporting, OpenSSL names and descriptions. This makes get_ciphers reflect the context selection and validates curves and Suite B against the active provider.
Keep set_ciphers from changing TLS 1.3 suites, preserve the provider's order, fix empty conjunctions, and keep FIPS initialization within the provider's filtered cipher and key-exchange defaults.
Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
Assisted-by: Claude
Assisted-by: CodeRabbit
Assisted-by: OpenAI Codex: GPT-5
* ssl: preserve OpenSSL OID lookup rules
Keep object-name lookup case-sensitive and canonicalize numeric OID arcs before consulting the generated OpenSSL object table.
Assisted-by: OpenAI Codex: GPT-5
* ssl: preserve cleartext after MemoryBIO shutdown
Read at most one TLS record while waiting for close_notify in BIO mode, leaving any post-TLS cleartext queued for the BIO owner.
Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
Assisted-by: OpenAI Codex: GPT-5
* ssl: reject empty cipher conjunction operands
Keep empty operands visible to the parser so malformed OpenSSL cipher strings such as AES+ and ALL++AES are rejected instead of normalized.
Assisted-by: CodeRabbit
Assisted-by: OpenAI Codex: GPT-5
* ssl: make Suite B cipher ids unambiguous
Spell out the u16 conversion so Windows-only iterator implementations cannot make collection type inference ambiguous.
Assisted-by: OpenAI Codex: GPT-5
* ssl: move native certificate loading behind host_env
Keep platform trust-store loading and filesystem predicates on the host boundary while exposing backend-neutral certificate bytes to the rustls integration.
Assisted-by: OpenAI Codex: GPT-5
* ssl: invalidate server config after cipher changes
Rebuild cached server configuration after set_ciphers updates the selected suites or Suite B key-exchange groups.
Assisted-by: CodeRabbit
Assisted-by: OpenAI Codex: GPT-5
* ssl: load default certificate sources independently
Honor SSL_CERT_FILE and SSL_CERT_DIR independently. Keep directory certificates as lazy capath candidates, and fall back to native roots only when no environment certificate file was loaded.
Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
Assisted-by: OpenAI Codex: GPT-5
---------
Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
|
||
|
Joshua Megnauth
|
bae642c196 |
host_env: Fix wcslen, leverage NonNull<T> more (#8386)
`strlen` was both unused and unneeded. Rust's `CStr` handles the length for us without needing to call into `libc` manually or implementing our own version of `strlen`. Our `strlen` implemented a fallback that incremented and dereferenced a pointer to check for NUL. This is slower than what `libc` usually does. For example, `musl` operates on words instead of individual bytes. This implementation also caused UB if the caller passed in a null pointer. `wcslen` is similar. The currently implementation can cause UB for null pointers. It's also slow from the same reason mentioned above. Luckily, the widestring crate can handle this for us. The crate panics on null pointers, so using it was a good excuse to use NonNull in more places. |
||
|
Joshua Megnauth
|
71f2ffc289 |
Use memchr consistently for NUL checks (#8372)
`memchr` is used throughout RustPython for searching through bytes expediently. However, for NUL checks, it's scantily used. Instead, our NUL checks either use `contains` or `memchr`. I switched all of the `contains(b'\0')` I could find to using memchr instead. I marked the failure paths as cold to hint to LLVM that interior NULs are truly exceptional. This should help branch prediction a bit which is nice for string/bytes functions since they are likely called a lot. |
||
|
lms0806
|
9436ba5067 |
time: use C altzone when available (#8357)
* time: use C altzone when available * Modified to also check TARGET_CC/CC_target. |
||
|
Jeong, YunWon
|
fafd9c75df |
more hostenv isolation (#7932)
* Move which crate dependency from vm to host_env
Add host_env::fs::which wrapper and update getpath.rs call site.
* Move readline module from vm to host_env
Move crates/vm/src/readline.rs to crates/host_env/src/readline.rs and
re-export it from rustpython_vm::readline. Move rustyline dependency
from vm to host_env. Drop the host_env feature gates on history I/O
inside the moved module.
* Move gethostname and mac_address dependencies from stdlib to host_env
Add host_env::socket::hostname and host_env::socket::mac_address
wrappers. Update _socket.gethostname and _uuid.get_node_id call sites.
* Re-export libc constants and types through host_env
Move direct libc constant and type references in stdlib (faulthandler,
fcntl, multiprocessing, posixshmem, select, syslog) to use host_env
re-exports. Replace libc::c_int and libc::c_uint in stdlib with
core::ffi equivalents.
* Re-export mmap libc constants through host_env
Move direct libc MADV_*, MAP_*, PROT_*, EOVERFLOW references in
stdlib::mmap to host_env::mmap re-exports. Replace libc::c_int with
core::ffi::c_int.
* Re-export locale and termios libc constants through host_env
Move direct libc LC_*, ABDAY_*, ABMON_*, TIOC*, FIO* and related
constants in stdlib::locale and stdlib::termios to host_env
re-exports. Replace libc::c_char in stdlib::locale and
stdlib::openssl::cert with core::ffi::c_char.
* Replace libc c-types and FILE wrappers in openssl with host_env
Replace libc::c_int, c_long, c_uchar, c_uint, c_ulong, c_void,
c_char in stdlib::openssl with core::ffi equivalents. Replace
libc::ENOENT with host_env::errno::errors::ENOENT. Add
host_env::fileutils::{CFile, fclose} and route the load_dh_params
fclose call and PEM_read_DHparams FILE pointer type through them.
* Fix Windows build: expose EOVERFLOW, SIGSEGV, SIGFPE on Windows
host_mmap::EOVERFLOW and host_faulthandler::{SIGSEGV, SIGFPE}
re-exports were gated to cfg(unix) but stdlib uses them from Windows
and platform-agnostic code paths.
* Re-export remaining libc types and constants through host_env
Move direct libc references in stdlib (resource, posixsubprocess,
grp, socket) to host_env re-exports. Replace libc::c_long,
libc::c_longlong, libc::c_char in stdlib::socket with core::ffi
equivalents. Add resource::{RLIMIT_*, RUSAGE_*, c_long, rlim_t,
rlimit, timeval}, posix::{c_char, pid_t}, grp::gid_t, and
socket::{AF_*, SOCK_STREAM, sa_family_t, socklen_t, sockaddr_*}
re-exports.
* Move socket2 and dns-lookup dependencies from stdlib to host_env
Re-export socket2 as host_env::socket::raw and dns-lookup as
host_env::socket::dns. Update _socket call sites to route through
host_env aliases.
* Move system-configuration dependency from stdlib to host_env
Re-export the macOS system-configuration crate as
host_env::system_configuration and update _scproxy call sites.
* Restrict CAN/ALG sockaddr structs to target_os linux
libc on Android (bionic) does not define sockaddr_can, so importing it
for target_os = "android" fails to compile. Keep AF_ALG/AF_CAN for both
linux and android, and gate sockaddr_alg/sockaddr_can to linux only.
Assisted-by: Claude
|
||
|
Joshua Megnauth
|
f32e96ce90 |
host_env: os.mkdir for Windows, Redox (#8128)
Python supports `mkdir` on Windows. I deferred to calling Rust's implementation for now. However, like `rename`, Python's implementation supports additional features that are currently unsupported on RustPython. I added a note for future reference. Redox supports `mkdirat` which significantly cleans up the implementation. |
||
|
Jeong, YunWon
|
40b2da6037 |
more hostenv isolation (#7886)
* Convert host_env Windows path/argv params from raw *const u16 to &WideCStr * Migrate remaining winapi raw u16 pointer signatures to typed references * Migrate winreg pub unsafe fn string parameters to typed references * Add ToPyException impls for host_env error types (PyPy wrap_oserror analog) * Add CheckLibcResult helper and apply to socket/fcntl/shm/posix_wasi * Add Win32 BOOL/HANDLE check helpers; apply check helpers across host_env * Apply Win32/libc check helpers to overlapped/testconsole/os.rs * Apply Win32 check helpers to winapi.rs (partial) * Apply Win32 check helpers across more winapi.rs functions * Apply Win32 check helpers to nt.rs (partial) * Add CheckWin32Sentinel helper; apply to nt.rs INVALID_HANDLE_VALUE/INVALID_FILE_ATTRIBUTES patterns * Add OwnedHandle / HandleToOwned helper; apply to mmap create_named_mapping leak path * Use OwnedHandle RAII in nt::pipe to eliminate manual cleanup on error path * Use OwnedHandle in nt::chmod_follow; hoist HandleToOwned import * Drop rustix dependency from vm crate Remove unused IntoPyException impl for rustix::io::Errno and the rustix entry in crates/vm/Cargo.toml. rustix is now only depended on by host_env. * Fix CI failures: cross-platform regressions - winapi.rs: pass None to create_event_w; the recent Option<&WideCStr> migration left one call site still passing a raw null pointer. - exceptions.rs: gate ToPyException for LockfError with cfg(any(unix, target_os = "wasi")), matching host_env::fcntl's own cfg. The previous cfg let it compile on wasm32-unknown-unknown where host_env::fcntl does not exist. - io_unsupported.rs: derive Eq on FileMode alongside PartialEq to satisfy clippy::derive_partial_eq_without_eq. * Fix CI failures: cfg gates and unused imports - exceptions.rs: gate ToPyException for LockfError with cfg(all(unix, not(target_os = "redox"))) to match the type's own cfg in host_env/src/fcntl.rs (LockfError is not built on wasi). - signal.rs: CheckLibcResult is only used in unix-gated functions; split import so it is not pulled in for windows. - mmap.rs: remove CheckWin32Handle from imports; no longer used after switching to HandleToOwned-based RAII. - overlapped.rs: remove INVALID_HANDLE_VALUE from connect_pipe import; the call now uses .check_valid(). * Fix CI failures: rustfmt and windows unused import - signal.rs: reorder cfg-gated imports per rustfmt. - socket.rs: gate ToPyException import to cfg(all(unix, not(target_os = "redox"))); it is only used inside sendmsg which has the same gate, so it was unused on windows. * Push remaining libc/extern callsites from vm into host_env Add host_env wrappers and replace the corresponding vm call sites: - host_env::errno::strerror_string for libc::strerror - host_env::io::write_stderr_raw for libc::write(STDERR_FILENO,...) - host_env::locale::localeconv_data reused from vm::format - host_env::os::abort for the inline abort extern - host_env::os::urandom wraps getrandom; getrandom moves from vm to host_env - host_env::posix::lchmod for the macOS/BSD lchmod extern - host_env::posix::fcopyfile for the macOS fcopyfile extern - host_env::nt::wputenv for the Windows _wputenv extern vm/format.rs's get_locale_info now uses host_env on both unix and windows instead of the unix-only libc::localeconv path. * Move time tz state and winsound FFI into host_env - host_env::time::tz: wraps the libc tzset/timezone/daylight/tzname globals on non-msvc, non-wasm32 targets. vm::stdlib::time now reads these via the typed wrappers instead of declaring its own externs. - host_env::winsound (windows): exposes PlaySoundW (via a typed PlaySoundSource enum), Beep, and MessageBeep. vm::stdlib::winsound drops its inline FFI block and routes through host_env. * Migrate unsetenv to host_env::nt::wputenv; rustfmt - vm::stdlib::os::unsetenv had a second _wputenv call site that still referenced the removed inline extern. Route it through host_env::nt::wputenv like putenv. - rustfmt fixups in exceptions.rs (boolean chain layout) and the two winsound files. * Address PR review comments - host_env::winapi::create_process: assert that the command_line buffer is NUL-terminated and that the env block ends with a double-NUL, matching the Win32 CreateProcessW contract. - stdlib::overlapped CreateEvent: replace WideCString::from_str_truncate with the fallible from_str(), so embedded NULs in the event name surface as ValueError instead of being silently truncated. - vm::exceptions::ReadlinkError::NotSymbolicLink now maps to OSError (matches Win32 ERROR_NOT_A_REPARSE_POINT semantics) rather than ValueError. - winreg::ConnectRegistry: route the non-zero return through the existing os_error_from_windows_code helper so the resulting exception carries the real winerror/message instead of a generic OSError. * Fix CI failures and address review follow-ups CI failures: - rustfmt cleanup in exceptions.rs after the ReadlinkError change. - vm/stdlib/os.rs: drop unused ToWideString import that the wputenv migration left behind. - vm/stdlib/winsound.rs: replace explicit `&*buf` with `&buf` to satisfy clippy::explicit_auto_deref. - Lib/test/test_format.py, Lib/test/test_types.py: drop the now-stale expectedFailureIfWindows decorators on the locale-format tests; the Windows path now reads real `localeconv` data via host_env so these tests pass. Review follow-ups: - host_env::winapi::create_process: switch the new buffer terminator checks from `assert!` to fallible validators returning `io::ErrorKind::InvalidInput`, so bad inputs stay recoverable at the API boundary. - host_env::winsound::play_sound: reject `Memory(_)` together with `SND_ASYNC` (lifetime-unsafe) and `SND_MEMORY` without a `Memory(_)` source. Expand `PlaySoundError` into a variant enum. - vm::stdlib::_winapi::CreateProcess: route the Win32 path/argv strings through `as_wtf8().to_wide_cstring()` like the rest of the Windows API surface; `expect_str()` could panic on Python strings containing lone surrogates. |
||
|
Joshua Megnauth
|
ea9a3d0623 |
Use correct dirs crate; drop uname for rustix (#7906)
RustPython used an ancient, unmaintained version of the `dirs` crate. This pulled in `winapi-rs` on Windows, which is an old crate that has been deprecated in favor of `windows-rs`. `windows-rs` is maintained by Microsoft. Bumping `dirs` to the latest version removes more `winapi-rs` from Cargo.lock. As for `uname`, `rustix` handles it safely so the additional crate isn't needed. |
||
|
Joshua Megnauth
|
2c558fa749 |
rustix/windows-sys for page size; drop maplit (#7876)
The `page_size` crate is a simple libc wrapper for Unix and uses `winapi-rs` for Windows. `windows-sys` and `windows-rs` are the modern alternatives for the unmaintained `winapi-rs`. Both crates are maintained by Microsoft - they're official. Getting the page size is a simple call for both Unix and Windows. Besides Unix and Windows, I also added the page size for wasm32 which the `page_size` crate did not support. `wasm32`'s page size is a constant that is defined by the spec, so I hard coded it without adding additional dependencies. Finally, I dropped `maplit` which is seven years old and only used in one place. Calling `collect()` with a single item iterator is idiomatic as well as better in this case because Rust can optimize it. `maplit` called `HashMap::insert` which over allocates to amortize future allocs. |
||
|
Jeong, YunWon
|
bf588b3c4e | more hostenv impl (#7604) | ||
|
fanninpm
|
8b2bf3d90d |
Move dependency to workspace for host_env crate (#7719)
* Add dependency from `host_env` crate to main workspace * Declare dependency as workspace = true |
||
|
|
f51f27c949 |
Create rustpython-host-env (#7582)
* Create rustpython-host-env crate; move host abstractions out of common
Move os, crt_fd, fileutils, windows, macros modules from
rustpython-common into the new rustpython-host-env crate.
This isolates host OS API wrappers behind a crate boundary
with zero Python runtime dependency.
- Add crates/host_env to workspace
- Drop nix, windows-sys, widestring deps from common
- Wire vm and stdlib to depend on rustpython-host-env
- Migrate all imports from common::{os,crt_fd,fileutils,windows}
to rustpython_host_env::
* refactor: extract host helpers
Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/48d1e64d-37ce-409f-b511-8e61a349665c
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
* lint: enforce direct host API boundaries
Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/97225fb7-7b3d-4197-a77c-eb44aead5b13
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
* refactor: extract remaining host env helpers
Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/d96f57e1-b196-4460-9983-97d5ff118835
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
* fix: clean extracted host env follow-up
Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/d96f57e1-b196-4460-9983-97d5ff118835
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
* fix: document env mutation safety
Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/d96f57e1-b196-4460-9983-97d5ff118835
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
* refactor: split host fs helpers from fileutils
Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/c57424c5-0e1d-490a-82b3-2d2f6c8cf2cd
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
* fix: resolve latest host env ci regressions
Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/899eb717-ebc6-4a4a-870c-2a15c5f33e02
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
* fix: resolve remaining windows clippy host fs calls
Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/12f32740-8173-4b10-a1d6-00b29e90a8ec
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
* host_env
---------
Co-authored-by: Jeong, YunWon <jeong@youknowone.org>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
|