* Add deterministic profiler and canonical Python 3.15 tools
Use the merged operation critical sections with interpreter-bound profiler
entry checks and short payload borrows. Trace owned code references for GC.
Import the canonical tracing and offline profiling output closure via the
repository updater, retaining existing test markers.
Assisted-by: dot:GPT-6.1
* Clarify standalone profiling license cross-reference
Preserve license terms and copyright notices; point to the reproduced
Apache text below instead of CPython's original OpenSSL section above.
Assisted-by: dot:GPT-6.1
* Defer unsupported sampling package from profiler extraction
Keep the deterministic tracing profiler and its canonical tools together.
Defer the entirely new sampling production, tests, assets and their license
as one closure until native record and binary-collector support is ready.
Retain canonical pstats for deterministic empty-profile handling; its
upstream sampled-file sorting issue is not patched here.
Assisted-by: dot:GPT-6.1
* Stage profiling root as a namespace until sampling is available
Omit the whole newly added canonical package initializer rather than
advertise an absent sampling module or hand-edit canonical __all__.
Explicit profiling.tracing and cProfile imports remain available.
The namespace root intentionally lacks canonical metadata/automatic exports.
This addresses test___all__ failures in the prior Linux, macOS and Windows
CI runs. No test assertions, skips or canonical file contents are changed.
Assisted-by: dot:GPT-6.1
* Match canonical static builtin profiler labels and metadata
Normalize the raw builtin receiver instead of its call-argument accessor.
Snapshot static method module metadata at construction, matching CPython,
so explicit module mutation and deletion remain observable. Avoid a
profiler-time fallback that would obscure deleted metadata.
Correct float.__getformat__ to CPython's classmethod calling convention,
including subclass binding, rather than special-casing its profiler label.
Assisted-by: dot:GPT-6.1
* Support collision-safe frozen package containers
Generate empty frozen packages only for source namespace directories with
real frozen descendants. Preserve genuine modules and packages across
frozen-library registrations while retaining real-real replacement order.
Record the container marker in FrozenModule and its library envelope;
legacy module/package encodings remain unchanged. This is closed-tree
frozen packaging, not full PEP 420 namespace merging.
Add structural codec, mock generator, and registration-priority checks.
* Revert frozen package containers before restoring canonical profiling root
The container implementation changed filesystem namespace precedence.
Restore the existing importer and frozen envelope unchanged; the following
offline sampling closure restores the canonical regular package root.
This reverses d5a8049234.
Assisted-by: dot:GPT-6.1
* Restore canonical profiling package with offline sampling codec
Provide native records, status constants and genuine uncompressed binary
reader/writer APIs for canonical offline tools. Keep the unsupported live
process capability absent and preserve existing canonical test gates.
Restore the complete selected sampling production/test/assets closure via
update_lib, including its asyncio frame-schema dependency. The prior frozen
container extension is reverted separately; canonical packages work with
both filesystem and frozen imports without changing namespace semantics.
Assisted-by: dot:GPT-6.1
* Keep platform-dependent C long normalization explicit
Allow only the c_long-to-i32 conversion lint: Windows uses the same type,
while LP64 requires the existing truncation to match CPython normalization.
No runtime behavior or test assertions change.
Assisted-by: dot:GPT-6.1
* Include profiling notices in library and desktop packages
Bundle a byte-identical copy of the root profiling notice in rustpython-pylib
and include the root notice in configured desktop resources. Avoid a symlink
so Windows checkouts preserve the actual notice text. Terms are unchanged.
Assisted-by: dot:GPT-6.1
* Keep profiling asset license notice in pylib crate
* Serve frozen profiling assets through native resource readers
* Preserve canonical profiling asset line endings on checkout
---------
Co-authored-by: youknowdot <dot@youknowone.org>
* Report errors from #[pyclass] trait items
The trait branch of impl_pyclass_impl built its own ImplContext, so the
errors collected while extracting trait items were never emitted: an
invalid #[pymethod] or #[pygetset] in a #[pyclass] trait was left out of
the class without a compile error. The trait branch now uses the shared
context whose errors are emitted.
Assisted-by: Claude Code:claude-opus-5-5
* Pass the object to #[pymethod] and #[pygetset] receivers outside vm
Outside crates/vm, where impl Py<T> cannot be written, #[pymethod] and
#[pygetset] receivers that took &self become zelf: &Py<Self> inside the
existing impl T blocks. Where Rust code calls one of these methods on a
payload value, the logic stays in a payload method and the Python method
forwards to it.
Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Reject payload receivers on #[pymethod] and #[pygetset]
A #[pymethod] or #[pygetset] item with a self receiver is now a compile
error when self is the payload: in an impl for T rather than Py<T> or
PyRef<T>, and in a #[pyclass] trait. The error points to taking
zelf: &Py<Self> or zelf: PyRef<Self>, or defining the item in impl Py<T>.
Assisted-by: Claude Code:claude-opus-5-5
* Pass the object to the layout test getter
The subclass layout test's extra getter is a #[pygetset] on the
payload. Take zelf: &Py<Self> so the receiver is the object.
Assisted-by: Grok:4.7
* Set builtin __doc__, __module__, and datetime signatures
Expose empty doc-db member entries as ''. Heap __dict__ and __weakref__
getsets carry their descriptor docs. Member repr uses the qualified
type name.
__module__ is collections for the deque iterators, _csv for Dialect,
ctypes for ArgumentError and CField, _lzma for LZMAError, operator for
the getter types, pickle for PickleBuffer, _ssl for MemoryBIO,
SSLSession, _SSLContext and _SSLSocket, weakref for the proxy types,
and posix (nt on Windows) for times_result and uname_result.
unicodedata's copied UCD functions set __module__ to unicodedata.
_testinternalcapi function docs and weakproxy.__str__ use the stored
text. datetime replace defaults render as unchanged, __replace__ is
($self, /, **changes) and binds through replace, now defaults tz to
None, and fromisoformat and tzinfo methods name the parameter object.
Assisted-by: Grok:grok-4.7
* Move doctext helpers above the test module
db_slice and docs sat after mod tests.
Remove expectedFailure from test_html_doc, test_text_doc,
test_help_output_redirect, test_method_aliases, and test_long_signatures.
Assisted-by: Grok:grok-4.7
Use `//` instead of `///` on Python-exposed items so their comments are not
published as `__doc__`.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Skip Python-defined classes when collecting builtin type docs
generate.py recorded every object.__subclasses__() entry as builtins.<name>
after importing the whole stdlib, so classes written in Python (tkinter,
unittest, tarfile, ...) ended up in the doc database. Classes that are
mutable heap types without C method descriptors and whose module is not a
C extension are now skipped, and their entries are removed from
data.inc.rs.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Store database docstrings xz-compressed behind an optional doc feature
- rustpython-doc's build script compresses all database docstrings into
one xz blob with an offset index; identical texts share one offset.
- Macros emit blob references for database docs; vm decompresses the
blob on first access.
- New `doc` cargo feature (default on) in rustpython, rustpython-vm and
rustpython-stdlib. Without it the blob and xz are not linked and
database-backed __doc__ is None; __text_signature__ is unaffected.
- Native types without a docstring get __doc__ = None in their dict.
- Workflows that build with --no-default-features enable `doc` where
tests or releases need docstrings.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Read PyTuple payloads through as_slice()
- Add Py<PyTuple<R>>::as_slice(); it is generic over R, so typed tuples use
the same view.
- Callers reach slice methods, indexing and slice arguments through
as_slice().
- PyTuple's own sequence and mapping slots and marshal's set_marshal_item reach
the remaining PyTuple methods through the payload field.
- The Py<T>::payload field is pub(crate).
Assisted-by: Claude Code:claude-opus-5-5
* Read PyBytes payloads through as_bytes()
- Callers read bytes through Py<PyBytes>::as_bytes().
- bytes.rs reaches PyBytesInner and the remaining PyBytes methods through
the payload field; fspath and os call payload.contains_nuls().
- DictKey for [u8] falls back to the PyObject key_eq; it previously resolved
to its own key_eq through Deref.
Assisted-by: Claude Code:claude-opus-5-5
* Copy capi tuple arguments with as_slice().to_vec()
Assisted-by: Claude Code:claude-opus-5-5
* Read sqlite3 Row tuples through as_slice()
Assisted-by: Claude Code:claude-opus-5-5
* Read platform-specific tuples and bytes through as_slice() and as_bytes()
Assisted-by: Claude Code:claude-opus-5-5
* Read PyInt and PyFloat payloads through as_bigint() and to_f64()
Add `Py<PyInt>::as_bigint()`, `Py<PyInt>::try_to_primitive()` and
`Py<PyFloat>::to_f64()`, which forward to the payload methods of the same
name. Reads of the `value` field and primitive conversions go through
these views; other PyInt and PyFloat methods are called through the
`payload` field inside rustpython-vm and through payload() outside it.
Assisted-by: Claude Code:claude-opus-5-5
* Read PyComplex payloads through as_complex()
Add Py<PyComplex>::as_complex() and use it at the call sites that
read the payload directly.
Assisted-by: Claude Code:claude-opus-5-5
* Move PyInt conversion helpers to Py<PyInt>
try_to_i64_fast(), to_str_radix_10(), as_u32_mask(), as_u64_mask(),
try_to_primitive() and try_to_primitive_raw() only read the integer
value, so they are defined on Py<PyInt> and callers no longer go through
the payload. PyRange computes its protocol length from the BigInt
directly instead of building a PyInt payload.
Assisted-by: Claude Code:claude-opus-5-5
* Share one conversion between try_to_primitive() and its OverflowError variant
Py<PyInt>::try_to_primitive() and try_to_primitive_in_range(), renamed
from try_to_primitive_raw(), both call to_primitive(). They differ only
in how a negative value for an unsigned type is reported: ValueError for
try_to_primitive(), OverflowError for try_to_primitive_in_range(). The
target type is bound with `for<'a> TryFrom<&'a BigInt>`, so the methods
no longer tie a lifetime to &self.
Assisted-by: Claude Code:claude-opus-5-5
* Move PyInt bitwise operators to Py<PyInt>
__xor__, __or__, __and__ and their int_op() helper are defined on
Py<PyInt>. PyBool calls them without going through the payload.
Assisted-by: Claude Code:claude-opus-5-5
* Move PyBytes operators to Py<PyBytes>
__add__, __contains__, __getitem__ and __mod__ are defined on
Py<PyBytes>. The subscript, contains and remainder slots call them
without going through the payload.
Assisted-by: Claude Code:claude-opus-5-5
* Move PyInt general_op() and modpow() to Py<PyInt>
The power slot calls them on Py<PyInt> without going through the
payload.
Assisted-by: Claude Code:claude-opus-5-5
* Resolve class attribute docs at macro time and drop rustpython-doc from vm
- #[pyclass] and #[pystruct_sequence] emit a sorted PyClassDef::ATTR_DOCS
table from the doc database; attribute and slot docs look up that table.
- rustpython-vm no longer depends on rustpython-doc.
- _operator.attrgetter, itemgetter, methodcaller and object get
constructor Args with FromArgs::PARAMS, replacing the hardcoded class
signature docs in class.rs.
- methodcaller() with no arguments raises "methodcaller needs at least one
argument, the method name".
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Drop default values that equal the field type's default
Bare `default` on integer and bool fields already renders the zero value
in the text signature, so `default = 0` and `default = false` are
replaced with bare `default`.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Return the empty str singleton from Context::new_str
Assisted-by: Claude:claude-opus-5-5
* Render CPython defaults in generated text signatures
Arguments whose default rendered as `<unrepresentable>`, while the
same callable has a representable default in CPython, now render that
default:
- Where the Rust value can be a literal, the argument becomes a plain
field with `default = <literal>`.
- The remaining cases keep their Rust default and add `py_default`
with a comment. These are wrappers such as ArgIntoFloat or
ArgIntoBool, values that need the VM, enums, and Option where None
means another value.
Byte string literals render as Python bytes. A function registered
under several names gets an internal doc per name, so its __doc__ no
longer carries the signature header.
Removed expectedFailure from test_pydoc tests that now pass.
Assisted-by: Grok:grok-4.7
Assisted-by: Claude:claude-opus-5-5
* Match native callable signatures to CPython names and kinds
Rename parameters and adjust positional-only / keyword-only kinds and
defaults of native functions and methods so their generated text
signatures match CPython. Remove expectedFailure markers from tests
that now pass.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Fix Windows console read size and keep type.__prepare__ permissive
Assisted-by: Claude:claude-opus-5-5
* Derive signature defaults from typed Rust values
Param.default is now a DefaultRepr rendered by the const writer.
Literals, integer and bool constants become typed defaults, and other
default expressions use the type's const fn py_default. A bare
`optional` renders as None, so `default = None` rewrites are reverted.
Address review comments: zlib max_length defaults, get_ca_certs
binary_form keyword, getpeercert der, WASM ssl certificate keywords,
fstatvfs fd-only, and _sre endpos shown as sys.maxsize.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Generate class constructor text signatures at compile time
A #[pyclass] impl composes the type's internal doc from the Args of
its Initializer or Constructor, so type.__text_signature__ and
inspect.signature work for native classes. __doc__ stays the plain
doc. Struct sequences get (iterable=(), /) and __reduce__ (self, /).
Hand-parsed constructors are converted to FromArgs structs, and
_weakref.proxy is split from ProxyType.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Remove expectedFailure from test_hmac tests that now pass
Assisted-by: Claude:claude-opus-5-5
* Match remaining native signatures and count missing CPython signatures as matches
Give lock.__enter__ and RLock.__enter__ their own no-argument methods,
add FromArgs for start_new_thread and start_joinable_thread keeping
their errors, take RTLD_GLOBAL/RTLD_LOCAL from libc, and align names,
kinds and <unrepresentable> defaults where CPython's signature is
invalid (bytes.hex, reduce, warn, link, setlocale, add_done_callback,
...). getaddrinfo accepts keyword arguments. whats_left treats a
callable CPython has no signature for as matching.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Drop unfulfilled dead_code expectation on os.link dir_fd args
Assisted-by: Claude:claude-opus-5-5
* Distinguish OptionalArg and Option in signature defaults
An `optional` field's signature default now comes from its type:
Option<T> renders None, while OptionalArg<T> (a missing argument that
is not any Python value) renders <unrepresentable>. Fields where a
missing argument behaves exactly like None become Option<T>; fields
whose missing state differs keep OptionalArg. Redundant
py_default = "<unrepresentable>" entries are removed.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Remove expectedFailure from test_rlcompleter.test_attr_matches
Assisted-by: Claude:claude-opus-5-5
* Use Option for optional arguments whose missing and None states match
OptionalOption and OptionalArg<Option<T>> fields that treat a missing
argument like None become Option<T> with #[pyarg(optional)], which renders
=None in the text signature. Plain optional parameters of that kind move
into small FromArgs structs. Uses where missing and None differ (reduce
initial, StringIO newline, TextIOWrapper.reconfigure, set_asyncgen_hooks)
keep OptionalOption.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Render ::NAME defaults by name and restore named constant defaults
- default = ::NAME writes NAME into the text signature and stores
Into::into(NAME) as the Rust value.
- A one-field tuple-struct argument pattern such as Fildes(fd) names the
parameter after its binding, so fcntl and termios keep destructuring.
- zlib, lzma, syslog, mmap and select defaults use their named constants
again instead of literals or <unrepresentable>.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Name FromArgs fields after their Python parameters
Drop #[pyarg(name = ...)] where the Python name is a valid Rust field name
and rename the field instead. The option stays only for keywords (fn, loop,
final, type, self) and the non-snake_case memLevel.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Build literal defaults from Python values and type <unrepresentable>
- default = LIT on a non-primitive field converts the literal through
ToPyObject and TryFromObject when the argument is missing, so object
fields no longer need a vm expression plus a matching py_default.
- A bare default on a primitive integer, bool or float field renders its
zero value.
- DefaultRepr::Unrepresentable replaces the <unrepresentable> text, and
py_default = "<unrepresentable>" is a compile error.
- pyexpat.ParserCreate distinguishes a missing intern from intern=None,
mmap AccessMode renders its value, and sqlite blobopen names rowid.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Resolve native docstrings from the doc database at compile time
- rustpython-doc stores its table as a sorted const slice with const fn
lookups (get, get_attr, class_attr_doc) instead of a phf map, and the
update-doc-db workflow emits that form.
- Class, method, getset, module function and struct sequence docs, and
per-class slot wrapper docs, are chosen by const lookups in the derive
output. The database text wins over Rust doc comments.
- Macros reach the table through rustpython_vm::__exports, so only the vm
depends on rustpython-doc.
- Remove expectedFailure markers from doctest and pydoc tests that pass.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Keep time.gmtime, localtime and ctime without a text signature
Their [seconds] parameter has no text signature, and unittest.mock relies
on inspect.signature failing for them. Take OptionalOption again so the
default stays unrepresentable, and treat None like a missing argument.
Assisted-by: Claude:claude-opus-5-5
* Borrow tp_descr_get/tp_init arguments and by-value public helpers
- DescrGetFunc and GetDescriptor::descr_get take
(&PyObject, Option<&PyObject>, Option<&PyObject>); impls call
.to_owned() where they store or return an argument.
- InitFunc takes &PyObject and Initializer::init takes &Py<Self>.
- VirtualMachine::get_attribute_opt, print_exception,
call_get_descriptor(_specific) and other helpers flagged by
clippy::needless_pass_by_value take &PyObject / &Py<T>; callers in
crates/, examples/ and tests/ drop the clones made for the call.
Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Grok CLI:grok-4.7
* Pass the borrowed zelf to internal_close in WindowsConsoleIO init
Assisted-by: Claude Code:claude-opus-5-5
* Fix borrowed signatures missed by the rebase
OrderedDict.init, classmethod_descriptor.descr_get, and the warnings
get_attribute_opt callers still used owned handles.
Assisted-by: Grok CLI:grok-4.7
Parameters, return types, closure annotations, Fn and Iterator bounds,
trait methods and struct fields that borrowed an owning handle now borrow
the object: &PyObjectRef -> &PyObject, &PyRef<T> / &PyXxxRef -> &Py<T>.
Existing .clone() calls on those values became .to_owned().
- Move PyTuple::extract_tuple into impl Py<PyTuple>; FromPyTuple::from_pytuple
takes &Py<PyTuple>.
- DictIter and IntoIterator for &PyDict use &Py<PyDict>; apply_shared_ns
takes &Py<PyDict>.
- PyExactSizeIterator, calculate_suggestions, collection_repr and the cformat
tuple helpers take iterators over &PyObject; callers map with |o| &**o.
- Return types changed: TypeNamespace::as_dict, Context::cached_int,
struct_error_type, PyBoundMethod::function_obj/self_obj,
AsRef for ArgIndex. as_dict, function_obj and self_obj are no longer
const fn.
- asyncgenerator unbox, csv parse_lineterminator and _ctypes bytes
parameter take &Py<T> instead of &T.
Assisted-by: Claude Code:claude-opus-5-5
Assisted-by: Grok CLI:grok-4.7
* host_env: export winsound, spawn, and remaining SSL_OP flags
Assisted-by: Claude
* stdlib: take winsound, nt spawn, and SSL_OP names from host_env
Assisted-by: Claude
* host_env: export ssl alerts, sysexits, socket extras
Assisted-by: Claude
* stdlib: take remaining host_env and common constants
Assisted-by: Claude
* Remove unused workspace exitcode dependency
Assisted-by: Claude
* host_env: export PROTO, access, reparse, wait flags
Assisted-by: Claude
* stdlib: take PROTO, access, reparse, wait from host_env
Assisted-by: Claude
* host_env: keep rustls off the rustls-free ssl surface
ssl is MemoryBIO, constants, OID, and ALPN. ssl-rustls is
the rustls engine. Drop the wasm32 rustls-pki-types web
dep; the browser crate already enables that feature.
Assisted-by: Claude
* host_env: drop rustls-pki-types from the lockfile
Assisted-by: Claude
* host_env: name the rustls engine feature rustls
ssl is the rustls-free surface. rustls is the engine.
stdlib's ssl-rustls product feature still selects this.
Assisted-by: Claude
* host_env: share thread, locale, loader and BSTR
Add Windows thread name, locale-info/wcscoll, CompareStringOrdinal,
LoadLibraryEx/GetProcAddress/FreeLibrary, BSTR and COM helpers, and
AF_HYPERV UUID conversion. Wire _thread, locale, ctypes LoadLibrary
and WMI BSTRs through those wrappers. Ensure =X: before spawnve/execve.
Assisted-by: Claude
* host_env: expose rustls-free ssl core on wasm
Compile MemoryBIO, protocol constants, OID tables, hostname
checks, and ALPN parsing without the rustls feature so wasm
and other hosts can implement _ssl on the shared types.
Assisted-by: Claude
* stdlib: add wasm _ssl on rustls-free host_env
Install _ssl on wasm32 against rustpython_host_env::ssl.
MemoryBIO, constants, OID, hostname, and ALPN work. Context
settings are stored; wrap/handshake and rustls-only loaders
raise SSLError until a wasm TLS engine is wired.
Assisted-by: Claude
* host_env: address #8702 review comments
Cache-miss Windows ctypes lookup uses GetProcAddress. FreeLibrary
propagates OSError. start_new_thread returns GetThreadId. Export
_thread._get_name and drop pyfunction doc comments. Reject
embedded NULs in Windows locale.strcoll/strxfrm. Terminate
wcscoll/wcsxfrm and SetThreadDescription inputs.
wasm _ssl accepts wrap_bio owner and get_ca_certs(binary_form),
and MemoryBIO.read(-1) returns all pending bytes.
Assisted-by: Claude
* host_env: gate the whole ssl module on feature ssl
Compile rustpython_host_env::ssl only with the ssl feature.
Rustls crates stay native-only, so enabling ssl on wasm does
not pull rustls. Inside the module, rustls files follow
not(wasm32) rather than a second feature flag.
Assisted-by: Claude
* host_env: fix Windows types and thread names
Cast SysAllocStringLen to *mut u16 and RPC status to u32 so
windows-gnu and MSVC agree. Export _NAME_MAXLEN, truncate names
at NUL and the OS byte cap, and decode get_name with fsdecode.
Assisted-by: Claude
* host_env: compile rustls engine on WASI
Keep rustls-free MemoryBIO/constants/OID/ALPN on all wasm
targets. Compile the rustls engine on WASI, where UnixTime::now
exists. Allow the rustpython ssl feature on wasm without
ssl-rustls or ssl-openssl.
Assisted-by: Claude
* compiler: pick the earliest CPython parse override
A later incompatible-prefix scan must not hide an earlier
number or bracket diagnostic. Parser-driven tokenization
stops at the first failure.
Assisted-by: Claude
* host_env: address remaining #8702 review comments
Map a missing Windows export to a symbol-load error, keep
LoadLibrary's Win32 code on the OSError, drop incomplete
UTF-16 pairs when truncating thread names, hide set_name
and _get_name off linux/macOS/Windows, and let ssl-rustls
enable host_env.
Assisted-by: Claude
* stdlib: add wasm _socket and read-all MemoryBIO
Register a rustls-free-style _socket shim on wasm32 so
Lib/ssl.py can import. Treat MemoryBIO.read(-1) as read-all,
matching _ssl_MemoryBIO_read_impl.
Assisted-by: Claude
* host_env: fix clippy, WASI ssl cfg, and wasm rustls
Defer unclosed-bracket overrides until no token diagnostic
exists, keep WASI rustls and wasm ssl modules exclusive,
preserve lone Windows surrogates, compile the rustls engine
on browser wasm, and clear clippy denials.
Assisted-by: Claude
* host_env: fix Windows clippy and wasm UnixTime
Gate thread-name byte truncation to linux/macOS, allow
GetProcAddress/FreeLibrary pointer deref, and enable
rustls-pki-types web time on browser wasm.
Assisted-by: Claude
* host_env: lock rustls-pki-types web for wasm
Assisted-by: Claude
* host_env: silence Windows unused thread id and shear
Assisted-by: Claude
* stdlib: drop Windows locale clippy needless wraps
Windows clippy failed on locale.strcoll/strxfrm needless return
and _getdefaultlocale always wrapping Ok.
Assisted-by: Claude
* compiler: keep lexer errors over print hints
print x; 0x must report an invalid hex literal. Ranking by source
offset let the earlier print hint win. Treat a finished radix
literal as valid so print x; 0x1 still gets the print message.
Assisted-by: Claude
* compiler: keep decode errors over unclosed openers
Ranking by offset made ( '\\N' report an unclosed paren and
wiped the unclosed-bracket flag on f'{, so compile snippets
raised SyntaxError instead of incomplete input. Export wasm
socket._accept for Lib/socket.py.
Assisted-by: Claude
* wasm: drop duplicate _socket; split locale cfgs
_socket lives in rustpython-stdlib's socket_wasm. The wasm crate
only re-exported that file when freeze-stdlib was off. Split
strcoll, strxfrm, and getencoding into cfg'd function defs.
Assisted-by: Claude
* compiler: keep decode blockers over unclosed openers
(print x; '\\N' must keep the print hint. A later malformed
escape still suppresses the EOF opener even when print wins.
Assisted-by: Claude
* compiler: keep earlier f-string over later lexer errors
f'{x'; ' and f'{x'; 0x must not report the trailing quote or
hex literal. Decode and f-string diagnostics now compete with
lexer failures by offset.
Assisted-by: Claude
* compiler: rank decode vs lexer vs print by class
Unterminated quotes no longer hide format-spec newlines or
empty f-string fields. Print is considered only after the
final winner is not a lexer failure, so print x; f'{x'; 0x
keeps the print hint. Line-continuation uses CPython wording.
Assisted-by: Claude
* compiler: keep unclosed paren over line-cont at EOF
test_exceptions wants '(' was never closed at the opener when
a final backslash follows. Skip escaped quotes in the bracket
scan so {\'a\'} still reports the continuation error. Un-xfail
test_join_daemon_thread_in_finalization.
Assisted-by: Claude
* test_threading: xfail daemon join finalization off Linux
The test passes on Linux. macOS and Windows leave an ignored
deallocator exception on stderr.
Assisted-by: Claude
* vm: hang daemon threads at interpreter finalize
Forced-off daemons must stay not-done so join() during
finalization GC raises PythonFinalizationError and
Thread.is_alive() stays true. SystemExit made the handle
look finished on macOS/Windows. Drop the Lib/test xfail.
Assisted-by: Claude
(cherry picked from commit d1c4925534)
Assisted-by: Codex:gpt-6
* address review: drain plaintext before BIO EOF
Skip feed_tls([]) while rustls still has application
data, matching the native SSL read EOF order.
Assisted-by: Grok:grok-4.6
* Honor asyncgen ag_closed and asend reuse
Keep ag_closed apart from a finished frame. aclose()
sets it before GeneratorExit. asend() already-running
closes the awaitable so it does not warn unawaited.
Assisted-by: Grok:grok-4.6
(cherry picked from commit 823f6c6e0e)
Assisted-by: Codex:gpt-6
* address review: assignment scan and wasm TLS
Skip comments and lambda defaults in the condition
assignment scanner. Restrict the suite-header colon
fallback to yield targets. Capture __del__ before
calling it. Store browser socket timeouts as f64 and
accept integers. Fail closed when wasm verify_mode is
not CERT_NONE. Export getpeercert and feed MemoryBIO EOF.
Assisted-by: Grok:grok-4.6
* address review: parenthesized yield suite header
Walk through grouping parentheses before requiring
yield after a suite-header colon.
Assisted-by: Grok:grok-4.6
* Honor close unraisable and deallocator messages
A failed close lookup on a yield-from target is unraisable.
Ignored GeneratorExit from finalize gets a traceback and the
closing-generator err_msg. __del__ uses the deallocator message.
Assisted-by: Grok:grok-4.6
* address review: claim, ag_running, frame steal
Keep the running claim through close() cleanup. ag_running
reads running_async. aclose ignore-yield does not mark the
generator closed. Steal the frame slot before uniqueness
and expose it with try_to_owned.
Assisted-by: Grok:grok-4.6
* Honor wrapper docs, yield assign, throw context
method-wrapper exposes the slot __doc__. Parenthesized yield
assignment uses the invalid-target messages. throw() restores
without overwriting __context__, then chains only from the
generator's own exc_info slot.
Assisted-by: Grok:grok-4.6
* Remove passing expectedFailure markers
method-wrapper docs, deallocator messages, and help pager
output now match, so drop the leftover markers.
Assisted-by: Grok:grok-4.6
* Honor wrapper signatures and assignment diagnostics
Slot docs include text signatures. method-wrapper compares and
hashes by identity. Keyword genexp and if/while assignment use
the invalid-target messages.
Assisted-by: Grok:grok-4.6
* address review: aclose, unraisable, frame publish
aclose ignore-yield does not close the generator. Format
unraisable object is None. close() leaves traceback to the
caller; finalize attaches the generator frame. Publish the
iframe before the lock-free slot.
Assisted-by: Grok:grok-4.6
* Honor member descriptors and wrapper annotations
Assisted-by: Grok:grok-4.6
* address review: getitem, bool doc, if assign
Assisted-by: Grok:grok-4.6
* Honor eval-mode assignment invalid syntax
Assisted-by: Grok:grok-4.6
* Supply getset and member docs from rustpython-doc
Assisted-by: Grok:grok-4.6
* Enable rustls SSL on WASI targets
Compile the rustls _ssl module for WASI and stub
socket waits plus the native certificate store.
Assisted-by: Grok:grok-4.6
* Add browser TLS _ssl in rustpython-wasm
Register rustls _ssl and a MemoryBIO-oriented _socket
from rustpython-wasm. Use rustls-rustcrypto because
ring cannot compile for wasm32-unknown-unknown.
Assisted-by: Grok:grok-4.6
* Define SSL protocol and cert constants as enums
Keep OpenSSL bitmask defines as integer constants.
Assisted-by: Grok:grok-4.6
* Ignore rustls deps cargo-shear cannot see
The rustpython-wasm _ssl sources are compiled through
a symlink, so shear does not count their crate uses.
Assisted-by: Grok:grok-4.6
* Replace wasm ssl symlink with a BIO module
Drop the stdlib ssl.rs link and keep a rustpython-wasm
_ssl that only covers MemoryBIO wrap_bio TLS.
Assisted-by: Grok:grok-4.6
* common: share the zlib stream engine
Move the VM-independent zlib-rs stream owner and state machine into a feature-gated rustpython-common module. Make rustpython-stdlib adapt that engine instead of carrying its own flate2 implementation, including native stream copy/deepcopy, Z_BLOCK flushes, and wbits=0 decompression.
Assisted-by: Codex:GPT-5
* common: share the lzma stream engine
Move the VM-independent liblzma stream owner, filter properties, and compression state into a target-gated rustpython-common module. Keep rustpython-stdlib as the Python object and exception adapter, and remove its direct xz dependencies and lzma-only generic compressor machinery.
Assisted-by: Codex:GPT-5
* common: address compression engine reviews
* Align compiler futures and annotations with CPython
Handle Barry parsing consistently across AST, type-comment, REPL, and WASM paths. Preserve deferred annotation source and scope metadata, and make _symtable conversion cached and linear-time.
Assisted-by: Codex:gpt-5.4
* Fix f-string debug ranges and update Barry/annotate test expectations
Compute the debug f-string source range from the raw leading and
trailing text instead of the comment-stripped text, so a comment inside
a multi-line replacement field no longer shifts the emitted LOAD_CONST
location.
Restrict the obsolete-operator location shift to `<>` by also requiring
a `>` at the reported offset; without it any `ExpectedExpression` that
followed a `<` (`2 <;`) was reported one column to the left.
Add `CodeFlags::FUTURE_MASK` and use it in builtins, the shell, and the
WASM VM instead of repeating the eight-flag union, and collapse the
duplicated Barry diagnostic sequences into `BarrySource::diagnostic`.
Update the `__annotate__` disassembly snapshot to LOAD_FAST_BORROW and
replace the RustPython-only `barry_as_FLUFL` co_flags assertion in
extra_tests with the CPython-matching one.
Assisted-by: Claude Code:claude-opus-5
* Restore the annotated function in __annotate__ qualnames
Signature annotation scopes are compiled in the scope enclosing the
annotated function, so the function name has to be folded back into the
qualname: `f.__annotate__`, `C.m.__annotate__`,
`outer.<locals>.inner.__annotate__`. test_type_annotations
test_annotate_qualname covers this.
test_pyrepl test_future_barry_as_flufl now passes, so drop its
expectedFailure marker.
Assisted-by: Claude Code:claude-opus-5
* Pass safepoints in the attached-thread interpreter tests
`main_and_subinterpreter_run_sections_overlap` parked its workers on a
condvar while attached, and `busy_main_interpreter_does_not_block_subinterpreter`
looped over protocol calls that never reach a safepoint. An attached
thread that never reaches a safepoint cannot be suspended, so a
concurrent process-wide stop-the-world (from the gc test running in
parallel) never completes and the already-stopped sibling interpreter
stays stopped; the second worker then waits in `wait_while_suspended`
until the 30s deadline.
Both loops now call `vm.check_signals()` each iteration, and the condvar
wait uses `wait_timeout` so the lock is released between safepoints.
Reproduced on Linux (4 CPUs): the two tests with `--test-threads=2`
failed 2/40 before and 0/40 after; the full `rustpython-vm` lib binary
failed 1/30 before and 0/30 after.
Assisted-by: Claude Code:claude-opus-5
* Report the obsolete `<>` operator from one place
`<>` was located twice: `BarrySource` recorded the token ranges it
rewrote to `!=`, and `cpython_parse_diagnostic_override` separately
peeked at the bytes around an `ExpectedExpression` location. Both
produced "invalid syntax" spanning the whole operator.
`prepare_barry_as_flufl_source` now records the operator ranges in the
non-Barry case too -- by plain text search, since nothing is rewritten
there -- so `BarrySource::invalid_legacy_operator` covers both, and the
byte-peeking helper is gone. It also defers to an unclosed bracket
earlier in the source, which the diagnostic chain it used to sit in did
for it.
A `<>` that starts a statement now spans both characters: the parser
stops at the `<` rather than one character in, which the byte peek did
not handle.
Assisted-by: Claude Code:claude-opus-5
* Drop a py object's payload before its class
`PyInner` declares `typ` before `payload`, so `drop_in_place` dropped the
class first and `PyAtomicRef::drop` left it null. A `PyWeak` payload is
still linked into the weakref list of the object it points at until its
own `Drop` unlinks it under the stripe lock, which happens after that.
`WeakRefList::add` walks the same list looking for a proxy to reuse and
reads `node.class()` off every node it passes, so it could dereference
the null class of a node on its way out.
`PyInner::dealloc` now runs the two destructors in the other order at all
three of its exit paths, with a compile-time check that no other field
has one.
Reproduced by the new test in about 0.1s, as the panic CI hit in the
c-api `weakrefobject::tests::new_proxy` run:
panicked at crates/vm/src/object/ext.rs:312:
unsafe precondition(s) violated: hint::unreachable_unchecked must
never be reached
Assisted-by: Claude Code:claude-opus-5
* Allow lone-surrogate keyword keys in f(**d)
`f(**{'\udc81': 2})` raised `TypeError: keywords must be strings` even
though the key is a valid `str`, because `KwArgs` stored keys as Rust
`String` (strict UTF-8) and `collect_ex_args` narrowed each key to
`PyUtf8Str` (valid UTF-8 required). CPython only checks that the key is a
`str` (`Py_TPFLAGS_UNICODE_SUBCLASS`), never encoding validity.
Change `KwArgs<T>`'s key type from `String` to `Wtf8Buf` (the WTF-8
storage `PyStr` already uses) and relax the keyword-key downcast from
`PyUtf8Str` to `PyStr` in `collect_ex_args`, `from_vectorcall`/
`from_vectorcall_owned` (which previously panicked on surrogate keys),
the `functools.partial` keyword merge, and the C-API `dict_to_kwargs`.
`Wtf8Buf` borrows only as `Wtf8`, so inherent `get`/`contains_key`/
`swap_remove`/`shift_remove(&str)` on `KwArgs` restore the `&str` lookup
interface `String: Borrow<str>` used to provide (via the zero-cost
`Wtf8::new` cast), and a generic `FromIterator<(K: Into<Wtf8Buf>, T)>`
keeps construction sites unchanged. WTF-8 awareness stays localized to
`function/argument.rs`.
Fixes#8228
Assisted-by: Claude Code:claude-opus-4-8
* Avoid cloning kwargs keys in _ast, borrow via as_ref instead
`new_str` only needs a `&Wtf8`, so pass `key.as_ref()` rather than
cloning the key into an owned `Wtf8Buf`. The key is reused afterwards
(error message, `intern_str`), so a borrow is the right fit.
Assisted-by: Claude Code:claude-opus-4-8
* Align compiler and AST behavior with CPython
Update the compiler pipeline, AST module support, and related VM compile/eval paths toward CPython 3.14 behavior.
Use upstream Ruff parser crates directly and keep RustPython-specific syntax preflight handling in the compile path.
Refresh opcode metadata, snapshots, and targeted tests for the aligned bytecode and AST behavior.
* fix
* Fix PR 8138 CI failures
* Fix match mapping wildcard scan complexity
* fixes
* Use Ruff runtime AST fields
* Minimize residual patch diff
* refactor error handling
* Remove _ast conversion context wrappers
* Clean up _ast conversion glue and compiler warning escalation
Rename the Node conversion parameter `ctx` to `vm` throughout the _ast
module.
Remove the unused compile_program, compile_program_single,
compile_block_expression, and compile_expression forwarders; compile_top
stays as the entry point.
Stash an escalated compiler SyntaxWarning so a non-SyntaxWarning category
propagates unchanged instead of being rewritten to SyntaxError, matching
PyErr_ExceptionMatches(SyntaxWarning) in compiler_warn. Drop the now-unused
CompileWarningError::into_codegen_error.
Replace the PositionalArguments two-variant enum with a struct holding a
shared range and a PositionalArgumentsKind enum.
Assisted-by: Claude
* Represent compiler flags as a CompilerFlags bitflags
Replace the loose PY_CF_*/CO_* i32 constants in compile_mode with a
CompilerFlags bitflags covering the PyCF_* flags and __future__
CO_FUTURE_* bits. The Python-visible ast.PyCF_* attribute values are
derived from it via bits(); parser input modes stay plain consts.
builtins::compile and the compile_string flag handling test flags
through CompilerFlags::contains instead of raw bit math.
Assisted-by: Claude
* Fix PR 8138 test regressions
* Fix PR 8138 CI failures
* Avoid shell interpolation in Linux deps action
* Fix review findings in compiler/AST changes
- frame.rs: compute the MATCH_CLASS class name lazily so a successful
match no longer takes the type name lock or allocates; the
__match_args__-not-a-tuple branch reuses it instead of recomputing.
- _ast/constant.rs: add the missing leading space to the with_recursion
label ("maximum recursion depth exceeded during compilation").
- sys.set_int_max_str_digits: restore the "maxdigits must be 0 or larger
than {n}" error text.
- codegen: set CO_MAXBLOCKS to 20 in both ir.rs and compile.rs.
- _ast/exception.rs: extract the duplicated excepthandler node guard
into ensure_excepthandler_node.
- codegen: correct the validate_keywords doc comment.
- Drop redundant "CPython" qualifiers from comments added in this branch.
Assisted-by: Claude
* Object header slimming: prefix allocation for ObjExt
Extract dict, weak_list, and slots fields from PyInner<T> into a
separate ObjExt struct allocated as a prefix before PyInner using
Layout::extend(). Objects that don't need these fields (int, str,
float, list, tuple, dict, etc.) skip the prefix entirely.
- Add HAS_WEAKREF flag to PyTypeFlags for per-type weakref control
- Add HAS_EXT bit to GcBits indicating prefix presence
- Define ObjExt struct with dict, weak_list, slots
- Shrink PyInner header from ~80-88 bytes to ~32 bytes for lightweight objects
- Update all accessor methods to go through ext_ref()
- Update bootstrap type hierarchy to use prefix allocation
- Add __weakref__ getset descriptor for heap types
- Set HAS_WEAKREF on builtin types that support weak references
- Remove test_weak_keyed_bad_delitem expectedFailure (now passes)
* Add HAS_WEAKREF to _asyncio Future/Task, rename weakref helpers
- Add HAS_WEAKREF flag to PyFuture and PyTask (matches CPython)
- Rename subtype_getweakref/setweakref to subtype_get_weakref/set_weakref
to fix cspell unknown word lint
* Add HAS_WEAKREF to array, deque, _grouper; remove expectedFailure markers
- Add HAS_WEAKREF to PyArray and PyDeque (matches CPython)
- Add HAS_WEAKREF to PyItertoolsGrouper (internal use by groupby)
- Remove 6 expectedFailure markers from test_dataclasses for weakref/slots
tests that now pass
* Add HAS_WEAKREF to code, union, partial, lock, IO, mmap, sre, sqlite3, typevar types
Add HAS_WEAKREF flag to built-in types that support weakref:
- PyCode, PyUnion, PyPartial, Lock, RLock
- All IO base/concrete classes (_IOBase, _RawIOBase, _BufferedIOBase,
_TextIOBase, BufferedReader, BufferedWriter, BufferedRandom,
BufferedRWPair, TextIOWrapper, StringIO, BytesIO, FileIO,
WindowsConsoleIO)
- PyMmap, sre Pattern, sqlite3 Connection/Cursor
- TypeVar, ParamSpec, ParamSpecArgs, ParamSpecKwargs, TypeVarTuple
Remove 3 expectedFailure markers from test_descr for now-passing tests.
* Add HAS_DICT to type flags and handle non-METHOD/CLASS in descr_get
- Add HAS_DICT flag to PyType (type metaclass) alongside existing
HAS_WEAKREF. All type objects are instances of type and need dict
support, matching CPython's PyType_Type.
- Replace unimplemented!() in PyMethodDescriptor::descr_get with
fallback to bind obj directly, matching CPython's method_get which
uses PyCFunction_NewEx for non-METH_METHOD methods.
* Fix ext detection, HeapMethodDef ownership, WASM error
- Remove HAS_EXT gc_bits flag; detect ext from type flags
using raw pointer reads to avoid Stacked Borrows violations
- Store HeapMethodDef owner in payload instead of dict hack
- Clear dict entries in gc_clear_raw to break cycles
- Add WASM error fallback when exception serialization fails
* Remove unsafe ctx cast in make_class
Use Context::genesis() directly in make_class to obtain
&'static Context, eliminating the raw pointer cast.
* make_class -> make_static_type
Defer locals dict allocation for function frames until first access.
Most function frames only use fastlocals and never touch the locals
dict, so this skips one dict heap allocation per function call.
Also remove a redundant code.clone() in invoke_with_locals.
Func call ~23% faster, method call ~15% faster in benchmarks.
