8 Commits
Author SHA1 Message Date
youknowdot feca6f2004 Align collections and testing libraries with Python 3.15 (#9192)
* Align collections and testing libraries with Python 3.15

Refresh twelve production modules and canonical tests through scripts/update_lib at CPython d220409a13b77de7a6c30abe16ea90db01528edc. Preserve existing RustPython unittest helpers and test guards. Accept frozendict in native defaultdict and OrderedDict union slots, matching upstream PyAnyDict_Check without changing constructors, factories, or update order.

Validated as a local stack on the exact #9189 regex/dataclasses/typing prerequisite; this dependency avoids eager typing.ByteString deprecation during unittest warning tests.

Assisted-by: OpenAI Codex: model version unavailable

* Synchronize regression runner with doctest subtest reporting

Refresh the canonical test_regrtest consumer and its direct runner/pythoninfo closure through scripts/update_lib at CPython d220409a13b77de7a6c30abe16ea90db01528edc. Preserve both existing expected-failure markers. This aligns the runner test with doctest reporting two failed examples and one skipped example.

Assisted-by: OpenAI Codex: model version unavailable
2026-10-19 20:07:29 +09:00
28393e932d Implement PEP 814 frozendict (#8951)
* Restore consolidated frozendict implementation on current main

Preserve reverse dictionary iterator fixes and the 3.15 frame imports. Keep library refreshes separate pending the runtime dependency sequencing decision.

Assisted-by: OpenAI Codex:model identifier unavailable

* Support scoped canonical test updates in update_lib

Index conditional test classes and methods without crossing namespaces, reject ambiguous targets, and preserve inherited override indentation. Add hash-pinned whole-test copying that verifies execution context and exact source bytes, with focused Python fixtures.

Assisted-by: OpenAI Codex:model identifier unavailable

* Align frozendict copy support and tests with CPython 3.15

Import exact-final copy.py with update_lib; defer Mapping ABC registration to the later library stage. Generate only proven marker corrections, retain observed debug recursion-limit expected failures, and copy the complete canonical builtin-doc discovery test using hash-pinned update_lib copy-test.

Assisted-by: OpenAI Codex:model identifier unavailable

* Preserve decorator rationale during canonical test copying

Reject differing decorator source blocks even when their ASTs match. Cover inline and multiline marker comments, guarded async methods, and CLI no-write behavior.

Assisted-by: OpenAI Codex:model identifier unavailable

* Finish frozendict with borrowed PyAnyDict views

Use the landed PyRef target model for a transparent dict/frozendict header
view, retaining objects explicitly with PyRef<PyAnyDict>. Keep targets
sealed and preserve mutable-only entry points. Borrow frame globals from
the object header rather than movable reference wrappers.

Unify exact-TypeError key diagnostics with final CPython semantics,
including callback failure propagation and no explicit cause. Keep the
Python regressions in builtin_dict and test reference layout/ownership
in Rust. Preserve existing canonical library updates.

Assisted-by: OpenAI Codex:model identifier unavailable

* Align frozendict with CPython and use CPython 3.15 tests

- Name the target type in unhashable key errors through
  PyDict::update_arg (dict_update_arg), replacing the separate
  PyFrozenDict::merge_into copy of the merge logic.
- Split the | operator into dict_or (_PyDict_Or) and frozendict_or.
- Return the argument from frozendict(frozendict) only when the called
  type is frozendict.
- Call through the vectorcall slot in PyCallable::invoke when there
  are no keyword arguments, instead of special-casing frozendict.
- Add ensure_builtins (_PyEval_EnsureBuiltins) for eval/exec, and
  insert __builtins__ for source strings regardless of globals type.
- Read __warningregistry__ and __name__ from stored globals entries and
  accept only None or str for the module name in setup_context.
- Remove the unused PyFrozenDict::from_object.
- Migrate test_dict.py, mapping_tests.py, test_copy.py and
  test_builtin.py to v3.15.0 with update_lib, and add
  test_import/data/syntax_warnings.py.
- Drop builtin_dict.py frozendict tests covered by those modules,
  keeping the descriptor and namespace integration tests.

Assisted-by: Claude Code:claude-opus-5-5

* Update crates/vm/src/builtins/dict.rs

---------

Co-authored-by: Jeong YunWon <jeong@youknowone.org>
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
2026-10-14 16:42:56 +09:00
Jeong, YunWon 3c654204f5 Match native text signatures to CPython names, kinds and defaults (#8815)
* Render CPython defaults in generated text signatures

Arguments whose default rendered as `<unrepresentable>`, while the
same callable has a representable default in CPython, now render that
default:

- Where the Rust value can be a literal, the argument becomes a plain
  field with `default = <literal>`.
- The remaining cases keep their Rust default and add `py_default`
  with a comment. These are wrappers such as ArgIntoFloat or
  ArgIntoBool, values that need the VM, enums, and Option where None
  means another value.

Byte string literals render as Python bytes. A function registered
under several names gets an internal doc per name, so its __doc__ no
longer carries the signature header.

Removed expectedFailure from test_pydoc tests that now pass.

Assisted-by: Grok:grok-4.7
Assisted-by: Claude:claude-opus-5-5

* Match native callable signatures to CPython names and kinds

Rename parameters and adjust positional-only / keyword-only kinds and
defaults of native functions and methods so their generated text
signatures match CPython. Remove expectedFailure markers from tests
that now pass.

Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7

* Fix Windows console read size and keep type.__prepare__ permissive

Assisted-by: Claude:claude-opus-5-5

* Derive signature defaults from typed Rust values

Param.default is now a DefaultRepr rendered by the const writer.
Literals, integer and bool constants become typed defaults, and other
default expressions use the type's const fn py_default. A bare
`optional` renders as None, so `default = None` rewrites are reverted.

Address review comments: zlib max_length defaults, get_ca_certs
binary_form keyword, getpeercert der, WASM ssl certificate keywords,
fstatvfs fd-only, and _sre endpos shown as sys.maxsize.

Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7

* Generate class constructor text signatures at compile time

A #[pyclass] impl composes the type's internal doc from the Args of
its Initializer or Constructor, so type.__text_signature__ and
inspect.signature work for native classes. __doc__ stays the plain
doc. Struct sequences get (iterable=(), /) and __reduce__ (self, /).
Hand-parsed constructors are converted to FromArgs structs, and
_weakref.proxy is split from ProxyType.

Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7

* Remove expectedFailure from test_hmac tests that now pass

Assisted-by: Claude:claude-opus-5-5

* Match remaining native signatures and count missing CPython signatures as matches

Give lock.__enter__ and RLock.__enter__ their own no-argument methods,
add FromArgs for start_new_thread and start_joinable_thread keeping
their errors, take RTLD_GLOBAL/RTLD_LOCAL from libc, and align names,
kinds and <unrepresentable> defaults where CPython's signature is
invalid (bytes.hex, reduce, warn, link, setlocale, add_done_callback,
...). getaddrinfo accepts keyword arguments. whats_left treats a
callable CPython has no signature for as matching.

Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7

* Drop unfulfilled dead_code expectation on os.link dir_fd args

Assisted-by: Claude:claude-opus-5-5

* Distinguish OptionalArg and Option in signature defaults

An `optional` field's signature default now comes from its type:
Option<T> renders None, while OptionalArg<T> (a missing argument that
is not any Python value) renders <unrepresentable>. Fields where a
missing argument behaves exactly like None become Option<T>; fields
whose missing state differs keep OptionalArg. Redundant
py_default = "<unrepresentable>" entries are removed.

Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7

* Remove expectedFailure from test_rlcompleter.test_attr_matches

Assisted-by: Claude:claude-opus-5-5

* Use Option for optional arguments whose missing and None states match

OptionalOption and OptionalArg<Option<T>> fields that treat a missing
argument like None become Option<T> with #[pyarg(optional)], which renders
=None in the text signature. Plain optional parameters of that kind move
into small FromArgs structs. Uses where missing and None differ (reduce
initial, StringIO newline, TextIOWrapper.reconfigure, set_asyncgen_hooks)
keep OptionalOption.

Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7

* Render ::NAME defaults by name and restore named constant defaults

- default = ::NAME writes NAME into the text signature and stores
  Into::into(NAME) as the Rust value.
- A one-field tuple-struct argument pattern such as Fildes(fd) names the
  parameter after its binding, so fcntl and termios keep destructuring.
- zlib, lzma, syslog, mmap and select defaults use their named constants
  again instead of literals or <unrepresentable>.

Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7

* Name FromArgs fields after their Python parameters

Drop #[pyarg(name = ...)] where the Python name is a valid Rust field name
and rename the field instead. The option stays only for keywords (fn, loop,
final, type, self) and the non-snake_case memLevel.

Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7

* Build literal defaults from Python values and type <unrepresentable>

- default = LIT on a non-primitive field converts the literal through
  ToPyObject and TryFromObject when the argument is missing, so object
  fields no longer need a vm expression plus a matching py_default.
- A bare default on a primitive integer, bool or float field renders its
  zero value.
- DefaultRepr::Unrepresentable replaces the <unrepresentable> text, and
  py_default = "<unrepresentable>" is a compile error.
- pyexpat.ParserCreate distinguishes a missing intern from intern=None,
  mmap AccessMode renders its value, and sqlite blobopen names rowid.

Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7

* Resolve native docstrings from the doc database at compile time

- rustpython-doc stores its table as a sorted const slice with const fn
  lookups (get, get_attr, class_attr_doc) instead of a phf map, and the
  update-doc-db workflow emits that form.
- Class, method, getset, module function and struct sequence docs, and
  per-class slot wrapper docs, are chosen by const lookups in the derive
  output. The database text wins over Rust doc comments.
- Macros reach the table through rustpython_vm::__exports, so only the vm
  depends on rustpython-doc.
- Remove expectedFailure markers from doctest and pydoc tests that pass.

Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7

* Keep time.gmtime, localtime and ctime without a text signature

Their [seconds] parameter has no text signature, and unittest.mock relies
on inspect.signature failing for them. Take OptionalOption again so the
default stays unrepresentable, and treat None like a missing argument.

Assisted-by: Claude:claude-opus-5-5
2026-09-26 10:39:04 +09:00
Jeong, YunWon 53a906c916 Match settrace events, f_lineno jumps, and type dict lookup (#8758)
* Keep type dict keys and report current f_lasti

Snapshot the MRO during lookup so non-string keys can run __eq__,
skip subclass updates when set_bases is reentered, make module
__dict__ a readonly member, and forbid deleting exception __dict__.
f_lasti is the current opcode; line events skip NO_LOCATION units.

Assisted-by: Grok:grok-4.6

* Fire settrace call events from RESUME

PY_START/PY_RESUME become a 'call' event only at the first RESUME
of a frame activation, so lasti is the resume unit and f_trace_lines
is honored. Unskip passing TraceTestCase methods.

Assisted-by: Grok:grok-4.6

* Record what_event so f_lineno jumps match line traces

Store the current monitoring event while a trace callback
runs, reject jumps from call/return/exception, unwind tracer
errors in the traced frame, and fire call on PY_THROW.
Unskip JumpTestCase except the infinite-loop and five remaining xfails.

Assisted-by: Grok:grok-4.6

* Fire remaining settrace events from opcodes and jumps

Keep GET_ANEXT's iterator in mark_stacks, dispatch non-call
events through f_trace, fire PY_YIELD/PY_RETURN from the
opcodes, emit same-line LINE on backward jumps, and raise at
flattened call sites. Unskip the rest of test_sys_settrace.

Assisted-by: Grok:grok-4.6

* Fix trampoline raise panic and unxfail passing traces

Do not decode inline-cache units as opcodes when a flattened
call raises. Remove expectedFailure from test_trace and
test_regrtest coverage tests that now pass.

Assisted-by: Grok:grok-4.6

* Fire profile returns without a per-frame f_trace

setprofile must see PY_RETURN/PY_YIELD even when f_trace is unset.
Drop EXPECTED_FAILURE from the pdb doctest and unxfail
test_unfinished_generator.

Assisted-by: Grok:grok-4.6
2026-09-21 18:11:32 +09:00
Jeong, YunWon f7d50795e5 Reduce per-instruction overhead in interpreter loop (#7315)
* Reduce per-instruction overhead in interpreter loop

- Defer prev_line update to when tracing is active
- Inline skip_caches_if_fallthrough, compute cache_entries once per instruction

* Unmark test_pdb_set_trace EXPECTED_FAILURE

prev_line deferral fix corrects pdb line tracking, making
these doctests pass.
2026-03-03 18:01:18 +09:00
Jeong, YunWon 25f545f059 apply tests
Assisted-by: Codex:gpt-6
2026-02-11 16:50:23 +09:00
CPython Developers 182fc3d79c Update doctest from v3.14.2 2026-02-11 16:47:03 +09:00
Terry Tianlin Luan cd012f9b2e Added doctest tests (#6598) 2025-12-31 09:43:10 +09:00