Commit Graph
73 Commits
Author SHA1 Message Date
Jeong, YunWonandLee Dogeon afad92bf1b Add OrderedDict, marshal, inspect, and chflags bitflagset (#8780)
* Implement _collections.OrderedDict

Extracted from #6799: native OrderedDict as a dict subclass,
iterator mutation tracking, and mapping-view set operations.

* Add OrderedDict, warnings source lines, and socket sendmsg

Implement _collections.OrderedDict as a dict subclass with a
linked list of insertion-order nodes.

Pack struct F/D as native float/double complex pairs, and report
unexpected __replace__ field names as a list repr.

Register encodings.latin_1 at startup. Reject NUL in input()
prompts. Copy non-exact type dicts in mapping order.

Fetch warning source lines through _bless_my_loader and unicode
splitlines. Keep the last warnings.filters list after deletion.

Snapshot sendmsg sequences before converting buffers, parse
ancillary ints with __index__, and dispatch AF_CAN by protocol.

Store sys.trace and sys.profile on each ThreadSlot so
_settraceallthreads/_setprofileallthreads are visible to
sys.gettrace/getprofile on other threads.

Use <frozen importlib._bootstrap> as frozen importlib
co_filename. Keep HEAPTYPE socket objects GC-tracked.

Assisted-by: Grok:grok-4.6

* Load marshal via readinto and dump sets in dumps order

marshal.load fills a scratch buffer with readinto on a
contiguous writable memoryview, and rejects a returned length
other than the requested size.

set and frozenset dumps write elements in sorted(v,
key=marshal.dumps) order.

Assisted-by: Grok:grok-4.6

* Add classmethod_descriptor and ABC IS_ABSTRACT flags

Give METH_CLASS methods a distinct classmethod_descriptor type.
Store Py_TPFLAGS_IS_ABSTRACT from __abstractmethods__ and look
up that attribute only in the type's own dict.

Expose defaultdict.default_factory as a member descriptor.
Inherit bool's unused number slots from int. Mark set and
frozenset __contains__ as METH_COEXIST.

Parse classmethod, staticmethod, object.__new__, and heap type
text signatures. Register interactive source in linecache.

Assisted-by: Grok:grok-4.6

* address review: odict reuse, descr call, new wrapper

Reuse odict node slots and walk the live list in next_after.
Require both | operands to be dicts. Drop the odict iterator
on GC clear. Bind classmethod descriptors through descr_get
and update IS_ABSTRACT under the type lock. Give odict keys
and items sq_contains. Walk tp_new_wrapper past new_wrapper
so HEAPTYPE socket subclasses can call socket.__new__.
Split FreeBSD Bluetooth protocol constants from Linux values.

Assisted-by: Grok:grok-4.6

* Represent chflags bits with bitflagset

Assisted-by: Grok:grok-4.6

* Represent access and mode bits with bitflagset

Assisted-by: Grok:grok-4.6

* Continue unfinished REPL suites as incomplete input

Assisted-by: Grok:grok-4.6

* Add pickle and os.stat inspect signatures

Give _pickle.Pickler the clinic constructor text signature and
accept fix_imports and buffer_callback as positional-or-keyword.
Allow #[pyfunction(text_signature)] and use it so os.stat reports
(path, *, dir_fd=None, follow_symlinks=True). Uncomment import
_pickle and drop the pickle inspect expectedFailures.

Assisted-by: Grok:grok-4.6

* Add Unpickler, dump/load, and lstat signatures

Give _pickle Unpickler, dump, dumps, load, and loads clinic
text signatures, and os.lstat (path, *, dir_fd=None).

Assisted-by: Grok:grok-4.6

* Complete OrderedDict with insertion-order links

Move the native OrderedDict into ordered_dict.rs and track order
with a linked list of nodes, keeping mapping-view set operations.

Assisted-by: Grok:grok-4.6

* address CI: stat casts, HCI_DATA_DIR, passing xfails

Compare stat permission bits as Mode so Linux clippy does not
flag a u32-to-u32 cast. Export HCI_DATA_DIR with the Bluetooth
socket option values. Drop expectedFailure from tests that now
pass.

Assisted-by: Grok:grok-4.6

* Fix wasm teardown deadlock and bluetooth constants

Assisted-by: Grok:grok-4.6

* Clear process-wide eval breaker bits in STW test

Assisted-by: Grok:grok-4.6

* Drop exclusive-held panic that tripped wasm clippy

Assisted-by: Grok:grok-4.6

* Fix OrderedDict contains and export Windows BDADDR

Assisted-by: Grok:grok-4.6

* Add cmp_to_key and hash-once LRU cache

_functools.cmp_to_key now returns KeyWrapper. LRU lookups store the
key hash on _lru_list_elem so __hash__ is not called again on hits
or eviction.

Assisted-by: Grok:grok-4.6

* Allow subclass keywords on list, tuple, and float

list.__init__ rejects keywords only when tp_new is still list's.
tuple/float.__new__ reject them only when tp_init is still the base's.

Assisted-by: Grok:grok-4.6

* Fix CI lint, test re-export, and structseq members

Drop the duplicate clear_eval_breaker_for_test re-export.
Format structseq imports. Expose struct sequence fields as
readonly members so assignment raises "readonly attribute".

Assisted-by: Grok:grok-4.6

* Allow subclass keywords on itertools and filter

Positional-only constructors ignore extra keywords when a subclass
has replaced tp_init, matching list/tuple/float.

Assisted-by: Grok:grok-4.6

* Remove text_signature derive parameter

Drop the string override on #[pyfunction] and #[pyclass]. Signatures
are generated from the Rust parameter list only. Restore inspect
expectedFailures that depended on the override.

Assisted-by: Grok:grok-4.6

* Rename set contains_method to __contains__

Keep a borrowed contains helper for slots and C-API. The coexist
pymethod is named __contains__.

Assisted-by: Grok:grok-4.6

* Use MonitoringEvent instead of EVENT_* aliases

Drop the unused MonitoringEvents bitflags and the EVENT_PY_START
style mask constants. Call sites use the enum and .mask().

Assisted-by: Grok:grok-4.6

* Store warnings filters as PyListRef

The list already serializes its elements. Drop the extra mutex around
the cached list pointer.

Assisted-by: Grok:grok-4.6

* Re-export clear_eval_breaker_for_test after rebase

Keep the STW test helper visible without bringing back stop_bit_set.

Assisted-by: Grok:grok-4.6

* Fix groupby reentrancy and islice arguments

Give _grouper its own tgtkey and take currvalue after the key
comparison. Accept __index__ on islice bounds, reject step 0, and
drain the source up to stop.

Assisted-by: Grok:grok-4.6

* SETREF warnings filters, onceregistry, and defaultaction

Keep the last list, dict, or str after the module attributes are
deleted by swapping the interpreter cache pointer. Do not store a
None fallback for _warnings_context.

Assisted-by: Grok:grok-4.6

* Iterate OrderedDict inner dict via next_entry

&PyDict is not an iterator after the borrowed-ref signature change.

Assisted-by: Grok:grok-4.6

* Remove expectedFailure from testBluetoothConstants

Linux now exports BTPROTO_HCI and the rest of the set.

Assisted-by: Grok:grok-4.6

* Borrow get_filter arguments as PyObject

Match the warn_explicit call site that passes borrowed
handles after the internal-helper borrow change.

Assisted-by: Grok:grok-4.6

* Fix bool slots and SemHandle safety docs

Leave unused bool number slots unset so inspect reports
__add__ on int. Document SemHandle::from_raw safety.

Assisted-by: Grok:grok-4.6

* Keep sendmsg errors and drop duplicate linecache

Replace sendmsg iterable TypeError only. Register REPL
source in linecache once per compiled input.

Assisted-by: Grok:grok-4.6

* Nest REPL parse or-patterns for clippy

Assisted-by: Grok:grok-4.6

* Mark pydoc os.stat signature test expectedFailure

Assisted-by: Grok:grok-4.6

---------

Co-authored-by: Lee Dogeon <dev.moreal@gmail.com>
2026-09-24 12:21:26 +09:00
Jeong, YunWon 7bc3ef6c0a ExceptionGroup fields, OSError use-init, and live f_lineno (#8738)
* Add ExceptionGroup fields and OSError use-init

BaseExceptionGroup stores msg, excs, and excs_str; args keep
the original sequence. OSError defers parse when a subclass
overrides only __init__. characters_written can be deleted.
Exception args/traceback/cause/context reject invalid
setattr/delattr. UnicodeError.object is a deletable field.
except* remainder projection copies traceback/context/cause.
SyntaxError.__init__ clears omitted location attributes.

Assisted-by: Grok:grok-4.6

* Fix OSError payload init and UnicodeError construction

new_payload_exception now runs slot_new so BlockingIOError
parses characters_written. new_exception falls back to
invoke_exception for extra-payload types. Mixed tab/space
UnexpectedIndentation is TabError. AttributeError pickle
omits obj.

Assisted-by: Grok:grok-4.6

* address review: classify TabError from error line

Assisted-by: Grok:grok-4.6

* Add new_simple_exception; fill SyntaxError and f_lineno

new_exception constructs extra-payload types; BaseException-sized
types use new_simple_exception. Non-UTF-8 SyntaxError sets location
and text. Codegen errors load text via ProgramText. Keep Missing
parentheses capitalization. f-string adjacent atoms report a missing
comma. f_lineno may be None; lasti is not restored on unwind.

Assisted-by: Grok:grok-4.6

* address review: skip f-string continuation keywords

Assisted-by: Grok:grok-4.6

* Fix clippy map_unwrap_or in Non-UTF-8 offset

Assisted-by: Grok:grok-4.6

* Rename encode error ctor and exception setters

new_unicode_encode_error_real is new_unicode_encode_error.
set___traceback__/__cause__/__context__ are pygetset setters;
Rust uses set_traceback/set_cause/set_context.

Assisted-by: Grok:grok-4.6

* Use set_context in native pickle

Assisted-by: Grok:grok-4.6

* Collapse nested if in asyncio traceback restore

Assisted-by: Grok:grok-4.6

* Use current instruction for f_lineno addr2line

lasti is stored as the next instruction index. addr2line of
that byte offset is past RETURN_VALUE, so f_lineno was None
and stdlib_threading_current_frames hung on leftover workers.

Assisted-by: Grok:grok-4.6

* Look up format args, dir class, and frame builtins

Consume printf args before rejecting unknown conversions.
object.__dir__ uses getattr of __class__/__dict__.
_PyEval_GetBuiltin looks up iter/__import__ on the frame.

Assisted-by: Grok:grok-4.6

* Fix builtin lookups after rebase and unxfail passing tests

Assisted-by: Grok:grok-4.6

* Restore RERAISE lasti and SyntaxError locations

RERAISE with oparg restores lasti from the exception table
before unwind, so leftover f_lineno matches the original raise.
Handler lookup still uses the raising instruction.

Indentation errors outrank print/exec missing-parentheses.
BOM-with-non-utf8, bytes literals, and global+nonlocal now
report the same offsets as the parser/symtable.

object.__dir__ keeps string keys; type.__type_params__ skips
the getset wrapper stored on builtin types.

Assisted-by: Grok:grok-4.6

* Unxfail syntax/import tests and bind jump locals

IMPORT_NAME passes fromlist None instead of an empty tuple.
Jumping f_lineno binds None into unbound localsplus slots.
str lstrip/rstrip copy subclasses to exact str when unchanged.

Assisted-by: Grok:grok-4.6

* Fix import_from callers and rename frame lineno

sqlite3.iterdump now passes fromlist as a PyObject.
FrameObject::lineno is the i32 helper; f_lineno is the getset.

Assisted-by: Grok:grok-4.6

* Pass sqlite fromlist without Into conversion

from_list.into() could not infer Into<PyObjectRef> at the
import_from call. The typed tuple already converts.

Assisted-by: Grok:grok-4.6
2026-09-20 14:22:58 +09:00
Jeong, YunWon a1ba6a662c Align compiler and AST behavior with CPython (#8138)
* Align compiler and AST behavior with CPython

Update the compiler pipeline, AST module support, and related VM compile/eval paths toward CPython 3.14 behavior.

Use upstream Ruff parser crates directly and keep RustPython-specific syntax preflight handling in the compile path.

Refresh opcode metadata, snapshots, and targeted tests for the aligned bytecode and AST behavior.

* fix

* Fix PR 8138 CI failures

* Fix match mapping wildcard scan complexity

* fixes

* Use Ruff runtime AST fields

* Minimize residual patch diff

* refactor error handling

* Remove _ast conversion context wrappers

* Clean up _ast conversion glue and compiler warning escalation

Rename the Node conversion parameter `ctx` to `vm` throughout the _ast
module.

Remove the unused compile_program, compile_program_single,
compile_block_expression, and compile_expression forwarders; compile_top
stays as the entry point.

Stash an escalated compiler SyntaxWarning so a non-SyntaxWarning category
propagates unchanged instead of being rewritten to SyntaxError, matching
PyErr_ExceptionMatches(SyntaxWarning) in compiler_warn. Drop the now-unused
CompileWarningError::into_codegen_error.

Replace the PositionalArguments two-variant enum with a struct holding a
shared range and a PositionalArgumentsKind enum.

Assisted-by: Claude

* Represent compiler flags as a CompilerFlags bitflags

Replace the loose PY_CF_*/CO_* i32 constants in compile_mode with a
CompilerFlags bitflags covering the PyCF_* flags and __future__
CO_FUTURE_* bits. The Python-visible ast.PyCF_* attribute values are
derived from it via bits(); parser input modes stay plain consts.

builtins::compile and the compile_string flag handling test flags
through CompilerFlags::contains instead of raw bit math.

Assisted-by: Claude

* Fix PR 8138 test regressions

* Fix PR 8138 CI failures

* Avoid shell interpolation in Linux deps action

* Fix review findings in compiler/AST changes

- frame.rs: compute the MATCH_CLASS class name lazily so a successful
  match no longer takes the type name lock or allocates; the
  __match_args__-not-a-tuple branch reuses it instead of recomputing.
- _ast/constant.rs: add the missing leading space to the with_recursion
  label ("maximum recursion depth exceeded during compilation").
- sys.set_int_max_str_digits: restore the "maxdigits must be 0 or larger
  than {n}" error text.
- codegen: set CO_MAXBLOCKS to 20 in both ir.rs and compile.rs.
- _ast/exception.rs: extract the duplicated excepthandler node guard
  into ensure_excepthandler_node.
- codegen: correct the validate_keywords doc comment.
- Drop redundant "CPython" qualifiers from comments added in this branch.

Assisted-by: Claude
2026-06-26 18:22:08 +09:00
Shahar Naveh 6eae1dde6c Update some tests to 3.14.5 (#7991) 2026-06-02 18:00:54 +09:00
Shahar Naveh c22c74e631 Add builtin.PythonFinalizationError (#7966)
* Add PythonFinalizationError to builtins

* Patch failing tests (unrelated)

* Unmark passing test

* Update `exception_hierarchy.txt` to 3.14.5
2026-05-24 19:58:16 +09:00
Lee Dogeon f5b1ea7c60 Preserve imaginary zero signs when adding real values to complex numbers (#7421)
* Preserve imaginary zero signs when adding real values to complex numbers

* Refactor complex_add with match expression

* Correct complex real subtract op

* Remove unnecessary vm arugment
2026-03-15 17:04:23 +09:00
Lee Dogeon 3f1ce34422 Match divmod zero-division message with CPython (#7426) 2026-03-14 20:21:23 +09:00
Lee Dogeon 257cf6fa0e Match __module__ setter behavior to CPython (#7424)
* Remove stale __firstlineno__ when updating type.__module__

* Register and use __firstlineno__ as identifier
2026-03-14 15:48:48 +09:00
Lee Dogeon ae73aab6a4 Handle chr() range checks without Rust integer overflow (#7422) 2026-03-14 15:47:45 +09:00
Lee Dogeon 3106c30d58 Make singleton builtin types immutable (#7423) 2026-03-14 09:45:43 +09:00
김은빈 76556540d1 Add strict parameter to map() builtin (#7405)
* Add strict parameter to map() builtin

* Refactor map IterNext to match zip style
2026-03-12 15:02:53 +00:00
Jeong, YunWon 48e4e7c661 Implement more ast features (#6986)
Assisted-by: Codex:gpt-6
2026-02-05 16:20:22 +09:00
Shahar Naveh da736dcf5f Update test_builtin.py from 3.14.2 (#6979)
* Update `test_builtin.py` from 3.14.2

* Mark haning/segfault tests

* Patch failing tests
2026-02-03 21:55:55 +09:00
Jeong, YunWonandClaude Sonnet 4.5 3814348943 dealloc and finalize_modules (#6934)
* rewrite finalize_modules with phased algorithm

Replace the absence of module finalization during interpreter shutdown
with a 5-phase algorithm matching pylifecycle.c finalize_modules():

1. Set special sys attributes to None, restore stdio
2. Set all sys.modules values to None, collect module dicts
3. Clear sys.modules dict
4. Clear module dicts in reverse import order (2-pass _PyModule_ClearDict)
5. Clear sys and builtins dicts last

This ensures __del__ methods are called during shutdown and modules are
cleaned up in reverse import order without hardcoded module names.

* dealloc the rigth way

* fix finalize_modules: only clear __main__ dict, mark daemon thread tests as expected failure

Without GC, clearing all module dicts during finalization causes __del__
handlers to fail (globals are None). Restrict Phase 4 to only clear
__main__ dict — other modules' globals stay intact for their __del__
handlers.

Mark test_daemon_threads_shutdown_{stdout,stderr}_deadlock as expected
failures — without GC+GIL, finalize_modules clears __main__ globals
while daemon threads are still running.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-02 10:10:58 +09:00
Jeong, YunWon 8d901a7300 Implement bool(NotImplemented) 2026-01-17 19:21:10 +09:00
Padraic Fanning 98d8ebcc1d Skip test causing worker bug in test_builtin 2026-01-10 22:10:35 -05:00
Terry Tianlin Luan 75838e567e Update pty + tty + associated libraries - v3.13.10 (#6630)
* Update tty + added the test from v3.13.10

* Updated pty.py + test
2026-01-05 14:38:04 +09:00
Jeong, YunWon 7edc3bba5d tp_itemsize (#6544) 2025-12-28 10:27:58 +09:00
Jeong, YunWon 9b2ad34a08 check surrogates (#6547) 2025-12-28 00:06:29 +09:00
Jeong, YunWon 75ecd72428 test_builtin.test_import (#6546)
* fix warnings

* fix test_import
2025-12-27 21:38:12 +09:00
Jeong, YunWon 4592787946 get_inheritable, dup for windows (#6343) 2025-12-08 23:55:03 +09:00
Noa 7d05f881c4 Have rustpython_literal::escape support wtf8 2025-03-28 11:26:29 +09:00
Snowapril aa0353a501 Fix best_base to select proper base class (#5324)
* add __basicsize__ getter

* modify best_base function to get proper base

* inherit basicsize from the base type

---------

Signed-off-by: snowapril <sinjihng@gmail.com>
2024-05-19 15:48:56 +09:00
hydrogen602 61feb43aba test_builtin.py: test_type_qualname fix 2024-05-13 14:18:18 +09:00
hydrogen602 af8bed6d3f filter out doc strings with opt level 2 2024-04-25 16:11:04 +09:00
Jonathan Rotter 3313fdeb32 test_builtin.py test_compile unit test fix (#5251)
* compile accepts memoryview now

* Update test_compile to what it is in CPython3.12

* compile optimize flag

* added undocumented flags to flag validator
2024-04-23 14:07:02 +09:00
Jeong, YunWon 49cfcd817d remove some skipIf and enable shutil CI for windows (#5235)
* mark failing shtuil tests in windows

* unsetenv raise more informed error

* Remove a few skipIf for windows
2024-04-20 03:00:27 +09:00
Moreal 21c5eae717 Add testcase for __ne__ method 2024-04-14 13:07:13 +09:00
Ankit Kumar Pandey 058f8c5500 Add fork support functions (#4972) 2023-06-01 15:29:16 +09:00
Ankit Kumar Pandey 322aa6887a add support for os.fork and related functions (#4877) 2023-04-25 02:39:54 +09:00
Jaeyoung Ahn 096a3dc645 Fix builtins (has,get,set)attr type error messages (#4776) 2023-03-27 15:22:11 +09:00
JaeyoungAhn 44da572893 uncomment annotation 2023-03-25 22:19:15 +09:00
JaeyoungAhn f750c0ca06 added back the skipIf for windows 2023-03-25 22:13:50 +09:00
JaeyoungAhn 0873e451a8 update test_builtin.py from cpython v3.11.2 2023-03-25 20:30:40 +09:00
JaeyoungAhn 0eea760443 Update test_builtin.py from CPython v3.11.2 2023-03-25 20:17:27 +09:00
yt2b 42e6eecf7f Remove the decorator 2022-12-28 22:18:21 +09:00
Devon Hollowood 052b87b6e7 Fix compatibility bug for min/max builtins
This was as simple as fixing the error messages produced when zero
arguments are passed.
2022-10-19 22:34:15 -07:00
Gyubong e9970edd77 Add built-in breakpoint function (#4122) 2022-08-23 21:45:01 +09:00
Jeong Yunwon 3d27f163e4 adjust failing markers for test_{builtin,decimal,float,range} 2022-07-18 04:38:26 +09:00
CPython developers 33690be35b Update tests from CPython 3.10.5 2022-07-18 04:07:35 +09:00
Jeong YunWon bd4f7ab3f9 Merge pull request #3734 from gnsxun/map-mod
Add map.reduce()
2022-05-22 13:52:54 +09:00
gnsxun d1f22fe4c3 delete expectedFailtures 2022-05-22 12:40:57 +09:00
Lithium 53185cdfb8 Fixed test_bytearray_translate Error (#3730) 2022-05-22 10:57:12 +09:00
Hyun Park dfa4bdc695 add filter.__reduce__ (#3732) 2022-05-21 19:08:20 +09:00
Moreal d6236dcdcc Unmark correct tests 2022-04-24 05:31:01 +09:00
Dean Li 29c953d9a6 test: update test.support and test_io to 3.10
Initially this is only meant to be a small update for `test_io.py`.
But it turns out that `test.support` need a lot of cleanup especially
for `test.support.warnings_helper`.
2022-04-18 22:50:55 +08:00
Daniel Watkins 2960ebc0d1 test/test_builtin.py: run most of test_type_name
The remaining failures are due to what appears to be a fundamental
difference between how RustPython and CPython handle surrogate
characters/codepoints-which-aren't-characters.  They are wrapped with
`self.assertRaises(AssertionError)` so we will notice once they start
passing.

https://github.com/RustPython/RustPython/issues/935 is most likely
related.
2022-04-16 00:38:12 +09:00
Dean Li 5ee4fb899b test: use os_helper 2021-11-28 20:51:32 +08:00
Jeong YunWon 913b78ca44 Revert "Merge pull request #3433 from deantvv/test-update"
This reverts commit 9fa5c5ac66, reversing
changes made to e7fa32c687.
2021-11-17 17:06:51 +09:00
Dean Li 49a5805d11 test: use os_helper 2021-11-13 02:18:33 +00:00