Commit Graph
53 Commits
Author SHA1 Message Date
youknowdot 7d15b50cd4 Align weak-reference collection with Python 3.15 finalization (#9168)
* Align weak-reference collection with Python 3.15 finalization

Trace and retain owned callback edges independently of weak referent links.
Clear callback-bearing weakrefs before finalizers, suppress callbacks from
unreachable weakrefs, and close remaining weakref ingress after resurrection
checks while the world is stopped. Publish and unlink intrusive nodes safely.

Refresh weakref and canonical weakref/GC/finalization tests through update_lib
from pinned CPython 3.15.0. Retire four obsolete guards only after fresh body
passes, retaining applicable RustPython limitations.

Assisted-by: OpenAI Codex:model version unavailable

* Refresh canonical IO tests for weakref finalization

Migrate the complete CPython 3.15.0 test_io package at d220409 with the
fail-closed legacy-map updater, preserving all ten existing guards. Retire
the seven consolidated standalone suites and update io/file dependency
paths with mapping regression tests.

The canonical GC tests ignore ResourceWarning without retaining its
source, so callback-free weakrefs correctly reflect surviving owners.
This closes the stale test dependency exposed by the lifecycle change.

Assisted-by: OpenAI Codex:model version unavailable

* Protect weakref callbacks with the original referent stripe

Follow CPython's free-threading weakref locking model instead of keeping
an independent callback RwLock. Retain the original referent stripe key
so callback access stays synchronized after GC clears the referent.
Unlink and steal callbacks under that stripe, then release ownership
outside the lock. Preserve synchronization for running-world GC paths
and gc.get_referents while keeping ordinary weakref upgrades unchanged.

Assisted-by: OpenAI Codex:model version unavailable

* Defer weakref GC tracking until referent publication

Keep freshly allocated weakrefs out of GC's object lists until they are
linked and their referent is published. Release the referent stripe before
tracking, since tracking can collect and invoke finalizers. Generic reuse
losers remain untracked.

Exercise the allocator boundary directly for reference, proxy, callable
proxy and subclass payloads, and ensure completed constructors track their
objects. This prevents observers from retaining a half-built dead weakref
that later becomes live.

Assisted-by: OpenAI Codex:model version unavailable

* Retain the original weakref stripe lock like CPython

Store a safe handle to the selected static weakref stripe instead of
retaining and rehashing the referent address. Keep the same lock after
referent clearing, matching CPython free-threaded weakrefs_lock. The
nonthreaded handle is zero-sized and introduces no locking.

Preserve lock selection, acquire/release ordering, fork reset, deferred
GC publication, and callback-owner release outside the stripe. No new
locks or unsafe code are needed.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-17 23:53:07 +09:00
youknowdot fc9f4a9541 Update importlib to CPython 3.15 with native bootstrap callback (#9169)
* Refresh canonical importlib and preserve native import callback

Cache the original builtin __import__ callback so CPython 3.15 importlib's
hierarchical locking uses the existing native parent-initialization path.
Refresh the complete canonical importlib and related test/helper families
from CPython d220409a13b77de7a6c30abe16ea90db01528edc through update_lib.

Preserve both nondecorator importlib guards in the updater. Add explicit
hash-pinned removal of reviewed obsolete canonical files, with full
preflight, dry-run, idempotence and refusal tests. Preserve empty fixtures.

Assisted-by: OpenAI Codex:model version unavailable

* Complete canonical inspect and types closure for importlib metadata

Refresh whole canonical inspect/types production and related test families
through update_lib after hosted CI exposed stale __cached__ consumers.
Add coroutine and async-generator state getters, with closed-state priority
for reentrant frame-local finalizers, and expose the existing frame-locals
proxy type with its CPython class documentation.

Preserve the optional _datetime and unittest import blocks through a narrow
migration adapter. Retain canonical assertions and existing guards; record
the missing LazyImportType expectation before adding a conditional expected
failure, and capability-skip only the two absent-_testcapi CLI fixtures.

Assisted-by: OpenAI Codex:model version unavailable

* Reject canonical removal junctions and read UTF-8 manifests

Reject junctions alongside symlinks in the existing path-component preflight and revalidation. Decode removal manifests explicitly as UTF-8. Add regressions for whole-batch refusal, revalidation, and Unicode dry-run/apply/idempotence.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-17 14:40:13 +09:00
youknowdot 19d2fe3af7 Align bytes and bytearray operations with Python 3.15 (#9165)
Correct quote escaping, replacement count keyword binding, signed-minimum hex grouping, and clear storage/export handling. Migrate canonical bytes coverage through fail-closed preservation of the existing RustPython customizations, and retire only two freshly verified formatting guards.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-17 11:33:14 +09:00
youknowdot 1f4e9ffaf6 Align OS argument conversion and syscall handling with Python 3.15 (#9146)
* Align OS argument conversion and syscall handling with Python 3.15

Import canonical os tests through update_lib, retaining existing guards. Preserve raw descriptor validation, errno, audit ordering, signal retries and unlocked sendfile buffers.

Validation: independent source review passed; matching release build and nine bounded canonical cases pending.

Assisted-by: OpenAI Codex:model version unavailable

* Import canonical root-user test support for OS package

Add hash-pinned selective assignment migration within test.support and import the three Python 3.15.0 privilege helpers plus their public exports. Preserve unrelated support helpers and existing test markers.

Validation: 177 host updater tests pass; byte-identical canonical helper provenance verified. Independent source review passed. Native import validation pending integration onto current main.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-16 10:06:13 +09:00
youknowdot 9e18d7be6b Preserve explicit customizations during canonical library migration (#9142)
* Preserve explicit customizations during canonical library migration

Add bounded runtime, call, and doctest preservation plus explicit remaining-marker metadata. Support mapped legacy test-package migrations without losing ordered guards, and exclude cache directories from both migration paths while retaining canonical assets.

Validation: 53 bounded host-Python tooling tests passed. Tests inspect source and temporary files without executing migrated fixtures. Existing copy-library cache protection is unchanged. No Lib or interpreter changes.

Assisted-by: OpenAI Codex:model version unavailable

* Integrate migration preservation into existing updater modules

Keep migration orchestration and guarded package moves in cmd_migrate, and marker/doctest metadata in patch_spec. Reuse AST anchor validation and existing test modules, removing all eleven newly introduced helper, test, and data files.

Require caller-selected doctest metadata instead of embedding syntax-case policy. Reject guarded legacy doctests before package writes. Preserve the prior 53 test scenarios and add seven focused metadata regressions; all 60 bounded host tooling tests passed. No Lib, workflow, or native changes.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 19:54:38 +09:00
youknowdot 498adff3a9 Align JSON array hooks and libraries with Python 3.15 (#9135)
Generate the JSON library and selected tests with update_lib from CPython d220409a13b77de7a6c30abe16ea90db01528edc, preserving native bootstrap workarounds and test markers. Exclude generated cache directories at the library copy boundary.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 07:32:30 +00:00
youknowdot 92e20da822 Fix update_lib marker preservation for inline classes (#9131)
* Preserve inherited test markers in inline class migrations

Expand canonical inline class suites before appending retained inherited overrides, and validate generated syntax. Run focused migration-tool tests in a path-filtered Python-only CI job.

Assisted-by: OpenAI Codex:model version unavailable

* Run migration tool tests through existing CI conditions

Keep Rust build gates unchanged and run focused Python migration tests on Linux for tool-only changes. Remove the separate workflow.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 13:54:46 +09:00
youknowdot 0f127b07e9 Add Python 3.15 math APIs using pymath (#9122)
* Add Python 3.15 math APIs using pymath

Pin pymath to merged dd1e9a4cf26d68caf3b549c09476f52e356e16a3
and expose isnormal, issubnormal, signbit, fmax and fmin through thin
wrappers. Share the existing integer functions through math.integer
and _math_integer, matching CPython module identity and metadata.

Migrate test_math.py and test_math_integer.py via scripts/update_lib
from CPython d220409a13b77de7a6c30abe16ea90db01528edc, preserving the
existing Windows remainder marker. Add bounded Python snippet coverage.

AI assistance: OpenAI Codex implemented and source-reviewed this change,
ran update_lib, and performed focused automated validation. No human
manual testing is claimed.

Assisted-by: OpenAI Codex:model version unavailable

* Track integer math tests in library updates

Assisted-by: OpenAI Codex:model version unavailable
2026-10-15 10:02:44 +09:00
28393e932d Implement PEP 814 frozendict (#8951)
* Restore consolidated frozendict implementation on current main

Preserve reverse dictionary iterator fixes and the 3.15 frame imports. Keep library refreshes separate pending the runtime dependency sequencing decision.

Assisted-by: OpenAI Codex:model identifier unavailable

* Support scoped canonical test updates in update_lib

Index conditional test classes and methods without crossing namespaces, reject ambiguous targets, and preserve inherited override indentation. Add hash-pinned whole-test copying that verifies execution context and exact source bytes, with focused Python fixtures.

Assisted-by: OpenAI Codex:model identifier unavailable

* Align frozendict copy support and tests with CPython 3.15

Import exact-final copy.py with update_lib; defer Mapping ABC registration to the later library stage. Generate only proven marker corrections, retain observed debug recursion-limit expected failures, and copy the complete canonical builtin-doc discovery test using hash-pinned update_lib copy-test.

Assisted-by: OpenAI Codex:model identifier unavailable

* Preserve decorator rationale during canonical test copying

Reject differing decorator source blocks even when their ASTs match. Cover inline and multiline marker comments, guarded async methods, and CLI no-write behavior.

Assisted-by: OpenAI Codex:model identifier unavailable

* Finish frozendict with borrowed PyAnyDict views

Use the landed PyRef target model for a transparent dict/frozendict header
view, retaining objects explicitly with PyRef<PyAnyDict>. Keep targets
sealed and preserve mutable-only entry points. Borrow frame globals from
the object header rather than movable reference wrappers.

Unify exact-TypeError key diagnostics with final CPython semantics,
including callback failure propagation and no explicit cause. Keep the
Python regressions in builtin_dict and test reference layout/ownership
in Rust. Preserve existing canonical library updates.

Assisted-by: OpenAI Codex:model identifier unavailable

* Align frozendict with CPython and use CPython 3.15 tests

- Name the target type in unhashable key errors through
  PyDict::update_arg (dict_update_arg), replacing the separate
  PyFrozenDict::merge_into copy of the merge logic.
- Split the | operator into dict_or (_PyDict_Or) and frozendict_or.
- Return the argument from frozendict(frozendict) only when the called
  type is frozendict.
- Call through the vectorcall slot in PyCallable::invoke when there
  are no keyword arguments, instead of special-casing frozendict.
- Add ensure_builtins (_PyEval_EnsureBuiltins) for eval/exec, and
  insert __builtins__ for source strings regardless of globals type.
- Read __warningregistry__ and __name__ from stored globals entries and
  accept only None or str for the module name in setup_context.
- Remove the unused PyFrozenDict::from_object.
- Migrate test_dict.py, mapping_tests.py, test_copy.py and
  test_builtin.py to v3.15.0 with update_lib, and add
  test_import/data/syntax_warnings.py.
- Drop builtin_dict.py frozendict tests covered by those modules,
  keeping the descriptor and namespace integration tests.

Assisted-by: Claude Code:claude-opus-5-5

* Update crates/vm/src/builtins/dict.rs

---------

Co-authored-by: Jeong YunWon <jeong@youknowone.org>
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
2026-10-14 16:42:56 +09:00
Shahar Naveh 9c027d4f67 Update test/signalinterproctester.py to 3.14.6 (#8173)
* Update `test/signalinterproctester.py` to 3.14.6

* Add `signalinterproctester.py` as a test dependency of `test_signal.py`
2026-06-26 18:24:20 +09:00
Shahar Naveh 38d17bc12c scripts/update_lib migrate to preserve patches on classes (#8057) 2026-06-10 11:17:50 +09:00
Shahar Naveh 4812ea819e Update test_xpickle.py to 3.14.5 (#7901) 2026-05-18 00:23:34 +09:00
Shahar Naveh 9c5a674fee Update pickle.py to 3.14.5 (#7895)
* Update `pickle.py` to 3.14.5

* Update pickle deps with new file

* Mark failing tests
2026-05-17 19:49:42 +09:00
Shahar Naveh a35079967a Fix comment confusion bug in update_lib (#7802)
* Use `unittest.expectedSuccess` to mark failing tests

* Fix comment find logic

* Add test for it
2026-05-10 00:27:51 +09:00
Shahar Naveh fb6142621f Fix libs deps check (#7732)
* Convert parsing logic to python

* Trigger job. (DROP ME BEFORE MERGE!)

* Missing EOF

* Add found modules to set

* Suggestion by @fanninpm

* Fix missing trailing slash

* Ensure no `.py` at module name

* Strip any file auffix

* Revert "Trigger job. (DROP ME BEFORE MERGE!)"

This reverts commit 8cf9651a24.

* Handle quoted args
2026-05-02 08:14:55 +09:00
Shahar Naveh b1df45bd09 Update test_unicodedata.py to 3.14.4 (#7750)
* Update `test_unicodedata.py` to 3.14.4

* Add test asset

* Mark failing tests

* Add `expectedSuccess` docorator

* Add support for nee decorator in patch_spec

* Reapply patches
2026-05-01 19:48:41 +09:00
Jeong, YunWonandgithub-actions[bot] 1471e7559d [update_lib] fast date lookup for todo (#7299)
* [update_lib] fast date lookup for todo

* add deps

* Auto-format: ruff format

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-02 01:23:52 +09:00
Shahar Naveh cf66ed14b5 Don't return frozenset for modules (#7276) 2026-03-01 09:43:34 +09:00
Shahar Naveh 9b29cb426a More accurate test dependencies (#7261) 2026-02-28 22:11:07 +09:00
Shahar Naveh 5a58e2d038 Reapply "ast.NodeVisitor for import tracking" (#7241)
* Reapply "`ast.NodeVisitor` for import tracking (#7229)" (#7230)

This reverts commit a47572c89e.
2026-02-28 10:37:00 +09:00
CPython Developers 70942d3333 Update test_monitoring from CPython 3.14.3 2026-02-27 04:36:46 +09:00
Jeong, YunWon 35e897e58a Revert "ast.NodeVisitor for import tracking (#7229)" (#7230)
This reverts commit 804a7e42fc.
2026-02-27 03:55:13 +09:00
Shahar Naveh 07159eb9a1 ast.NodeVisitor for import tracking (#7229) 2026-02-27 03:07:57 +09:00
Shahar Naveh 0ea84cdf12 update_lib test format fix + deps for atexit & eintr (#7227)
* update_lib deps for atexit & eintr

* Fix test format
2026-02-26 12:03:47 +09:00
Jeong, YunWon a1f469db16 Update scripts/update_lib/cmd_quick.py 2026-02-13 15:20:19 +09:00
Jeong, YunWon 44d9c70d18 [update_lib] _pyio deps 2026-02-13 14:00:00 +09:00
Jeong, YunWon 8cd5ea073e [update_lib] fix hard_deps resolution and fix commit not to miss test data (#7058)
* [update_lib] fix hard_dep not to unclude other tests

* [update_lib] commit test data
2026-02-09 14:41:15 +09:00
Jeong, YunWon ea1b917712 [update_lib] todo shows last updated date (#7053) 2026-02-09 09:39:01 +09:00
Jeong, YunWon c370438d81 non-code migration 2026-02-05 00:21:34 +09:00
Jeong, YunWon 9104cbdbd4 [update_lib] auto-mark original contents recovery (#6960) 2026-02-02 16:03:26 +09:00
Jeong, YunWonandgithub-actions[bot] ab34f0a1e0 Make auto-mark output deterministic and fix blank line leak (#6957)
* Make auto-mark output deterministic and fix blank line leak

Sort set iteration in build_patches and dict iteration in
_iter_patch_lines Phase 2 so expectedFailure markers are
always added in alphabetical order.

Include preceding blank line in _method_removal_range so
removing a super-call override doesn't leave behind the
blank line that was added with the method.

* deps code

* auto-mark handles crash better

* Auto-format: ruff format

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-02-02 13:53:01 +09:00
Jeong, YunWon 68ebb61f7b fix: Include hard_deps in git commit (#6940)
Bug: update_lib quick ast was not committing _ast_unparse.py
because git_commit() only added lib_path and test_paths,
but not hard_deps.

Fixed by:
- Add hard_deps parameter to git_commit()
- Collect hard_deps from DEPENDENCIES in main()
- Add hard_deps to paths_to_add in git_commit()
2026-02-02 00:47:29 +09:00
Jeong, YunWon 153b0da60d automark add reasons to reason-less expectedFailures and handles parent better (#6936) 2026-02-01 22:58:07 +09:00
Jeong, YunWon 0546bb0410 more deps grouping (#6931) 2026-02-01 17:01:56 +09:00
github-actions[bot] 9338509a71 Auto-format: ruff format 2026-01-30 20:50:37 +00:00
Jeong, YunWon 922b644116 [update_lib] fix async detection 2026-01-31 00:59:29 +09:00
Jeong, YunWon 528d6573e0 [update_lib] Fix async func auto-mark (#6903) 2026-01-31 00:00:41 +09:00
Jeong, YunWon 6a3643cdde Improve deps output (#6874)
* Improve deps output: [x]/[ ] sync status, TODO counts

- Replace ambiguous [+] with [x] (synced) / [ ] (not synced)
- Add (TODO: n) suffix for test files with expectedFailure/skip markers

* Refactor update_lib: extract shared utilities
2026-01-26 18:35:35 +09:00
Jeong, YunWon 9071147db5 [update_lib] Apply test grouping to deps (#6867)
* quick resolve test path too

* Fix migrate resolve

* Remove test/ from deps data

* resolve test to lib
2026-01-24 18:40:57 +02:00
Jeong, YunWon f35791ec64 Fix update_lib tests (#6866) 2026-01-24 22:47:03 +09:00
Jeong, YunWon 0cc8f63849 [update_lib] deps grouping (#6854) 2026-01-24 21:29:11 +09:00
Jeong, YunWon 691d2816f9 update_lib todo also shows test todo (#6859)
also tracking untracked files considered in  #6775
2026-01-24 16:04:36 +09:00
Noa f842fbe25d Use std::fmt::from_fn (#6850)
New in 1.93 - replaces our ad-hoc equivalent types.
2026-01-24 09:21:45 +09:00
Jeong, YunWon 180c68e76b improve deps CI formatting and name resolution (#6847)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Improvements**
* Enhanced test module dependency tracking in automated checks to
properly identify and report test-related dependencies.
* Improved dependency resolution logic with better module name mapping
and deduplication for clearer dependency reporting.
* Refined formatting of dependency information and todo items for
improved readability.
* Dependency review comments now automatically refresh with the latest
information when updated.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-01-24 00:32:32 +09:00
9b56aa5b60 [update_lib] show dependent tests in deps subcommand (#6828)
---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jeong YunWon <jeong@youknowone.org>
2026-01-23 08:13:16 +09:00
Jeong, YunWon d690b2d65e [update_lib] todo subcommand (#6823)
* [update_lib] todo

* better CI comment
2026-01-21 09:00:45 +09:00
Jeong, YunWon 58c804309b [update_lib] show deps (#6821)
* show_deps

* soft deps tree

* show deps CI
2026-01-21 01:25:30 +09:00
Jeong, YunWonandgithub-actions[bot] 5ef8d48e5a update_lib hard dependency resolver (#6817)
* update_lib __init__ handling

* dependency

* Auto-format: ruff format

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-01-20 23:52:28 +09:00
github-actions[bot] f1fd1e9121 Auto-format: ruff format 2026-01-20 11:38:26 +09:00
Jeong, YunWon 41cdc5cd5b BooleanOptionalAction 2026-01-20 11:38:26 +09:00