40 Commits
Author SHA1 Message Date
youknowdot 7d15b50cd4 Align weak-reference collection with Python 3.15 finalization (#9168)
* Align weak-reference collection with Python 3.15 finalization

Trace and retain owned callback edges independently of weak referent links.
Clear callback-bearing weakrefs before finalizers, suppress callbacks from
unreachable weakrefs, and close remaining weakref ingress after resurrection
checks while the world is stopped. Publish and unlink intrusive nodes safely.

Refresh weakref and canonical weakref/GC/finalization tests through update_lib
from pinned CPython 3.15.0. Retire four obsolete guards only after fresh body
passes, retaining applicable RustPython limitations.

Assisted-by: OpenAI Codex:model version unavailable

* Refresh canonical IO tests for weakref finalization

Migrate the complete CPython 3.15.0 test_io package at d220409 with the
fail-closed legacy-map updater, preserving all ten existing guards. Retire
the seven consolidated standalone suites and update io/file dependency
paths with mapping regression tests.

The canonical GC tests ignore ResourceWarning without retaining its
source, so callback-free weakrefs correctly reflect surviving owners.
This closes the stale test dependency exposed by the lifecycle change.

Assisted-by: OpenAI Codex:model version unavailable

* Protect weakref callbacks with the original referent stripe

Follow CPython's free-threading weakref locking model instead of keeping
an independent callback RwLock. Retain the original referent stripe key
so callback access stays synchronized after GC clears the referent.
Unlink and steal callbacks under that stripe, then release ownership
outside the lock. Preserve synchronization for running-world GC paths
and gc.get_referents while keeping ordinary weakref upgrades unchanged.

Assisted-by: OpenAI Codex:model version unavailable

* Defer weakref GC tracking until referent publication

Keep freshly allocated weakrefs out of GC's object lists until they are
linked and their referent is published. Release the referent stripe before
tracking, since tracking can collect and invoke finalizers. Generic reuse
losers remain untracked.

Exercise the allocator boundary directly for reference, proxy, callable
proxy and subclass payloads, and ensure completed constructors track their
objects. This prevents observers from retaining a half-built dead weakref
that later becomes live.

Assisted-by: OpenAI Codex:model version unavailable

* Retain the original weakref stripe lock like CPython

Store a safe handle to the selected static weakref stripe instead of
retaining and rehashing the referent address. Keep the same lock after
referent clearing, matching CPython free-threaded weakrefs_lock. The
nonthreaded handle is zero-sized and introduces no locking.

Preserve lock selection, acquire/release ordering, fork reset, deferred
GC publication, and callback-owner release outside the stripe. No new
locks or unsafe code are needed.

Assisted-by: OpenAI Codex:model version unavailable
2026-10-17 23:53:07 +09:00
김은빈 8b65a5455f feat: reuse weak callable proxy (#8618) 2026-08-30 17:28:46 +09:00
Shahar Naveh a6d88b144e Update some libs and tests to 3.14.7 (#8593)
* test_weakref

* test_inspect

* socket

* test_copy

* pickle

* test_ast

* string

* zstd

* test_py_compile

* test_os

* mark failing test

* mark failing

* fix markers
2026-08-25 22:01:24 +09:00
김은빈 d26972e3bc weakref: reuse proxy objects (#8307)
* weakref: reuse callback-less proxy objects like CPython

Make PyWeakProxy a #[repr(transparent)] view sharing PyWeak's payload
(manual PyPayload with shared PAYLOAD_TYPE_ID + class check), so a proxy
is the weakref-list node itself and add()'s reuse applies to it
directly. Port CPython's insert_weakref/get_basic_refs slot discipline:
genericref at head, generic proxy right after it, with callback/class
re-verificationbefore reuse. Constructor moves to a slot_new override
since reuse returns an existing object. Drops the hidden __weakproxy
marker subclass and the proxy branch in PyWeakref_GetRef.

Assisted-by: Claude Code:claude-opus-4-8

* weakref: always validate class in basic-proxy slot lookup

* weakref: rustfmt find_generic_proxy_ptr

* weakref: unify reuse check shape
2026-07-18 23:47:03 +09:00
김은빈 abc35e5f83 weakref: treat explicit None callback as no callback (#8298) 2026-07-17 02:51:36 +00:00
Copilotandyouknowone 76755524df Add __callback__ property to weakref type (#7590)
* Initial plan

* Add __callback__ property to PyWeak for weakref compatibility

Add get_callback() method to PyWeak in core.rs that safely reads the
callback field under the stripe lock. Expose it as a read-only
__callback__ property on the weakref type, matching CPython behavior:
- Returns the callback function when one was provided and referent is alive
- Returns None when no callback was provided
- Returns None after the referent has been collected
- Raises AttributeError on assignment (read-only)

Remove @unittest.expectedFailure from test_callback_attribute and
test_callback_attribute_after_deletion in test_weakref.py.

Add regression tests to extra_tests/snippets/stdlib_weakref.py.

Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/a8689daa-4476-4645-a935-0e13c7f7bb42

Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>

* Fix ruff formatting in stdlib_weakref.py snippet

Agent-Logs-Url: https://github.com/RustPython/RustPython/sessions/4995198f-e083-4dac-823a-166fcf54adc4

Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
2026-04-13 23:58:58 +09:00
김은빈 85d4aeaa45 Fix weakref proxy number protocol, right-side ops, and non-iterator error (#7410) 2026-03-13 11:36:16 +09:00
Jeong, YunWon 99060c1dcd Object header slimming: prefix allocation for ObjExt (#7334)
* Object header slimming: prefix allocation for ObjExt

Extract dict, weak_list, and slots fields from PyInner<T> into a
separate ObjExt struct allocated as a prefix before PyInner using
Layout::extend(). Objects that don't need these fields (int, str,
float, list, tuple, dict, etc.) skip the prefix entirely.

- Add HAS_WEAKREF flag to PyTypeFlags for per-type weakref control
- Add HAS_EXT bit to GcBits indicating prefix presence
- Define ObjExt struct with dict, weak_list, slots
- Shrink PyInner header from ~80-88 bytes to ~32 bytes for lightweight objects
- Update all accessor methods to go through ext_ref()
- Update bootstrap type hierarchy to use prefix allocation
- Add __weakref__ getset descriptor for heap types
- Set HAS_WEAKREF on builtin types that support weak references
- Remove test_weak_keyed_bad_delitem expectedFailure (now passes)

* Add HAS_WEAKREF to _asyncio Future/Task, rename weakref helpers

- Add HAS_WEAKREF flag to PyFuture and PyTask (matches CPython)
- Rename subtype_getweakref/setweakref to subtype_get_weakref/set_weakref
  to fix cspell unknown word lint

* Add HAS_WEAKREF to array, deque, _grouper; remove expectedFailure markers

- Add HAS_WEAKREF to PyArray and PyDeque (matches CPython)
- Add HAS_WEAKREF to PyItertoolsGrouper (internal use by groupby)
- Remove 6 expectedFailure markers from test_dataclasses for weakref/slots
  tests that now pass

* Add HAS_WEAKREF to code, union, partial, lock, IO, mmap, sre, sqlite3, typevar types

Add HAS_WEAKREF flag to built-in types that support weakref:
- PyCode, PyUnion, PyPartial, Lock, RLock
- All IO base/concrete classes (_IOBase, _RawIOBase, _BufferedIOBase,
  _TextIOBase, BufferedReader, BufferedWriter, BufferedRandom,
  BufferedRWPair, TextIOWrapper, StringIO, BytesIO, FileIO,
  WindowsConsoleIO)
- PyMmap, sre Pattern, sqlite3 Connection/Cursor
- TypeVar, ParamSpec, ParamSpecArgs, ParamSpecKwargs, TypeVarTuple

Remove 3 expectedFailure markers from test_descr for now-passing tests.

* Add HAS_DICT to type flags and handle non-METHOD/CLASS in descr_get

- Add HAS_DICT flag to PyType (type metaclass) alongside existing
  HAS_WEAKREF. All type objects are instances of type and need dict
  support, matching CPython's PyType_Type.
- Replace unimplemented!() in PyMethodDescriptor::descr_get with
  fallback to bind obj directly, matching CPython's method_get which
  uses PyCFunction_NewEx for non-METH_METHOD methods.

* Fix ext detection, HeapMethodDef ownership, WASM error

- Remove HAS_EXT gc_bits flag; detect ext from type flags
  using raw pointer reads to avoid Stacked Borrows violations
- Store HeapMethodDef owner in payload instead of dict hack
- Clear dict entries in gc_clear_raw to break cycles
- Add WASM error fallback when exception serialization fails
2026-03-06 01:19:22 +09:00
Jeong, YunWon 6cf1d32f93 skip flaky weakref test (#7286) 2026-03-01 15:28:23 +09:00
Jeong, YunWon 8a74956590 more algorithm-independent GC infra (#7194)
* mark poluting tests

* GC-infra independent to EBR

* trashcan

* add overflow guard to inc(), #[must_use] on dec()/safe_inc(), trashcan debug_assert, weakref generic re-check
2026-02-22 21:31:42 +09:00
Jeong, YunWon 7271f56c28 skip flaky test_threaded_weak_key_dict_copy (#7010) 2026-02-05 23:47:09 +09:00
Jeong, YunWon fea26e456f impl more nt (#6984)
* mpl new features

* windows encodings

* impl nt functions

* revert

* codecs

* fix codecs
2026-02-04 09:53:02 +09:00
Jeong, YunWonandgithub-actions[bot] 60bec8a561 rework weakref (#6916)
* Replace WeakListInner with inline atomic weakref list and stripe locks

Remove heap-allocated WeakListInner (OncePtr<PyMutex<WeakListInner>>).
WeakRefList now holds two inline atomic pointers (head, generic).
PyWeak.parent replaced with wr_object pointing directly to referent.
Add weakref_lock module with AtomicU8 spinlock array for thread safety.
Rewrite upgrade/clear/drop_inner/count/get_weak_references with stripe lock.
Make Pointers methods public in linked_list.rs.
Remove expectedFailure from test_subclass_refs_dont_replace_standard_refs.

* Auto-format: cargo fmt --all

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-02-01 12:12:33 +09:00
Jeong, YunWon 714d1ce58b gc module internal structure and API (#6910)
* gc module internal structure and API

Add gc_state module with GcState, GcGeneration, GcDebugFlags, GcStats.
Replace gc module stubs with working API backed by gc_state.
Add gc_callbacks and gc_garbage to Context.
Add is_gc_tracked, gc_finalized, gc_get_referents to PyObject.
Collection is stubbed (returns 0) — actual algorithm to follow.

* fix dict/weakref/generators

* unmark test_asyncio

* apply review
2026-02-01 08:51:39 +09:00
CPython Developers 0446bc71a0 Update weakref from v3.14.2 2026-01-30 19:21:10 +09:00
Jeong, YunWon 479bb7af1a skip weakref crash test 2026-01-20 17:33:07 +09:00
Terry Tianlin Luan 68d65a7280 Updated token + weakref libraries + weakref test - v3.13.11 (#6633)
* Updated the token library

* Updated the weakref library

* Annotated the failing weakref test
2026-01-04 10:17:12 +09:00
Jeong, YunWon 3d7e521acd introduce slot_wrapper (#4884) 2025-12-24 17:29:47 +09:00
Jiseok CHOI ff35dcd95a feat(vm/slot): implement Py_TPFLAGS_MANAGED_DICT for class objects (#5949) 2025-07-20 14:11:36 +09:00
Shahar Naveh 97e85b220e Update test_{dict,weakref}.py from 3.13.5 (#5963)
* Update test_dict.py from 3.13.5

* Update `test_weakref.py` from 3.13.5
2025-07-14 14:21:59 +09:00
Jeong YunWon d1b7dc551c skip flaky test_weakref 2025-04-30 14:32:43 +09:00
jinnarajin 050f0bd0ad weakref.__reversed__ fixed (#4780) 2023-05-19 23:43:07 +09:00
jyj afffb13902 Add Hash of WeakProxy for test\test_Weakref.py's "test_proxy_hash "
Fix Hash of WeakProxy : remove "?" ending of the code

Fix weakproxy.rs for test_proxy_hash to solve error
2023-03-26 12:12:38 +09:00
jyj 95f3c0d69d Add Iter of WeakProxy for test\test_weakref.py's "test_proxy_next" 2023-03-26 11:32:36 +09:00
Jeong YunWon bad1c6f0b8 adjust filaing test markers 2022-07-25 04:10:59 +09:00
CPython developers 1d92f78b4c Update test_weakref from CPython 3.10.5 2022-07-25 04:10:59 +09:00
KaratusandKim, Yeonwoo 3664986629 Fix weakref richcompare (#3913)
Co-Authored-By: Kim, Yeonwoo <oow214@gmail.com>
2022-07-18 20:59:46 +09:00
Kim, YeonWoo 4ac218f90e Add weakproxy bytes 2022-07-16 14:34:28 +09:00
Kim, YeonWoo 27884e2d5f Add weakproxy bool 2022-07-16 13:47:35 +09:00
Kim, YeonWoo 21a7369953 Add weakproxy mapping 2022-07-15 00:49:20 +09:00
Dean Li 14727e6432 test: mark win32 failure test 2021-11-29 21:03:03 +08:00
Noa 2a87b893fb Reuse standard no-cb weakref 2021-11-15 16:08:47 -06:00
Noa f9d9a1e15e Implement weakref._remove_dead_weakref 2021-11-15 16:08:47 -06:00
Noa ec0a658d20 Implement weakref.getweakrefs 2021-11-15 16:08:47 -06:00
Noa e51cb82ff0 Unskip tests 2021-11-15 16:08:46 -06:00
jfh 103195b277 Remove Python gc stub, add rust stub. 2021-10-21 19:56:20 +03:00
Padraic Fanning abead8c553 Skip doctests 2021-07-10 11:48:57 -04:00
Padraic Fanning 245869b01f Mark erroring/failing tests 2021-07-09 23:55:41 -04:00
Padraic Fanning 911b2b61d1 Skip panicking tests 2021-07-09 23:22:42 -04:00
Padraic Fanning f4011b1662 Add test_weakref from CPython 3.8 2021-07-09 23:22:41 -04:00