40 Commits
Author SHA1 Message Date
Jeong, YunWon 38de7462c0 Fix Constants newtype usage in init_cleanup_code 2026-03-20 22:47:55 +09:00
Jeong, YunWon 76e6ece941 address review: check datastack space for extra_bytes, require CO_OPTIMIZED in vectorcall fast path 2026-03-20 22:47:55 +09:00
Jeong, YunWon fb0dfa102c address review: invalidate init cache on type modification, add cspell words 2026-03-20 22:47:55 +09:00
Jeong, YunWon 2ea5960998 update cspell config (#7452)
* add allowCompoundWords

* cspell dict
2026-03-17 20:18:31 +09:00
Jeong, YunWon 2bb9173caf Suspend Python threads before fork() (#7364)
* Suspend Python threads before fork()

Add stop-the-world thread suspension around fork() to prevent
deadlocks from locks held by dead parent threads in the child.

- Thread states: DETACHED / ATTACHED / SUSPENDED with atomic CAS
  transitions matching _PyThreadState_{Attach,Detach,Suspend}
- stop_the_world / start_the_world: park all non-requester threads
  before fork, resume after (parent) or reset (child)
- allow_threads (Py_BEGIN/END_ALLOW_THREADS): detach around blocking
  syscalls (os.read/write, waitpid, Lock.acquire, time.sleep) so
  stop_the_world can force-park via CAS
- Acquire/release import lock around fork lifecycle
- zero_reinit_after_fork: generic lock reset for parking_lot types
- gc_clear_raw: detach dict instead of clearing entries
- Lock-free double-check for descriptor cache reads (no read-side
  seqlock); write-side seqlock retained for writer serialization
- fork() returns PyResult, checks PythonFinalizationError, calls
  sys.audit
2026-03-07 20:20:16 +09:00
Jeong, YunWon 2f2a7da185 Add method cache (MCACHE) for MRO lookups (#7313)
4096-entry direct-mapped cache keyed by (tp_version_tag, interned_name_ptr)
using lock-free SeqLock pattern for concurrent access.

- find_name_in_mro() checks cache before MRO walk
- has_name_in_mro() avoids cloning on cache hit
- Auto-assigns version tags on cache miss
- Invalidation via modified() (tp_version_tag=0) with value drops
- TYPE_CACHE_CLEARING flag suppresses re-population during GC drops
- Clear method cache during GC collection to break reference cycles
2026-03-03 23:32:15 +09:00
Jeong, YunWon c315033091 Implement LOAD_ATTR inline caching with adaptive specialization (#7292)
* Implement LOAD_ATTR inline caching with adaptive specialization

Add type version counter (tp_version_tag) to PyType with subclass
invalidation cascade. Add cache read/write methods (u16/u32/u64)
to CodeUnits. Implement adaptive specialization in load_attr that
replaces the opcode with specialized variants on first execution:

- LoadAttrMethodNoDict: cached method lookup for slotted types
- LoadAttrMethodWithValues: cached method with dict shadow check
- LoadAttrInstanceValue: direct dict lookup skipping descriptors

Specialized opcodes guard on type_version_tag and deoptimize back
to generic LOAD_ATTR with backoff counter on cache miss.

* Add BINARY_OP and CALL adaptive specialization

BINARY_OP: Specialize int add/subtract/multiply and float
add/subtract/multiply with type guards and deoptimization.

CALL: Add func_version to PyFunction, specialize simple
function calls (CallPyExactArgs, CallBoundMethodExactArgs)
with invoke_exact_args fast path that skips FuncArgs
allocation and fill_locals_from_args.

* Lazy quickening for adaptive specialization counters

Move counter initialization from compile-time to RESUME execution,
matching CPython's _PyCode_Quicken pattern. Store counter in CACHE
entry's arg byte to preserve op=Instruction::Cache for dis/JIT.
Add PyCode.quickened flag for one-time initialization.

* Add Instruction::deoptimize() and CodeUnits::original_bytes()

- deoptimize() maps specialized opcodes back to their base adaptive variant
- original_bytes() produces deoptimized bytecode with zeroed CACHE entries
- co_code now returns deoptimized bytes, _co_code_adaptive returns current bytes
- Marshal serialization uses original_bytes() instead of raw transmute

* Fix monitoring and specialization interaction

- cache_entries() returns correct count for instrumented opcodes
- deoptimize() maps instrumented opcodes back to base
- quicken() skips adaptive counter for instrumented opcodes
- instrument_code Phase 3 deoptimizes specialized opcodes and
  clears CACHE entries to prevent stale pointer dereferences

* Address review: bounds checks, UB fix, version overflow, error handling

- Add bounds checks to read_cache_u16/u32/u64
- Fix quicken() aliasing UB by using &mut directly
- Add JumpBackwardJit/JumpBackwardNoJit to deoptimize()
- Guard can_specialize_call with NEWLOCALS flag check
- Use compare_exchange_weak for version tag to prevent wraparound
- Propagate dict lookup errors in LoadAttrMethodWithValues
- Apply adaptive backoff on version tag assignment failure
- Remove duplicate imports in frame.rs
2026-03-02 16:36:57 +09:00
Jeong, YunWon 6950baf687 more algorithm-independent GC infra (#7194)
* mark poluting tests

* GC-infra independent to EBR

* trashcan

* add overflow guard to inc(), #[must_use] on dec()/safe_inc(), trashcan debug_assert, weakref generic re-check
2026-02-22 21:31:42 +09:00
Jeong, YunWon 7b866f66e9 Add missing _winapi functions (#7185)
* Add missing _winapi functions and fix WinHandle int conversion

Add 13 functions: ReadFile, SetNamedPipeHandleState, CreateFileMapping,
OpenFileMapping, MapViewOfFile, UnmapViewOfFile, VirtualQuerySize,
CopyFile2, ResetEvent, CreateMutex, OpenEventW, LoadLibrary,
_mimetypes_read_windows_registry.

Add constants: INVALID_HANDLE_VALUE, FILE_MAP_READ/WRITE/COPY/EXECUTE.

Change WinHandle integer type from usize to isize so negative values
like INVALID_HANDLE_VALUE (-1) can be passed from Python.

* Align _winapi module with CPython

- Rename winapi.rs to _winapi.rs with #[path] attribute
- Rename CreateMutex to CreateMutexW
- Add missing constants: ERROR_ACCESS_DENIED, ERROR_PRIVILEGE_NOT_HELD,
  PROCESS_ALL_ACCESS, 10 STARTF_ constants, LOCALE_NAME_SYSTEM_DEFAULT,
  LOCALE_NAME_USER_DEFAULT, COPY_FILE_DIRECTORY
- Fix OpenMutexW return type and ReleaseMutex param type to use WinHandle

* Fix ReadFile/WriteFile overlapped keyword argument

Use FromArgs structs so overlapped parameter can be passed as
a keyword argument (overlapped=True), matching the CPython API.

* Remove extra constants and LoadLibrary not in CPython _winapi

Remove 19 constants (WAIT_ABANDONED, CREATE_ALWAYS, CREATE_NEW,
OPEN_ALWAYS, TRUNCATE_EXISTING, FILE_ATTRIBUTE_NORMAL, 8 FILE_FLAG_*,
3 FILE_SHARE_*, NMPWAIT_NOWAIT, NMPWAIT_USE_DEFAULT_WAIT) and
LoadLibrary function that are not present in CPython's _winapi module.

* Fix utf8_mode default to 0 and add PYTHONUTF8 env var support

Default utf8_mode was incorrectly set to 1, causing text-mode
subprocess to always decode as UTF-8 instead of locale encoding.
Changed default to 0 to match CPython 3.13 behavior on Windows.
Added PYTHONUTF8 environment variable handling with -X utf8 override.

* Fix CopyFile2 to raise proper OSError subclass

Use std::io::Error::from_raw_os_error instead of vm.new_os_error so
that winerror attribute is set and errno-to-exception mapping works
(e.g. ERROR_ACCESS_DENIED → PermissionError).

* Fix syntax_non_utf8 test to not depend on locale encoding

Use explicit encoding='latin-1' so the test works regardless of
the system locale (e.g. C/POSIX locale uses ASCII by default).
2026-02-22 19:54:27 +09:00
Jeong, YunWon 38283becaa Fix typing, typevar, genericalias, and symboltable (#7091)
- TypeVar/ParamSpec repr: use infer_variance flag
- ParamSpec: add type_check on bound argument
- ParamSpecArgs/Kwargs: use equality instead of identity
- NoDefault: change to IMMUTABLETYPE flag
- subscript_generic: wrap TypeVarTuple in Unpack
- symboltable: selective name mangling in type param scopes
- symboltable: fix double default scanning for non-generic fns
- Unmark 4 passing tests in test_type_params
2026-02-14 09:15:57 +09:00
Jeong, YunWonandgithub-actions[bot] f777416870 initial sandbox (#7035)
* Add host_env feature for sandbox isolation

Introduce a `host_env` feature flag that gates all host environment
access (filesystem, network, signals, processes). When disabled,
the VM operates in sandbox mode:

- _io module always available; FileIO gated by host_env
- SandboxStdio provides lightweight stdin/stdout/stderr via Rust std::io
- BytesIO/StringIO/BufferedIO/TextIOWrapper work without host_env
- open() returns UnsupportedOperation in sandbox
- stdlib modules (os, socket, signal, etc.) gated by host_env
- CI checks both host_env ON and OFF builds

* Auto-format: ruff check --select I --fix

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-02-08 13:58:30 +00:00
Jeong, YunWon 4ca2da4987 Merge pull request #6838 from youknowone/asyncio
Add _asyncio and add test_asyncio
2026-01-29 22:35:30 +09:00
Jeong, YunWon 95abec4c21 More Load instructions including LoadLocals, LoadFastBorrow (#6886)
* fix stdlib_io

* more scpell dict

* LoadLocals

* LoadFastBorrow

* more ops
2026-01-29 19:33:46 +09:00
Jeong, YunWon fcca0feb70 Add missing windows APIs 2026-01-29 16:48:33 +09:00
8443b2c97e Add POSIX shared memory module for multiprocessing on Unix (#6498)
---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: youknowone <69878+youknowone@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-12-25 14:54:44 +09:00
Jeong, YunWon 70b93898d4 ctypes overhaul (#6450) 2025-12-19 14:10:55 +09:00
Jeong, YunWon 6a3dff63bb Downcastable (#5986)
* simplify int power

* downcastasble

* deprecate payload*
2025-07-16 18:00:18 +09:00
Jeong YunWon e25c2856ad cspell ignore target/** 2025-07-01 19:05:29 +09:00
Lee Dogeon 910077d7ae Implement _stat module (#5847)
* Implement _stat module

* Allow 'FIRMLINK' term
2025-06-28 03:15:46 +09:00
Jeong YunWon 3673372d3d Fix cspell warnings 2025-06-27 12:32:59 +09:00
Jeong, YunWon 4ae2936a45 fix more cspell warnings (#5689) 2025-04-11 12:08:07 +09:00
Ashwin Naren b81ae9b954 More cspell fixes (#5670) 2025-04-11 09:37:20 +09:00
Jeong YunWon d7113e11db Fix more cspell warnings 2025-04-04 21:45:03 +09:00
Jeong YunWon 7ac61f3840 fix cspell warnings 2025-04-04 16:15:54 +09:00
Jeong YunWon 10d2837041 rework dicts 2025-03-27 02:15:03 +09:00
Jeong YunWon 372e683063 disable cspell from files with a bunch of OS jargons 2025-03-27 01:47:17 +09:00
Jeong YunWon 549cce24c8 Add #[pystruct(skip)] 2025-03-26 22:40:43 +09:00
Jeong YunWon 6ca4685fee __objclass__ 2024-07-29 14:30:46 +09:00
Jeong, YunWon d975c51b96 implement more warnings (#5077) 2023-10-04 23:42:37 +09:00
Jeong YunWon e95eafe02a clean up io.{String,Bytes}IO 2023-03-19 04:57:07 +09:00
Jeong YunWon b35861005e more cspell (#4702) 2023-03-17 14:02:24 +09:00
Jeong YunWon 310a83427c cspell derive 2023-03-16 22:42:03 +09:00
Jeong YunWon 21afc1933a Update cspell for compiler 2023-03-16 22:39:09 +09:00
Jeong YunWon e800af70bb update cspell dict 2023-03-16 22:08:14 +09:00
Jeong YunWon 089c8b7b0a Refactor zlib using new Level type 2023-03-16 00:52:39 +09:00
Jeong YunWon 71e11950d9 rename preconversor -> conversion to follow cpython name 2023-03-01 04:51:47 +09:00
Jeong YunWon 10de265b4f Add basic _imp.find_frozen 2023-02-24 14:34:50 +09:00
Jeong YunWon daca1f38cf Fix cspell warnings 2023-02-18 14:02:18 +09:00
Jeong YunWon d4207e2936 Add test_generator_expression_argument 2023-02-11 05:20:39 +09:00
Jeong YunWon 1848c45302 first cspell dict 2023-01-09 19:57:23 +09:00