22 Commits
Author SHA1 Message Date
Jeong, YunWonanddot 2a09ee02bd Align compiler and VM bytecode contracts with CPython 3.15 (#9105)
* Align compiler and VM with final CPython 3.15 bytecode

Reconstruct the compiler-only baseline from validated487ae47 on the current main and frozen mapping prerequisitec78d811. Preserve the full source delta while dropping obsolete prerequisite ancestry; detailed old-to-new provenance is retained in the recovery artifact.

Assisted-by: OpenAI Codex: model identifier unavailable

* Fix final script validation, exception cleanup and private cache contracts

Replay reviewed compiler CI repairs 2191a9d, 1f1fe04, a2dacaf on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable

* Align canonical compiler fixtures and generator frame state

Replay reviewed compiler CI repairs 476bb74 on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable

* Preserve code-name owners and final warning and constant-fold contracts

Replay reviewed compiler CI repairs 6482adf on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable

* Skip unavailable monitoring C API tests before tool registration

Replay reviewed compiler CI repairs 246c309 on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable

* Assert exact final constant-list iterable folding

Replay reviewed compiler CI repairs 7ecedcf on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable

* Align local monitoring events and exception lifecycle state with final 3.15

Replay reviewed compiler CI repairs ec206b1, 191fe58 on the frozen current prerequisite. Intermediate compile/diagnostic corrections are combined where they address the same failure.

Assisted-by: OpenAI Codex: model identifier unavailable

* Align 3.15 code objects and codegen with CPython

- Store co_consts, co_names, co_localsplusnames, co_localspluskinds,
  co_linetable, co_exceptiontable and co_name as objects on PyCode;
  build co_code and the varnames/cellvars/freevars tuples lazily in
  _co_cached, with co_code deoptimized through deopt_code.
- Move the compiler constant cache (_PyCode_ConstantKey,
  const_cache_insert) to builtins/code/const_cache.rs.
- Write every code field with w_object in marshal and pass the read
  field objects to MarshalBag::make_code through CodeFields.
- Emit RETURN_GENERATOR/POP_TOP in enter_scope and place SETUP_CLEANUP
  after them; drop the generator part of insert_prefix_instructions.
- Add PyCode::first_traceable (_co_firsttraceable) for
  _PyFrame_IsIncomplete and sys.monitoring.
- Follow the CPython DISABLE branches in sys.monitoring, the
  check_matched signature in warnings, the 3.15 % format error
  messages for str and bytes, code_returns_only_none, and the FOR_ITER
  stack analysis.
- Compute IMPORT_NAME flags with in_exception_handler.
- Replace the linetable/exceptiontable byte tests in codegen with
  dis(show_positions=True) insta snapshots in _opcode.
- Remove the opcode metadata verification tooling, byte-exact
  scripts, private cache tests and redundant snippets.
- Restore test_code.py and test_monitoring.py to v3.15.0 plus
  RUSTPYTHON markers.

Assisted-by: Claude Code:claude-opus-5-5

* Remove generator snippet checks covered by Lib/test

Drop extra_tests/snippets/generator_iterator_prefix.py. Its loop,
delegation, async, f_back and never-awaited checks are covered by
test_generators, test_coroutines, test_asyncgen, test_yield_from and
test_monitoring. Move the remaining checks into existing files:
genexpr prefix tracing and monitoring into stdlib_sys.py, and the
caller frame seen from the eager iterator into stdlib_sys_getframe.py.

Assisted-by: Claude Code:claude-opus-5-5

* Emit the generator prefix for top-level await modules

Module code compiled with PyCF_ALLOW_TOP_LEVEL_AWAIT and marked
ste_coroutine now starts with RETURN_GENERATOR; POP_TOP. The prefix and
RESUME emission move into emit_scope_prefix, shared by enter_scope and
the module entry points, as in codegen_enter_scope. Fixes
test_builtin.test_compile_top_level_await.

Assisted-by: Claude Code:claude-opus-5-5

* Allow the c_long cast in _opcode.stack_effect

c_long is i32 on Windows, where clippy reports the cast to i32 as
unnecessary.

Assisted-by: Claude Code:claude-opus-5-5

* Update malachite crates to 0.14.0

pymath requires malachite-bigint ^0, so the example projects, which
have no lock file, resolve it to 0.14.0 while the workspace pinned
0.13.0, and fail to build.

Assisted-by: Claude Code:claude-opus-5-5

---------

Co-authored-by: dot <dot@youknowone.org>
2026-10-14 22:38:42 +09:00
Jeaeun Kim 980be72e4f jit: preserve return types for recursive calls (#8499)
Assisted-by: Codex:5.6-sol
2026-08-12 16:24:44 +09:00
Jeaeun Kim 1c9daa0ef6 Add JIT support for returning None (#8479) 2026-08-10 20:45:16 +09:00
Shahar Naveh 9a0291188c Clippy rules (test related) (#7968) 2026-05-25 22:03:18 +09:00
Shahar Naveh acf12468c4 Clippy nursery lints (#7875) 2026-05-15 10:38:36 +00:00
Shahar Naveh 54be99bbc9 Clippy warn uninlined_format_args & redundant_else (#7873) 2026-05-15 16:52:06 +09:00
Shahar Naveh 27dffed1e3 Add map_unwrap_or clippy rule (#7829) 2026-05-12 00:51:23 +09:00
Shahar Naveh c3afe01fb6 Warn on unreachable_pub (#7762) 2026-05-04 10:18:24 +09:00
Jeong, YunWon 844fdf3a8d Bytecode parity (#7536)
* Add CFG block splitting, jump threading, backward jump normalization, genexpr StopIteration wrapper

- split_blocks_at_jumps: split blocks at branch points so each has one exit
- jump_threading: thread jumps through single-jump blocks (flowgraph.c jump_thread)
- Backward conditional jump normalization: invert and create NOT_TAKEN+JUMP block
- Follow empty blocks in jump-to-return optimization (next_nonempty_block)
- Add PEP 479 StopIteration handler to compile_comprehension for generators

* Add ConstantData::Slice and constant slice folding

- Add Slice variant to ConstantData and BorrowedConstant
- Fold constant slices (x[:3], x[1:4]) into LOAD_CONST(slice(...))
- Marshal serialization/deserialization for Slice type
- Box::leak in borrow_obj_constant for PySlice roundtrip

* Add duplicate_exits_without_lineno (disabled) and Block: Clone

Prepare infrastructure for exit block duplication optimization.
Currently disabled pending stackdepth integration.

* Improve codegen bytecode parity
2026-03-30 18:50:58 +09:00
Jeong, YunWon 13a0623eb1 Adapt bitflagset to use enum (#7419) 2026-03-14 09:44:02 +09:00
Shahar Naveh 5bda4fdaca Newtype ConstIdx, Constants (#7358)
* Newtype ConstIdx, Constants

* Set generic
2026-03-09 11:47:24 +09:00
Shahar Naveh 80e0403729 Bytecode enum named oparg (#7294)
* Align real opcodes

* Align pseudoes

* Add `consti` to known words
2026-03-03 21:33:59 +00:00
Jeong, YunWon 1b89b8f793 instruction CACHE (#7251)
* Emit CACHE code units in bytecode to match CPython 3.14

- Add cache_entries() method to Instruction enum
- Emit CACHE code units after opcodes in finalize_code
- Handle NO_LOCATION (line=-1) in linetable for CACHE entries
- Account for CACHE entries in exception table generation
- Skip CACHE entries in VM execution loop (with jump detection)
- Handle CACHE in InstrumentedLine/InstrumentedInstruction/InstrumentedForIter/InstrumentedNotTaken
- Skip CACHE in monitoring instrumentation phases
- Update co_branches() for cache-adjusted offsets
- Restore _cache_format in Lib/opcode.py
- Remove expectedFailure from test_c_call, test_start_offset

* Use relative jump offsets and fix bytecode layout

- Convert jump arguments from absolute to relative offsets
  in frame.rs, monitoring.rs, and stack_analysis
- Add jump_relative_forward/backward helpers to ExecutingFrame
- Resolve pseudo jump instructions before offset fixpoint loop
- Emit NOP for break, continue, pass to match line-tracing
- Fix async for: emit EndAsyncFor with correct target, add NotTaken
- Fix comprehension if-cleanup to use separate block
- Fix super() source range for multi-line calls
- Fix NOP removal to preserve line-marker NOPs
- Fix InstrumentedLine cache skipping after re-dispatch
- Match InstrumentedResume/YieldValue in yield_from_target
- Remove CALL_FUNCTION_EX cache entry from opcode.py
- Remove resolved expectedFailure markers

* Align CPython 3.14 LOAD_GLOBAL null-bit and RERAISE semantics

* Remove redundant CPython-referencing comments

Clean up comments that unnecessarily mention CPython per project
convention. Replace with concise descriptions of the behavior itself.
2026-03-01 14:51:54 +09:00
Jeong, YunWon 061c709434 cold block reordering and jump normalization (#7210)
* cold block reordering and jump normalization

Add mark_cold, push_cold_blocks_to_end, and normalize_jumps
passes to the codegen CFG pipeline. Use JumpNoInterrupt for
exception handler exit paths in try-except-finally compilation.

* mark test_peepholer
2026-02-24 08:53:07 +09:00
Jeong, YunWon e0cceaf31c Remove ReturnConst/Break/Continue ops (#6816)
* Remove ReturnConst

* Remove break/continue ops
2026-01-21 00:03:43 +09:00
Jeong, YunWon eb50246b41 fix annotation (#6767) 2026-01-18 15:24:17 +09:00
Jeong, YunWon 6df3753229 Py 3.14 changes fix (#6755)
* marshal v5

* conditional blclk

* fix jit don't use lossy string

* add varname

* symboltable takes responsibility of __debug__
2026-01-18 01:02:40 +09:00
Jeong, YunWon 33689c1d0f Fix jit failure 2026-01-17 19:22:00 +09:00
Shahar Naveh c2bfdf30bd Assign opcode ids (#6637) 2026-01-09 22:52:47 +09:00
Jeong, YunWon dac236dac0 more no_std clippy (#6587) 2025-12-30 15:15:02 +09:00
Shahar Naveh fef9de22c4 Remove Rotate* & Duplicate* instructions (#6303)
* Remove `Instruction::Duplicate2?`

* Remove `Instruction::Rotate*` instructions

* Fix jit

* Update snapshot
2025-11-29 00:46:27 +09:00
Shahar Naveh cc2e84b9fc Move jit -> crates/jit (#6262) 2025-11-15 18:26:24 +09:00