Index conditional test classes and methods without crossing namespaces, reject ambiguous targets, and preserve inherited override indentation. Add hash-pinned whole-test copying that verifies execution context and exact source bytes, with focused Python fixtures.
Assisted-by: OpenAI Codex:model identifier unavailable
* Target Python 3.15 using the final upstream reference
Update active version and comparison configuration to Python 3.15.0. Preserve the alpha implementation status, actual bytecode format, and existing libraries pending separate migration. Align generated library-upgrade workflow pins without changing permissions or actions.
Assisted-by: OpenAI Codex:model identifier unavailable
* Adapt essential snippet contracts to Python 3.15 final
Preserve existing coverage while following verified final behavior for script metadata, builtin signatures and loading, AST fixture construction, memoryview diagnostics, mmap search bounds, and struct format errors. The unchanged final comparator exposes the earlier expectations; focused native baseline failures establish the required implementation changes.
Assisted-by: OpenAI Codex:model identifier unavailable
* Keep final-reference regression paths and remove expired sysconfig argument
Preserve reverse multiplication syntax and native exact overflow diagnostics. Accept the two source-proven CPython 3.15 specialization messages only in affected reverse str/bytes/tuple cases. Use an actually out-of-range Windows datetime errno fixture. Remove the sysconfig check_home argument at its existing canonical 3.15 removal test boundary.
Assisted-by: OpenAI Codex:model identifier unavailable
* Keep README wording unchanged in Python 3.15 transition
Assisted-by: OpenAI Codex:model identifier unavailable
* Match Python 3.15 repetition specialization and keep universal tests
Use exact sequence-type guards and preserve generic fallback diagnostics. Validate both cold and warmed expressions without implementation-specific assertions. Reject stale CPython source checkouts and cross-version library caches.
Assisted-by: OpenAI Codex:model identifier unavailable
* Limit in-place specialization additions to exact sequence repetition
Assisted-by: OpenAI Codex:model identifier unavailable
* Keep subclass regression scoped to repetition dispatch
Assisted-by: OpenAI Codex:model identifier unavailable
* Provision final CPython 3.15 with pinned uv builds
Assisted-by: OpenAI Codex:model identifier unavailable
* Enforce Python 3.15 AST constructor requirements
Reject retired missing-field and arbitrary-keyword forms while preserving defaults and explicit placeholders. Sync only matching final CPython constructor test hunks.
Assisted-by: OpenAI Codex:model identifier unavailable
* Keep seeded Python ahead of its Windows DLL directory
Assisted-by: OpenAI Codex:model identifier unavailable
* Keep opcode metadata tied to the implemented bytecode format
The interpreter target and reference tests use Python 3.15.0, but the current Rust opcode enum retains the 3.14 bytecode format. Generate metadata from its matching source rather than importing unsupported instruction families.
Assisted-by: OpenAI Codex:model identifier unavailable
* Align CSV, logarithm and decimal formatting with Python 3.15
Propagate dialect lookup errors except AttributeError, retry integer logarithms after float overflow, and update the decimal conversion diagnostic. Exercise the final reference contracts without implementation-specific branches.
Assisted-by: OpenAI Codex:model identifier unavailable
* Bind the logarithm index result before borrowing its integer
Assisted-by: OpenAI Codex:model identifier unavailable
* Align required canonical test expectations with Python 3.15
Take only demonstrated contract changes from CPython v3.15.0, d220409a13b77de7a6c30abe16ea90db01528edc: removed module __cached__, Struct format exception classes, and decimal-format diagnostic. Preserve unrelated tests and expectations.
Assisted-by: OpenAI Codex:model identifier unavailable
* Track pending frozendict pickle support during the 3.15 transition
Keep the two existing frozendict pickle tests running as expected failures until #8951 supplies the builtin. Unexpected success will signal removal of these temporary markers.
Assisted-by: OpenAI Codex:model identifier unavailable
* Support final Python 3.15 Windows negative timestamps
Use FILETIME for pre-epoch conversions and preserve precise Windows timestamp errors.
Assisted-by: OpenAI Codex:model identifier unavailable
* Derive historical Windows timezone offsets from wall time
FILETIME reports historical DST as unknown, so calculate the actual offset rather than selecting standard-time bias.
Assisted-by: OpenAI Codex:model identifier unavailable
* Identify decimal formatting arguments in Python 3.15 errors
Track consumed positional arguments, including star width and precision, and preserve mapping key representations in decimal conversion diagnostics.
Assisted-by: OpenAI Codex:model identifier unavailable
* Reuse mapping format keys for lookup and error context
Assisted-by: OpenAI Codex:model identifier unavailable
* Borrow mapping diagnostic keys as Python objects
Assisted-by: OpenAI Codex:model identifier unavailable
* Avoid empty components in the reference Python library path
Assisted-by: OpenAI Codex:model identifier unavailable
* Isolate RustPython bytecode caches during the 3.15 transition
Assisted-by: OpenAI Codex:model identifier unavailable
* Sync final Windows timestamp test expectations
Import only the final CPython platform-limit guards and mktime roundtrip expectation required by the existing Windows CI suite.
Assisted-by: OpenAI Codex:model identifier unavailable
* Restore official Python setup for 3.15 final
Use the existing pinned setup-python action now that its official catalog supplies CPython 3.15.0. Remove the temporary standalone provisioning action and its sparse-checkout entry while retaining interpreter outputs and requested versions.
Assisted-by: OpenAI Codex:model identifier unavailable
* Align merged runtime contracts with Python 3.15 final
Restrict mappingproxy ordering, use final codec registry normalization, and prefer index conversion before float conversion for sleep. Retain universal Python assertions and import the matching final codec/time canonical test hunks plus the non-ASCII codec-name deprecation warning.
Upstream source: CPython d220409a13b77de7a6c30abe16ea90db01528edc. The existing codec normalization implementation and bytecode contract remain unchanged.
Assisted-by: OpenAI Codex:model identifier unavailable
* Limit Windows launcher version check to its interpreter
Exclude only the CPython py.exe version-identity case from RustPython CI. The installed launcher reports its own final version, while RustPython correctly remains alpha. Preserve the other 45 launcher cases and all RustPython version assertions without modifying Lib.
Assisted-by: OpenAI Codex:model identifier unavailable
---------
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
* Add a native _zoneinfo module
ZoneInfo reads TZif tuples from zoneinfo._common.load_data. The stdlib
module implements utcoffset, dst, tzname, fromutc, and the zone cache.
whats_left compares that class. Cache tests no longer expect failure.
Assisted-by: Grok:grok-4.7
* Fix ZoneInfo layout and keep its cache on the module
utcoffset, dst, and tzname take the object. Zone data is boxed so the
payload lines up with tzinfo on 32-bit. The strong, weak, and timedelta
caches live on each interpreter's _zoneinfo module.
Assisted-by: Grok:grok-4.7
Mask sys.executable, sys.version_info, the install scheme, and set order in signatures. Compare decimal and zoneinfo with the pure-Python fallback. Print the referent type's module in weakref repr.
Assisted-by: Grok:grok-4.7
* Render CPython defaults in generated text signatures
Arguments whose default rendered as `<unrepresentable>`, while the
same callable has a representable default in CPython, now render that
default:
- Where the Rust value can be a literal, the argument becomes a plain
field with `default = <literal>`.
- The remaining cases keep their Rust default and add `py_default`
with a comment. These are wrappers such as ArgIntoFloat or
ArgIntoBool, values that need the VM, enums, and Option where None
means another value.
Byte string literals render as Python bytes. A function registered
under several names gets an internal doc per name, so its __doc__ no
longer carries the signature header.
Removed expectedFailure from test_pydoc tests that now pass.
Assisted-by: Grok:grok-4.7
Assisted-by: Claude:claude-opus-5-5
* Match native callable signatures to CPython names and kinds
Rename parameters and adjust positional-only / keyword-only kinds and
defaults of native functions and methods so their generated text
signatures match CPython. Remove expectedFailure markers from tests
that now pass.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Fix Windows console read size and keep type.__prepare__ permissive
Assisted-by: Claude:claude-opus-5-5
* Derive signature defaults from typed Rust values
Param.default is now a DefaultRepr rendered by the const writer.
Literals, integer and bool constants become typed defaults, and other
default expressions use the type's const fn py_default. A bare
`optional` renders as None, so `default = None` rewrites are reverted.
Address review comments: zlib max_length defaults, get_ca_certs
binary_form keyword, getpeercert der, WASM ssl certificate keywords,
fstatvfs fd-only, and _sre endpos shown as sys.maxsize.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Generate class constructor text signatures at compile time
A #[pyclass] impl composes the type's internal doc from the Args of
its Initializer or Constructor, so type.__text_signature__ and
inspect.signature work for native classes. __doc__ stays the plain
doc. Struct sequences get (iterable=(), /) and __reduce__ (self, /).
Hand-parsed constructors are converted to FromArgs structs, and
_weakref.proxy is split from ProxyType.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Remove expectedFailure from test_hmac tests that now pass
Assisted-by: Claude:claude-opus-5-5
* Match remaining native signatures and count missing CPython signatures as matches
Give lock.__enter__ and RLock.__enter__ their own no-argument methods,
add FromArgs for start_new_thread and start_joinable_thread keeping
their errors, take RTLD_GLOBAL/RTLD_LOCAL from libc, and align names,
kinds and <unrepresentable> defaults where CPython's signature is
invalid (bytes.hex, reduce, warn, link, setlocale, add_done_callback,
...). getaddrinfo accepts keyword arguments. whats_left treats a
callable CPython has no signature for as matching.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Drop unfulfilled dead_code expectation on os.link dir_fd args
Assisted-by: Claude:claude-opus-5-5
* Distinguish OptionalArg and Option in signature defaults
An `optional` field's signature default now comes from its type:
Option<T> renders None, while OptionalArg<T> (a missing argument that
is not any Python value) renders <unrepresentable>. Fields where a
missing argument behaves exactly like None become Option<T>; fields
whose missing state differs keep OptionalArg. Redundant
py_default = "<unrepresentable>" entries are removed.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Remove expectedFailure from test_rlcompleter.test_attr_matches
Assisted-by: Claude:claude-opus-5-5
* Use Option for optional arguments whose missing and None states match
OptionalOption and OptionalArg<Option<T>> fields that treat a missing
argument like None become Option<T> with #[pyarg(optional)], which renders
=None in the text signature. Plain optional parameters of that kind move
into small FromArgs structs. Uses where missing and None differ (reduce
initial, StringIO newline, TextIOWrapper.reconfigure, set_asyncgen_hooks)
keep OptionalOption.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Render ::NAME defaults by name and restore named constant defaults
- default = ::NAME writes NAME into the text signature and stores
Into::into(NAME) as the Rust value.
- A one-field tuple-struct argument pattern such as Fildes(fd) names the
parameter after its binding, so fcntl and termios keep destructuring.
- zlib, lzma, syslog, mmap and select defaults use their named constants
again instead of literals or <unrepresentable>.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Name FromArgs fields after their Python parameters
Drop #[pyarg(name = ...)] where the Python name is a valid Rust field name
and rename the field instead. The option stays only for keywords (fn, loop,
final, type, self) and the non-snake_case memLevel.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Build literal defaults from Python values and type <unrepresentable>
- default = LIT on a non-primitive field converts the literal through
ToPyObject and TryFromObject when the argument is missing, so object
fields no longer need a vm expression plus a matching py_default.
- A bare default on a primitive integer, bool or float field renders its
zero value.
- DefaultRepr::Unrepresentable replaces the <unrepresentable> text, and
py_default = "<unrepresentable>" is a compile error.
- pyexpat.ParserCreate distinguishes a missing intern from intern=None,
mmap AccessMode renders its value, and sqlite blobopen names rowid.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Resolve native docstrings from the doc database at compile time
- rustpython-doc stores its table as a sorted const slice with const fn
lookups (get, get_attr, class_attr_doc) instead of a phf map, and the
update-doc-db workflow emits that form.
- Class, method, getset, module function and struct sequence docs, and
per-class slot wrapper docs, are chosen by const lookups in the derive
output. The database text wins over Rust doc comments.
- Macros reach the table through rustpython_vm::__exports, so only the vm
depends on rustpython-doc.
- Remove expectedFailure markers from doctest and pydoc tests that pass.
Assisted-by: Claude:claude-opus-5-5
Assisted-by: Grok:grok-4.7
* Keep time.gmtime, localtime and ctime without a text signature
Their [seconds] parameter has no text signature, and unittest.mock relies
on inspect.signature failing for them. Take OptionalOption again so the
default stays unrepresentable, and treat None like a missing argument.
Assisted-by: Claude:claude-opus-5-5
* Record a module's docstring mismatches on their own
They were only kept when the module also had missing items or signature
mismatches, so annotationlib, base64, ntpath and _ctypes reported none.
Whether a module counts as implemented still depends on missing items
and signatures alone.
Assisted-by: Claude Code:claude-opus-5
* Report a builtin type method that differs as a mismatch
A method whose signature or docstring differed was listed under
"builtin items" as if it were missing, with both extra_info dicts as the
reason. That section feeds the website's builtin_items.csv. Differences
now go to mismatched_items and mismatched_doc_items, keyed by type name,
like module items. Nothing reaches this path yet because builtin
signatures are still skipped; without this change, lifting that skip
turns the one "builtin items" line into 455.
Assisted-by: Claude Code:claude-opus-5
* Compare signatures and docstrings of native callables
extra_info() skipped anything inspect treats as builtin since cc2c46b1b
(#2410), when RustPython had no __text_signature__ for builtins. It has
one now, so --signature and --doc cover functions and methods written in
Rust. Modules with a mismatched signature go from 45 to 134, and 160 of
the 710 mismatched items are ones CPython itself cannot parse, which the
--signature listing already skips.
__builtins__ is no longer compared. inspect.signature() evals defaults in
the owning module's namespace and eval() adds __builtins__ there, so
CPython's builtins module gained one after list.index was inspected and
it showed up as missing.
Assisted-by: Claude Code:claude-opus-5
* Scan the members of native classes in each module
A module was only compared one level deep, so a native class counted as
present whatever its methods were. Classes a module owns and that are
not Python-defined are now opened too, and their own members are
compared as "Class.member": 438 classes and 4,833 members, 1,033 of
which have a signature --signature reports.
A class is native when it has no __firstlineno__, which a class
statement records, and no Python functions of its own, which rules out
namedtuple-style classes made by type(). BUILTIN_TYPES moves to module
level so the builtin types are not reported twice, an alias like
array.ArrayType is skipped, and CPython's __annotations_cache__ and
__vectorcalloffset__ are left out.
RustPython follows the CPython list instead of deciding again. A member
only inherited is listed as missing with "(inherited)", as builtin type
methods are. A class missing here, or implemented as something else such
as _sha3's hash functions, is reported by its own line only.
Assisted-by: Claude Code:claude-opus-5
* Report a module that exists but cannot be imported
dir_of_mod_or_error() ran dir() on the exception import_module() returns,
so a failed import looked like a module whose every item is missing:
sqlite3 alone filled 198 of those lines, and CPython's own _ios_support
was compared as a module holding ImportError's attributes. Returning the
exception reaches the "exists but not importable" line that was already
written for it, and drops 529 lines from "stdlib items".
Assisted-by: Claude Code:claude-opus-5
* Compare a module's own docstring
extra_info() reports a docstring for callables, and a module is not one,
so the __doc__ item both sides record was None and a module docstring
never counted as a mismatch. Reading it with getdoc(), the way a
callable's is read, brings 221 module docstrings into the comparison.
sys is the one that differs; base64 and imaplib differ here only
because their Lib copies track 3.14.7 while this run used 3.14.6.
Non-callables keep reporting None. getdoc() on a plain value answers
with its type's docstring, which would turn every string constant into
a mismatch.
Assisted-by: Claude Code:claude-opus-5
* Decide a member's ownership by the class dictionary
attr_is_not_inherited() compared the attribute's value with each base's.
That took an inherited classmethod for the class's own, since a bound
method compares unequal across classes, and dropped an own attribute
whose value matched a base's default None: 394 of CPython's in all,
ast.FunctionDef.returns among them. On the RustPython side it called
AttributeError.name, a class attribute set to None, inherited.
Ownership now means the class dictionary holds the name. An inherited
member is reported only when what it resolves to differs from CPython's
own, so dict_items.__doc__, a slot CPython fills with None that
RustPython resolves to None without one, stays quiet. "(inherited)"
lines drop from 196 to 54, and ast.alias.end_lineno is among the real
gaps that appear.
Assisted-by: Claude Code:claude-fable-5-1
* Run gc_collect ten times per benchmark iteration
Assisted-by: Grok:4.6
* Reuse the prebuilt rustpython for CI whats_left
whats_left was invoking cargo build --release without threading,
so the snippets job rebuilt the interpreter after the earlier
release build.
Assisted-by: Grok:4.6
* Split CI cargo run and test into build then execute
Assisted-by: Grok:4.6
* Compile doctests in the CI build step
cargo test --no-run skips doctests, so rustdoc compilation
was still happening in the run step.
Assisted-by: Grok:4.6
* Drop cron-ci.yaml from this PR
A pull_request path filter matches the whole PR, so every
push was starting the periodic benchmark and coverage jobs.
Assisted-by: Grok:4.6
* Run doctests in their own CI step
cargo test --doc --no-run is rejected, so doctests cannot
be compiled without also executing them.
Assisted-by: Grok:4.6
* Do not run benchmark targets in rust tests
--all-targets includes Criterion benches, which cargo test
does not run by default.
Assisted-by: Grok:4.6
* Add script to run upstream pyperformance against RustPython
pyperformance can't be installed as-is on RustPython since pyperf hard-depends
on psutil, a C extension RustPython cannot build or load (no CPython C-API /
extension-module support). Since pyperf already disables psutil usage on
interpreters reporting Py_GIL_DISABLED=1 (which RustPython does, having no
GIL), a functionality-free pure-Python psutil stub is enough to satisfy pip's
dependency resolution and unblock any pure-Python benchmark.
scripts/pyperformance/run_all.py drives pyperformance (under a host CPython
venv) with RustPython as the --python target, installs the stub via
PIP_FIND_LINKS + --inherit-environ, and catalogs pass/fail/timeout per
benchmark with an automatic failure classifier (C-extension build blocked vs.
a genuine exception raised by the benchmark itself).
Assisted-by: Claude Code:claude-sonnet-5
* pyperformance: support targeting any Python and comparing catalogs
Generalize run_all.py from a RustPython-only runner to a --python/--label
pair, so the same script can catalog a real CPython (with --no-psutil-stub,
since it has a genuine psutil) alongside RustPython. Each label gets its own
results/<label>/ subdirectory.
Add compare.py, which loads two labels' catalogs and writes a per-benchmark
Markdown table with both targets' status/mean and the candidate/baseline
time ratio, plus the median slowdown across benchmarks both targets passed.
Assisted-by: Claude Code:claude-sonnet-5
* ci: compare base, head and CPython with pyperformance on one runner
A ratio between two pyperformance runs measured on different machines can
move further than the change under review did, so a PR's numbers are only
worth reading when base, head and CPython were measured back to back on the
same runner. This adds a job that does exactly that -- two release builds
kept side by side with the stdlib of their own commit, then three runs with
nothing in between -- and posts the table as a PR comment.
`pr_diff.py` renders the three catalogs `run_all.py` writes: per benchmark
the CPython time, both RustPython times, both ratios against CPython, and
the head/base change, with anything under 3% shown as `~` because a shared
runner is not quiet enough to say more.
The comment is handed to a `workflow_run` companion the same way
`codspeed-comment.yaml` does it, since a `pull_request` run from a fork gets
a read-only token however the workflow is configured.
The job is opt-in behind a `run:pyperformance` label: it runs 28 benchmarks
three times over and costs well over an hour, which most PRs do not need.
`run_all.py` now forwards RUSTPYTHONPATH through pyperf, which a binary
copied away from its checkout needs in order to find the stdlib at all; the
two names go in one comma-separated `--inherit-environ`, because repeating
the flag keeps only the last.
Assisted-by: Claude Code:claude-fable-5-1
* ci: comment from the pyperformance run itself when the PR allows it
The `workflow_run` companion only starts firing once it is on the default
branch, so a pull request that introduces it -- this one -- would upload the
table and never show it. A pull request opened from this repo carries a
token that may write comments, so post it from the run that measured it and
leave the companion to the forks that genuinely cannot.
Both paths upsert on the same marker, so a fork PR that reaches the
companion after this step was skipped still lands one comment.
Assisted-by: Claude Code:claude-fable-5-1
* ci: address zizmor findings and fall back to head vs. CPython
- Pin `actions/github-script` to the commit the rest of the repo pins it to;
the hash carried over from an in-flight branch pointed elsewhere than the
`v9.0.0` comment claimed (ref-version-mismatch).
- Read the PR number, the CPython path and version, and the two commit shas
from `env` rather than expanding them into the shell (template-injection).
- Record the `workflow_run` trigger in `.github/zizmor.yml` next to the
existing `pull_request_target` entry, with the same justification: the
workflow reads one artifact and writes a comment, and never checks out or
runs pull request code (dangerous-triggers).
Building or measuring the base commit is now non-fatal, and `pr_diff.py`
renders head against CPython alone when the base column has nothing in it --
a base that will not build should still leave a usable table rather than
failing the job. The time and ratio columns are also named apart, since
`| base | head | base | head |` gave no clue which pair was which.
Assisted-by: Claude Code:claude-fable-5-1
* ci: derive pyperformance comment target from trusted workflow_run metadata
pr_number.txt came from the artifact a `pull_request` run uploads, which a
fork could tamper with to make this privileged `workflow_run` comment on an
arbitrary PR/issue. Resolve the PR from context.payload.workflow_run instead
(falling back to a commit-associated open PR lookup for workflow_dispatch
runs), and escape any HTML comments already present in diff.md so they can't
be mistaken for the marker.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* ci: grant actions:read to the pyperformance comment workflow
download-artifact needs actions:read to fetch another run's artifact by
run-id. Specifying a permissions block sets every unlisted scope to none, so
without this the cross-run download silently failed for fork PRs -- the
scenario this workflow exists for -- continue-on-error masked it, and the
comment step found no diff.md and returned without posting.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* pyperformance: kill the whole process group on a benchmark timeout
subprocess.run's own timeout only terminates the direct pyperformance
process; pyperf re-execs the target interpreter as a child, which can
survive the timeout and keep burning CPU into the next benchmark's
measurement. Run each benchmark in its own session/process group instead,
and on timeout SIGKILL the whole group via os.killpg.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* pyperformance: compute the head-vs-base cell from head_vs_base directly
The cell was derived from base_vs_cpython and head_vs_cpython, both of which
are None when a benchmark has no CPython time -- so it rendered "-" even
though base and head both had valid times and head_vs_base was already
computed. Use that precomputed ratio instead.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* pyperformance: invalidate the resumed catalog when the target executable changes
catalog.json records results without any fingerprint of target_python, and
resume skips benchmarks by name alone, so a local rebuild (or swapping the
target) under the same --label could silently reuse stale measurements.
Store a sha256 of the executable's contents alongside the catalog and, when
it no longer matches, ignore the existing catalog and re-run everything.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* ci: fall back to github.sha for the head checkout on workflow_dispatch
github.event.pull_request.head.sha is empty on a workflow_dispatch run (no
pull_request payload), so `git checkout --quiet "" --` failed with "fatal:
invalid reference" on this non-optional step, aborting every manual run
before it could measure or render results.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* pyperformance: resolve --out/--cache-dir to absolute paths
Every benchmark subprocess runs with cwd=work_dir (under --cache-dir), so a
relative --out or --cache-dir passed on the command line resolved against
that directory instead of the one run_all.py was invoked from, breaking
catalog output, the pyperformance binary path, and PIP_FIND_LINKS. Both
default to absolute paths already; resolve them up front so a relative
override behaves the same way.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* pyperformance: persist catalog invalidation before the new fingerprint
The fingerprint file was rewritten before the invalidated catalog was ever
written back to disk. An interruption between the two (e.g. a crash during
the first benchmark of the re-run) left a fingerprint matching the new
executable pointing at a catalog.json still full of results measured
against the old one, so the next run would trust it as fresh. Write the
reset (empty) catalog first, then record the fingerprint.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* pyperformance: show benchmarks that fail on both base and head
Rows were dropped whenever neither side had a mean, which also hid a
benchmark that failed or timed out identically on base and head -- e.g. a
C-extension gap unrelated to the PR under review. Only skip a benchmark
never attempted on either commit; keep rows with a fail/timeout status so
they still show up in the table.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* pyperformance: write catalog.json atomically
write_catalog runs after every single benchmark, and Path.write_text
truncates the file before writing the replacement. An interruption
mid-write left catalog.json both missing every earlier result and invalid
JSON, so the next run's resume logic failed outright at json.loads(). Write
to a temporary sibling file and rename it into place instead.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* pyperformance: fix passed count and median population in the summary table
"Benchmarks passed" counted len(base_ratios)/len(head_ratios), which require
a *CPython* ratio too -- a CPython-only failure hid an otherwise-passing
benchmark from both sides' counts. Count actual "ok" status per side
instead.
The two medians were also computed over independently-filtered
populations: whichever benchmarks had a valid ratio on that side alone. A
benchmark that regressed to failure on only one side dropped out of only
that side's median, which could shift it well beyond what any surviving
benchmark's timing actually changed by. Restrict both medians to
benchmarks with a valid ratio on both sides, and label the population size.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* pyperformance: include RUSTPYTHONPATH's stdlib in the executable fingerprint
The catalog-invalidation fingerprint hashed only the executable's bytes, but
a RustPython binary copied away from its checkout finds its stdlib through
RUSTPYTHONPATH -- pyperf re-execs the target interpreter into a venv and
forwards that variable so it can still find Lib/. Changing that path, or
editing the files under it, without rebuilding the binary left resume
trusting results measured against a now-different stdlib. Fold its contents
into the fingerprint too.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* pyperformance: require an explicit ok/fail status for pass/fail transitions
only_head/only_base flagged a benchmark as newly passing or no longer
passing based only on whether the other side had a mean, so a benchmark
never attempted on one side (e.g. a partial --benchmarks run) was reported
as a regression or fix it never was. Require the other side's status to be
an actual fail/timeout, not just missing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* pyperformance: don't gate the head/base geometric mean on a CPython ratio
The summary panel -- pass counts and the head/base geometric mean, neither
of which need a CPython comparison -- only rendered when at least one
benchmark also had a valid ratio against CPython on both sides. A build
with a perfectly good head-vs-base comparison but no common CPython result
lost the whole panel, including the one aggregate that didn't depend on
CPython at all.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* pyperformance: normalize RUSTPYTHONPATH and extract main() into named steps
Fix: a relative RUSTPYTHONPATH was fingerprinted relative to this script's
invocation directory but resolved by pyperf's subprocess relative to
work_dir (its cwd), so a relative value could be hashed against the wrong
directory entirely, or fail to resolve at runtime. Resolve it to an
absolute path up front, alongside --out and --cache-dir.
Refactor: main() had accumulated argparse setup, executable resolution, and
the fingerprint/resume block inline, each wrapped in an explanatory comment
-- a sign the function was doing too much rather than that it needed more
narration. Extracted parse_args(), resolve_target_python(), and
load_resumable_catalog() so main() reads as a short sequence of named
steps and each comment sits next to the one function it actually explains.
No behavior change; verified with unit tests against the extracted
functions (fingerprint reacting to RUSTPYTHONPATH, resume picking up a
prior catalog, invalidation on an executable change).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* pyperformance: split pr_diff.py's main() and fix a status-transition gap
Fix: only_head was still gated on have_base, which only asks whether *any*
base benchmark produced a timing -- so a benchmark that flipped from
failing to passing went unreported whenever every other base benchmark
also happened to fail. Status transitions don't need that gate; they
already require an explicit ok/fail/timeout status on both sides.
Refactor: main() had grown into one long function mixing argument parsing,
catalog loading, row computation, and three separate rendering concerns.
Split it into parse_args(), load_base_catalog(), build_rows(),
render_intro(), render_summary_panel(), render_table(), and
find_transitions(), each documenting the one thing it decides. No behavior
change beyond the fix above; re-verified against every scenario exercised
in the last several fixes (dual-fail rows, geo mean without a CPython
ratio, unattempted benchmarks, and the have_base gap) plus the existing
manual smoke tests.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* pyperformance: don't resolve an empty RUSTPYTHONPATH to the cwd
The normalization only checked "RUSTPYTHONPATH" in os.environ, so a set-but-
empty value (RUSTPYTHONPATH="") resolved to the current directory via
Path("").resolve(). executable_fingerprint() would then recursively hash
every file under wherever run_all.py was invoked from -- including its own
results directory while writing to it -- causing spurious catalog
invalidation and a lot of pointless hashing. Check for a nonempty value
instead, matching the truthiness check already used everywhere else this
variable is read.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
* docs(pyperformance): distinguish the host runner from the benchmarked target
A reviewer read "pyperformance cannot be installed as-is on RustPython" and
reasonably asked why RustPython would need to support pyperformance's CLI at
all, since it's normally run as a command under any Python (see
https://github.com/RustPython/RustPython/pull/8691#discussion_r3997957942).
It doesn't: the pyperformance CLI (the host process) always runs under a
real CPython regardless of --python. What actually needs psutil/pyperf
importable is the *benchmark* subprocess -- pyperf re-execs the --python
target and that subprocess itself does `import pyperf`. Spell out that
host/worker split up front instead of leading with "cannot be installed".
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* ci: hold a CodSpeed run to the machine main's baseline was measured on
CodSpeed compares a run against a base run and refuses to call the
comparison meaningful when the two ran on different machines: the runner
uploads a `SystemInfo` record beside the profile carrying the OS, the
architecture, the CPU's vendor id, brand and flags, the physical core
count and the installed memory, and a hosted pool hands out whatever
machine is free. A simulated measurement is fixed against how fast the
machine is, but not against what it makes the process execute -- glibc
selects `memcpy` and the string routines through IFUNC on CPU flags, a
new stable rustc emits different code, and these benchmarks run each
program under CPython through pyo3 as well as under RustPython, so the
runner's default interpreter is an input too.
`scripts/codspeed-environment.py` records those fields plus the rustc,
libc and CPython versions, and compares a run against a recording. The
benchmark job now restores the recording main last wrote, records over it
when the ref is main, and compares against it otherwise; the install,
build and measure steps carry `if: steps.environment.outputs.match !=
'false'`, so a run that drew a different machine finishes green having
uploaded nothing. A missing or unfetchable recording is read as "nothing
to disagree with" rather than as a mismatch.
Assisted-by: Claude
* ci: read an unreadable environment recording as not a match
A cache miss, an eviction or a cache-service outage leaves the reference file
absent, and the comparison then reported `match=true`: every guarded step ran
and the run uploaded a measurement without having identified the machine it was
taken on.
An absent recording now reports `match=false`, and says so as a `::warning`
rather than the `::notice` a mismatch emits, so a job that measures nothing
because it has nothing to compare against is distinguishable from one that
measures nothing because the machine differs.
Assisted-by: Claude
* common: add PyPy CJK codec engines
Assisted-by: Codex:GPT-5
* common: mark CJK mapping tables generated
Assisted-by: Codex:GPT-5
* common: add reusable byte and text engines
Assisted-by: Codex:GPT-5
* common: record opt-in engine dependencies
Assisted-by: Codex:GPT-5
* stdlib: add `_multibytecodec` and the six `_codecs_*` modules
Wire the CJK codec engines in `rustpython-common` up to the VM. `_multibytecodec`
holds `MultibyteCodec`, `MultibyteIncrementalEncoder`, `MultibyteIncrementalDecoder`,
`MultibyteStreamReader` and `MultibyteStreamWriter`, the encode/decode drivers and
the error-handler protocol; each `_codecs_*` module exposes `getcodec` for the
encodings of its region.
`rustpython-stdlib` now enables the `cjk-codecs` feature of `rustpython-common`.
Add `cjk::decode_reset`, which the decoder `reset()` methods need. It differs from
building a fresh decoder state: ISO 2022 keeps its G1..G3 designations and the
escape-throughout flag, and resets only G0 and the shifted flag.
Drop the markers from the CJK codec tests and from the `test_codecs`, `test_io`,
`test_email`, `test_zipfile` and `test_traceback` cases that needed a CJK codec.
`test_codecs.BasicUnicodeTest.{test_basics,test_decoder_state}` stay expected
failures for an unrelated reason: `codecs.charmap_encode` rejects a `None` mapping.
Assisted-by: Claude Code:claude-opus-5
* stdlib: use the byte and text engines from `rustpython-common`
`binascii`, `_json` and `_socket`'s address converters now call the engines in
`rustpython-common` instead of carrying their own copies, and `rustpython-stdlib`
enables the `binascii`, `inet` and `json` features for them. That leaves
`base64` and `crc32fast` with no caller in `rustpython-stdlib`, and
`crates/stdlib/src/json/machinery.rs`, the last of the json_in_type-derived code,
goes away with its last caller.
The address converters change behaviour. `inet_aton` reads the lenient forms
(`127.1`, `16909060`, `0x7f.1`, `0177.0.0.1`), `inet_pton` rejects an octet with a
redundant leading zero and a group of five hex digits, `inet_ntop` writes the
IPv4-compatible form as `::c000:201`, and an address family with no converter
raises `EAFNOSUPPORT` instead of a bare `OSError`.
Give the JSON engine the closing-quote scan and the allocation-free `\uXXXX`
escape it needs to carry `_json`'s workload. Against the code it replaces,
`json.dumps` runs 2.3x faster with `ensure_ascii=True` and 1.7x without it, and
`json.loads` 1.2x.
`binascii.a2b_base64` now reports the messages `binascii.c` does: no
`error decoding base64: ` prefix, and the data-character count in parentheses.
Assisted-by: Claude Code:claude-opus-5
* test: enable the Windows cp932 codec case
The shared CJK engine now satisfies the Windows code-page adapter, so keeping this test marked as an expected failure turns its success into a CI failure.
Assisted-by: Codex:GPT-5
* Align CFG cleanup bytecode with CPython
* Bytecode parity: fblock unwind, fstring join, folding, scope
- compile.rs: unwind_fblock_stack returns whether a finally ran so
return-statement emission can adjust location handling; restructure
try/except/finally cleanup to preserve or drop boundary NOPs based on
whether the body falls through; rework f-string lowering with
count/join helpers; remove the per-collection-type heuristic for
AST-level folding and defer to flowgraph passes; add several folding
helpers and a ComprehensionLoopControl enum.
- ir.rs: re-run unary/binop folding around tuple folding, add
reorder_conditional_scope_exit_and_jump_back_blocks and several block
classification helpers, add MAX_STR_SIZE, change is_exit_without_lineno
to take the block list.
- symboltable.rs: in analyze_cells, remove names owned as cells in
function-like scopes from the parent's free set; mark lambda scope
type explicitly.
* Refine CFG scope-exit backedge ordering
* Convert parsing logic to python
* Trigger job. (DROP ME BEFORE MERGE!)
* Missing EOF
* Add found modules to set
* Suggestion by @fanninpm
* Fix missing trailing slash
* Ensure no `.py` at module name
* Strip any file auffix
* Revert "Trigger job. (DROP ME BEFORE MERGE!)"
This reverts commit 8cf9651a24.
* Handle quoted args
* Update `test_unicodedata.py` to 3.14.4
* Add test asset
* Mark failing tests
* Add `expectedSuccess` docorator
* Add support for nee decorator in patch_spec
* Reapply patches
* Align bytecode codegen structure with CPython 3.14
* Bytecode parity - constant folding, annotation ordering, superinstruction alignment
- Add BoolOp constant folding with short-circuit semantics in compile_expression
- Add constant truthiness evaluation for assert statement optimization
- Disable const collection/boolop folding in starred unpack and assignment contexts
- Move annotation block generation after body with AnnotationsPlaceholder splicing
- Reorder insert_superinstructions to run before push_cold_blocks (matching flowgraph.c)
- Lower LOAD_CLOSURE after superinstructions to avoid false LOAD_FAST_LOAD_FAST
- Add ToBool before PopJumpIf in comparisons and chained compare cleanup blocks
- Unify annotation dict building to always use incremental BuildMap + StoreSubscr
- Add TrueDivide constant folding for integer operands
- Fold constant sets to Frozenset (not Tuple) in try_fold_constant_collection
- Add PyVmBag for frozenset constant materialization in code objects
- Add remove_redundant_const_pop_top_pairs pass and peephole const+branch folding
- Emit Nop for skipped constant expressions and constant-true asserts
- Preserve comprehension local ordering by source-order bound name collection
- Simplify annotation scanning in symboltable (remove simple-name gate)
* Fix CI regressions in marshal and fast-local ops
* impl more
* Align bytecode codegen with CPython structure
* Bytecode parity - comprehension/except scope ordering, load_fast_borrow fixes
- Reorder comprehension symbol-table walk so the outermost iterator
registers its sub_tables in the enclosing scope before the comp
scope, and rescan elt/ifs in CPython's order. Codegen peeks past the
outermost iterator's nested scopes to find the comprehension table.
- For plain try/except, emit handler sub_tables before the else block
so codegen's linear sub_table cursor stays aligned.
- Rename `collect_simple_annotations` to `collect_annotations` and
evaluate non-simple annotations during __annotate__ compilation to
preserve source-order side effects while keeping the simple-name
index stable.
- Dedupe equivalent code constants in `arg_constant` and add a
structural equality check on `CodeObject`.
- Disable LOAD_FAST_BORROW for the tail end block when a try has a
bare `except:` clause, and have `new_block` inherit the flag from
the current block.
- Remove `cfg!(debug_assertions)` guard around the
`optimize_load_fast_borrow` start-depth check so mismatches are
handled (return instead of assert) in release builds.
- Collapse nop-only blocks that precede a return epilogue and hoist
the prior line number into the next real instruction so the
line table matches.
- Unmark now-passing `test_consts_in_conditionals`,
`test_load_fast_unknown_simple`,
`test_load_fast_known_because_already_loaded`, and PEP 646 f3/f4
annotation checks.
* Bytecode parity - try/except line tracking, assert 0 shape
- In `compile_try_except`, drop the leading Nop and set the end
block's source range from the last orelse/body statement so line
events after the try fall on the right line.
- Recognise constant-false asserts as the direct-raise shape (no
ToBool/PopJumpIfFalse) and flip the test assertion accordingly.
- Extend `remove_redundant_nops_in_blocks` to also look through a
trailing nop before a return-epilogue pair (LoadConst/ReturnValue
or LoadSmallInt/ReturnValue) so the epilogue keeps the correct
line number.
- Rename `preds` to `predecessor_blocks` in the LOAD_FAST_BORROW
disable pass and add a test-only `debug_late_cfg_trace` helper.
- Regenerate the `nested_double_async_with` snapshot: the tail
reference to `stop_exc` now emits LOAD_FAST instead of
LOAD_FAST_BORROW.
* Bytecode parity - iter folding, break/continue line, cold inlining
- Fold a constant list iterable into a constant tuple in for-loop
iterable position, matching the CPython optimizer, and strip a
redundant LIST_TO_TUPLE immediately before GET_ITER in the IR
peephole pass.
- Emit a Nop at the break/continue source range before unwinding
so line events land on the break/continue statement instead of
the following instruction.
- Drop `propagate_disable_load_fast_borrow`; the forward propagation
was over-zealous and the per-block inheritance in `new_block` plus
the bare-except marker are enough.
- Relax `inline_small_or_no_lineno_blocks` so small exit blocks at
the tail of a cold block are always inlined, not just return
epilogues.
- Add codegen tests covering the LIST_TO_TUPLE/GET_ITER peephole and
the late-CFG trace helper for a for-loop list-literal iterable.
- Use POP_TOP instead of POP_ITER for for-loop break/return cleanup
- Expand duplicate_end_returns to clone final return for jump predecessors
- Restrict late jump threading pass to unconditional jumps only
- Skip exception blocks in inline/reorder passes
- Simplify threaded_jump_instr NoInterrupt handling
* Fix exception handling: except* chaining, finally cleanup, RERAISE
- Align except* bytecode chaining
- Fix exception state model and finally handler cleanup
- Fix RERAISE to only pop exception, preserve values below
* Port IR optimization passes from flowgraph.c
- BUILD_TUPLE n + UNPACK_SEQUENCE n elimination
- Dead store elimination within basic blocks
- apply_static_swaps for SWAP reduction
* Add bytecode comparison and disassembly dump scripts
- compare_bytecode.py: compare CPython vs RustPython bytecode output
- dis_dump.py: extract disassembly in normalized JSON format
* use `prek` for unified linting
* Fix actionlint error
* Generate metadata when specific files change
* `check_redundant_patches.py` to accept glob path
* Test
* revert defective changes
* use `rustfmt` over `cargo fmt` for individual files
* debug reviewdog
* rustfmt
* Move comment to correct location
* defevtive fmt test
* Fail with reviewdog
* fix reviewdog perms
* Try to use present token
* without checks oerms
* put normal perms
* fmt
* [update_lib] fast date lookup for todo
* add deps
* Auto-format: ruff format
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
dot