ci: add pyperformance benchmarking scripts and workflow (#8691)

* Add script to run upstream pyperformance against RustPython

pyperformance can't be installed as-is on RustPython since pyperf hard-depends
on psutil, a C extension RustPython cannot build or load (no CPython C-API /
extension-module support). Since pyperf already disables psutil usage on
interpreters reporting Py_GIL_DISABLED=1 (which RustPython does, having no
GIL), a functionality-free pure-Python psutil stub is enough to satisfy pip's
dependency resolution and unblock any pure-Python benchmark.

scripts/pyperformance/run_all.py drives pyperformance (under a host CPython
venv) with RustPython as the --python target, installs the stub via
PIP_FIND_LINKS + --inherit-environ, and catalogs pass/fail/timeout per
benchmark with an automatic failure classifier (C-extension build blocked vs.
a genuine exception raised by the benchmark itself).

Assisted-by: Claude Code:claude-sonnet-5

* pyperformance: support targeting any Python and comparing catalogs

Generalize run_all.py from a RustPython-only runner to a --python/--label
pair, so the same script can catalog a real CPython (with --no-psutil-stub,
since it has a genuine psutil) alongside RustPython. Each label gets its own
results/<label>/ subdirectory.

Add compare.py, which loads two labels' catalogs and writes a per-benchmark
Markdown table with both targets' status/mean and the candidate/baseline
time ratio, plus the median slowdown across benchmarks both targets passed.

Assisted-by: Claude Code:claude-sonnet-5

* ci: compare base, head and CPython with pyperformance on one runner

A ratio between two pyperformance runs measured on different machines can
move further than the change under review did, so a PR's numbers are only
worth reading when base, head and CPython were measured back to back on the
same runner. This adds a job that does exactly that -- two release builds
kept side by side with the stdlib of their own commit, then three runs with
nothing in between -- and posts the table as a PR comment.

`pr_diff.py` renders the three catalogs `run_all.py` writes: per benchmark
the CPython time, both RustPython times, both ratios against CPython, and
the head/base change, with anything under 3% shown as `~` because a shared
runner is not quiet enough to say more.

The comment is handed to a `workflow_run` companion the same way
`codspeed-comment.yaml` does it, since a `pull_request` run from a fork gets
a read-only token however the workflow is configured.

The job is opt-in behind a `run:pyperformance` label: it runs 28 benchmarks
three times over and costs well over an hour, which most PRs do not need.

`run_all.py` now forwards RUSTPYTHONPATH through pyperf, which a binary
copied away from its checkout needs in order to find the stdlib at all; the
two names go in one comma-separated `--inherit-environ`, because repeating
the flag keeps only the last.

Assisted-by: Claude Code:claude-fable-5-1

* ci: comment from the pyperformance run itself when the PR allows it

The `workflow_run` companion only starts firing once it is on the default
branch, so a pull request that introduces it -- this one -- would upload the
table and never show it. A pull request opened from this repo carries a
token that may write comments, so post it from the run that measured it and
leave the companion to the forks that genuinely cannot.

Both paths upsert on the same marker, so a fork PR that reaches the
companion after this step was skipped still lands one comment.

Assisted-by: Claude Code:claude-fable-5-1

* ci: address zizmor findings and fall back to head vs. CPython

- Pin `actions/github-script` to the commit the rest of the repo pins it to;
  the hash carried over from an in-flight branch pointed elsewhere than the
  `v9.0.0` comment claimed (ref-version-mismatch).
- Read the PR number, the CPython path and version, and the two commit shas
  from `env` rather than expanding them into the shell (template-injection).
- Record the `workflow_run` trigger in `.github/zizmor.yml` next to the
  existing `pull_request_target` entry, with the same justification: the
  workflow reads one artifact and writes a comment, and never checks out or
  runs pull request code (dangerous-triggers).

Building or measuring the base commit is now non-fatal, and `pr_diff.py`
renders head against CPython alone when the base column has nothing in it --
a base that will not build should still leave a usable table rather than
failing the job. The time and ratio columns are also named apart, since
`| base | head | base | head |` gave no clue which pair was which.

Assisted-by: Claude Code:claude-fable-5-1

* ci: derive pyperformance comment target from trusted workflow_run metadata

pr_number.txt came from the artifact a `pull_request` run uploads, which a
fork could tamper with to make this privileged `workflow_run` comment on an
arbitrary PR/issue. Resolve the PR from context.payload.workflow_run instead
(falling back to a commit-associated open PR lookup for workflow_dispatch
runs), and escape any HTML comments already present in diff.md so they can't
be mistaken for the marker.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* ci: grant actions:read to the pyperformance comment workflow

download-artifact needs actions:read to fetch another run's artifact by
run-id. Specifying a permissions block sets every unlisted scope to none, so
without this the cross-run download silently failed for fork PRs -- the
scenario this workflow exists for -- continue-on-error masked it, and the
comment step found no diff.md and returned without posting.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* pyperformance: kill the whole process group on a benchmark timeout

subprocess.run's own timeout only terminates the direct pyperformance
process; pyperf re-execs the target interpreter as a child, which can
survive the timeout and keep burning CPU into the next benchmark's
measurement. Run each benchmark in its own session/process group instead,
and on timeout SIGKILL the whole group via os.killpg.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* pyperformance: compute the head-vs-base cell from head_vs_base directly

The cell was derived from base_vs_cpython and head_vs_cpython, both of which
are None when a benchmark has no CPython time -- so it rendered "-" even
though base and head both had valid times and head_vs_base was already
computed. Use that precomputed ratio instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* pyperformance: invalidate the resumed catalog when the target executable changes

catalog.json records results without any fingerprint of target_python, and
resume skips benchmarks by name alone, so a local rebuild (or swapping the
target) under the same --label could silently reuse stale measurements.
Store a sha256 of the executable's contents alongside the catalog and, when
it no longer matches, ignore the existing catalog and re-run everything.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* ci: fall back to github.sha for the head checkout on workflow_dispatch

github.event.pull_request.head.sha is empty on a workflow_dispatch run (no
pull_request payload), so `git checkout --quiet "" --` failed with "fatal:
invalid reference" on this non-optional step, aborting every manual run
before it could measure or render results.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* pyperformance: resolve --out/--cache-dir to absolute paths

Every benchmark subprocess runs with cwd=work_dir (under --cache-dir), so a
relative --out or --cache-dir passed on the command line resolved against
that directory instead of the one run_all.py was invoked from, breaking
catalog output, the pyperformance binary path, and PIP_FIND_LINKS. Both
default to absolute paths already; resolve them up front so a relative
override behaves the same way.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* pyperformance: persist catalog invalidation before the new fingerprint

The fingerprint file was rewritten before the invalidated catalog was ever
written back to disk. An interruption between the two (e.g. a crash during
the first benchmark of the re-run) left a fingerprint matching the new
executable pointing at a catalog.json still full of results measured
against the old one, so the next run would trust it as fresh. Write the
reset (empty) catalog first, then record the fingerprint.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* pyperformance: show benchmarks that fail on both base and head

Rows were dropped whenever neither side had a mean, which also hid a
benchmark that failed or timed out identically on base and head -- e.g. a
C-extension gap unrelated to the PR under review. Only skip a benchmark
never attempted on either commit; keep rows with a fail/timeout status so
they still show up in the table.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* pyperformance: write catalog.json atomically

write_catalog runs after every single benchmark, and Path.write_text
truncates the file before writing the replacement. An interruption
mid-write left catalog.json both missing every earlier result and invalid
JSON, so the next run's resume logic failed outright at json.loads(). Write
to a temporary sibling file and rename it into place instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* pyperformance: fix passed count and median population in the summary table

"Benchmarks passed" counted len(base_ratios)/len(head_ratios), which require
a *CPython* ratio too -- a CPython-only failure hid an otherwise-passing
benchmark from both sides' counts. Count actual "ok" status per side
instead.

The two medians were also computed over independently-filtered
populations: whichever benchmarks had a valid ratio on that side alone. A
benchmark that regressed to failure on only one side dropped out of only
that side's median, which could shift it well beyond what any surviving
benchmark's timing actually changed by. Restrict both medians to
benchmarks with a valid ratio on both sides, and label the population size.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* pyperformance: include RUSTPYTHONPATH's stdlib in the executable fingerprint

The catalog-invalidation fingerprint hashed only the executable's bytes, but
a RustPython binary copied away from its checkout finds its stdlib through
RUSTPYTHONPATH -- pyperf re-execs the target interpreter into a venv and
forwards that variable so it can still find Lib/. Changing that path, or
editing the files under it, without rebuilding the binary left resume
trusting results measured against a now-different stdlib. Fold its contents
into the fingerprint too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* pyperformance: require an explicit ok/fail status for pass/fail transitions

only_head/only_base flagged a benchmark as newly passing or no longer
passing based only on whether the other side had a mean, so a benchmark
never attempted on one side (e.g. a partial --benchmarks run) was reported
as a regression or fix it never was. Require the other side's status to be
an actual fail/timeout, not just missing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* pyperformance: don't gate the head/base geometric mean on a CPython ratio

The summary panel -- pass counts and the head/base geometric mean, neither
of which need a CPython comparison -- only rendered when at least one
benchmark also had a valid ratio against CPython on both sides. A build
with a perfectly good head-vs-base comparison but no common CPython result
lost the whole panel, including the one aggregate that didn't depend on
CPython at all.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* pyperformance: normalize RUSTPYTHONPATH and extract main() into named steps

Fix: a relative RUSTPYTHONPATH was fingerprinted relative to this script's
invocation directory but resolved by pyperf's subprocess relative to
work_dir (its cwd), so a relative value could be hashed against the wrong
directory entirely, or fail to resolve at runtime. Resolve it to an
absolute path up front, alongside --out and --cache-dir.

Refactor: main() had accumulated argparse setup, executable resolution, and
the fingerprint/resume block inline, each wrapped in an explanatory comment
-- a sign the function was doing too much rather than that it needed more
narration. Extracted parse_args(), resolve_target_python(), and
load_resumable_catalog() so main() reads as a short sequence of named
steps and each comment sits next to the one function it actually explains.
No behavior change; verified with unit tests against the extracted
functions (fingerprint reacting to RUSTPYTHONPATH, resume picking up a
prior catalog, invalidation on an executable change).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* pyperformance: split pr_diff.py's main() and fix a status-transition gap

Fix: only_head was still gated on have_base, which only asks whether *any*
base benchmark produced a timing -- so a benchmark that flipped from
failing to passing went unreported whenever every other base benchmark
also happened to fail. Status transitions don't need that gate; they
already require an explicit ok/fail/timeout status on both sides.

Refactor: main() had grown into one long function mixing argument parsing,
catalog loading, row computation, and three separate rendering concerns.
Split it into parse_args(), load_base_catalog(), build_rows(),
render_intro(), render_summary_panel(), render_table(), and
find_transitions(), each documenting the one thing it decides. No behavior
change beyond the fix above; re-verified against every scenario exercised
in the last several fixes (dual-fail rows, geo mean without a CPython
ratio, unattempted benchmarks, and the have_base gap) plus the existing
manual smoke tests.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* pyperformance: don't resolve an empty RUSTPYTHONPATH to the cwd

The normalization only checked "RUSTPYTHONPATH" in os.environ, so a set-but-
empty value (RUSTPYTHONPATH="") resolved to the current directory via
Path("").resolve(). executable_fingerprint() would then recursively hash
every file under wherever run_all.py was invoked from -- including its own
results directory while writing to it -- causing spurious catalog
invalidation and a lot of pointless hashing. Check for a nonempty value
instead, matching the truthiness check already used everywhere else this
variable is read.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

* docs(pyperformance): distinguish the host runner from the benchmarked target

A reviewer read "pyperformance cannot be installed as-is on RustPython" and
reasonably asked why RustPython would need to support pyperformance's CLI at
all, since it's normally run as a command under any Python (see
https://github.com/RustPython/RustPython/pull/8691#discussion_r3997957942).

It doesn't: the pyperformance CLI (the host process) always runs under a
real CPython regardless of --python. What actually needs psutil/pyperf
importable is the *benchmark* subprocess -- pyperf re-execs the --python
target and that subprocess itself does `import pyperf`. Spell out that
host/worker split up front instead of leading with "cannot be installed".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy8J7LpmdYjesTDonNoYkv

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

scripts/pyperformance/README.md

@@ -0,0 +1,100 @@
+# Running upstream pyperformance against RustPython
+
+Two different interpreters are involved whenever
+[pyperformance](https://github.com/python/pyperformance) is pointed at
+RustPython, and only one of them is RustPython itself:
+
+- **The `pyperformance` CLI** (`pyperformance run --python <target>`) is the
+  *host* process -- it parses arguments, decides which benchmarks to run, and
+  collects results. It always runs under a real CPython, regardless of what
+  `--python` points at; RustPython never has to run this command.
+- **The benchmark being measured** runs as the `--python` target instead.
+  `pyperf`'s `Runner` re-execs that target as a subprocess to actually run the
+  timing loop, and the benchmark script that subprocess runs does
+  `import pyperf` itself to call the measurement API. So it's RustPython, as
+  that subprocess, that needs to import `pyperf` -- not run `pyperformance`.
+
+`pyperf` hard-depends on `psutil`, a C-extension package, and RustPython has
+no CPython C-API / extension-module loading support (`_imp.create_dynamic` /
+`_imp.exec_dynamic` are not implemented, and `_sysconfigdata` doesn't provide
+a full compiler config, so even building a C extension from source fails).
+That's what breaks: not "installing the pyperformance CLI on RustPython" --
+RustPython importing `pyperf` as a benchmark subprocess.
+
+Workaround: `pyperf` already disables psutil usage on interpreters that
+report `Py_GIL_DISABLED=1` in `sysconfig` (see `pyperf._utils.USE_PSUTIL`) --
+this is exactly what RustPython reports, since it has no GIL. So `pyperf`
+never actually calls into psutil at runtime on RustPython; a
+functionality-free pure-Python `psutil` stub (see `stub_psutil/`) is enough
+to satisfy pip's dependency resolution and let installation proceed.
+
+This only unblocks *pure-Python* benchmarks. Any benchmark whose own
+workload needs a real C extension (numpy, lxml, greenlet, PyYAML's C
+accelerator, ...) will still fail to install -- and that failure is a real,
+separate finding about RustPython's lack of C-extension support, not an
+artifact of this workaround.
+
+## Usage
+
+```shell
+# Build RustPython first (SSL feature needed for --install-pip, not required here)
+cargo build --release
+
+# Run every benchmark pyperformance knows about against it, and write a catalog
+python3 scripts/pyperformance/run_all.py --rustpython target/release/rustpython
+
+# Or just a subset
+python3 scripts/pyperformance/run_all.py --benchmarks nqueens,richards,pyflate
+```
+
+Output goes to `scripts/pyperformance/results/`:
+- `CATALOG.md` -- human-readable pass/fail table with the mean time and a
+  failure reason for each benchmark.
+- `catalog.json` -- the same data, machine-readable.
+- `raw/<benchmark>.json` -- pyperf's own result file for benchmarks that
+  passed.
+
+The run resumes automatically: re-running the script skips any benchmark
+already present in `catalog.json`. Pass `--force` to redo everything.
+
+A host CPython venv (pyperformance itself only runs under a real CPython;
+RustPython is just the `--python` target it benchmarks) and the stub wheel
+are cached under `target/pyperformance/` (gitignored) and reused across
+runs.
+
+See `python3 scripts/pyperformance/run_all.py --help` for all options
+(`--timeout`, `--rigorous` instead of `--fast`, `--cache-dir`, ...).
+
+## Comparing against real CPython
+
+`run_all.py` can target any Python executable, not just RustPython, via
+`--python` + `--label`. For a real CPython you don't need (and don't want)
+the psutil stub -- pass `--no-psutil-stub` so it installs and uses the
+genuine `psutil`:
+
+```shell
+python3 scripts/pyperformance/run_all.py \
+    --python "$(command -v python3.13)" --label cpython3.13 --no-psutil-stub
+
+python3 scripts/pyperformance/run_all.py \
+    --python target/release/rustpython --label rustpython
+```
+
+Each `--label` gets its own subdirectory under `results/` (its own
+`catalog.json`, `CATALOG.md`, `raw/`), so multiple targets' catalogs coexist.
+
+Then compare two catalogs:
+
+```shell
+python3 scripts/pyperformance/compare.py --baseline cpython3.13 --candidate rustpython
+```
+
+This writes `results/COMPARE-rustpython-vs-cpython3.13.md`: a per-benchmark
+table with both targets' status/mean and the candidate/baseline time ratio,
+plus the median slowdown across benchmarks both targets passed.
+
+Note: pass an actual interpreter binary to `--python`, not a version-manager
+shim (e.g. an `asdf`/`mise`/`pyenv` shim) -- pyperformance runs a helper
+script through it directly, which some shims don't support. If unsure what a
+`python3` on your PATH really resolves to, use
+`python3 -c "import sys; print(sys.executable)"` and pass that path.

scripts/pyperformance/compare.py

@@ -0,0 +1,155 @@
+#!/usr/bin/env python3
+"""Compare two `run_all.py` catalogs (e.g. RustPython vs. CPython) side by side.
+
+Usage
+-----
+    # Catalog a real CPython too (no psutil stub needed -- it has a real one):
+    python3 scripts/pyperformance/run_all.py --python "$(which python3.13)" \\
+        --label cpython3.13 --no-psutil-stub
+
+    # RustPython catalog, if not already generated:
+    python3 scripts/pyperformance/run_all.py --python target/release/rustpython \\
+        --label rustpython
+
+    # Compare them:
+    python3 scripts/pyperformance/compare.py --baseline cpython3.13 --candidate rustpython
+
+Writes `scripts/pyperformance/results/COMPARE-<candidate>-vs-<baseline>.md`.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+from pathlib import Path
+
+SCRIPT_DIR = Path(__file__).resolve().parent
+DEFAULT_RESULTS_DIR = SCRIPT_DIR / "results"
+
+TIME_RE = re.compile(r"^\s*([\d.]+)\s*(ns|us|ms|sec)\b")
+UNIT_SECONDS = {"ns": 1e-9, "us": 1e-6, "ms": 1e-3, "sec": 1.0}
+
+
+def parse_mean_seconds(mean: str | None) -> float | None:
+    """Parse a pyperf "<value> <unit> +- <value> <unit>" string into seconds."""
+    if not mean:
+        return None
+    m = TIME_RE.match(mean)
+    if not m:
+        return None
+    value, unit = m.groups()
+    return float(value) * UNIT_SECONDS[unit]
+
+
+def load_catalog(results_dir: Path, label: str) -> dict[str, dict]:
+    catalog_json = results_dir / label / "catalog.json"
+    if not catalog_json.exists():
+        raise SystemExit(
+            f"no catalog for label {label!r} at {catalog_json} -- run "
+            f"`run_all.py --label {label} ...` first"
+        )
+    return {r["benchmark"]: r for r in json.loads(catalog_json.read_text())}
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
+    )
+    parser.add_argument(
+        "--baseline",
+        required=True,
+        help="label of the baseline catalog, e.g. cpython3.13",
+    )
+    parser.add_argument(
+        "--candidate",
+        required=True,
+        help="label of the catalog to compare against it, e.g. rustpython",
+    )
+    parser.add_argument(
+        "--results-dir",
+        type=Path,
+        default=DEFAULT_RESULTS_DIR,
+        help="parent directory containing <label>/catalog.json (default: %(default)s)",
+    )
+    parser.add_argument(
+        "--out",
+        type=Path,
+        default=None,
+        help="output Markdown file (default: <results-dir>/COMPARE-<candidate>-vs-<baseline>.md)",
+    )
+    args = parser.parse_args()
+
+    baseline = load_catalog(args.results_dir, args.baseline)
+    candidate = load_catalog(args.results_dir, args.candidate)
+    names = sorted(set(baseline) | set(candidate))
+
+    rows = []
+    both_ok = []
+    for name in names:
+        b = baseline.get(name)
+        c = candidate.get(name)
+        b_status = b["status"] if b else "missing"
+        c_status = c["status"] if c else "missing"
+        b_mean = parse_mean_seconds(b["mean"]) if b else None
+        c_mean = parse_mean_seconds(c["mean"]) if c else None
+        ratio = None
+        if b_mean and c_mean:
+            ratio = c_mean / b_mean
+            both_ok.append(ratio)
+        rows.append(
+            {
+                "benchmark": name,
+                "baseline_status": b_status,
+                "baseline_mean": b["mean"] if b else "",
+                "candidate_status": c_status,
+                "candidate_mean": c["mean"] if c else "",
+                "ratio": ratio,
+                "candidate_detail": (c or {}).get("detail") or "",
+            }
+        )
+
+    out = (
+        args.out or args.results_dir / f"COMPARE-{args.candidate}-vs-{args.baseline}.md"
+    )
+    lines = []
+    lines.append(f"# {args.candidate} vs {args.baseline} -- pyperformance comparison")
+    lines.append("")
+    lines.append(
+        f"Baseline: **{args.baseline}** ({len([r for r in baseline.values() if r['status'] == 'ok'])}/"
+        f"{len(baseline)} passed). Candidate: **{args.candidate}** "
+        f"({len([r for r in candidate.values() if r['status'] == 'ok'])}/{len(candidate)} passed)."
+    )
+    if both_ok:
+        both_ok.sort()
+        n = len(both_ok)
+        median = (
+            both_ok[n // 2] if n % 2 else (both_ok[n // 2 - 1] + both_ok[n // 2]) / 2
+        )
+        lines.append("")
+        lines.append(
+            f"Median slowdown on the {len(both_ok)} benchmarks both passed: "
+            f"**{median:.2f}x** ({args.candidate} time / {args.baseline} time)."
+        )
+    lines.append("")
+    lines.append(
+        f"| Benchmark | {args.baseline} | {args.candidate} | {args.candidate}/{args.baseline} | Notes |"
+    )
+    lines.append("|---|---|---|---|---|")
+    for r in rows:
+        b_cell = r["baseline_mean"] or f"({r['baseline_status']})"
+        c_cell = r["candidate_mean"] or f"({r['candidate_status']})"
+        ratio_cell = f"{r['ratio']:.2f}x" if r["ratio"] else ""
+        notes = r["candidate_detail"].replace("|", "\\|").replace("\n", " ")
+        lines.append(
+            f"| {r['benchmark']} | {b_cell} | {c_cell} | {ratio_cell} | {notes} |"
+        )
+    lines.append("")
+
+    out.parent.mkdir(parents=True, exist_ok=True)
+    out.write_text("\n".join(lines) + "\n")
+    print(f"wrote {out}")
+
+
+if __name__ == "__main__":
+    main()

scripts/pyperformance/pr_diff.py

@@ -0,0 +1,266 @@
+#!/usr/bin/env python3
+"""Render a base/head/CPython pyperformance comparison as a PR comment.
+
+`compare.py` puts one catalog next to one other catalog, which is the right
+shape for "how far is RustPython from CPython". A pull request needs a third
+column -- the base commit -- and it needs all three measured *in the same CI
+job*, back to back on one runner: a ratio between two numbers that were
+measured on different machines can move by more than the change under review
+did, so a cross-run comparison cannot tell a real regression from a runner
+that happened to be slower that day.
+
+Reads `<results-dir>/<label>/catalog.json` for the three labels and writes a
+Markdown comment body (and optionally the same data as JSON).
+
+Usage:
+
+    python3 scripts/pyperformance/pr_diff.py \\
+        --cpython cpython --base base --head head \\
+        --results-dir "$RUNNER_TEMP/pyperf-results" \\
+        --md diff.md --json diff.json
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import statistics
+from pathlib import Path
+
+from compare import load_catalog, parse_mean_seconds
+
+MARKER = "<!-- pyperformance-local-diff -->"
+
+# Below this, a head/base difference is indistinguishable from run-to-run
+# noise on a shared CI runner, so it is rendered as "~" rather than a number
+# that invites over-reading.
+NOISE = 0.03
+
+
+def fmt_ratio(value: float | None) -> str:
+    return "-" if value is None else f"{value:.2f}x"
+
+
+def fmt_change(base: float | None, head: float | None) -> str:
+    """Head vs. base as a percentage, blanked out inside the noise floor."""
+    if not base or not head:
+        return "-"
+    change = head / base - 1.0
+    if abs(change) < NOISE:
+        return "~"
+    return f"{change * 100:+.1f}%"
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
+    )
+    parser.add_argument("--cpython", default="cpython", help="CPython catalog label")
+    parser.add_argument("--base", default="base", help="base-commit catalog label")
+    parser.add_argument("--head", default="head", help="head-commit catalog label")
+    parser.add_argument(
+        "--results-dir",
+        type=Path,
+        required=True,
+        help="parent directory containing <label>/catalog.json",
+    )
+    parser.add_argument("--md", type=Path, required=True, help="Markdown output file")
+    parser.add_argument("--json", type=Path, help="optional JSON output file")
+    parser.add_argument(
+        "--cpython-version",
+        default="CPython",
+        help="version string to label the CPython column with",
+    )
+    return parser.parse_args()
+
+
+def load_base_catalog(results_dir: Path, label: str) -> tuple[dict, bool]:
+    """Load the base-commit catalog, if any.
+
+    The base commit is the column that makes this a review aid rather than a
+    status report, but it is also the one that can go missing: it may fail to
+    build, or predate something the benchmarks now need. The caller falls
+    back to head against CPython alone when nothing here produced a usable
+    time, rather than failing the whole comparison.
+    """
+    base = {}
+    if (results_dir / label / "catalog.json").exists():
+        base = load_catalog(results_dir, label)
+    have_base = any(parse_mean_seconds(r.get("mean")) for r in base.values())
+    return base, have_base
+
+
+def build_rows(cpython: dict, base: dict, head: dict) -> list[dict]:
+    rows = []
+    for name in sorted(set(cpython) | set(base) | set(head)):
+        c = parse_mean_seconds((cpython.get(name) or {}).get("mean"))
+        b = parse_mean_seconds((base.get(name) or {}).get("mean"))
+        h = parse_mean_seconds((head.get(name) or {}).get("mean"))
+        rows.append(
+            {
+                "benchmark": name,
+                "cpython_mean": (cpython.get(name) or {}).get("mean") or "",
+                "base_mean": (base.get(name) or {}).get("mean") or "",
+                "head_mean": (head.get(name) or {}).get("mean") or "",
+                "base_status": (base.get(name) or {}).get("status", "missing"),
+                "head_status": (head.get(name) or {}).get("status", "missing"),
+                "base_vs_cpython": (b / c) if (b and c) else None,
+                "head_vs_cpython": (h / c) if (h and c) else None,
+                "head_vs_base": (h / b) if (h and b) else None,
+            }
+        )
+    return rows
+
+
+def render_intro(have_base: bool, cpython_version: str) -> list[str]:
+    if have_base:
+        return [
+            "All three interpreters were measured back to back in this one job, "
+            "so they share the exact same CPU and kernel. Ratios are "
+            f"`time / {cpython_version} time` -- lower is better. A "
+            f"head/base difference smaller than {NOISE:.0%} is shown as `~`; CI "
+            "runners are not quiet enough to read more into it than that."
+        ]
+    return [
+        "No usable result for the base commit -- it did not build, or none "
+        "of its benchmarks produced a time -- so this is head against "
+        f"{cpython_version} alone, both measured back to back in this "
+        "one job. Ratios are `time / {} time`; lower is better.".format(cpython_version)
+    ]
+
+
+def render_summary_panel(
+    rows: list[dict], have_base: bool, cpython_version: str
+) -> list[str]:
+    """The "| | base | head |" mini-table: pass counts, the median slowdown
+    vs. CPython, and the head/base geometric mean.
+
+    Each row is independently gated on having the data it needs: the two
+    medians are restricted to benchmarks with a valid ratio on *both* sides,
+    so a benchmark that regressed to failure on only one side can't drop out
+    of only that side's population and skew the comparison; "Benchmarks
+    passed" and the geometric mean don't depend on CPython at all, so they
+    must not disappear just because no benchmark had a usable CPython
+    comparison.
+    """
+    if not have_base:
+        return []
+
+    common_vs_cpython = [
+        r for r in rows if r["base_vs_cpython"] and r["head_vs_cpython"]
+    ]
+    base_ratios = [r["base_vs_cpython"] for r in common_vs_cpython]
+    head_ratios = [r["head_vs_cpython"] for r in common_vs_cpython]
+    base_passed = sum(1 for r in rows if r["base_status"] == "ok")
+    head_passed = sum(1 for r in rows if r["head_status"] == "ok")
+    both = [r for r in rows if r["head_vs_base"]]
+
+    out = ["| | base | head |", "| --- | ---: | ---: |"]
+    if base_ratios and head_ratios:
+        out.append(
+            f"| Median slowdown vs. {cpython_version} "
+            f"({len(common_vs_cpython)} common) "
+            f"| {statistics.median(base_ratios):.2f}x "
+            f"| {statistics.median(head_ratios):.2f}x |"
+        )
+    out.append(f"| Benchmarks passed | {base_passed} | {head_passed} |")
+    if both:
+        geo = statistics.geometric_mean([r["head_vs_base"] for r in both])
+        out.append(f"| Geometric mean, head/base ({len(both)} common) | | {geo:.3f} |")
+    out.append("")
+    return out
+
+
+def render_table(rows: list[dict], have_base: bool, cpython_version: str) -> list[str]:
+    if have_base:
+        out = [
+            f"| Benchmark | {cpython_version} | base | head "
+            f"| base/{cpython_version} | head/{cpython_version} "
+            "| head vs. base |",
+            "| --- | ---: | ---: | ---: | ---: | ---: | ---: |",
+        ]
+    else:
+        out = [
+            f"| Benchmark | {cpython_version} | head | head/{cpython_version} |",
+            "| --- | ---: | ---: | ---: |",
+        ]
+
+    for r in sorted(rows, key=lambda r: r["head_vs_base"] or r["head_vs_cpython"] or 0):
+        # A row with no mean on either side but a fail/timeout status is still
+        # worth showing -- e.g. a C-extension gap that fails identically on
+        # base and head. Only a benchmark never attempted on either commit
+        # (no entry in that catalog at all) has nothing to report.
+        if r["base_status"] == "missing" and r["head_status"] == "missing":
+            continue
+        head_cell = r["head_mean"] or "({})".format(r["head_status"])
+        if not have_base:
+            out.append(
+                f"| {r['benchmark']} | {r['cpython_mean'] or '-'} | {head_cell} "
+                f"| {fmt_ratio(r['head_vs_cpython'])} |"
+            )
+            continue
+        base_cell = r["base_mean"] or "({})".format(r["base_status"])
+        out.append(
+            f"| {r['benchmark']} | {r['cpython_mean'] or '-'} "
+            f"| {base_cell} | {head_cell} "
+            f"| {fmt_ratio(r['base_vs_cpython'])} | {fmt_ratio(r['head_vs_cpython'])} "
+            f"| {fmt_change(1.0, r['head_vs_base'])} |"
+        )
+    return out
+
+
+def find_transitions(rows: list[dict]) -> tuple[list[str], list[str]]:
+    """Benchmarks whose pass/fail status flipped between base and head.
+
+    A transition claim needs an actual *fail/timeout* status on the other
+    side, not just a missing entry: a benchmark never attempted there (e.g.
+    a partial --benchmarks run) isn't a regression or a fix, just no data
+    point. This intentionally doesn't gate on `have_base` (whether *any*
+    base benchmark produced a timing) -- a benchmark can still validly flip
+    from failing to passing even when every other base benchmark failed too.
+    """
+    only_head = [
+        r["benchmark"]
+        for r in rows
+        if r["head_status"] == "ok" and r["base_status"] in ("fail", "timeout")
+    ]
+    only_base = [
+        r["benchmark"]
+        for r in rows
+        if r["base_status"] == "ok" and r["head_status"] in ("fail", "timeout")
+    ]
+    return only_head, only_base
+
+
+def main() -> None:
+    args = parse_args()
+
+    cpython = load_catalog(args.results_dir, args.cpython)
+    head = load_catalog(args.results_dir, args.head)
+    base, have_base = load_base_catalog(args.results_dir, args.base)
+    rows = build_rows(cpython, base, head)
+
+    title = (
+        "base vs. head vs. CPython (same runner)"
+        if have_base
+        else "head vs. CPython (same runner)"
+    )
+    out = [MARKER, f"### pyperformance: {title}", ""]
+    out += render_intro(have_base, args.cpython_version)
+    out.append("")
+    out += render_summary_panel(rows, have_base, args.cpython_version)
+    out += render_table(rows, have_base, args.cpython_version)
+
+    only_head, only_base = find_transitions(rows)
+    if only_head:
+        out += ["", f"Newly passing on head: {', '.join(only_head)}."]
+    if only_base:
+        out += ["", f"No longer passing on head: {', '.join(only_base)}."]
+
+    args.md.write_text("\n".join(out) + "\n")
+    if args.json:
+        args.json.write_text(json.dumps(rows, indent=2) + "\n")
+
+
+if __name__ == "__main__":
+    main()

scripts/pyperformance/run_all.py

@@ -0,0 +1,549 @@
+#!/usr/bin/env python3
+"""Run the upstream pyperformance benchmark suite (https://github.com/python/pyperformance)
+against a Python executable -- RustPython, a real CPython, or both -- and
+catalog which benchmarks pass, fail, or time out under each.
+
+Background
+----------
+pyperformance cannot be installed as-is on RustPython: its runtime dependency
+`pyperf` hard-depends on `psutil`, a C-extension package. RustPython has no
+CPython C-API / extension-module loading support (no `_imp.create_dynamic` /
+`_imp.exec_dynamic`, no compiler config vars such as LDCXXSHARED in
+`_sysconfigdata`), so building or loading any C extension fails.
+
+Workaround: `pyperf` itself already disables psutil usage on interpreters
+that report `Py_GIL_DISABLED=1` (see `pyperf._utils.USE_PSUTIL`), which is
+exactly what RustPython reports (it has no GIL). So a functionality-free,
+pure-Python "psutil" stub (see stub_psutil/) is enough to satisfy pip's
+dependency resolution -- pyperf never actually calls into it at runtime on
+RustPython. This unblocks any *pure-Python* pyperformance benchmark; any
+benchmark whose own workload (not just pyperf) requires a real C extension
+(e.g. lxml, numpy, greenlet) will still fail, and that failure is a genuine
+finding: it marks a real C-extension gap in RustPython, not a tooling
+artifact of this script. A real CPython target doesn't need this workaround
+at all -- pass --no-psutil-stub for it, so it installs and uses the genuine
+psutil.
+
+This script:
+  1. Ensures a *host* CPython venv with pyperformance installed (pyperformance
+     itself only runs under a real CPython; the --python target is just the
+     interpreter it benchmarks, which may be that same CPython or RustPython).
+  2. Builds the pure-Python psutil stub wheel once (skipped with --no-psutil-stub).
+  3. Runs every benchmark pyperformance knows about, one at a time, against
+     the given --python target, with a timeout per benchmark.
+  4. Writes a JSON + Markdown catalog of the results under results/<label>/.
+
+Usage
+-----
+    python3 scripts/pyperformance/run_all.py \\
+        --python target/release/rustpython --label rustpython
+
+    python3 scripts/pyperformance/run_all.py \\
+        --python "$(command -v python3.13)" --label cpython3.13 --no-psutil-stub
+
+Then compare two labels' catalogs with compare.py (see its docstring).
+
+Re-running resumes from results/<label>/catalog.json by default (skips
+benchmarks already recorded); pass --force to redo everything.
+"""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import os
+import re
+import shutil
+import signal
+import subprocess
+from pathlib import Path
+
+SCRIPT_DIR = Path(__file__).resolve().parent
+REPO_ROOT = SCRIPT_DIR.parent.parent
+STUB_PSUTIL_DIR = SCRIPT_DIR / "stub_psutil"
+
+DEFAULT_CACHE_DIR = REPO_ROOT / "target" / "pyperformance"
+DEFAULT_OUT_DIR = SCRIPT_DIR / "results"
+DEFAULT_TIMEOUT = 180
+
+
+def log(msg: str) -> None:
+    print(f"[pyperformance-runner] {msg}", flush=True)
+
+
+def executable_fingerprint(path: Path) -> str:
+    """Hash of the target executable's contents -- and, when RUSTPYTHONPATH
+    points it at a stdlib copy, that stdlib's contents too, since that's the
+    other half of what pyperf actually runs. Used to invalidate a resumed
+    catalog when a local rebuild (or a different target under the same
+    --label) replaces either one from under the earlier results.
+    """
+    digest = hashlib.sha256()
+    digest.update(path.read_bytes())
+    rustpythonpath = os.environ.get("RUSTPYTHONPATH")
+    if rustpythonpath:
+        for file in sorted(Path(rustpythonpath).rglob("*")):
+            if file.is_file():
+                digest.update(str(file.relative_to(rustpythonpath)).encode())
+                digest.update(file.read_bytes())
+    return digest.hexdigest()
+
+
+def find_host_python() -> str:
+    for candidate in (
+        "python3.13",
+        "python3.12",
+        "python3.11",
+        "python3.10",
+        "python3",
+    ):
+        path = shutil.which(candidate)
+        if path:
+            return path
+    raise SystemExit("No usable host CPython found on PATH (need python3.10+)")
+
+
+def ensure_host_venv(cache_dir: Path) -> Path:
+    venv_dir = cache_dir / "host-venv"
+    pip_marker = venv_dir / "pyvenv.cfg"
+    pyperf_installed = False
+    if pip_marker.exists():
+        pip_bin = venv_dir / "bin" / "pip"
+        result = subprocess.run(
+            [str(pip_bin), "show", "pyperformance"],
+            capture_output=True,
+            text=True,
+        )
+        pyperf_installed = result.returncode == 0
+    if not pyperf_installed:
+        log(
+            f"setting up host venv at {venv_dir} (this runs pyperformance's CLI; "
+            "RustPython is only the --python target it benchmarks)"
+        )
+        shutil.rmtree(venv_dir, ignore_errors=True)
+        venv_dir.parent.mkdir(parents=True, exist_ok=True)
+        host_python = find_host_python()
+        subprocess.run([host_python, "-m", "venv", str(venv_dir)], check=True)
+        pip_bin = venv_dir / "bin" / "pip"
+        subprocess.run([str(pip_bin), "install", "-q", "-U", "pip"], check=True)
+        subprocess.run([str(pip_bin), "install", "-q", "pyperformance"], check=True)
+    return venv_dir
+
+
+def ensure_stub_psutil_wheel(host_venv: Path, cache_dir: Path) -> Path:
+    stub_pkgs_dir = cache_dir / "stub-pkgs"
+    existing = list(stub_pkgs_dir.glob("psutil-*.whl"))
+    if existing:
+        return stub_pkgs_dir
+    log(
+        "building pure-Python psutil stub wheel (see scripts/pyperformance/stub_psutil/)"
+    )
+    stub_pkgs_dir.mkdir(parents=True, exist_ok=True)
+    pip_bin = host_venv / "bin" / "pip"
+    subprocess.run(
+        [
+            str(pip_bin),
+            "wheel",
+            str(STUB_PSUTIL_DIR),
+            "-w",
+            str(stub_pkgs_dir),
+            "--no-deps",
+            "-q",
+        ],
+        check=True,
+    )
+    return stub_pkgs_dir
+
+
+def list_benchmarks(host_venv: Path) -> list[str]:
+    pyperformance_bin = host_venv / "bin" / "pyperformance"
+    result = subprocess.run(
+        [str(pyperformance_bin), "list"],
+        capture_output=True,
+        text=True,
+        check=True,
+    )
+    names = []
+    for line in result.stdout.splitlines():
+        line = line.strip()
+        if line.startswith("- "):
+            names.append(line[2:].strip())
+    return names
+
+
+MEAN_RE = re.compile(r"Mean \+- std dev:\s*(.+)")
+FAILED_BUILD_RE = re.compile(r"Failed to build (\S+)")
+FAILED_WHEEL_RE = re.compile(r"Failed building wheel for (\S+)")
+LDCXXSHARED_RE = re.compile(
+    r"Unexpected None in config vars: \[('LDCXXSHARED'[^\]]*)\]"
+)
+# The root-cause exception raised by the *benchmark script itself* (running under
+# RustPython), as opposed to pyperformance's own wrapper exceptions
+# (e.g. "RuntimeError: Benchmark died", "RuntimeError: ... failed with exit code ...")
+# which are always the LAST exception in the combined output, not the first.
+PYPERFORMANCE_WRAPPER_ERRORS = {
+    "Benchmark died",
+}
+EXCEPTION_LINE_RE = re.compile(
+    r"^(?:\w+\.)*(\w*(?:Error|Exception)): (.+)$", re.MULTILINE
+)
+
+
+def classify_failure(combined_output: str) -> str:
+    if LDCXXSHARED_RE.search(combined_output):
+        pkg_match = FAILED_WHEEL_RE.search(combined_output) or FAILED_BUILD_RE.search(
+            combined_output
+        )
+        pkg = pkg_match.group(1) if pkg_match else "unknown package"
+        return f"C-extension build blocked (no compiler config / no CPython C-API) building {pkg}"
+    pkg_match = FAILED_WHEEL_RE.search(combined_output) or FAILED_BUILD_RE.search(
+        combined_output
+    )
+    if pkg_match:
+        return f"failed to build/install {pkg_match.group(1)}"
+    # Prefer the first real exception in the log: it's raised by the benchmark
+    # script running under RustPython, before pyperformance's own wrapper
+    # exceptions (always last) obscure it with a generic "Benchmark died".
+    for exc_type, exc_msg in EXCEPTION_LINE_RE.findall(combined_output):
+        if exc_msg.strip() in PYPERFORMANCE_WRAPPER_ERRORS:
+            continue
+        if "failed with exit code" in exc_msg:
+            continue
+        return f"{exc_type}: {exc_msg.strip()}"[:300]
+    m = re.search(r"^(ERROR: .*)$", combined_output, re.MULTILINE)
+    if m:
+        return m.group(1)[:200]
+    tail = "\n".join(combined_output.strip().splitlines()[-5:])
+    return tail[:400] or "unknown failure"
+
+
+def run_one_benchmark(
+    pyperformance_bin: Path,
+    target_python: Path,
+    bench: str,
+    work_dir: Path,
+    out_dir: Path,
+    stub_pkgs_dir: Path | None,
+    timeout: int,
+    extra_args: list[str],
+) -> dict:
+    result_json = out_dir / "raw" / f"{bench}.json"
+    result_json.parent.mkdir(parents=True, exist_ok=True)
+    if result_json.exists():
+        result_json.unlink()
+
+    env = os.environ.copy()
+    cmd = [
+        str(pyperformance_bin),
+        "run",
+        "--python",
+        str(target_python),
+        "-b",
+        bench,
+        "-o",
+        str(result_json),
+        *extra_args,
+    ]
+    inherited = []
+    if stub_pkgs_dir is not None:
+        # Work around the target interpreter's lack of a C-extension psutil (see
+        # module docstring). Real CPython doesn't need this -- pass
+        # stub_pkgs_dir=None for it so it installs and uses the real psutil.
+        env["PIP_FIND_LINKS"] = str(stub_pkgs_dir)
+        inherited.append("PIP_FIND_LINKS")
+    if "RUSTPYTHONPATH" in env:
+        # A RustPython binary copied away from its checkout (as CI does when it
+        # keeps one build of each commit around) can only find the stdlib
+        # through this variable, and pyperf re-executes the target interpreter
+        # in a venv of its own, so it has to survive that hop too.
+        inherited.append("RUSTPYTHONPATH")
+    if inherited:
+        # One comma-separated flag, not one flag per variable: pyperf's
+        # `--inherit-environ` is a plain (non-appending) option, so repeating it
+        # keeps only the last name.
+        cmd += ["--inherit-environ", ",".join(inherited)]
+
+    log(f"running {bench} ...")
+    # A new session makes this process the leader of its own process group, so
+    # a timeout can kill the whole group -- pyperf re-execs the target
+    # interpreter as a child, and `subprocess.run`'s own timeout handling only
+    # ever reaches the direct child, leaving that descendant free to keep
+    # burning CPU into the next benchmark's measurement.
+    proc = subprocess.Popen(
+        cmd,
+        cwd=work_dir,
+        env=env,
+        stdout=subprocess.PIPE,
+        stderr=subprocess.PIPE,
+        text=True,
+        start_new_session=True,
+    )
+    try:
+        stdout, stderr = proc.communicate(timeout=timeout)
+    except subprocess.TimeoutExpired:
+        os.killpg(proc.pid, signal.SIGKILL)
+        proc.communicate()  # reap the process, discard its output
+        return {
+            "benchmark": bench,
+            "status": "timeout",
+            "detail": f"exceeded {timeout}s timeout",
+            "mean": None,
+        }
+
+    combined = (stdout or "") + "\n" + (stderr or "")
+    if proc.returncode == 0 and result_json.exists():
+        mean_match = MEAN_RE.search(combined)
+        return {
+            "benchmark": bench,
+            "status": "ok",
+            "detail": None,
+            "mean": mean_match.group(1).strip() if mean_match else None,
+        }
+
+    return {
+        "benchmark": bench,
+        "status": "fail",
+        "detail": classify_failure(combined),
+        "mean": None,
+    }
+
+
+def write_catalog(results: list[dict], out_dir: Path, label: str) -> None:
+    out_dir.mkdir(parents=True, exist_ok=True)
+    catalog_json = out_dir / "catalog.json"
+    # Write-then-rename rather than truncate-in-place: this is called after
+    # every single benchmark, so an interruption mid-write must not leave
+    # catalog.json half-written -- that would both lose every earlier result
+    # in the file and make the next run's json.loads() fail outright.
+    tmp_path = out_dir / "catalog.json.tmp"
+    tmp_path.write_text(json.dumps(results, indent=2, sort_keys=False) + "\n")
+    tmp_path.replace(catalog_json)
+
+    ok = [r for r in results if r["status"] == "ok"]
+    fail = [r for r in results if r["status"] == "fail"]
+    timeout = [r for r in results if r["status"] == "timeout"]
+
+    lines = []
+    lines.append(f"# pyperformance on {label} -- benchmark catalog")
+    lines.append("")
+    lines.append(
+        f"Generated by `scripts/pyperformance/run_all.py --label {label}`. "
+        f"`{label}` was benchmarked as the `--python` target of upstream "
+        "`pyperformance`. See the script docstring for the psutil-stub workaround "
+        "used for interpreters without CPython C-API / C-extension support."
+    )
+    lines.append("")
+    lines.append(
+        f"**{len(ok)} passed, {len(fail)} failed, {len(timeout)} timed out** "
+        f"out of {len(results)} benchmarks."
+    )
+    lines.append("")
+    lines.append(f"| Benchmark | Status | Mean ({label}) | Notes |")
+    lines.append("|---|---|---|---|")
+    for r in sorted(results, key=lambda r: (r["status"] != "ok", r["benchmark"])):
+        status_label = {"ok": "✅ pass", "fail": "❌ fail", "timeout": "⏱ timeout"}[
+            r["status"]
+        ]
+        mean = r["mean"] or ""
+        detail = (r["detail"] or "").replace("|", "\\|").replace("\n", " ")
+        lines.append(f"| {r['benchmark']} | {status_label} | {mean} | {detail} |")
+    lines.append("")
+
+    catalog_md = out_dir / "CATALOG.md"
+    catalog_md.write_text("\n".join(lines) + "\n")
+    log(f"wrote {catalog_json} and {catalog_md}")
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
+    )
+    parser.add_argument(
+        "--python",
+        "--rustpython",
+        dest="python",
+        type=Path,
+        default=REPO_ROOT / "target" / "release" / "rustpython",
+        help="path to the Python (or RustPython) executable to benchmark "
+        "(default: target/release/rustpython)",
+    )
+    parser.add_argument(
+        "--label",
+        type=str,
+        default=None,
+        help="name for this target, used for the results subdirectory and report "
+        "headings (default: the executable's basename, e.g. 'rustpython' or "
+        "'python3.13')",
+    )
+    parser.add_argument(
+        "--no-psutil-stub",
+        action="store_true",
+        help="don't inject the pure-Python psutil stub -- use this for a real "
+        "CPython target, which can install and use the genuine psutil",
+    )
+    parser.add_argument(
+        "--timeout",
+        type=int,
+        default=DEFAULT_TIMEOUT,
+        help="per-benchmark timeout in seconds (default: %(default)s)",
+    )
+    parser.add_argument(
+        "--out",
+        type=Path,
+        default=DEFAULT_OUT_DIR,
+        help="parent directory to write <label>/catalog.json and <label>/CATALOG.md into",
+    )
+    parser.add_argument(
+        "--cache-dir",
+        type=Path,
+        default=DEFAULT_CACHE_DIR,
+        help="directory for the host venv and stub wheel cache (default: target/pyperformance)",
+    )
+    parser.add_argument(
+        "--benchmarks",
+        type=str,
+        default=None,
+        help="comma-separated list of benchmarks to run (default: everything pyperformance knows)",
+    )
+    parser.add_argument(
+        "--fast",
+        action="store_true",
+        default=True,
+        help="pass --fast to pyperformance run (default: on, for a quicker catalog run)",
+    )
+    parser.add_argument(
+        "--rigorous",
+        action="store_true",
+        help="pass --rigorous to pyperformance run instead of --fast",
+    )
+    parser.add_argument(
+        "--force",
+        action="store_true",
+        help="re-run benchmarks even if already present in an existing catalog.json",
+    )
+    return parser.parse_args()
+
+
+def resolve_target_python(python_arg: Path) -> Path:
+    target_python = shutil.which(str(python_arg)) or str(python_arg)
+    target_python = Path(target_python)
+    if not target_python.exists():
+        raise SystemExit(
+            f"Python executable not found: {python_arg} "
+            "(for RustPython, build it first, e.g. `cargo build --release --features ssl`)"
+        )
+    return target_python.resolve()
+
+
+def load_resumable_catalog(
+    out_dir: Path, target_python: Path, label: str
+) -> dict[str, dict]:
+    """Load catalog.json to resume from, keyed by benchmark name -- unless
+    the target executable (or the stdlib RUSTPYTHONPATH points it at) has
+    changed since that catalog was written, in which case it is invalidated
+    (and every benchmark re-run) instead of trusted.
+    """
+    out_dir.mkdir(parents=True, exist_ok=True)
+    fingerprint_path = out_dir / ".executable-fingerprint"
+    catalog_json = out_dir / "catalog.json"
+    current_fingerprint = executable_fingerprint(target_python)
+    stale = (
+        fingerprint_path.exists()
+        and fingerprint_path.read_text().strip() != current_fingerprint
+    )
+
+    all_results: dict[str, dict] = {}
+    if stale:
+        log(
+            f"target executable for label {label!r} changed since the last run; "
+            "ignoring the existing catalog and re-running all benchmarks"
+        )
+        # Persist the invalidated (empty) catalog before recording the new
+        # fingerprint below -- otherwise a crash between the two writes would
+        # leave a fingerprint that matches the *new* executable pointing at a
+        # catalog.json still full of results measured against the old one.
+        write_catalog([], out_dir, label)
+    elif catalog_json.exists():
+        for r in json.loads(catalog_json.read_text()):
+            all_results[r["benchmark"]] = r
+
+    fingerprint_path.write_text(current_fingerprint + "\n")
+    return all_results
+
+
+def main() -> None:
+    args = parse_args()
+
+    target_python = resolve_target_python(args.python)
+    label = args.label or target_python.name
+
+    # Every benchmark subprocess runs with cwd=work_dir (below); a relative
+    # --out, --cache-dir, or RUSTPYTHONPATH would then resolve against that
+    # directory instead of the one this script was invoked from.
+    args.out = args.out.resolve()
+    args.cache_dir = args.cache_dir.resolve()
+    if os.environ.get("RUSTPYTHONPATH"):
+        os.environ["RUSTPYTHONPATH"] = str(Path(os.environ["RUSTPYTHONPATH"]).resolve())
+
+    out_dir = args.out / label
+    extra_args = ["--rigorous"] if args.rigorous else ["--fast"]
+
+    host_venv = ensure_host_venv(args.cache_dir)
+    stub_pkgs_dir = (
+        None
+        if args.no_psutil_stub
+        else ensure_stub_psutil_wheel(host_venv, args.cache_dir)
+    )
+    pyperformance_bin = host_venv / "bin" / "pyperformance"
+
+    benchmarks = (
+        [b.strip() for b in args.benchmarks.split(",") if b.strip()]
+        if args.benchmarks
+        else list_benchmarks(host_venv)
+    )
+    log(f"{len(benchmarks)} benchmarks to run against {label} ({target_python})")
+
+    work_dir = args.cache_dir / "work" / label
+    work_dir.mkdir(parents=True, exist_ok=True)
+
+    # Starts as *every* benchmark previously recorded (not just this
+    # invocation's --benchmarks subset), so a partial/targeted re-run never
+    # drops earlier results from catalog.json/CATALOG.md -- it only updates
+    # the entries it actually re-ran.
+    all_results = load_resumable_catalog(out_dir, target_python, label)
+
+    for bench in benchmarks:
+        if bench in all_results and not args.force:
+            log(f"skipping {bench} (already in catalog; use --force to redo)")
+            continue
+        r = run_one_benchmark(
+            pyperformance_bin,
+            target_python,
+            bench,
+            work_dir,
+            out_dir,
+            stub_pkgs_dir,
+            args.timeout,
+            extra_args,
+        )
+        log(
+            f"  -> {bench}: {r['status']}"
+            + (f" ({r['detail']})" if r["detail"] else "")
+        )
+        all_results[bench] = r
+        write_catalog(
+            list(all_results.values()), out_dir, label
+        )  # incremental, so a crash keeps progress
+
+    write_catalog(list(all_results.values()), out_dir, label)
+    ran = [all_results[b] for b in benchmarks]
+    ok = sum(1 for r in ran if r["status"] == "ok")
+    log(
+        f"done: {ok}/{len(ran)} requested benchmarks passed "
+        f"({len(all_results)} total in catalog for {label})"
+    )
+
+
+if __name__ == "__main__":
+    main()

scripts/pyperformance/stub_psutil/pyproject.toml

@@ -0,0 +1,15 @@
+[build-system]
+requires = ["setuptools>=61"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "psutil"
+version = "999.0.0"
+description = "Stub psutil (pure Python, no C extension) used to unblock pip installs of pyperf on interpreters without CPython C-API support, such as RustPython"
+requires-python = ">=3.8"
+
+[tool.setuptools]
+package-dir = {"" = "src"}
+
+[tool.setuptools.packages.find]
+where = ["src"]

scripts/pyperformance/stub_psutil/src/psutil/__init__.py

@@ -0,0 +1,41 @@
+"""Stub psutil.
+
+pyperf hard-depends on psutil (a C-extension package RustPython cannot build
+or load, since RustPython has no CPython C-API/extension-module support).
+
+pyperf itself already disables psutil usage on interpreters that report
+Py_GIL_DISABLED=1 in sysconfig (see pyperf._utils.USE_PSUTIL), which is the
+value RustPython reports since it has no GIL. So on RustPython, pyperf never
+actually calls into psutil at runtime -- this stub only needs to satisfy
+pip's dependency resolution, not provide working functionality.
+
+Do not use this against a real Python/pyperf run where USE_PSUTIL is True:
+every function here raises immediately.
+"""
+
+
+class AccessDenied(Exception):
+    pass
+
+
+class NoSuchProcess(Exception):
+    pass
+
+
+REALTIME_PRIORITY_CLASS = None
+
+
+def _unsupported(*_a, **_k):
+    raise NotImplementedError(
+        "stub psutil: no real implementation; only present to satisfy "
+        "pip dependency resolution on interpreters without C-extension support"
+    )
+
+
+cpu_count = _unsupported
+boot_time = _unsupported
+
+
+class Process:
+    def __init__(self, *_a, **_k):
+        _unsupported()