From cd75e2a6dff25807f43d50198039fca895719894 Mon Sep 17 00:00:00 2001 From: Bas Schoenmaeckers <7943856+bschoenmaeckers@users.noreply.github.com> Date: Thu, 11 Jun 2026 05:13:25 +0200 Subject: [PATCH] Convert claude skills to more generic agent skills (#8064) Assisted-by: Codex:gpt-6 --- .agents/skills/apple-container/SKILL.md | 63 +++++++ .../skills/investigate-test-failure/SKILL.md | 53 ++++++ .agents/skills/upgrade-pylib-next/SKILL.md | 34 ++++ .agents/skills/upgrade-pylib/SKILL.md | 102 ++++++++++++ .claude/commands/apple-container.md | 46 ----- .claude/commands/investigate-test-failure.md | 49 ------ .claude/commands/upgrade-pylib-next.md | 33 ---- .claude/commands/upgrade-pylib.md | 157 ------------------ 8 files changed, 252 insertions(+), 285 deletions(-) create mode 100644 .agents/skills/apple-container/SKILL.md create mode 100644 .agents/skills/investigate-test-failure/SKILL.md create mode 100644 .agents/skills/upgrade-pylib-next/SKILL.md create mode 100644 .agents/skills/upgrade-pylib/SKILL.md delete mode 100644 .claude/commands/apple-container.md delete mode 100644 .claude/commands/investigate-test-failure.md delete mode 100644 .claude/commands/upgrade-pylib-next.md delete mode 100644 .claude/commands/upgrade-pylib.md diff --git a/.agents/skills/apple-container/SKILL.md b/.agents/skills/apple-container/SKILL.md new file mode 100644 index 000000000..fed7ce1e2 --- /dev/null +++ b/.agents/skills/apple-container/SKILL.md @@ -0,0 +1,63 @@ +--- +name: apple-container +description: Run RustPython tests in Apple's `container` CLI Linux environment. Use when user asks to run or compare Linux test results on macOS. +allowed-tools: Bash(container:*) Bash(cargo:*) Read Grep Glob +--- + +# Run Tests in Linux Container (Apple `container` CLI) + +Run RustPython tests inside a Linux container using Apple's `container` CLI. +NEVER use Docker, Podman, or any other container runtime. Only use the `container` command. + +## Arguments + +- Test command to run (examples: `test_io`, `test_codecs -v`, `test_io -v -m "test_errors"`) + +## Prerequisites + +- The `container` CLI is installed via `brew install container`. +- The dev image `rustpython-dev` is already built. + +## Workflow + +1. Check whether the test container is already running: + + ```shell + container list 2>/dev/null | grep rustpython-test + ``` + +2. Start the container if it is not running: + + ```shell + container run -d --name rustpython-test -m 8G -c 4 \ + --mount type=bind,source="$(pwd)",target=/workspace \ + -w /workspace rustpython-dev sleep infinity + ``` + +3. Run the requested test command inside the container: + + ```shell + container exec rustpython-test cargo run --release -- -m test + ``` + +4. Report results: + +- Show pass/fail summary and expected failures / unexpected successes. +- Highlight new failures compared to macOS results, if available. +- Do not stop or remove the container after testing. + +## Notes + +- The workspace is bind-mounted, so local code changes are immediately available in the container. +- Use `container exec rustpython-test sh -c "..."` for any command inside the container. +- Rebuild after code changes with: + + ```shell + container exec rustpython-test sh -c "cargo build --release" + ``` + +- Stop the container when explicitly requested: + + ```shell + container rm -f rustpython-test + ``` diff --git a/.agents/skills/investigate-test-failure/SKILL.md b/.agents/skills/investigate-test-failure/SKILL.md new file mode 100644 index 000000000..e63dfeeec --- /dev/null +++ b/.agents/skills/investigate-test-failure/SKILL.md @@ -0,0 +1,53 @@ +--- +name: investigate-test-failure +description: Investigate a failing RustPython test, compare with CPython behavior, and decide whether to implement a fix or prepare incompatibility issue details. +allowed-tools: Bash(python3:*) Bash(cargo run:*) Bash(gh issue create:*) Read Grep Glob Bash(git add:*) Bash(git commit:*) Bash(cargo fmt:*) Bash(git diff:*) Task +--- + +# Investigate Test Failure + +Investigate why a specific test is failing and determine whether it can be fixed now or should be reported as an incompatibility issue. + +## Arguments + +- Failed test identifier (example: `test_inspect.TestGetSourceBase.test_getsource_reload`) + +## Workflow + +1. Analyze the failure cause: + +- Read the test code. +- Analyze failure message and traceback. +- Check related RustPython implementation code. + +2. Verify behavior in CPython: + +- Run the test with `python3 -m unittest`. +- Document expected behavior/output. + +3. Determine fix feasibility: + +- Simple fix (import issues, small logic bugs): implement fix, run formatting and checks, review changes, and commit. +- Complex fix (major missing feature): gather issue information and report to user. + +Pre-commit review process: + +- Run `git diff` to review local changes. +- Use a focused subagent review to compare implementation against CPython behavior and check for missed edge cases. +- Commit only after review passes. + +4. For complex issues, collect issue information using `.github/ISSUE_TEMPLATE/report-incompatibility.md`: + +- Feature: missing or broken Python feature. +- Minimal reproduction code. +- CPython behavior (`python3`). +- RustPython behavior (`cargo run`). +- Python documentation reference link. + +Report the collected information to the user. Create an issue only when explicitly requested. + +Example issue command: + +```shell +gh issue create --template report-incompatibility.md --title "..." --body "..." +``` diff --git a/.agents/skills/upgrade-pylib-next/SKILL.md b/.agents/skills/upgrade-pylib-next/SKILL.md new file mode 100644 index 000000000..e68e73165 --- /dev/null +++ b/.agents/skills/upgrade-pylib-next/SKILL.md @@ -0,0 +1,34 @@ +--- +name: upgrade-pylib-next +description: Pick the next CPython stdlib module ready for upgrade based on update_lib todo output and skip modules with open upgrade PRs. +allowed-tools: Skill(upgrade-pylib) Bash(gh pr list:*) Bash(cargo run:*) +--- + +# Upgrade Next Python Library + +Find the next Python library module ready for upgrade and then run the `upgrade-pylib` workflow for it. + +## Workflow + +1. Get current TODO status: + +```shell +cargo run --release -- scripts/update_lib todo 2>/dev/null +``` + +2. Get open upgrade PRs: + +```shell +gh pr list --search "Update in:title" --json number,title --template '{{range .}}#{{.number}} {{.title}}{{"\\n"}}{{end}}' +``` + +3. From TODO output, select modules in this priority order: + +- `[ ] [no deps]` +- `[ ] [0/n]` where dependencies are all upgraded (for example `[0/3]`, `[0/5]`) + +4. Skip modules that already have an open upgrade PR. + +5. Run upgrade for selected module using the `upgrade-pylib` workflow. + +If no modules match, report that eligible modules are blocked by dependencies. diff --git a/.agents/skills/upgrade-pylib/SKILL.md b/.agents/skills/upgrade-pylib/SKILL.md new file mode 100644 index 000000000..84400e5b7 --- /dev/null +++ b/.agents/skills/upgrade-pylib/SKILL.md @@ -0,0 +1,102 @@ +--- +name: upgrade-pylib +description: Upgrade a Python standard library module from CPython into RustPython using scripts/update_lib and then triage and mark remaining failures. +allowed-tools: Bash(git add:*) Bash(git commit:*) Bash(git diff:*) Bash(cargo run:*) Bash(python3 scripts/update_lib quick:*) Bash(python3 scripts/update_lib auto-mark:*) +--- + +# Upgrade Python Library from CPython + +Upgrade a Python standard library module from CPython to RustPython. + +## Arguments + +- Library name or path (examples: `inspect`, `asyncio`, `json`, `cpython/Lib/inspect.py`, `cpython/Lib/json/`) + +## Important: Report Tool Issues First + +If `scripts/update_lib` shows a bug, missing automation, or unexpected behavior, stop the upgrade and report the tooling issue first, including: + +1. What you were trying to do: + +- Library name. +- Full command used. + +2. What went wrong or what is missing. +3. Expected vs actual behavior. + +## Workflow + +1. Run quick upgrade with update_lib: + +- `python3 scripts/update_lib quick ` +- or `python3 scripts/update_lib quick cpython/Lib/.py` +- or `python3 scripts/update_lib quick cpython/Lib//` + +This step copies library files, patches tests while preserving markers where possible, runs tests, auto-marks new failures, removes stale `@unittest.expectedFailure`, and creates a commit. + +If warnings like `WARNING: TestCFoo does not exist in remote file` appear, class structure changed and some markers must be restored manually. + +2. Review diff and restore RustPython-specific changes: + +- Run `git diff Lib/test/test_`. +- Restore only changes with explicit `RUSTPYTHON` comments. +- Do not restore unrelated upstream CPython changes. + +3. Investigate failing dependent tests: + +- Get dependencies: + + ```shell + cargo run --release -- scripts/update_lib deps + ``` + +- Find direct dependent test modules from the `- [ ] :` line. +- Run dependent tests and collect failures: + + ```shell + cargo run --release -- -m test 2>&1 | grep -E "^(FAIL|ERROR):" + ``` + +- For each failing test identifier, investigate with the `investigate-test-failure` workflow. + +4. Mark remaining failures with auto-mark: + +- `python3 scripts/update_lib auto-mark Lib/test/test_.py --mark-failure` +- or `python3 scripts/update_lib auto-mark Lib/test/test_/ --mark-failure` + +Note: `--mark-failure` marks all current failures including regressions; review carefully. + +5. Handle panics manually: + +- For tests that panic/crash/hang, use `@unittest.skip("TODO: RUSTPYTHON; ...")` rather than expectedFailure. + +6. Handle class-specific failures: + +- If only one subclass fails (for example `TestCFoo` but not `TestPyFoo`), move marker to the failing subclass override. + +7. Commit test-fix markers as a separate commit: + +```shell +git add -u && git commit -m "Mark failing tests" +``` + +## Example usage + +```text +upgrade-pylib inspect +upgrade-pylib json +upgrade-pylib asyncio +upgrade-pylib cpython/Lib/inspect.py +upgrade-pylib cpython/Lib/json/ +``` + +## Notes + +- The `cpython/` directory should contain the CPython source used for sync. +- `scripts/update_lib` modes: + - `quick`: patch + auto-mark + - `migrate`: patch only + - `auto-mark`: mark failures only + - `copy-lib`: copy library files only +- Patching transfers `@unittest.expectedFailure` and `@unittest.skip` with `TODO: RUSTPYTHON` markers where possible. +- The tool does not preserve every RustPython-specific change; always review and restore explicit RUSTPYTHON-marked logic. diff --git a/.claude/commands/apple-container.md b/.claude/commands/apple-container.md deleted file mode 100644 index b0f04c81f..000000000 --- a/.claude/commands/apple-container.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -allowed-tools: Bash(container *), Bash(cargo *), Read, Grep, Glob ---- - -# Run Tests in Linux Container (Apple `container` CLI) - -Run RustPython tests inside a Linux container using Apple's `container` CLI. -**NEVER use Docker, Podman, or any other container runtime.** Only use the `container` command. - -## Arguments -- `$ARGUMENTS`: Test command to run (e.g., `test_io`, `test_codecs -v`, `test_io -v -m "test_errors"`) - -## Prerequisites - -The `container` CLI is installed via `brew install container`. -The dev image `rustpython-dev` is already built. - -## Steps - -1. **Check if the container is already running** - ```shell - container list 2>/dev/null | grep rustpython-test - ``` - -2. **Start the container if not running** - ```shell - container run -d --name rustpython-test -m 8G -c 4 \ - --mount type=bind,source=.,target=/workspace \ - -w /workspace rustpython-dev sleep infinity - ``` - -3. **Run the test inside the container** - ```shell - container exec rustpython-test sh -c "cargo run --release -- -m test $ARGUMENTS" - ``` - -4. **Report results** - - Show test summary (pass/fail counts, expected failures, unexpected successes) - - Highlight any new failures compared to macOS results if available - - Do NOT stop or remove the container after testing (keep it for reuse) - -## Notes -- The workspace is bind-mounted, so local code changes are immediately available -- Use `container exec rustpython-test sh -c "..."` for any command inside the container -- To rebuild after code changes, run: `container exec rustpython-test sh -c "cargo build --release"` -- To stop the container when done: `container rm -f rustpython-test` diff --git a/.claude/commands/investigate-test-failure.md b/.claude/commands/investigate-test-failure.md deleted file mode 100644 index e9d6b2d2d..000000000 --- a/.claude/commands/investigate-test-failure.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -allowed-tools: Bash(python3:*), Bash(cargo run:*), Read, Grep, Glob, Bash(git add:*), Bash(git commit:*), Bash(cargo fmt:*), Bash(git diff:*), Task ---- - -# Investigate Test Failure - -Investigate why a specific test is failing and determine if it can be fixed or needs an issue. - -## Arguments -- `$ARGUMENTS`: Failed test identifier (e.g., `test_inspect.TestGetSourceBase.test_getsource_reload`) - -## Steps - -1. **Analyze failure cause** - - Read the test code - - Analyze failure message/traceback - - Check related RustPython code - -2. **Verify behavior in CPython** - - Run the test with `python3 -m unittest` to confirm expected behavior - - Document the expected output - -3. **Determine fix feasibility** - - **Simple fix** (import issues, small logic bugs): Fix code → Run `cargo fmt --all` → Pre-commit review → Commit - - **Complex fix** (major unimplemented features): Collect issue info and report to user - - **Pre-commit review process**: - - Run `git diff` to see the changes - - Use Task tool with `general-purpose` subagent to review: - - Compare implementation against cpython/ source code - - Verify the fix aligns with CPython behavior - - Check for any missed edge cases - - Proceed to commit only after review passes - -4. **For complex issues - Collect issue information** - Following `.github/ISSUE_TEMPLATE/report-incompatibility.md` format: - - - **Feature**: Description of missing/broken Python feature - - **Minimal reproduction code**: Smallest code that reproduces the issue - - **CPython behavior**: Result when running with python3 - - **RustPython behavior**: Result when running with cargo run - - **Python Documentation link**: Link to relevant CPython docs - - Report collected information to the user. Issue creation is done only upon user request. - - Example issue creation command: - ``` - gh issue create --template report-incompatibility.md --title "..." --body "..." - ``` diff --git a/.claude/commands/upgrade-pylib-next.md b/.claude/commands/upgrade-pylib-next.md deleted file mode 100644 index 712b79433..000000000 --- a/.claude/commands/upgrade-pylib-next.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -allowed-tools: Skill(upgrade-pylib), Bash(gh pr list:*) ---- - -# Upgrade Next Python Library - -Find the next Python library module ready for upgrade and run `/upgrade-pylib` for it. - -## Current TODO Status - -!`cargo run --release -- scripts/update_lib todo 2>/dev/null` - -## Open Upgrade PRs - -!`gh pr list --search "Update in:title" --json number,title --template '{{range .}}#{{.number}} {{.title}}{{"\n"}}{{end}}'` - -## Instructions - -From the TODO list above, find modules matching these patterns (in priority order): - -1. `[ ] [no deps]` - Modules with no dependencies (can be upgraded immediately) -2. `[ ] [0/n]` - Modules where all dependencies are already upgraded (e.g., `[0/3]`, `[0/5]`) - -These patterns indicate modules that are ready to upgrade without blocking dependencies. - -**Important**: Skip any modules that already have an open PR in the "Open Upgrade PRs" list above. - -**After identifying a suitable module**, run: -``` -/upgrade-pylib -``` - -If no modules match these criteria, inform the user that all eligible modules have dependencies that need to be upgraded first. diff --git a/.claude/commands/upgrade-pylib.md b/.claude/commands/upgrade-pylib.md deleted file mode 100644 index d54305d26..000000000 --- a/.claude/commands/upgrade-pylib.md +++ /dev/null @@ -1,157 +0,0 @@ ---- -allowed-tools: Bash(git add:*), Bash(git commit:*), Bash(python3 scripts/update_lib quick:*), Bash(python3 scripts/update_lib auto-mark:*) ---- - -# Upgrade Python Library from CPython - -Upgrade a Python standard library module from CPython to RustPython. - -## Arguments -- `$ARGUMENTS`: Library name to upgrade (e.g., `inspect`, `asyncio`, `json`) - -## Important: Report Tool Issues First - -If during the upgrade process you encounter any of the following issues with `scripts/update_lib`: -- A feature that should be automated but isn't supported -- A bug or unexpected behavior in the tool -- Missing functionality that would make the upgrade easier - -**STOP the upgrade and report the issue first.** Describe: -1. What you were trying to do - - Library name - - The full command executed (e.g. python scripts/update_lib quick cpython/Lib/$ARGUMENTS.py) -2. What went wrong or what's missing -3. Expected vs actual behavior - -This helps improve the tooling for future upgrades. - -## Steps - -1. **Run quick upgrade with update_lib** - - Run: `python3 scripts/update_lib quick $ARGUMENTS` (module name) - - Or: `python3 scripts/update_lib quick cpython/Lib/$ARGUMENTS.py` (library file path) - - Or: `python3 scripts/update_lib quick cpython/Lib/$ARGUMENTS/` (library directory path) - - This will: - - Copy library files (delete existing `Lib/$ARGUMENTS.py` or `Lib/$ARGUMENTS/`, then copy from `cpython/Lib/`) - - Patch test files preserving existing RustPython markers - - Run tests and auto-mark new test failures (not regressions) - - Remove `@unittest.expectedFailure` from tests that now pass - - Create a git commit with the changes - - **Handle warnings**: If you see warnings like `WARNING: TestCFoo does not exist in remote file`, it means the class structure changed and markers couldn't be transferred automatically. These need to be manually restored in step 2 or added in step 3. - -2. **Review git diff and restore RUSTPYTHON-specific changes** - - Run `git diff Lib/test/test_$ARGUMENTS` to review all changes - - **Only restore changes that have explicit `RUSTPYTHON` comments**. Look for: - - `# XXX: RUSTPYTHON` or `# XXX RUSTPYTHON` - Comments marking RustPython-specific code modifications - - `# TODO: RUSTPYTHON` - Comments marking tests that need work - - Code changes with inline `# ... RUSTPYTHON` comments - - **Do NOT restore other diff changes** - these are likely upstream CPython changes, not RustPython-specific modifications - - When restoring, preserve the original context and formatting - -3. **Investigate test failures with subagent** - - First, get dependent tests using the deps command: - ``` - cargo run --release -- scripts/update_lib deps $ARGUMENTS - ``` - - Look for the line `- [ ] $ARGUMENTS: test_xxx test_yyy ...` to get the direct dependent tests - - Run those tests to collect failures: - ``` - cargo run --release -- -m test test_xxx test_yyy ... 2>&1 | grep -E "^(FAIL|ERROR):" - ``` - - For example, if deps output shows `- [ ] linecache: test_bdb test_inspect test_linecache test_traceback test_zipimport`, run: - ``` - cargo run --release -- -m test test_bdb test_inspect test_linecache test_traceback test_zipimport 2>&1 | grep -E "^(FAIL|ERROR):" - ``` - - For each failure, use the Task tool with `general-purpose` subagent to investigate: - - Subagent should follow the `/investigate-test-failure` skill workflow - - Pass the failed test identifier as the argument (e.g., `test_inspect.TestGetSourceBase.test_getsource_reload`) - - If subagent can fix the issue easily: fix and commit - - If complex issue: subagent collects issue info and reports back (issue creation on user request only) - - Using subagent prevents context pollution in the main conversation - -4. **Mark remaining test failures with auto-mark** - - Run: `python3 scripts/update_lib auto-mark Lib/test/test_$ARGUMENTS.py --mark-failure` - - Or for directory: `python3 scripts/update_lib auto-mark Lib/test/test_$ARGUMENTS/ --mark-failure` - - This will: - - Run tests and mark ALL failing tests with `@unittest.expectedFailure` - - Remove `@unittest.expectedFailure` from tests that now pass - - **Note**: The `--mark-failure` flag marks all failures including regressions. Review the changes before committing. - -5. **Handle panics manually** - - If any tests cause panics/crashes (not just assertion failures), they need `@unittest.skip` instead: - ```python - @unittest.skip("TODO: RUSTPYTHON; panics with 'index out of bounds'") - def test_crashes(self): - ... - ``` - - auto-mark cannot detect panics automatically - check the test output for crash messages - -6. **Handle class-specific failures** - - If a test fails only in the C implementation (TestCFoo) but passes in the Python implementation (TestPyFoo), or vice versa, move the marker to the specific subclass: - ```python - # Base class - no marker here - class TestFoo: - def test_something(self): - ... - - class TestPyFoo(TestFoo, PyTest): pass - - class TestCFoo(TestFoo, CTest): - # TODO: RUSTPYTHON - @unittest.expectedFailure - def test_something(self): - return super().test_something() - ``` - -7. **Commit the test fixes** - - Run: `git add -u && git commit -m "Mark failing tests"` - - This creates a separate commit for the test markers added in steps 2-6 - -## Example Usage -``` -# Using module names (recommended) -/upgrade-pylib inspect -/upgrade-pylib json -/upgrade-pylib asyncio - -# Using library paths (alternative) -/upgrade-pylib cpython/Lib/inspect.py -/upgrade-pylib cpython/Lib/json/ -``` - -## Example: Restoring RUSTPYTHON changes - -When git diff shows removed RUSTPYTHON-specific code like: -```diff --# XXX RUSTPYTHON: we don't import _json as fresh since... --cjson = import_helper.import_fresh_module('json') #, fresh=['_json']) -+cjson = import_helper.import_fresh_module('json', fresh=['_json']) -``` - -You should restore the RustPython version: -```python -# XXX RUSTPYTHON: we don't import _json as fresh since... -cjson = import_helper.import_fresh_module('json') #, fresh=['_json']) -``` - -## Notes -- The cpython/ directory should contain the CPython source that we're syncing from -- `scripts/update_lib` package handles patching and auto-marking: - - `quick` - Combined patch + auto-mark (recommended) - - `migrate` - Only migrate (patch), no test running - - `auto-mark` - Only run tests and mark failures - - `copy-lib` - Copy library files (not tests) -- The patching: - - Transfers `@unittest.expectedFailure` and `@unittest.skip` decorators with `TODO: RUSTPYTHON` markers - - Adds `import unittest # XXX: RUSTPYTHON` if needed for the decorators - - **Limitation**: If a class was restructured (e.g., method overrides removed), update_lib will warn and skip those markers -- The smart auto-mark: - - Marks NEW test failures automatically (tests that didn't exist before) - - Does NOT mark regressions (existing tests that now fail) - these are warnings - - Removes `@unittest.expectedFailure` from tests that now pass -- The script does NOT preserve all RustPython-specific changes - you must review `git diff` and restore them -- Common RustPython markers to look for: - - `# XXX: RUSTPYTHON` or `# XXX RUSTPYTHON` - Inline comments for code modifications - - `# TODO: RUSTPYTHON` - Test skip/failure markers - - Any code with `RUSTPYTHON` in comments that was removed in the diff -- **Important**: Not all changes in the git diff need to be restored. Only restore changes that have explicit `RUSTPYTHON` comments. Other changes are upstream CPython updates.