mirror of
https://github.com/astral-sh/ruff.git
synced 2026-10-28 10:45:37 +08:00
Add agent skills for working on ty (#25422)
## Summary This PR adds a `.agents/skills` directory to the repo, and adds three skills to help agents work on ty more effectively: - A skill to help them write better ty diagnostics - A skill to help them minimize a single ecosystem change more effectively - A skill to help them analyze and summarise ecosystem reports more effectively The skills have been designed so that they will (hopefully) work well whether a contributor is using Codex or Claude as their agent locally. Codex should discover these skills automatically if you're working on the repo; if you use Claude locally, you _should_ hopefully be prompted by Claude to install the local skills due to them being listed as recommended skills in `.claude/settings.json`. ## Test Plan I ran `/review` locally with Codex, instructing it to be as nitpicky as possible, until it could find no more issues with these skills.
This commit is contained in:
Alex Waygood authored and GitHub committed
1 parent
574e10752f
commit
7e1bc1e75f
9 files changed
+184
-14
No files matched your search
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"name": "ruff-agent-skills",
|
||||
"owner": {
|
||||
"name": "Ruff contributors"
|
||||
},
|
||||
"description": "Local Claude Code skills for working on ty in the Ruff repository",
|
||||
"plugins": [
|
||||
{
|
||||
"name": "ty-skills",
|
||||
"source": "./",
|
||||
"description": "Local skills for ty diagnostics and ecosystem workflows"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"name": "ty-skills",
|
||||
"description": "Local skills for working on ty in the Ruff repository"
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
---
|
||||
name: adding-ty-diagnostics
|
||||
description: Use when adding, updating, or reviewing ty checks, diagnostics, diagnostic messages, subdiagnostics, or concise output behavior.
|
||||
---
|
||||
|
||||
# Adding Ty Diagnostics
|
||||
|
||||
Use this skill when adding or changing a ty diagnostic, especially as part of a new ty check.
|
||||
|
||||
**Keep error messages concise.** Think about how the diagnostic will look on a narrow terminal screen.
|
||||
|
||||
Put extra detail in subdiagnostics or secondary annotations when that helps, but make sure the primary diagnostic is understandable on its own.
|
||||
|
||||
Always check that the diagnostic still makes sense when the user passes `--output-format=concise`.
|
||||
|
||||
If the error code is entirely new or if you have changed the documentation for the error code,
|
||||
you will need to run `cargo dev generate-all` after making your changes, to update the generated schema for ty
|
||||
and the generated `.md` documentation files.
|
||||
|
||||
Diagnostics should usually be tested using mdtests. If you are changing behaviour for an existing diagnostic,
|
||||
you should usually add your tests to a pre-existing `.md` file; otherwise, it may be appropriate to add a new
|
||||
`.md` file for your tests. Snapshot tests are only usually necessary for diagnostics that use secondary annotations
|
||||
or subdiagnostics. If you want to add a snapshot, inline `# snapshot` comments are preferred over the legacy
|
||||
`<!-- snapshot-annotations -->` directive.
|
||||
|
||||
When using the `declare_lint!` macro, the `status` field should be set to `LintStatus::stable(<next version of ty>)`.
|
||||
You should determine what the next version of ty will be by inspecting https://pypi.org/pypi/ty/json, finding what
|
||||
the latest release of ty is, and incrementing the patch version by one. For example, if the latest release of ty is `0.5.3`, the status should be `LintStatus::stable("0.5.4")`.
|
||||
@@ -0,0 +1,47 @@
|
||||
---
|
||||
name: minimizing-ty-ecosystem-changes
|
||||
description: Use when reproducing, investigating, or minimizing behavior changes in ty ecosystem or primer projects.
|
||||
---
|
||||
|
||||
# Minimizing Ty Ecosystem Changes
|
||||
|
||||
Use this skill when asked to reproduce a ty ecosystem change, investigate a behavior difference in a primer project, or minimize a reproducer from a ty ecosystem project.
|
||||
|
||||
## Reproduce First
|
||||
|
||||
From the repository root, clone the project and install its dependencies into `.venv`:
|
||||
|
||||
```sh
|
||||
uv run scripts/setup_primer_project.py <project-name> <some-temp-dir>
|
||||
```
|
||||
|
||||
Confirm the behavior difference reproduces before minimizing or explaining it.
|
||||
|
||||
## Minimize
|
||||
|
||||
When asked to minimize an ecosystem change, start from the reproduced project. Reduce the Python code until only the smallest code needed to demonstrate the behavior difference remains. Even if the cause looks obvious, **do not skip ahead** to an explanation or a hand-written reproducer. Use a rigorous, systematic process and verify after each reduction that the behavior difference still reproduces.
|
||||
|
||||
An ideal minimized reproducer:
|
||||
|
||||
- Is a single file that is as small as possible.
|
||||
- Has as few third-party imports as possible, ideally none.
|
||||
- Uses smaller third-party packages with fewer dependencies when third-party imports are unavoidable. For example, prefer an import from `numpy` over one from `pandas`.
|
||||
- Has as few first-party and standard-library imports as possible.
|
||||
- Keeps imports from modules with highly special-cased symbols only when they are necessary (such modules may include `typing`, `abc`, `enum`, `types`, or `typing_extensions`).
|
||||
- Uses an absolute minimum of "advanced"/complex typing or language features. For example, if the original code
|
||||
uses a walrus operator, but the behavior difference still reproduces without it, remove the walrus operator. If the original code uses a `Protocol` from `typing_extensions`, but the behavior difference still reproduces without it, remove the `Protocol`.
|
||||
|
||||
Use a systematic loop. You do not need to apply every tool in every iteration, but keep looping until none of the available minimization tools can reduce the reproducer further while preserving the behavior difference.
|
||||
|
||||
Available minimization tools include:
|
||||
|
||||
1. Remove files that are not needed for the difference.
|
||||
2. Strip imports, definitions, and statements from the remaining files.
|
||||
3. Cut and paste definitions from one file into another file to reduce first-party or third-party imports.
|
||||
4. Inline the relevant parts of third-party dependencies into first-party code to reduce third-party imports.
|
||||
5. Inline the relevant parts of stdlib definitions from typeshed into first-party code to reduce stdlib imports.
|
||||
|
||||
After applying a minimization tool, re-run the comparison between the feature branch and `main`.
|
||||
Keep a reduction only when the behavior difference still reproduces.
|
||||
|
||||
Stop looping only when no further reductions could be applied without making the behavior difference disappear.
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
name: summarise-ecosystem-results
|
||||
description: Use when asked to summarise or summarize ty ecosystem results for a Ruff PR, including from a PR number, PR URL, GitHub ecosystem-results comment, or full detailed HTML report.
|
||||
---
|
||||
|
||||
# Summarise Ecosystem Results
|
||||
|
||||
Use this skill when asked to summarise ecosystem results for a Ruff PR with the `ty` label.
|
||||
|
||||
## Find the Report
|
||||
|
||||
Accept any of these inputs:
|
||||
|
||||
- A PR number.
|
||||
- A GitHub PR URL.
|
||||
- A GitHub comment URL on a PR, such as `https://github.com/astral-sh/ruff/pull/25342#issuecomment-4525002693`.
|
||||
- A full detailed HTML ecosystem report URL.
|
||||
|
||||
Determine the PR number first. If the user gave only a PR number, open `https://github.com/astral-sh/ruff/pull/<number>`.
|
||||
|
||||
Find the ty ecosystem-results comment on the PR. Search PR comments for terms such as "ecosystem", "full report", "HTML report", and "detailed report". From that comment, open the linked full detailed HTML report.
|
||||
|
||||
Use the PR comment as the change list and the full detailed HTML report as the source of detailed evidence. When the report includes exact project revisions, use those revisions rather than current upstream checkouts.
|
||||
|
||||
## Minimize in Parallel
|
||||
|
||||
Before minimizing, load and apply the `minimizing-ty-ecosystem-changes` skill to each ecosystem change.
|
||||
|
||||
If possible, use subagents to parallelize this work. Decide how to batch changes so the overall task finishes as quickly as possible while still allowing each subagent to work methodically. Reasonable batching strategies include grouping related changes by project, diagnostic code, suspected cause, or report section, while keeping large groups split enough to avoid one slow subagent blocking the whole task.
|
||||
|
||||
If subagents are not available, batch the minimization work manually and minimize the batches sequentially. Keep batches small enough that each pass can still be checked carefully.
|
||||
|
||||
Give each subagent a self-contained assignment:
|
||||
|
||||
- The PR number, PR URL, ecosystem comment URL, and detailed HTML report URL.
|
||||
- The exact ecosystem changes assigned to that subagent.
|
||||
- The requirement to use the `minimizing-ty-ecosystem-changes` process rigorously.
|
||||
- The expected Markdown output format for each minimized change.
|
||||
|
||||
Each subagent should proceed methodically through all assigned changes. If a subagent moves on to a new change and that change appears very similar to one it has already minimized, it may skip the new change, but it must record why the skipped change appears to demonstrate the same behavior.
|
||||
|
||||
## Collect Results
|
||||
|
||||
After all subagents finish, collect their minimizations into one Markdown file at the repository root:
|
||||
|
||||
```text
|
||||
PR_<number>_ECOSYSTEM_SUMMARY.md
|
||||
```
|
||||
|
||||
Remove minimizations that appear to demonstrate the same behavior change. Prefer the smallest and clearest minimized reproducer, especially one that is single-file and has fewer imports.
|
||||
|
||||
At the top of the file, add prose summarising the distinct behavior changes demonstrated by the retained minimizations. Then include the retained minimizations with enough detail for a reader to understand and reproduce them.
|
||||
|
||||
For each retained minimization, include:
|
||||
|
||||
- The original project and report entry.
|
||||
- The diagnostic or behavior change on `main` versus the PR.
|
||||
- The minimized code.
|
||||
- The commands or comparison method used to verify the minimization.
|
||||
- Any source links needed to trace the example back to the PR comment or full report.
|
||||
|
||||
Present `PR_<number>_ECOSYSTEM_SUMMARY.md` as the finished product.
|
||||
@@ -1,4 +1,15 @@
|
||||
{
|
||||
"enabledPlugins": {
|
||||
"ty-skills@ruff-agent-skills": true
|
||||
},
|
||||
"extraKnownMarketplaces": {
|
||||
"ruff-agent-skills": {
|
||||
"source": {
|
||||
"source": "directory",
|
||||
"path": ".agents"
|
||||
}
|
||||
}
|
||||
},
|
||||
"hooks": {
|
||||
"SessionStart": [
|
||||
{
|
||||
|
||||
@@ -84,7 +84,8 @@ repos:
|
||||
- mdformat-footnote==0.1.3
|
||||
exclude: |
|
||||
(?x)^(
|
||||
docs/formatter/black\.md
|
||||
\.agents/skills/.*
|
||||
| docs/formatter/black\.md
|
||||
| docs/\w+\.md
|
||||
)$
|
||||
priority: 0
|
||||
@@ -109,7 +110,8 @@ repos:
|
||||
- id: markdownlint-fix
|
||||
exclude: |
|
||||
(?x)^(
|
||||
docs/formatter/black\.md
|
||||
\.agents/skills/.*
|
||||
| docs/formatter/black\.md
|
||||
| docs/\w+\.md
|
||||
)$
|
||||
priority: 1
|
||||
|
||||
@@ -54,17 +54,6 @@ Run ty:
|
||||
cargo run --bin ty -- check path/to/file.py
|
||||
```
|
||||
|
||||
## Reproducing and minimizing ty ecosystem changes
|
||||
|
||||
If asked to reproduce changes in the ty ecosystem, use this script to clone the project to some
|
||||
directory and install its dependencies into `.venv`:
|
||||
|
||||
```sh
|
||||
uv run scripts/setup_primer_project.py <project-name> <some-temp-dir>
|
||||
```
|
||||
|
||||
If asked to *minimize* a change in the ty ecosystem, you should start off with the above command to ensure that the change reproduces. You should then attempt to minimize the Python code required to demonstrate a behaviour difference between ty on your feature branch and ty on the main branch. Your minimization process should consist of systematically removing files from the cloned ecosystem project, and stripping content from existing files, until the behaviour difference between your branch and `main` no longer reproduces.
|
||||
|
||||
## Pull Requests
|
||||
|
||||
When working on ty, PR titles should start with `[ty]` and be tagged with the `ty` GitHub label.
|
||||
@@ -83,7 +72,6 @@ When working on ty, PR titles should start with `[ty]` and be tagged with the `t
|
||||
- Prefer let chains (`if let` combined with `&&`) over nested `if let` statements to reduce indentation and improve readability. At the end of a task, always check your work to see if you missed opportunities to use `let` chains.
|
||||
- If you *have* to suppress a Clippy lint, prefer to use `#[expect()]` over `[allow()]`, where possible. But if a lint is complaining about unused/dead code, it's usually best to just delete the unused code.
|
||||
- Use comments purposefully. Don't use comments to narrate code, but do use them to explain invariants and why something unusual was done a particular way.
|
||||
- When adding new ty checks, it's important to make error messages concise. Think about how an error message would look on a narrow terminal screen. Sometimes more detail can be provided in subdiagnostics or secondary annotations, but it's also important to make sure that the diagnostic is understandable if the user has passed `--output-format=concise`.
|
||||
- **Salsa incrementality (ty):** Any method that accesses `.node()` must be `#[salsa::tracked]`, or it will break incrementality. Prefer higher-level semantic APIs over raw AST access.
|
||||
- Run `cargo dev generate-all` after changing configuration options, CLI arguments, lint rules, or environment variable definitions, as these changes require regeneration of schemas, docs, and CLI references.
|
||||
- Don't prefix tests with `test_`.
|
||||
|
||||
@@ -44,6 +44,20 @@ We **require all use of AI in contributions to follow our
|
||||
|
||||
If your contribution does not follow the policy, it will be closed.
|
||||
|
||||
### Local skills for Codex or Claude
|
||||
|
||||
This repository includes local agent skills for ty development under `.agents/skills`.
|
||||
|
||||
Contributors using Codex for development should find that Codex auto-discovers the skills and uses
|
||||
them automatically when necessary. Claude Code users may be prompted to install the
|
||||
`ty-skills@ruff-agent-skills` local plugin. If the skills are not installed automatically, install
|
||||
them manually from the repository root by running these slash commands inside Claude Code:
|
||||
|
||||
```text
|
||||
/plugin marketplace add ./.agents
|
||||
/plugin install ty-skills@ruff-agent-skills
|
||||
```
|
||||
|
||||
## The Basics
|
||||
|
||||
### Prerequisites
|
||||
|
||||
Reference in new issue
Block a user