chore: add triage skill and tooling for issue/PR investigation

Adds a `triage` skill documenting how to investigate a GitHub issue or PR
and record a durable verdict, ahead of a sweep over the open PR backlog.

Write-ups live in the gitignored `.triage/` tree, one directory per ticket
(`.triage/issues/NNNN/`, `.triage/prs/NNNN/`), each holding `results.md`
plus whatever repros and fixtures the investigation needs. Every write-up
opens with YAML frontmatter so a 250-PR sweep stays greppable and can be
resumed from disk; `reindex.mjs` derives the index table from it.

The skill is discoverable by both Claude (`.claude/skills/`) and Codex
(via the `.codex/skills` symlink), with a pointer in AGENTS.md for agents
that don't auto-load skills.

Also included:

- `scripts/triage-signal.ts`, which ranks open issues and PRs by a
  synthesized engagement signal to prioritize the backlog. Committed here
  only because the pre-commit hook rejects any untracked file.
- Ignore `bun.lock` alongside the existing `bun.lockb`, and let `.fray/`
  self-ignore the way `.frizz/` already does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Colin McDonnellandClaude Opus 5 committed 2026-08-16 15:45:17 -07:00
1 parent eb4682c930
commit ab4a5db809
6 files changed
+947 -1

No files matched your search

+149
View File
@@ -0,0 +1,149 @@
---
name: triage
description: Investigate a GitHub issue or pull request in colinhacks/zod and write up a durable verdict. Use whenever asked to triage, investigate, review, evaluate, or form an opinion on an issue or PR (by number, URL, or "the open PR queue"), and when sweeping many of them in bulk. Covers where write-ups live on disk (.triage/issues/NNNN/, .triage/prs/NNNN/), the cheap-disqualifier pass that avoids spinning up a worktree for stale PRs, the worktree checkout procedure for PRs, and the results.md format that keeps a 250-PR sweep greppable and resumable.
---
# Investigating issues and PRs
Produce a write-up a maintainer can act on without re-reading the thread. The deliverable is always a file on disk, never just a chat message — sweeps span many sessions and agents, so anything living only in a transcript is lost.
**Read-only on GitHub.** Never comment, label, assign, close, review, approve, or merge unless the user explicitly asks in this session. You are forming an opinion, not publishing one.
## Where things live
Everything goes under `.triage/` in the **root repo**, which is gitignored — write freely, it never shows up in `git status`.
```
.triage/
index.md # the original issue sweep index — hand-maintained prose, still the canonical issue queue
issues/index.md # generated from frontmatter by reindex.mjs
issues/<N>/results.md # the write-up; scratch files live beside it
prs/index.md # generated
prs/<N>/results.md
```
One directory per ticket, named by bare GitHub number (no leading zeros, no `issue-`/`pr-` prefix). `results.md` is the write-up. Everything else you need — repro scripts, a saved `gh pr diff`, test fixtures, benchmark output, notes — goes in that same directory. Name scratch files descriptively (`repro.ts`, `diff.patch`, `bench-before.txt`); there is no naming rule beyond not colliding with `results.md`.
For an issue repro, `pnpm dev .triage/issues/<N>/repro.ts` from the root repo works directly.
A repro must sit **inside** the checkout it is testing, because module resolution walks up from the file, not from the cwd — a script under `.triage/` cannot resolve `zod` from a PR worktree. Keep the canonical copy in the ticket directory and copy it in to run: `cp .triage/prs/<N>/repro.ts <worktree>/repro-<N>.ts`. That is also what lets you run the identical file against both `main` and the PR and diff the output, which is the cheapest way to characterize a behavior change. For a probe needing GC or other flags, skip `pnpm dev` and invoke node directly: `node --expose-gc --conditions=@zod/source --import tsx repro-<N>.ts`.
### Writing to the root repo from a PR worktree
This is the one mistake that silently destroys work. A worktree has its **own** gitignored `.triage/`, so a relative write from inside it lands somewhere nobody will ever look.
Resolve the root once and always write through it:
```bash
ZOD_ROOT=$(git worktree list | head -1 | awk '{print $1}') # correct from inside any worktree
mkdir -p "$ZOD_ROOT/.triage/prs/<N>"
```
Every path you write must start with `$ZOD_ROOT`. Investigate in the worktree, record in the root.
## results.md format
Open with YAML frontmatter. It is what makes a 250-PR sweep aggregatable and resumable — an interrupted sweep is resumed by grepping `status`, never by re-reading write-ups.
```yaml
---
number: 5928
type: pr # pr | issue
status: done # in-progress | done
verdict: merge-with-changes
title: "fix: preserve brand through .optional()"
author: someuser
url: https://github.com/colinhacks/zod/pull/5928
investigated: 2026-08-08
---
```
`verdict` for **PRs** — one of:
- `merge-as-is` — correct, tested, in scope. Say so plainly.
- `merge-with-changes` — right idea, needs specific edits. Enumerate them concretely enough to apply.
- `needs-author-changes` — blocked on something only the author can supply (repro, rebase, scope cut).
- `decline` — out of scope or wrong approach. Give the one or two strongest reasons, plus the escape hatch users actually have.
- `superseded` — already fixed on `main` or landed via another PR. Name the commit or PR.
- `needs-maintainer-decision` — the code is fine; the call is a product/API judgment that is Colin's to make. State the tradeoff and your recommendation.
`verdict` for **issues** — reuse the vocabulary already established in `.triage/index.md`: `good-idea`, `already-fixed`, `needs-maintainer-input`, `not-recommended`, `needs-more-reproduction`, `duplicate` (add a `duplicate_of: NNNN` key alongside it).
Set `status: in-progress` in a stub the moment you start, so a parallel agent or a later session does not duplicate the work. Flip to `done` only when the write-up stands on its own.
After the frontmatter, write prose under these headings. Match the density of the existing issue write-ups in `.triage/issues/*/results.md` — they are the house style; skim one before your first write-up.
- `## Request` / `## Change` — what is actually being asked or changed, in your own words. Include the thread's evolution when maintainers or the author already moved the discussion.
- `## Current state` — what the code on `main` does **today**, with `file.ts:line` references. Most old tickets were filed against v3 or early v4 and are stale on arrival; this section is usually where the verdict comes from.
- `## Analysis` — correctness, edge cases, type-level implications, perf, and whether the tests actually cover the claim.
- `## Recommendation` — the action to take, and the substance of the reply to post. Do not draft it in maintainer voice unless asked; AGENTS.md owns that tone if you are asked to post.
## Investigating an issue
No worktree — read the code on `main`.
1. `gh issue view <N> -R colinhacks/zod --comments` and read the whole thread, including maintainer replies. A prior decision from Colin usually settles it.
2. Check for duplicates and prior art: `gh search issues --repo colinhacks/zod '<keywords>'`, and grep `.triage/issues/*/results.md` — the sweep may already cover it. Cross-link with a relative `../NNNN/results.md`.
3. Reproduce against current source before believing the report. Write the repro to `.triage/issues/<N>/repro.ts` and run `pnpm dev` on it. A large share of open issues are already fixed.
4. Brainstorm before concluding. Consider the fix, its blast radius, whether it belongs in core at all, and what userland escape hatch already exists.
5. Write `results.md`.
## Investigating a PR
Cheap pass first — most of a 250-PR backlog is resolved without ever checking out code.
```bash
gh pr view <N> -R colinhacks/zod --json number,title,author,state,isDraft,mergeable,mergeStateStatus,baseRefName,headRefName,headRepositoryOwner,maintainerCanModify,createdAt,updatedAt,additions,deletions,changedFiles,labels,comments
gh pr diff <N> -R colinhacks/zod
```
Disqualify early and write the write-up straight from this, no worktree, when the diff is small and self-evident, or when the PR targets v3 / duplicates a landed change / is already superseded by `main` / is abandoned with unresolved conflicts. Verify `superseded` against actual code on `main` — do not infer it from dates.
Otherwise check it out (this repo's convention, per AGENTS.md):
```bash
git fetch origin pull/<N>/head:pr-<N>
git worktree add ~/.cursor/worktrees/zod/pr-<N> pr-<N>
cd ~/.cursor/worktrees/zod/pr-<N>
pnpm install --frozen-lockfile # fast, the pnpm store is shared
```
Do **not** use `gh pr checkout --detach` — it detaches your current working tree instead of creating a worktree.
Then, in the worktree:
1. Read the diff in full context, not just the hunks. What did it touch that the author did not think about?
2. `git log --oneline main..HEAD` — is this rebased on anything current, or written against a version of the code that has since moved?
3. Run the tests the change implicates: `pnpm vitest run <path>`. Run the full suite only when the change is broad.
4. Judge test coverage against the repo's own bar: a feature or bug fix without a test is incomplete, and tests must be TypeScript.
5. Probe the edge cases the author's tests skip — write them into `$ZOD_ROOT/.triage/prs/<N>/` and run them. A passing suite over a shallow test is not evidence.
6. Write `$ZOD_ROOT/.triage/prs/<N>/results.md`.
**Clean up when done**, or worktrees accumulate across a large sweep:
```bash
cd "$ZOD_ROOT"
git worktree remove ~/.cursor/worktrees/zod/pr-<N> && git branch -D pr-<N>
```
Keep the worktree only when the verdict is `merge-with-changes` and you have been asked to push the changes; AGENTS.md covers pushing back to a contributor's fork, including preserving their commits.
## Sweeping in bulk
Seed the queue, then work it:
```bash
gh pr list -R colinhacks/zod --state open --limit 400 --json number,title,updatedAt
```
Resume by asking disk, not memory:
```bash
grep -l 'status: done' .triage/prs/*/results.md | wc -l # progress
grep -h '^verdict:' .triage/prs/*/results.md | sort | uniq -c # verdict spread
node .claude/skills/triage/scripts/reindex.mjs prs # regenerate .triage/prs/index.md
```
`reindex.mjs` rebuilds an index table from frontmatter, so the index is derived and never hand-edited for PRs. Pass `issues` or `prs`.
When parallelizing across sub-agents, give each agent a disjoint set of numbers and have each write only its own ticket directories. Never have two agents share a results file.
+86
View File
@@ -0,0 +1,86 @@
#!/usr/bin/env node
// Regenerate .triage/<kind>/index.md from the YAML frontmatter of every results.md.
// Usage: node .claude/skills/triage/scripts/reindex.mjs [issues|prs]
import { readdirSync, readFileSync, writeFileSync, existsSync } from "node:fs";
import { join } from "node:path";
import { execFileSync } from "node:child_process";
const kind = process.argv[2] ?? "prs";
if (kind !== "issues" && kind !== "prs") {
console.error("usage: reindex.mjs [issues|prs]");
process.exit(1);
}
const root = execFileSync("git", ["worktree", "list"], { encoding: "utf8" }).split("\n")[0].split(" ")[0];
const dir = join(root, ".triage", kind);
if (!existsSync(dir)) {
console.error(`no such directory: ${dir}`);
process.exit(1);
}
// Minimal frontmatter reader: flat `key: value` pairs only, which is all the format uses.
function frontmatter(file) {
const text = readFileSync(file, "utf8");
const m = text.match(/^---\r?\n([\s\S]*?)\r?\n---/);
if (!m) return null;
const out = {};
for (const line of m[1].split(/\r?\n/)) {
const kv = line.match(/^([A-Za-z_][\w-]*):\s*(.*)$/);
if (kv) out[kv[1]] = kv[2].trim().replace(/^["'](.*)["']$/, "$1");
}
return out;
}
const rows = [];
for (const entry of readdirSync(dir, { withFileTypes: true })) {
if (!entry.isDirectory() || !/^\d+$/.test(entry.name)) continue;
const file = join(dir, entry.name, "results.md");
if (!existsSync(file)) continue;
const fm = frontmatter(file);
rows.push({
number: Number(entry.name),
status: fm?.status ?? "unknown",
verdict: fm?.verdict ?? "—",
title: fm?.title ?? "",
author: fm?.author ?? "",
});
}
rows.sort((a, b) => b.number - a.number);
const noun = kind === "prs" ? "pull" : "issues";
const counts = {};
for (const r of rows) counts[r.verdict] = (counts[r.verdict] ?? 0) + 1;
const done = rows.filter((r) => r.status === "done").length;
const lines = [
`# ${kind === "prs" ? "Open PR" : "Open issue"} triage index`,
"",
`Generated by \`.claude/skills/triage/scripts/reindex.mjs\` — do not hand-edit. Derived from the frontmatter of \`.triage/${kind}/*/results.md\`.`,
"",
`${rows.length} written up, ${done} marked \`done\`.`,
"",
"## Verdicts",
"",
...Object.entries(counts)
.sort((a, b) => b[1] - a[1])
.map(([v, n]) => `- \`${v}\` — ${n}`),
"",
"## Entries",
"",
"| # | Verdict | Status | Title | Author | Write-up |",
"| --- | --- | --- | --- | --- | --- |",
...rows.map((r) =>
[
`[#${r.number}](https://github.com/colinhacks/zod/${noun}/${r.number})`,
`\`${r.verdict}\``,
r.status,
r.title.replace(/\|/g, "\\|"),
r.author ? `@${r.author}` : "",
`[results.md](${r.number}/results.md)`,
].join(" | ").replace(/^/, "| ").concat(" |")
),
"",
];
writeFileSync(join(dir, "index.md"), lines.join("\n"));
console.log(`wrote ${join(dir, "index.md")} (${rows.length} entries)`);
+1
View File
@@ -0,0 +1 @@
../.claude/skills
+2 -1
View File
@@ -11,6 +11,7 @@ scratch
workspace.code-workspace
.netlify
bun.lockb
bun.lock
out.png
logos
.zed
@@ -24,7 +25,7 @@ tsconfig.vitest-temp.json
*.tgz
open-issues-triage.md
triage/
.triage/
tmp/
.worktrees/
research/
+6
View File
@@ -98,6 +98,12 @@ git push origin main
The release workflow only fires on changes under `packages/zod/package.json`, `packages/zod/src/**`, or the workflow file itself, so the bump must include `package.json`. Watch the Actions tab to confirm `build_and_publish` succeeds.
## Triaging issues and PRs
Follow the `triage` skill at [`.claude/skills/triage/SKILL.md`](.claude/skills/triage/SKILL.md) whenever you're asked to investigate, triage, or form an opinion on an issue or PR. It is the single source of truth for the procedure. Claude and Codex both auto-discover it (Codex via the `.codex/skills` symlink); read it directly if your agent doesn't.
Write-ups live in the gitignored `.triage/` tree — one directory per ticket, `.triage/issues/NNNN/results.md` and `.triage/prs/NNNN/results.md`, with scratch files and repros alongside. When investigating a PR from its worktree, resolve the root repo first (`git worktree list | head -1`) and write results back there; a relative write lands in the worktree's own ignored `.triage/` where nobody will find it.
## Iterating on a contributor PR in a worktree
When asked to make changes on top of an open PR (e.g. as a maintainer review suggestion), use a worktree so `main` stays clean:
+703
View File
@@ -0,0 +1,703 @@
/**
* Ranks open issues and PRs by a synthesized engagement signal.
*
* pnpm dev scripts/triage-signal.ts
* pnpm dev scripts/triage-signal.ts --repo colinhacks/zod --top 40
*
* Auth comes from the `gh` CLI (`gh auth token`).
*/
import { execFileSync } from "node:child_process";
import * as fs from "node:fs";
import * as path from "node:path";
/* -------------------------------------------------------------------------- */
/* config */
/* -------------------------------------------------------------------------- */
interface Args {
repo: string;
top: number;
outDir: string;
page: number;
}
function parseArgs(argv: string[]): Args {
const get = (flag: string) => {
const i = argv.indexOf(flag);
return i === -1 ? undefined : argv[i + 1];
};
return {
repo: get("--repo") ?? "colinhacks/zod",
top: Number(get("--top") ?? 40),
outDir: get("--out") ?? ".triage",
page: Number(get("--page") ?? 50),
};
}
/**
* Reaction weights. A reaction is one-per-person, so it is the cleanest
* "this affects me too" vote available. Positive reactions carry demand;
* the rest are ambiguous and count for much less.
*/
const UPVOTE_REACTIONS = new Set(["THUMBS_UP", "HEART", "ROCKET", "HOORAY"]);
/** Channel weights, applied to log-compressed counts so no one channel runs away. */
const W = {
upvote: 14,
softReaction: 5,
voice: 9,
comment: 5,
} as const;
/** Multipliers applied after the engagement score is assembled. */
const PENALTY = {
draft: 0.5,
stale: 0.7,
"needs-info": 0.6,
"working-as-intended": 0.3,
duplicate: 0.25,
dependencies: 0.2,
bot: 0.15,
} as const;
/** Flat bonus for confirmed security content — large enough to outrank any thread. */
const SECURITY_BONUS = 120;
const SECURITY_MAYBE_BONUS = 15;
/** Staleness decay on `updatedAt`: 1.0 fresh, ~0.68 at one year, floored so old pain still surfaces. */
const DECAY_FLOOR = 0.35;
const DECAY_HALF_LIFE_MONTHS = 12;
/* -------------------------------------------------------------------------- */
/* vulnerability detection */
/* -------------------------------------------------------------------------- */
/**
* Unambiguous markers. These name a security concept that has no innocent use,
* so a hit anywhere — title or body — promotes the item.
*/
const VULN_STRONG: [RegExp, string][] = [
[/\bCVE-\d{4}-\d{4,}/i, "CVE reference"],
[/\bGHSA-[0-9a-z]{4}-[0-9a-z]{4}-[0-9a-z]{4}\b/i, "GHSA reference"],
[/\bCWE-\d+\b/i, "CWE reference"],
[/prototype\s+pollut/i, "prototype pollution"],
[/\bprototype\s+(chain\s+)?(poison|injection|manipulat)/i, "prototype manipulation"],
[/regular\s+expression\s+denial\s+of\s+service/i, "ReDoS"],
[/catastrophic\s+backtrack/i, "catastrophic backtracking"],
[/exponential\s+backtrack/i, "catastrophic backtracking"],
[/\b(XSS|cross[\s-]site\s+scripting)\b/i, "XSS"],
[/\bRCE\b|remote\s+code\s+execution/i, "RCE"],
[/arbitrary\s+code\s+execution/i, "arbitrary code execution"],
[/security\s+(vulnerabilit|advisor|issue|flaw|report)/i, "security report"],
[/\bsupply[\s-]chain\b/i, "supply chain"],
];
/**
* Real security terms that also appear incidentally — a PR body noting "1 flaky
* timeout in the redos checker" is not a ReDoS report. These count only in the
* title, or when the body carries two different ones.
*/
const VULN_CONTEXTUAL: [RegExp, string][] = [
[/\b__proto__\b/, "__proto__"],
[/\bObject\.prototype\b/, "Object.prototype"],
[/\bReDoS\b/i, "ReDoS"],
[/\bdenial[\s-]of[\s-]service\b/i, "denial of service"],
[/\bvulnerabilit(y|ies)\b/i, "vulnerability"],
];
/** Weaker markers — flagged for a human look but not promoted. */
const VULN_WEAK: [RegExp, string][] = [
[/\bsanitiz(e|ing|ation)\b/i, "sanitization"],
[/\bexploit(able)?\b/i, "exploit"],
[/\buntrusted\s+(input|data|user)/i, "untrusted input"],
[/\bstack\s+overflow\b/i, "stack overflow"],
[/\bRangeError\b/, "RangeError"],
[/\bmalicious\b/i, "malicious"],
];
/** Label names that mean "security" regardless of content. */
const VULN_LABEL = /^(security|vulnerability|vuln|cve|advisory)$/i;
interface VulnVerdict {
level: "confirmed" | "possible" | "none";
reasons: string[];
}
function detectVuln(item: Item): VulnVerdict {
const confirmed: string[] = [];
for (const l of item.labels) {
if (VULN_LABEL.test(l)) confirmed.push(`label:${l}`);
}
// Only the leading slice of the body is scanned. Full bodies drag in stack traces
// and quoted source that trip every keyword.
const title = item.title;
const body = item.body.slice(0, 3000);
const hay = `${title}\n${body}`;
for (const [re, name] of VULN_STRONG) {
if (re.test(hay)) confirmed.push(name);
}
const inTitle: string[] = [];
const inBody: string[] = [];
for (const [re, name] of VULN_CONTEXTUAL) {
if (re.test(title)) inTitle.push(name);
else if (re.test(body)) inBody.push(name);
}
confirmed.push(...inTitle);
// Two different contextual terms in one body is a report, not a passing mention.
if (inBody.length >= 2) confirmed.push(...inBody);
if (confirmed.length) return { level: "confirmed", reasons: dedupe(confirmed) };
const weak = [...inBody];
for (const [re, name] of VULN_WEAK) {
if (re.test(hay)) weak.push(name);
}
if (weak.length) return { level: "possible", reasons: dedupe(weak) };
return { level: "none", reasons: [] };
}
function dedupe(xs: string[]): string[] {
return [...new Set(xs)];
}
/* -------------------------------------------------------------------------- */
/* fetch */
/* -------------------------------------------------------------------------- */
interface Item {
kind: "issue" | "pr";
number: number;
title: string;
url: string;
body: string;
author: string;
authorIsBot: boolean;
createdAt: string;
updatedAt: string;
labels: string[];
comments: number;
participants: number;
upvotes: number;
softReactions: number;
totalReactions: number;
// PR-only
isDraft?: boolean;
additions?: number;
deletions?: number;
changedFiles?: number;
reviews?: number;
reviewDecision?: string | null;
mergeable?: string;
}
interface Scored extends Item {
engagement: number;
decay: number;
penalty: number;
vuln: VulnVerdict;
/** Engagement alone, before the security bonus. */
baseScore: number;
score: number;
/** Verdict from a previous sweep's .triage/<kind>/<n>/results.md, if one exists. */
prior?: { verdict: string; status: string };
}
const REACTION_FRAGMENT = `reactionGroups { content users { totalCount } }`;
const ISSUE_QUERY = `
query($owner: String!, $name: String!, $cursor: String, $page: Int!) {
repository(owner: $owner, name: $name) {
issues(first: $page, after: $cursor, states: OPEN, orderBy: {field: CREATED_AT, direction: DESC}) {
pageInfo { hasNextPage endCursor }
nodes {
number title url bodyText createdAt updatedAt
author { login __typename }
comments { totalCount }
participants { totalCount }
labels(first: 20) { nodes { name } }
${REACTION_FRAGMENT}
}
}
}
}`;
const PR_QUERY = `
query($owner: String!, $name: String!, $cursor: String, $page: Int!) {
repository(owner: $owner, name: $name) {
pullRequests(first: $page, after: $cursor, states: OPEN, orderBy: {field: CREATED_AT, direction: DESC}) {
pageInfo { hasNextPage endCursor }
nodes {
number title url bodyText createdAt updatedAt
isDraft additions deletions changedFiles mergeable reviewDecision
author { login __typename }
comments { totalCount }
reviews { totalCount }
participants { totalCount }
labels(first: 20) { nodes { name } }
${REACTION_FRAGMENT}
}
}
}
}`;
let token: string | undefined;
function ghToken(): string {
token ??= execFileSync("gh", ["auth", "token"], { encoding: "utf8" }).trim();
return token;
}
async function graphql(query: string, variables: Record<string, unknown>): Promise<any> {
const res = await fetch("https://api.github.com/graphql", {
method: "POST",
headers: {
authorization: `bearer ${ghToken()}`,
"content-type": "application/json",
"user-agent": "zod-triage-signal",
},
body: JSON.stringify({ query, variables }),
});
if (!res.ok) throw new Error(`GitHub API ${res.status}: ${await res.text()}`);
const json = (await res.json()) as { data?: any; errors?: { message: string }[] };
if (json.errors?.length) throw new Error(json.errors.map((e) => e.message).join("; "));
return json.data;
}
function readReactions(groups: { content: string; users: { totalCount: number } }[]) {
let upvotes = 0;
let soft = 0;
for (const g of groups) {
const n = g.users.totalCount;
if (UPVOTE_REACTIONS.has(g.content)) upvotes += n;
else soft += n;
}
return { upvotes, soft, total: upvotes + soft };
}
function normalize(node: any, kind: "issue" | "pr"): Item {
const r = readReactions(node.reactionGroups ?? []);
return {
kind,
number: node.number,
title: node.title,
url: node.url,
body: node.bodyText ?? "",
author: node.author?.login ?? "ghost",
authorIsBot: node.author?.__typename === "Bot" || /\[bot\]$/.test(node.author?.login ?? ""),
createdAt: node.createdAt,
updatedAt: node.updatedAt,
labels: (node.labels?.nodes ?? []).map((l: { name: string }) => l.name),
comments: node.comments?.totalCount ?? 0,
participants: node.participants?.totalCount ?? 0,
upvotes: r.upvotes,
softReactions: r.soft,
totalReactions: r.total,
...(kind === "pr"
? {
isDraft: node.isDraft,
additions: node.additions,
deletions: node.deletions,
changedFiles: node.changedFiles,
reviews: node.reviews?.totalCount ?? 0,
reviewDecision: node.reviewDecision,
mergeable: node.mergeable,
}
: {}),
};
}
async function fetchAll(repo: string, kind: "issue" | "pr", page: number): Promise<Item[]> {
const [owner, name] = repo.split("/");
const query = kind === "issue" ? ISSUE_QUERY : PR_QUERY;
const key = kind === "issue" ? "issues" : "pullRequests";
const out: Item[] = [];
let cursor: string | null = null;
for (;;) {
const data = await graphql(query, { owner, name, cursor, page });
const conn = data.repository[key];
for (const node of conn.nodes) out.push(normalize(node, kind));
process.stderr.write(`\r ${kind}: ${out.length}`);
if (!conn.pageInfo.hasNextPage) break;
cursor = conn.pageInfo.endCursor;
}
process.stderr.write("\n");
return out;
}
/* -------------------------------------------------------------------------- */
/* scoring */
/* -------------------------------------------------------------------------- */
const MS_PER_MONTH = 1000 * 60 * 60 * 24 * 30.44;
function score(item: Item, now: number): Scored {
// Each channel is log-compressed independently so a single 200-comment thread
// can't outrank broad, quiet demand — and reactions stay the dominant term.
const voices = Math.max(0, item.participants - 1);
const discussion = item.comments + (item.reviews ?? 0);
const engagement =
W.upvote * Math.log1p(item.upvotes) +
W.softReaction * Math.log1p(item.softReactions) +
W.voice * Math.log1p(voices) +
W.comment * Math.log1p(discussion);
const monthsIdle = (now - Date.parse(item.updatedAt)) / MS_PER_MONTH;
const decay = DECAY_FLOOR + (1 - DECAY_FLOOR) * 0.5 ** (monthsIdle / DECAY_HALF_LIFE_MONTHS);
let penalty = 1;
if (item.isDraft) penalty *= PENALTY.draft;
if (item.authorIsBot) penalty *= PENALTY.bot;
for (const l of item.labels) {
const p = (PENALTY as Record<string, number>)[l];
if (p !== undefined) penalty *= p;
}
const vuln = detectVuln(item);
const bonus = vuln.level === "confirmed" ? SECURITY_BONUS : vuln.level === "possible" ? SECURITY_MAYBE_BONUS : 0;
const baseScore = engagement * decay * penalty;
return { ...item, engagement, decay, penalty, vuln, baseScore, score: baseScore + bonus };
}
/* -------------------------------------------------------------------------- */
/* duplicate clusters */
/* -------------------------------------------------------------------------- */
const STOPWORDS = new Set(
"a an the and or of to in for on with when is are be do does not no fix fixes fixed feat chore docs test tests refactor perf ci bug issue support add adds added allow allows make makes use using update updates from into via zod v3 v4 correctly properly".split(
" "
)
);
function titleTokens(title: string): Set<string> {
const stripped = title
.replace(/^\s*\w+(\([^)]*\))?!?:\s*/, "") // conventional-commit prefix
.replace(/\(#\d+\)/g, "")
.toLowerCase();
return new Set(
stripped
.split(/[^a-z0-9_]+/)
.filter((t) => t.length > 2 && !STOPWORDS.has(t))
.map((t) => t.replace(/(ing|ed|es|s)$/, ""))
);
}
function jaccard(a: Set<string>, b: Set<string>): number {
if (!a.size || !b.size) return 0;
let shared = 0;
for (const t of a) if (b.has(t)) shared++;
return shared / (a.size + b.size - shared);
}
/**
* Groups open items that appear to address the same thing, so a pile of
* near-identical drive-by PRs collapses into one decision.
*/
function findClusters(items: Scored[], threshold = 0.5): Scored[][] {
const toks = items.map((i) => titleTokens(i.title));
const parent = items.map((_, i) => i);
const find = (x: number): number => {
while (parent[x] !== x) {
parent[x] = parent[parent[x]];
x = parent[x];
}
return x;
};
for (let i = 0; i < items.length; i++) {
for (let j = i + 1; j < items.length; j++) {
if (jaccard(toks[i], toks[j]) >= threshold) parent[find(i)] = find(j);
}
}
const groups = new Map<number, Scored[]>();
for (let i = 0; i < items.length; i++) {
const root = find(i);
if (!groups.has(root)) groups.set(root, []);
groups.get(root)!.push(items[i]);
}
return [...groups.values()]
.filter((g) => g.length > 1)
.map((g) => g.sort((a, b) => b.score - a.score))
.sort((a, b) => b.length - a.length || b[0].score - a[0].score);
}
/* -------------------------------------------------------------------------- */
/* report */
/* -------------------------------------------------------------------------- */
function ago(iso: string, now: number): string {
const d = (now - Date.parse(iso)) / (1000 * 60 * 60 * 24);
if (d < 1) return "today";
if (d < 45) return `${Math.round(d)}d`;
if (d < 365) return `${Math.round(d / 30.44)}mo`;
return `${(d / 365.25).toFixed(1)}y`;
}
function link(i: Scored): string {
return `[#${i.number}](${i.url})`;
}
function esc(s: string): string {
return s.replace(/\|/g, "\\|").replace(/\r?\n/g, " ").trim();
}
function table(items: Scored[], now: number, opts: { pr?: boolean; base?: boolean } = {}): string {
const cols = ["#", "Score", "👍", "💬", "👥"];
if (opts.pr) cols.push("Diff");
cols.push("Age", "Idle", "Triaged", "Title");
const head = `| ${cols.join(" | ")} |\n|${cols.map(() => "---").join("|")}|`;
const rows = items.map((i) => {
const flags = [
i.vuln.level === "confirmed" ? "🔒" : i.vuln.level === "possible" ? "🔍" : "",
i.isDraft ? "📝" : "",
i.labels.includes("stale") ? "💤" : "",
]
.filter(Boolean)
.join("");
const cells = [
link(i),
(opts.base ? i.baseScore : i.score).toFixed(1),
String(i.upvotes),
String(i.comments + (i.reviews ?? 0)),
String(i.participants),
];
if (opts.pr) cells.push(`+${i.additions}/-${i.deletions}`);
cells.push(
ago(i.createdAt, now),
ago(i.updatedAt, now),
i.prior ? i.prior.verdict : "—",
`${flags ? `${flags} ` : ""}${esc(i.title)}`
);
return `| ${cells.join(" | ")} |`;
});
return [head, ...rows].join("\n");
}
function buildReport(all: Scored[], args: Args, now: number, advisories: Advisory[]): string {
const issues = all.filter((i) => i.kind === "issue");
const prs = all.filter((i) => i.kind === "pr");
const security = all.filter((i) => i.vuln.level === "confirmed").sort((a, b) => b.score - a.score);
const maybeSecurity = all.filter((i) => i.vuln.level === "possible").sort((a, b) => b.score - a.score);
// Security has its own section up top, so the engagement rankings below sort on
// engagement alone — otherwise the bonus buries every genuinely busy thread.
const byScore = (a: Scored, b: Scored) => b.baseScore - a.baseScore;
const L: string[] = [];
L.push(`# Triage signal report — ${args.repo}`);
L.push("");
L.push(
`Generated ${new Date(now).toISOString()} · ${issues.length} open issues · ${prs.length} open PRs · ${security.length} flagged security · ${advisories.length} draft advisories`
);
L.push("");
L.push(
"Legend: 👍 upvote reactions on the opening post · 💬 comments (+ reviews on PRs) · 👥 distinct participants · 🔒 security · 🔍 possible security · 📝 draft · 💤 stale"
);
L.push("");
L.push("## 1. Security");
L.push("");
if (advisories.length) {
L.push(`### Draft advisories in the Security tab (${advisories.length}) — these do not appear in the issue list`);
L.push("");
L.push("| GHSA | State | Severity | Summary |");
L.push("|---|---|---|---|");
for (const a of advisories) {
L.push(`| [${a.ghsa_id}](${a.html_url}) | ${a.state} | ${a.severity ?? "—"} | ${esc(a.summary)} |`);
}
L.push("");
}
if (security.length) {
L.push(`### Open issues/PRs matching security content (${security.length})`);
L.push("");
L.push(table(security, now));
L.push("");
for (const i of security) {
L.push(`- ${link(i)} — matched: ${i.vuln.reasons.join(", ")}`);
}
L.push("");
} else {
L.push("No open issue or PR matched a confirmed security pattern.");
L.push("");
}
L.push(`## 2. Top ${args.top} issues by signal`);
L.push("");
L.push(table(issues.sort(byScore).slice(0, args.top), now, { base: true }));
L.push("");
L.push(`## 3. Top ${args.top} pull requests by signal`);
L.push("");
L.push(table(prs.sort(byScore).slice(0, args.top), now, { pr: true, base: true }));
L.push("");
L.push("## 4. Highest raw demand (most upvoted issues, ignoring decay)");
L.push("");
L.push(
table(
issues
.filter((i) => i.upvotes > 0)
.sort((a, b) => b.upvotes - a.upvotes)
.slice(0, 20),
now
)
);
L.push("");
L.push("## 5. Busiest threads (most discussion, ignoring reactions)");
L.push("");
L.push(
table(
all
.slice()
.sort((a, b) => b.comments + (b.reviews ?? 0) - (a.comments + (a.reviews ?? 0)))
.slice(0, 20),
now
)
);
L.push("");
const clusters = findClusters(all);
if (clusters.length) {
const dupTotal = clusters.reduce((n, g) => n + g.length, 0);
L.push(`## 6. Likely duplicate clusters (${clusters.length} clusters, ${dupTotal} items)`);
L.push("");
L.push("Items whose titles overlap heavily — each cluster is probably one decision, not many.");
L.push("");
for (const g of clusters) {
L.push(`**${g.length}× — ${esc(g[0].title)}**`);
L.push("");
L.push(table(g, now));
L.push("");
}
}
const untriaged = all.filter((i) => !i.prior).sort(byScore);
L.push(`## 7. Highest signal with no triage write-up yet (${untriaged.length} untriaged)`);
L.push("");
L.push(`Nothing under \`${args.outDir}/{issues,prs}/<n>/results.md\` covers these. This is the work queue.`);
L.push("");
L.push(table(untriaged.slice(0, args.top), now, { base: true }));
L.push("");
if (maybeSecurity.length) {
L.push(`## 8. Possible security (weak keyword match, ${maybeSecurity.length})`);
L.push("");
L.push(table(maybeSecurity.slice(0, 25), now));
L.push("");
}
L.push("## Scoring");
L.push("");
L.push("```");
L.push(`engagement = ${W.upvote}·ln(1+upvotes) + ${W.softReaction}·ln(1+other reactions)`);
L.push(` + ${W.voice}·ln(1+participants-1) + ${W.comment}·ln(1+comments+reviews)`);
L.push(
`decay = ${DECAY_FLOOR} + ${(1 - DECAY_FLOOR).toFixed(2)} · 0.5^(months idle / ${DECAY_HALF_LIFE_MONTHS})`
);
L.push(`penalty = product of ${JSON.stringify(PENALTY)}`);
L.push(
`score = engagement · decay · penalty + ${SECURITY_BONUS} if security (+${SECURITY_MAYBE_BONUS} if possible)`
);
L.push("```");
L.push("");
L.push(
"Each channel is log-compressed on its own so one runaway thread can't dominate, and reactions outweigh comments because a reaction is one-per-person while a comment count can be two people arguing."
);
L.push("");
return L.join("\n");
}
/* -------------------------------------------------------------------------- */
/* existing triage write-ups */
/* -------------------------------------------------------------------------- */
/** Reads `verdict`/`status` out of the frontmatter of any existing .triage/*<N>/results.md. */
function loadPriorTriage(outDir: string): Map<string, { verdict: string; status: string }> {
const map = new Map<string, { verdict: string; status: string }>();
for (const kind of ["issues", "prs"] as const) {
const dir = path.join(outDir, kind);
if (!fs.existsSync(dir)) continue;
for (const entry of fs.readdirSync(dir)) {
const file = path.join(dir, entry, "results.md");
if (!fs.existsSync(file)) continue;
const fm = fs.readFileSync(file, "utf8").split(/^---\s*$/m)[1] ?? "";
map.set(`${kind === "issues" ? "issue" : "pr"}:${entry}`, {
verdict: /^verdict:\s*(.+)$/m.exec(fm)?.[1].trim() ?? "—",
status: /^status:\s*(.+)$/m.exec(fm)?.[1].trim() ?? "—",
});
}
}
return map;
}
/* -------------------------------------------------------------------------- */
/* advisories */
/* -------------------------------------------------------------------------- */
interface Advisory {
ghsa_id: string;
state: string;
severity: string | null;
summary: string;
html_url: string;
}
function fetchAdvisories(repo: string): Advisory[] {
try {
const raw = execFileSync("gh", ["api", `repos/${repo}/security-advisories`, "--paginate"], {
encoding: "utf8",
maxBuffer: 32 * 1024 * 1024,
});
const list = JSON.parse(raw) as Advisory[];
return list.filter((a) => a.state !== "closed" && a.state !== "published");
} catch {
// No admin/security access on this repo — not fatal.
return [];
}
}
/* -------------------------------------------------------------------------- */
/* main */
/* -------------------------------------------------------------------------- */
async function main() {
const args = parseArgs(process.argv.slice(2));
const now = Date.now();
process.stderr.write(`Fetching ${args.repo}...\n`);
const [issues, prs] = await Promise.all([
fetchAll(args.repo, "issue", args.page),
fetchAll(args.repo, "pr", args.page),
]);
const advisories = fetchAdvisories(args.repo);
const prior = loadPriorTriage(args.outDir);
const all = [...issues, ...prs]
.map((i) => {
const s = score(i, now);
s.prior = prior.get(`${i.kind}:${i.number}`);
return s;
})
// Ties are common once the security bonus flattens quiet reports; break by recency.
.sort((a, b) => b.score - a.score || Date.parse(b.updatedAt) - Date.parse(a.updatedAt));
fs.mkdirSync(args.outDir, { recursive: true });
const mdPath = path.join(args.outDir, "signal-report.md");
const jsonPath = path.join(args.outDir, "signal-report.json");
fs.writeFileSync(mdPath, buildReport(all, args, now, advisories));
fs.writeFileSync(
jsonPath,
`${JSON.stringify(
all.map(({ body, ...rest }) => rest),
null,
2
)}\n`
);
process.stderr.write(`\nWrote ${mdPath} and ${jsonPath}\n`);
}
main().catch((err) => {
process.stderr.write(`${err instanceof Error ? err.stack : String(err)}\n`);
process.exit(1);
});