mirrors/bun
0
0 Fork 0
mirror of https://github.com/oven-sh/bun.git synced 2026-10-25 21:40:30 +08:00
Code Activity
Files
main
bun/scripts/update-typescript-libs.ts
T
Jarred Sumnerandautofix-ci[bot] bbdc5a519e bun check - a TypeScript type checker builtin to Bun (#44361) 
### What does this PR do?

This adds a TypeScript type checker to Bun.

```sh
bun check                        # type check the project
bun --check src/index.ts         # type check, then run
bun build --check src/index.ts   # type check, then bundle
bun test --check                 # type check, then run the tests
```

It's a port of the type checker in
[typescript-go](https://github.com/microsoft/typescript-go) 7.0.2. You
get the same errors as `tsc`, with the same error codes, messages, lines
and columns. If `bun check` and `tsc` 7 disagree about an error, that's
a bug in Bun.

`packages/next` in the Next.js repo, 2,881 files:

| | CPU time | Peak memory |
| --- | --- | --- |
| `tsc` 6.0.2 | 14.8 s | 2.33 GB |
| `tsc` 7.0.2 (typescript-go) | 6.9 s | 1.78 GB |
| `bun check` | 2.4 s | 0.69 GB |

(How that was measured is under Performance.)

It only type checks. It doesn't emit JavaScript, `.d.ts` files,
sourcemaps or `.tsbuildinfo`. There's no language server, so your editor
keeps using TypeScript.

Most of this description is about how we tested it. What's still wrong
is at the bottom.

## Usage

### Check a project

`bun check` looks for the nearest `tsconfig.json` and checks whatever
`files`, `include` and `exclude` select. It uses all of your CPU cores.

```sh
$ bun check
src/index.ts(3,25): error TS2322: Type 'string' is not assignable to type 'number'.
Found 1 error in 1 file, checked 2 files [14.00ms]
$ echo $?
1
```

```sh
$ bun check
✓ No type errors in 2 files [14.00ms]
$ echo $?
0
```

Errors go to stdout. The summary goes to stderr.

If your `package.json` already has a `check` script, `bun check` still
runs that script, like it does today. `bun --check` with no file is the
type checker in every project. `"check": "bun check"` works too: inside
that script, and in the scripts it runs, `bun check` is the type
checker.

```sh
bun --check
```

`bun --filter '*' check` and `bun --workspaces check` still run each
package's `check` script.

### Check some files

```sh
bun check src/index.ts
bun check src/server packages/shared
```

Bun checks those files and everything they import. Each file is checked
in the project your editor uses for it. That's the nearest
`tsconfig.json`, or, if that one doesn't include the file but has
`references`, the referenced project that does. So it works in the
layout `create vite` gives you.

A directory means the part of the project that's in it, so `bun check .`
is the same as `bun check`. If the project has no files there, like `bun
check scripts` when `include` is `["src"]`, Bun checks every file in the
directory that isn't excluded, each with the `tsconfig.json` nearest to
it.

### Choose a tsconfig

```sh
bun check -p packages/server
bun check -p tsconfig.test.json
```

### Without a tsconfig

`bun check index.ts` works in an empty directory. These are the
defaults:

```json
{
  "lib": ["ESNext"], "target": "ESNext", "module": "Preserve", "moduleDetection": "force",
  "jsx": "react-jsx", "allowJs": true, "moduleResolution": "bundler",
  "allowImportingTsExtensions": true, "verbatimModuleSyntax": true,
  "noEmit": true, "strict": true, "skipLibCheck": true
}
```

In a monorepo with no `tsconfig.json` at the root, `bun check` checks
everything below the current directory. Each file gets the options of
the `tsconfig.json` nearest to it, and each file is only checked once.

### Project references

`references` are followed like `tsc -b` does, except you don't have to
build anything first and nothing gets written to disk. Each project is
checked with its own compiler options. Errors are printed one project at
a time, in build order. A file that 2 projects include is checked in
both, and its errors are printed once for each, like `tsc -b` does.

```sh
$ bun check
packages/app/src/index.ts(2,14): error TS2322: Type 'number' is not assignable to type 'string'.
Found 1 error in 1 file, checked 176 files across 2 projects [544.00ms]
```

### `--check`

`--check` type checks the entry point and everything it imports before
doing anything else. If there's a type error, Bun prints it, exits with
code 1, and your code doesn't run.

```sh
$ bun --check src/good.ts
hello Ada

$ bun --check src/bad.ts
src/bad.ts(3,21): error TS2322: Type 'string' is not assignable to type 'number'.
Found 1 error in 1 file, checked 2 files
$ echo $?
1
```

With `--watch`, every restart is checked first. After a type error Bun
waits for the next change in any file of the program, including files
that only have types. `--hot --check` restarts the process like
`--watch` does, so the code is checked again before it runs.

A file without an extension, `-e`, `-p` and a script on stdin are
checked as TypeScript, which is how Bun runs them. If a file imports an
HTML page, the scripts of that page are checked too. `--conditions` and
`--loader` apply to the check: with `--loader .js:ts`, every `.js` file
of your project is TypeScript.

`--check` doesn't cover files loaded with `--preload`. `bun check` does,
if your `tsconfig.json` includes them.

```sh
bun --watch --check src/index.ts
bun test --watch --check
```

```sh
$ bun build --check src/good.ts --outdir out
Bundled 2 modules

  good.js  131 bytes  (entry point)

$ bun build --check src/bad.ts --outdir out
3 | console.log(greet({ age: "36" }));
                        ^
error: TS2322: Type 'string' is not assignable to type 'number'.
    at /app/src/bad.ts:3:21

1 | export function greet(user: { name?: string; age: number }) {
                                                 ^
note: TS6500: The expected type comes from property 'age' which is declared here on type '{ name?: string | undefined; age: number; }'
   at /app/src/greet.ts:1:46
$ ls out
ls: out: No such file or directory
```

```sh
$ bun test --check
src/bad.test.ts(3,39): error TS2322: Type 'string' is not assignable to type 'number'.
Found 1 error in 1 file, checked 3 files
```

A `package.json` script has no entry point to start from, so Bun checks
the whole project first, like `bun check`. Same with `--filter`,
`--parallel` and `--sequential`.

```sh
bun run --check dev
bun --check --filter '*' build
```

`--tsconfig-override` applies to the check too, so the type checker
reads the same `tsconfig.json` as the bundler and the runtime. So does
`tsconfig` in `Bun.build`.

`Bun.build` takes `check: true`. A type error fails the build like any
other build error, and nothing is written. The check uses the
`conditions` and `loader` of the build.

```ts
const result = await Bun.build({
  entrypoints: ["./src/index.ts"],
  outdir: "./out",
  check: true,
  throw: false,
});

for (const log of result.logs) {
  console.error(`${log.position?.file}:${log.position?.line}: ${log.message}`);
}
// /app/src/index.ts:3: TS2322: Type 'string' is not assignable to type 'number'.
```

### Compiler options as flags

Any compiler option works as a flag, like in `tsc`. Flags override
`tsconfig.json`.

```sh
bun check --noUncheckedIndexedAccess
bun check --strict false
bun check --target es2022
```

### Output

In a terminal:

```
1 | import { greet } from "./user";
2 |
3 | const message = greet({ id: "1", name: "Ada" });
                            ^
error: TS2322: Type 'string' is not assignable to type 'number'.
    at src/index.ts:3:25

2 |   id: number;
      ^
note: The expected type comes from property 'id' which is declared here on type 'User'
   at src/user.ts:2:3

Found 1 error in 1 file, checked 2 files [14.00ms]
```

When there are more than 50 errors, identical errors are grouped and
sorted by how often they happen. `--all` prints every one.

```
1 | export const v0: number = "0";
                 ^
error: TS2322: Type 'string' is not assignable to type 'number'.
    at m1.ts:1:14
    60 times in 2 files
      30  m1.ts:1
      30  m2.ts:1
```

When stdout isn't a terminal, you get the same format as `tsc --pretty
false`, so problem matchers and scripts written for `tsc` keep working.

```
src/index.ts(3,25): error TS2322: Type 'string' is not assignable to type 'number'.
```

In GitHub Actions, errors are also printed as workflow commands so they
show up as annotations on the diff, also when the step runs in a
subdirectory.

```
::error file=src/index.ts,line=3,col=25,endLine=3,endColumn=27,title=TS2322::Type 'string' is not assignable to type 'number'.
```

With `AGENT=1`:

```
<error file="src/index.ts" line="3" column="25" code="TS2322">
Type 'string' is not assignable to type 'number'.
<source>
1 | import { greet } from "./user";
2 |
3 | const message = greet({ id: "1", name: "Ada" });
                            ^^
</source>
<related file="src/user.ts" line="2" column="3">The expected type comes from property 'id' which is declared here on type 'User'</related>
</error>
```

### What you need installed

```sh
bun add -d @types/bun
```

That's for `console`, `fetch`, `Bun` and `bun:test`. `bun init` installs
it for you.

You don't need the `typescript` package. TypeScript's `lib.*.d.ts` files
(`Array`, `Promise`, the DOM and so on) are built into Bun. They're
always the ones from TypeScript 7.0.2, no matter which version your
project has installed. Your editor may still want the package.

```sh
$ bun -p process.versions.typescript
7.0.2
```

### Flags

| Flag | What it does |
| --- | --- |
| `-p`, `--project <path>` | A `tsconfig.json`, or the directory that
holds one |
| `--pretty` / `--no-pretty` | Source around each error, or one line per
error |
| `--all` | Show every error instead of grouping identical ones above 50
|
| `--threads <n>` | Number of threads. Default: one per core |
| `--timing` | Print how long loading and checking took |
| `--cwd <path>` | Set the working directory |
| `--strict`, `--target`, ... | Any compiler option, as for `tsc` |

Docs are in `docs/runtime/check.mdx`.

## Also fixed

3 bugs that aren't in the type checker. We ran into them while testing
it, and they're on main too.

- `Bun.build({ tsconfig: "./custom.json" })` ignored the option. The 3
tests for it passed anyway, because in each of them the `tsconfig.json`
that Bun finds by itself says the same thing. Now that build reads the
file in place of every `tsconfig.json`, like `--tsconfig-override`. For
both, a directory means the `tsconfig.json` in it, like `tsc -p`. Other
builds in the same process, and the program's own imports, aren't
affected, even while it's running.
- `--tsconfig-override` printed `Internal error: directory mismatch for
directory "/app/custom.json", fd 3. You don't need to do anything, but
this indicates a bug.` every time, for `bun build` and for `bun run`.
- On Linux without `pidfd_open`, a child process that exits could
interrupt a system call of the main thread with `EINTR`. Code that
doesn't retry, like an addon or `bun:ffi`, saw a failed `read()`. It
also made `spawn.test.ts` fail in about half of the ASAN builds of this
branch.

## Intentional differences from `tsc`

- No emit. `noEmit`, `declaration` and `incremental` don't change
anything.
- `bun check` has no `--watch`. `bun --watch --check <file>` checks
before every restart.
- Language service plugins (`plugins` in `compilerOptions`) aren't
loaded.
- `listFiles`, `listFilesOnly` and `traceResolution` work. Other options
that only print things, like `explainFiles`, are ignored.
- Messages that tell you to run `npm i --save-dev` say `bun add -d`.
- The `lib.*.d.ts` files are built in. A path in one of them is printed
as `bundled:///libs/lib.dom.d.ts`.
- A misspelled compiler option gets a suggestion: `Unknown compiler
option 'strct'. Did you mean 'strict'?`
- If an import fails because a package in `package.json` isn't
installed, a note names the `package.json` and says how to run `bun
install` for it.
- If a syntax error or an invalid option stops the type check, like it
does in `tsc`, a note says how many files weren't checked: `Stopped
before type checking 2 files. Fix the errors above to see the rest.`
- TS2309 (`export =` in a module that has other exports) is also
reported when the other export is in a `declare module` block in another
file. `tsc` 6.0.2 reports it too. `tsc` 7.0.2 only does when both are in
one file.
- `acc = apply(run, ["x", acc])` in a loop, where the parameter is a
tuple: `tsc` 7.0.2 reports TS2345 because the array literal loses its
tuple type. `tsc` 6.0.2 doesn't, and `bun check` doesn't.
- You get the same errors no matter how many threads it uses or what
order the files are in. That isn't always true of `tsc`.

You can see it with 2 files:

```js
// a_user.js
const p = new Packet()
p.a = 1
```
```js
// z_packet.js
class Packet {
  constructor () {
    this.a = null
    this.c = null
  }
  own () { return this.c }
}
```

With `allowJs`, `checkJs` and `strict`:

| | TS7008 on `a` | TS7008 on `c` |
| --- | --- | --- |
| tsc 6.0.2 | yes | yes |
| tsgo 7.0.2 `--singleThreaded` or `--checkers 1` | yes | yes |
| tsgo 7.0.2, default | no | yes |
| all of the above, with `a_user.js` renamed to `zz_user.js` | no | yes
|
| `bun check`, either name | yes | yes |

### How did you verify your code works?

This is about 150,000 lines of new code, so there's a lot of room for
mistakes. A type checker that's wrong once in a thousand files is worse
than not having one, and "the tests pass" isn't a good enough reason to
trust it. Most of the work here went into trying to prove it wrong.

Everything below compares `bun check` against the real `tsc` from
TypeScript 7.0.2 on the same files. None of it compares against what we
think the answer should be.

#### Real projects

Each of these repos is checked with `tsc` 7.0.2 and with `bun check`,
once for every `tsconfig.json` in it. "Identical" means both report the
same set of (file, line, column, error code).

| Project | `tsconfig.json` files | Identical | Errors in `tsc` | Errors
in `bun check` | Messages worded differently |
| --- | ---: | ---: | ---: | ---: | ---: |
| [vercel/ai](https://github.com/vercel/ai) | 52 | 52 | 9,742 | 9,742 |
0 |
|
[anthropics/anthropic-sdk-typescript](https://github.com/anthropics/anthropic-sdk-typescript)
| 16 | 16 | 596 | 596 | 0 |
|
[apollographql/apollo-client](https://github.com/apollographql/apollo-client)
| 7 | 7 | 559 | 559 | 0 |
| [arktypeio/arktype](https://github.com/arktypeio/arktype) | 5 | 5 |
389 | 389 | 0 |
| [withastro/astro](https://github.com/withastro/astro) | 12 | 12 |
1,996 | 1,996 | 0 |
| [axios/axios](https://github.com/axios/axios) | 1 | 1 | 105 | 105 | 0
|
| [vuejs/core](https://github.com/vuejs/core) | 6 | 5 | 1,359 | 1,358 |
0 |
| [date-fns/date-fns](https://github.com/date-fns/date-fns) | 3 | 3 | 1
| 1 | 0 |
| [discordjs/discord.js](https://github.com/discordjs/discord.js) | 2 |
2 | 3 | 3 | 0 |
|
[drizzle-team/drizzle-orm](https://github.com/drizzle-team/drizzle-orm)
| 3 | 3 | 10,786 | 10,786 | 0 |
| [Effect-TS/effect](https://github.com/Effect-TS/effect) | 10 | 10 | 2
| 2 | 0 |
| [elysiajs/elysia](https://github.com/elysiajs/elysia) | 4 | 4 | 53 |
53 | 0 |
| [excalidraw/excalidraw](https://github.com/excalidraw/excalidraw) | 7
| 7 | 11,048 | 11,048 | 0 |
| [fastify/fastify](https://github.com/fastify/fastify) | 1 | 1 | 0 | 0
| 0 |
| [TanStack/form](https://github.com/TanStack/form) | 13 | 13 | 124 |
124 | 0 |
| [gcanti/fp-ts](https://github.com/gcanti/fp-ts) | 2 | 2 | 3 | 3 | 0 |
| [sindresorhus/got](https://github.com/sindresorhus/got) | 1 | 1 | 2 |
2 | 0 |
| [graphql/graphql-js](https://github.com/graphql/graphql-js) | 3 | 3 |
24 | 24 | 0 |
| [unjs/h3](https://github.com/unjs/h3) | 1 | 1 | 0 | 0 | 0 |
| [honojs/hono](https://github.com/honojs/hono) | 6 | 6 | 14,239 |
14,239 | 0 |
| [immerjs/immer](https://github.com/immerjs/immer) | 1 | 1 | 5 | 5 | 0
|
| [gcanti/io-ts](https://github.com/gcanti/io-ts) | 2 | 2 | 3 | 3 | 0 |
| [pmndrs/jotai](https://github.com/pmndrs/jotai) | 2 | 2 | 7 | 7 | 0 |
| [sindresorhus/ky](https://github.com/sindresorhus/ky) | 2 | 2 | 52 |
52 | 0 |
| [kysely-org/kysely](https://github.com/kysely-org/kysely) | 6 | 6 |
1,185 | 1,185 | 0 |
|
[langchain-ai/langchainjs](https://github.com/langchain-ai/langchainjs)
| 41 | 41 | 5,273 | 5,273 | 0 |
| [lit/lit](https://github.com/lit/lit) | 34 | 34 | 2,588 | 2,588 | 0 |
| [mikro-orm/mikro-orm](https://github.com/mikro-orm/mikro-orm) | 17 |
17 | 87,981 | 87,981 | 0 |
| [mobxjs/mobx](https://github.com/mobxjs/mobx) | 3 | 3 | 186 | 186 | 0
|
| [mswjs/msw](https://github.com/mswjs/msw) | 10 | 10 | 5,878 | 5,878 |
0 |
| [nestjs/nest](https://github.com/nestjs/nest) | 30 | 30 | 27,345 |
27,345 | 0 |
| [vercel/next.js](https://github.com/vercel/next.js) | 356 | 356 |
3,690 | 3,690 | 0 |
| [unjs/nitro](https://github.com/unjs/nitro) | 5 | 5 | 742 | 742 | 0 |
| [nuxt/nuxt](https://github.com/nuxt/nuxt) | 3 | 3 | 11 | 11 | 0 |
| [openai/openai-node](https://github.com/openai/openai-node) | 4 | 4 |
6 | 6 | 0 |
| [microsoft/playwright](https://github.com/microsoft/playwright) | 4 |
4 | 22 | 22 | 0 |
| [preactjs/preact](https://github.com/preactjs/preact) | 1 | 1 | 1 | 1
| 0 |
| [prisma/prisma](https://github.com/prisma/prisma) | 44 | 44 | 24,201 |
24,201 | 0 |
| [puppeteer/puppeteer](https://github.com/puppeteer/puppeteer) | 10 |
10 | 2,701 | 2,701 | 0 |
| [TanStack/query](https://github.com/TanStack/query) | 15 | 15 | 686 |
686 | 0 |
|
[react-hook-form/react-hook-form](https://github.com/react-hook-form/react-hook-form)
| 2 | 2 | 3 | 3 | 0 |
| [remix-run/react-router](https://github.com/remix-run/react-router) |
14 | 14 | 6,735 | 6,735 | 0 |
| [reduxjs/redux](https://github.com/reduxjs/redux) | 1 | 1 | 0 | 0 | 0
|
| [reduxjs/redux-toolkit](https://github.com/reduxjs/redux-toolkit) | 8
| 8 | 142 | 142 | 0 |
| [remeda/remeda](https://github.com/remeda/remeda) | 7 | 7 | 13 | 13 |
0 |
| [rollup/rollup](https://github.com/rollup/rollup) | 9 | 9 | 107 | 107
| 0 |
| [TanStack/router](https://github.com/TanStack/router) | 29 | 29 |
4,853 | 4,853 | 0 |
| [ReactiveX/rxjs](https://github.com/ReactiveX/rxjs) | 10 | 10 | 22,271
| 22,271 | 7 |
| [sequelize/sequelize](https://github.com/sequelize/sequelize) | 16 |
16 | 6,768 | 6,768 | 0 |
| [solidjs/solid](https://github.com/solidjs/solid) | 7 | 7 | 740 | 740
| 0 |
| [storybookjs/storybook](https://github.com/storybookjs/storybook) | 19
| 19 | 1,401 | 1,401 | 0 |
| [stripe/stripe-node](https://github.com/stripe/stripe-node) | 4 | 4 |
42 | 42 | 0 |
| [sveltejs/svelte](https://github.com/sveltejs/svelte) | 5 | 5 | 186 |
186 | 0 |
| [vercel/swr](https://github.com/vercel/swr) | 4 | 4 | 314 | 314 | 0 |
| [TanStack/table](https://github.com/TanStack/table) | 20 | 20 | 574 |
574 | 0 |
| [tldraw/tldraw](https://github.com/tldraw/tldraw) | 4 | 4 | 0 | 0 | 0
|
| [gvergnaud/ts-pattern](https://github.com/gvergnaud/ts-pattern) | 2 |
2 | 2 | 2 | 0 |
|
[total-typescript/ts-reset](https://github.com/total-typescript/ts-reset)
| 2 | 2 | 4 | 4 | 0 |
| [millsp/ts-toolbelt](https://github.com/millsp/ts-toolbelt) | 1 | 1 |
3 | 3 | 0 |
| [sindresorhus/type-fest](https://github.com/sindresorhus/type-fest) |
2 | 2 | 0 | 0 | 0 |
| [typeorm/typeorm](https://github.com/typeorm/typeorm) | 4 | 4 | 959 |
959 | 0 |
| [microsoft/TypeScript](https://github.com/microsoft/TypeScript) | 4 |
4 | 613 | 613 | 0 |
|
[typescript-eslint/typescript-eslint](https://github.com/typescript-eslint/typescript-eslint)
| 35 | 35 | 18,392 | 18,392 | 0 |
| [urql-graphql/urql](https://github.com/urql-graphql/urql) | 2 | 2 | 17
| 17 | 0 |
| [fabian-hiller/valibot](https://github.com/fabian-hiller/valibot) | 2
| 2 | 1 | 1 | 0 |
| [pmndrs/valtio](https://github.com/pmndrs/valtio) | 1 | 1 | 0 | 0 | 0
|
| [vitest-dev/vitest](https://github.com/vitest-dev/vitest) | 18 | 18 |
2,111 | 2,111 | 0 |
| [microsoft/vscode](https://github.com/microsoft/vscode),
`src/tsconfig.json` | 1 | 1 | 0 | 0 | 0 |
| vscode's extensions, tests and base configs | 105 | 105 | 5,685 |
5,685 | 0 |
| [statelyai/xstate](https://github.com/statelyai/xstate) | 8 | 8 | 545
| 545 | 0 |
| [jquense/yup](https://github.com/jquense/yup) | 1 | 1 | 1 | 1 | 0 |
| [colinhacks/zod](https://github.com/colinhacks/zod) | 10 | 10 | 191 |
191 | 0 |
| [pmndrs/zustand](https://github.com/pmndrs/zustand) | 1 | 1 | 0 | 0 |
0 |
| **all 72** | **1,103** | **1,102** | **286,267** | **286,266** | **7**
|

The error counts show how much there was to compare. They aren't what
you'd get building these projects properly. Each row adds up all of its
configs, each checked by itself, and most of the projects are installed
but not built, so imports of their own `dist` folders don't resolve. A
project with no errors only shows that `bun check` raises no false
alarms. One with thousands shows that it finds the same problems in the
same places. 82 of the configs have no errors in either.

VS Code's own build is `src/tsconfig.json`: 9,795 files, 0 errors in
both. The other 105 configs in that repo are its extensions, its tests
and base configs that only exist to be extended. The extensions'
dependencies aren't installed here, which is where 5,272 of those errors
come from.

There's no config that passes in one and fails in the other.

Not every one of those is a comparison of types. Of the 637 configs of
the 69 repos, `tsc` reports type errors in 494 (276,703 errors) and
nothing in 82. In the other 61 it stops at an error about the config
itself, most often `baseUrl`, which TypeScript 7 removed, and checks no
types at all. `bun check` stops there too, so those 61 only compare that
error. For 7 repos that's every config we have: discord.js, fp-ts,
io-ts, preact, ts-toolbelt, urql and yup.

The table counts an error as the same if the file, line, column and code
are. Comparing the whole text of all 277,975 errors, with the lines
under them, 9 are worded differently: the 7 in rxjs that are under Known
problems, and 1 in sequelize that 2 configs report, where `tsc` names
the type `string | (object & { url?: string; })` and `bun check` names
its alias.

For vscode, next.js and elysia that's every `tsconfig.json` in the repo.
The other 69 repos have 1,311 between them, and we dropped the ones that
give the same answer as another config in the same repo. Configs with
`references` (138 of them) are compared against `tsc -b`, the rest
against `tsc -p`.

That comparison is about which errors there are. For order and repeats
we took every config with `references`, 317 in 21 of the repos, and
compared the error lines one by one as they're printed. `tsc -b` repeats
a line in 26 of them, 1,047 lines in all, because 2 projects include the
same file. 316 are identical. The other one is the vercel-ai config
below, and it's identical to the second build.

2 places where the comparison is lenient:

- typescript-go's TS6059 and TS6307 (a file isn't under `rootDir`, or
isn't listed in the project) change from run to run. We ran it 6 times
per config. `bun check` is allowed to report what any of the 6 runs
reported, and has to report what all 6 did.
- One config (in vercel-ai) is compared against typescript-go's second
`-b` build instead of its first. Its first build caches "this file
doesn't exist" for outputs that it then writes, and reports 167 errors
that go away if you run it again.

Elysia's [Eden Treaty](https://elysiajs.com/eden/overview) turns the
type of a whole server into a typed client, which is about as hard on a
type checker as real code gets. Besides the repo, we check apps with 17,
50 and 200 routes, and a program that uses `edenFetch`, WebSockets,
macros, guards, file uploads, cookies, SSE and validators from zod,
valibot and arktype, each with correct and incorrect calls. Every error
and every message is identical. One of the apps is a test in this PR.

These are npm packages that publish their `.ts` sources and a
`tsconfig.json`, checked where they sit in `node_modules`:

| Package | Version | Files | Errors in `tsc` | Errors in `bun check` |
Messages worded differently |
| --- | --- | ---: | ---: | ---: | ---: |
| `<%= dasherize(name) %>` | 0.0.0 | 6 | 1 | 1 | 0 |
| `@actions/github-script` | 7.0.1 | 7 | 0 | 0 | 0 |
| `@arrows/array` | 1.4.1 | 97 | 0 | 0 | 0 |
| `@arrows/composition` | 1.2.2 | 28 | 4 | 4 | 0 |
| `@arrows/dispatch` | 1.0.3 | 11 | 0 | 0 | 0 |
| `@arrows/error` | 1.0.2 | 3 | 2 | 2 | 0 |
| `@arrows/multimethod` | 1.4.1 | 12 | 4 | 4 | 0 |
| `atomically` | 1.7.0 | 11 | 26 | 26 | 0 |
| `@aws-crypto/sha1-browser` | 5.2.0 | 5 | 14 | 14 | 0 |
| `@aws-crypto/sha256-browser` | 5.2.0 | 5 | 14 | 14 | 0 |
| `@aws-crypto/sha256-js` | 5.2.0 | 5 | 15 | 15 | 0 |
| `@aws-crypto/util` | 5.2.0 | 5 | 15 | 15 | 0 |
| `@azure/arm-appservice` | 15.0.0 | 69 | 12 | 12 | 0 |
| `@azure/arm-resources` | 5.2.0 | 25 | 2 | 2 | 0 |
| `@bcoe/v8-coverage` | 0.2.3 | 8 | 6 | 6 | 0 |
| `benny` | 3.7.1 | 13 | 10 | 10 | 0 |
| `@braintree/sanitize-url` | 7.1.1 | 4 | 1 | 1 | 0 |
| `@braintree/sanitize-url` | 7.1.2 | 4 | 1 | 1 | 0 |
| `comment-parser` | 1.2.4 | 35 | 25 | 25 | 0 |
| `comment-parser` | 1.4.1 | 35 | 27 | 27 | 0 |
| `comment-parser` | 1.4.5 | 35 | 27 | 27 | 0 |
| `comment-parser` | 1.4.6 | 35 | 27 | 27 | 0 |
| `comment-parser` | 1.4.7 | 35 | 27 | 27 | 0 |
| `comment-parser` | 1.4.8 | 35 | 1 | 1 | 0 |
| `comment-parser` | 1.4.9 | 35 | 1 | 1 | 0 |
| `@compodoc/ngd-core` | 2.1.1 | 3 | 0 | 0 | 0 |
| `@compodoc/ngd-transformer` | 2.1.3 | 3 | 36 | 36 | 0 |
| `convex` | 1.28.2 | 174 | 135 | 135 | 0 |
| `convex-helpers` | 0.1.104 | 30 | 1 | 1 | 0 |
| `csp_evaluator` | 1.1.1 | 22 | 0 | 0 | 0 |
| `devcert` | 1.2.3 | 11 | 14 | 14 | 0 |
| `dexie-react-hooks` | 4.4.0 | 23 | 0 | 0 | 0 |
| `editions` | 6.22.0 | 4 | 6 | 6 | 0 |
| `@electric-sql/pglite-socket` | 0.0.6 | 8 | 2 | 2 | 0 |
| `@electric-sql/pglite-socket` | 0.1.3 | 9 | 2 | 2 | 0 |
| `@electric-sql/pglite-socket` | 0.2.7 | 10 | 3 | 3 | 0 |
| `@electric-sql/pglite-tools` | 0.2.7 | 7 | 2 | 2 | 0 |
| `@electric-sql/pglite-tools` | 0.3.3 | 7 | 2 | 2 | 0 |
| `@embroider/reverse-exports` | 0.2.0 | 3 | 86 | 86 | 0 |
| `example-typescript` | 1.0.0 | 7 | 13 | 13 | 0 |
| `expo-asset` | 11.1.5 | 25 | 80 | 80 | 0 |
| `expo-asset` | 12.0.13 | 25 | 74 | 74 | 0 |
| `expo-asset` | 57.0.18 | 26 | 94 | 94 | 0 |
| `expo-constants` | 17.1.6 | 6 | 69 | 69 | 0 |
| `expo-constants` | 18.0.13 | 6 | 67 | 67 | 0 |
| `expo-constants` | 57.0.19 | 6 | 87 | 87 | 0 |
| `expo-file-system` | 18.1.10 | 18 | 79 | 79 | 0 |
| `expo-file-system` | 19.0.22 | 22 | 52 | 52 | 0 |
| `expo-file-system` | 57.0.7 | 36 | 70 | 70 | 0 |
| `expo-font` | 13.3.1 | 19 | 18 | 18 | 0 |
| `expo-font` | 14.0.11 | 21 | 56 | 56 | 0 |
| `expo-font` | 57.0.4 | 24 | 2 | 2 | 0 |
| `expo-haptics` | 15.0.8 | 4 | 53 | 53 | 0 |
| `expo-image` | 3.0.11 | 32 | 94 | 94 | 0 |
| `expo-keep-awake` | 14.1.4 | 4 | 5 | 5 | 0 |
| `expo-keep-awake` | 15.0.8 | 4 | 52 | 52 | 0 |
| `expo-keep-awake` | 57.0.2 | 4 | 70 | 70 | 0 |
| `expo-linking` | 8.0.12 | 13 | 69 | 69 | 0 |
| `expo-modules-autolinking` | 2.1.10 | 22 | 3 | 3 | 0 |
| `expo-modules-autolinking` | 3.0.25 | 40 | 2 | 2 | 0 |
| `expo-modules-autolinking` | 57.0.13 | 46 | 1 | 1 | 0 |
| `expo-modules-core` | 2.3.13 | 41 | 25 | 25 | 0 |
| `expo-modules-core` | 3.0.30 | 44 | 57 | 57 | 0 |
| `expo-modules-core` | 57.0.18 | 46 | 75 | 75 | 0 |
| `expo-splash-screen` | 31.0.13 | 4 | 53 | 53 | 0 |
| `expo-sqlite` | 14.0.6 | 14 | 34 | 34 | 0 |
| `expo-status-bar` | 3.0.9 | 5 | 56 | 56 | 0 |
| `expo-status-bar` | 57.0.1 | 6 | 72 | 72 | 0 |
| `expo-symbols` | 1.0.8 | 5 | 55 | 55 | 0 |
| `expo-system-ui` | 6.0.9 | 8 | 52 | 52 | 0 |
| `expo-web-browser` | 15.0.11 | 7 | 52 | 52 | 0 |
| `@expo/devcert` | 1.2.0 | 11 | 25 | 25 | 0 |
| `@expo/devcert` | 1.2.1 | 11 | 84 | 84 | 0 |
| `@expo/dom-webview` | 57.0.1 | 8 | 76 | 76 | 0 |
| `@expo/log-box` | 57.0.4 | 42 | 74 | 74 | 0 |
| `feed` | 4.2.2 | 12 | 0 | 0 | 0 |
| `@glimmer/component` | 2.1.1 | 4 | 5 | 5 | 0 |
| `@huggingface/jinja` | 0.3.4 | 6 | 0 | 0 | 0 |
| `human-id` | 4.1.1 | 3 | 0 | 0 | 0 |
| `human-id` | 4.1.3 | 3 | 0 | 0 | 0 |
| `human-id` | 4.2.0 | 3 | 0 | 0 | 0 |
| `human-id` | 4.2.1 | 3 | 0 | 0 | 0 |
| `import-in-the-middle` | 1.14.0 | 3 | 2 | 2 | 0 |
| `import-in-the-middle` | 1.15.0 | 3 | 2 | 2 | 0 |
| `import-path-rewrite` | 0.0.1 | 4 | 5 | 5 | 0 |
| `junit-xml` | 1.2.0 | 5 | 1 | 1 | 0 |
| `khroma` | 2.1.0 | 50 | 148 | 148 | 0 |
| `@layerup/layerup-security` | 1.6.0 | 5 | 0 | 0 | 0 |
| `@lokalise/node-api` | 12.8.0 | 226 | 1 | 1 | 0 |
| `merge-anything` | 2.4.4 | 3 | 6 | 6 | 0 |
| `mimetext` | 3.0.28 | 8 | 7 | 7 | 0 |
| `@monaco-editor/react` | 4.7.0 | 3 | 4 | 4 | 0 |
| `mongodb` | 6.21.0 | 131 | 64 | 64 | 0 |
| `mongodb` | 7.5.0 | 131 | 1 | 1 | 0 |
| `mongodb` | 7.6.0 | 132 | 1 | 1 | 0 |
| `mongodb` | 7.7.0 | 132 | 62 | 62 | 0 |
| `moo-color` | 1.0.3 | 12 | 33 | 33 | 0 |
| `next-mdx-remote-client` | 1.1.2 | 15 | 1 | 1 | 0 |
| `npx-import` | 1.1.4 | 4 | 1 | 1 | 0 |
| `oblivious-set` | 2.0.0 | 3 | 14 | 14 | 0 |
| `onigasm` | 2.2.5 | 5 | 33 | 33 | 0 |
| `openapi3-ts` | 2.0.2 | 12 | 11 | 11 | 0 |
| `opencontrol` | 0.0.6 | 5 | 52 | 52 | 0 |
| `path-data-parser` | 0.1.0 | 4 | 0 | 0 | 0 |
| `pickleparser` | 0.2.1 | 19 | 278 | 278 | 0 |
| `pino` | 10.3.0 | 17 | 3 | 3 | 0 |
| `pino` | 10.3.1 | 17 | 3 | 3 | 0 |
| `pino` | 8.17.2 | 17 | 42 | 42 | 0 |
| `pino` | 9.14.0 | 17 | 42 | 42 | 0 |
| `piscina` | 3.1.0 | 28 | 34 | 34 | 0 |
| `piscina` | 4.8.0 | 46 | 46 | 46 | 0 |
| `pkg-pr-new` | 0.0.53 | 4 | 8 | 8 | 0 |
| `pkg-pr-new` | 0.0.71 | 4 | 44 | 44 | 0 |
| `pkg-pr-new` | 0.0.72 | 4 | 12 | 12 | 0 |
| `pkg-pr-new` | 0.0.75 | 4 | 12 | 12 | 0 |
| `pkg-pr-new` | 0.0.87 | 4 | 48 | 48 | 0 |
| `pkg-pr-new` | 0.0.88 | 4 | 48 | 48 | 0 |
| `@pnpm/config.env-replace` | 1.1.0 | 3 | 11 | 11 | 0 |
| `@pnpm/network.ca-file` | 1.0.2 | 3 | 11 | 11 | 0 |
| `protractor` | 7.0.0 | 7 | 1 | 1 | 0 |
| `react-static-example-typescript` | None | 10 | 17 | 17 | 0 |
| `@redocly/openapi-core` | 1.34.20 | 278 | 49 | 49 | 0 |
| `resolve-package-path` | 1.2.7 | 4 | 116 | 116 | 0 |
| `resolve-package-path` | 2.0.0 | 4 | 116 | 116 | 0 |
| `resolve-package-path` | 3.1.0 | 4 | 104 | 104 | 0 |
| `rxjs` | 7.3.0 | 250 | 2 | 2 | 0 |
| `rxjs` | 7.8.1 | 251 | 1 | 1 | 0 |
| `rxjs` | 7.8.2 | 251 | 1 | 1 | 0 |
| `signal-polyfill` | 0.2.2 | 25 | 14 | 14 | 0 |
| `@stablelib/base64` | 1.0.1 | 3 | 15 | 15 | 0 |
| `@statelyai/inspect` | 0.4.0 | 11 | 22 | 22 | 0 |
| `@streamparser/json` | 0.0.21 | 29 | 1 | 1 | 0 |
| `@supabase/ssr` | 0.5.2 | 12 | 4 | 4 | 0 |
| `template` | 0.0.0 | 35 | 63 | 63 | 0 |
| `tunnel-rat` | 0.1.2 | 3 | 0 | 0 | 0 |
| `unfurl.js` | 6.4.0 | 14 | 16 | 16 | 0 |
| `uri-js` | 4.2.2 | 10 | 1 | 1 | 0 |
| `@vercel/agent-eval-playground` | 0.1.3 | 40 | 106 | 106 | 0 |
| `@verdaccio/core` | 8.0.0-next-8.1 | 19 | 3 | 3 | 0 |
| `@verdaccio/logger-prettify` | 8.0.0-next-8.0 | 9 | 15 | 15 | 0 |
| `@verdaccio/url` | 13.0.0-next-8.1 | 6 | 2 | 2 | 0 |
| `@verdaccio/utils` | 7.0.1-next-8.1 | 11 | 7 | 7 | 0 |
| `vite-plugin-externalize-deps` | 0.10.0 | 4 | 0 | 0 | 0 |
| `vscode-tas-client` | 0.3.3 | 7 | 0 | 0 | 0 |
| `@wdio/xvfb` | 9.27.2 | 5 | 16 | 16 | 0 |
| `webpod` | 0.0.2 | 4 | 8 | 8 | 0 |
| `with` | 7.0.2 | 3 | 10 | 10 | 0 |
| `workbox-core` | 7.3.0 | 30 | 34 | 34 | 0 |
| `workbox-core` | 7.4.1 | 30 | 34 | 34 | 0 |
| `@workflow/world` | 5.0.0-beta.27 | 29 | 4 | 4 | 0 |
| `yaml-ast-parser` | 0.0.43 | 38 | 167 | 167 | 0 |
| **all 151** |  | **4,096** | **4,496** | **4,496** | **0** |

We developed against most of the repos in the first table, so try it on
your own project:

```sh
bunx -p typescript@7.0.2 tsc --noEmit --pretty false > tsc.txt
bun check --no-pretty > bun.txt
diff tsc.txt bun.txt
```

If there's a difference, please open an issue.

#### Compiler options the projects don't use

Projects only exercise the options they've chosen. So the 69 repos are
checked again under 11 other sets of options.

| Options | Projects | Identical | Errors in `tsc` | Errors in `bun
check` |
| --- | ---: | ---: | ---: | ---: |
| every strictness flag on | 69 | 69 | 100,756 | 100,756 |
| `strict` off | 69 | 68 | 76,960 | 76,958 |
| `skipLibCheck` off | 69 | 69 | 90,609 | 90,609 |
| `checkJs` | 69 | 69 | 84,122 | 84,122 |
| `declaration` | 69 | 69 | 80,495 | 80,495 |
| `isolatedModules` with `verbatimModuleSyntax` | 69 | 69 | 80,622 |
80,622 |
| `isolatedDeclarations` | 62 | 62 | 34,982 | 34,982 |
| `erasableSyntaxOnly` with legacy decorators | 69 | 69 | 71,577 |
71,577 |
| `nodenext` | 69 | 69 | 85,238 | 85,238 |
| `"types": []` | 69 | 69 | 105,818 | 105,818 |
| `"lib": ["es5"]` | 69 | 69 | 83,872 | 83,872 |

typescript-go crashes or times out on 7 repos with
`isolatedDeclarations`, so those aren't compared. Of the 896,192 errors
both report across all 11 sets, 21 are worded differently.

#### TypeScript's own tests

typescript-go runs TypeScript's compiler and conformance tests on itself
(12,762 files). Some tests ask for several sets of compiler options,
which makes 13,101 runs. For each one it commits what it expects, as
files. `bun check` has to produce the same files, byte for byte.

| Baseline | What it holds | Tests | Identical |
| --- | --- | ---: | ---: |
| `.errors.txt` | every error, with the source around it | 13,101 |
13,101 |
| `.types` | the printed type of every expression and name | 12,463 |
12,463 |
| `.symbols` | the symbol of every name, with where it is declared |
12,463 | 12,463 |
| `.js` | the declaration files that are emitted, and the errors in them
| 13,032 | 13,032 |
| `.trace.json` | the log of how every import was resolved, with
`traceResolution` | 151 | 151 |

Not every test has every kind of baseline. The `.errors.txt` baselines
contain 966 different error codes.

A `.js` baseline also has the JavaScript that `tsc` emits. That part
isn't compared, because Bun has its own transpiler. The same goes for
the `.js.map` and `.sourcemap.txt` baselines. `bun check` doesn't write
declaration files for you either. It generates them in memory for
projects that other projects reference, and that's the code these
baselines test.

typescript-go's test runner skips 45 tests, and 8 more when it compares
what's emitted. This one skips the same ones.

The tests and the baselines are vendored in this PR as one file
(`test/cli/check/typescript-go/bundle.zst`, 8 MB, written by `sync.ts`
from a typescript-go tag). All 5 kinds run in CI:

```sh
bun bd test test/cli/check/conformance
```

#### Generated programs

TypeScript's tests only have a few forms of some things. Take this:

```ts
let s: S | null = mk();
while (s) {
  const cur = { v: s };
  s = cur.v;
}
```

`tsc` reports TS7022 on `cur`. Its type needs the type of `s`, which
needs what the loop assigns at the bottom, which needs `cur`. There are
thousands of ways to write that loop, and a type checker can pass every
one of TypeScript's tests and still say nothing about most of them.

So there are generators. Each one writes a cross product with one
function per line, runs `tsc` and `bun check` on it, and compares.
There's no expected output stored anywhere. `tsc` is the oracle.

These are in `test/cli/check/differential.test.ts`, and all of them
pass:

| What it generates | Functions | What has to match |
| --- | ---: | --- |
| narrowing: the reference, the guard, the control flow around it, what
happens before the use | 46,080 | every error and its message |
| variables without a type annotation | 29,484 | every error and its
message |
| calls with callbacks: the callee, the form of the callback, the
context of the call | 9,984 | every error and its message |
| circular references, directly and through a second declaration | 9,196
| every error and its message |
| a variable in a loop that's assigned back, in 12 kinds of loop |
20,304 | which loops have an error |
| index signatures: 14 kinds of key like `string & {}`, what they're
indexed by, how a name is written in a destructuring or an object
literal | 4,522 | every line `tsc` prints |
| index signatures: the key of the source and the key of the target, in
assignments, inference and redeclarations | 1,248 | every line `tsc`
prints |
| a circular reference next to a call with overloads and a callback |
3,888 | every line `tsc` prints |
| an interface that extends a mapped type of itself, in a `.d.ts` that
`skipLibCheck` hides: 13 forms of base type, 9 kinds of declaration, 36
uses | 4,212 | every line `tsc` prints |
| the same in a file that's checked, with the declaration before its use
| 4,212 | every line `tsc` prints |
| a conditional type inside another one that checks the same type, with
`infer` | 480 | every line `tsc` prints |
| a boolean: 13 ways to get one, 16 types it isn't assignable to, 8 ways
to assign it | 1,664 | every line `tsc` prints |
| a package with `export =`: what it exports, what `declare module` adds
to it, how it's imported, 3 messages | 144 | every line `tsc` prints |
| a parameter property that is a pattern: 5 modifiers, 5 patterns, 3
sets of options | 75 | every line `tsc` prints |
| a function from an index signature that is tested and not called: 11
receivers, 11 tests | 119 | every line `tsc` prints |
| a generator with literals where any type is expected: 14 contexts, 10
generators | 140 | every line `tsc` prints |
| a conditional or a mapped type over `this`: 9 types, 28 uses | 252 |
every line `tsc` prints |
| a type that is generic as an object, as an index, as both or as
neither: 13 types, 12 places | 156 | every line `tsc` prints |
| what a project with references imports: 5 imports, 5 sets of options,
3 sets of references | 75 | every line `tsc -b` prints, in order |

It also generates small projects, for file names that differ only in
case, like `import "./Button"` for `button.ts`. macOS and Windows take
those for the same file and Linux doesn't, so it runs on both kinds of
file system. Every case runs with `forceConsistentCasingInFileNames` on,
off and unset.

| What it generates | Cases | What has to match |
| --- | ---: | --- |
| how a file is referred to (8 forms of import and reference, packages,
`files`), in which spellings, in which order, from where | 150 | every
line `tsc` prints |
| the same where case matters, plus 2 real files like `Button.ts` and
`button.ts` | 167 | every line `tsc` prints |
| a project reference that's spelled differently from the directory, or
in 2 spellings | 27 | every line `tsc -b` prints |
| `tsconfig.json` with a value that isn't JSON, like `"strict": tru`: 28
kinds of value in 30 places | 840 | every line `tsc` prints |
| `tsconfig.json` whose root is a list, like `[{ "compilerOptions": {}
}]` | 9 | every line `tsc` prints |
| 3 projects with `references` and files that all of them include: the
order at the top, who references whom, a project that doesn't exist, a
type library that's missing | 189 | every line `tsc -b` prints, in order
|
| `import()` and `require()` with `import` or `require` somewhere in
their text, which changes what `tsc` says about why a file is in the
program | 44 | every line `tsc` prints |

The index signature and overload programs end with a line that has a
type error, and `tsc` has to report it. After a syntax error neither of
them checks anything, and then they'd agree about everything. CI runs 70
of the 840 `tsconfig.json` files, because each one takes 2 processes.
All 840 were compared on a laptop.

It runs in CI against TypeScript 7.0.2, which `test/package.json`
installs next to 6 as `typescript7`. You can also point it at another
`tsc`:

```sh
TSC=/path/to/typescript-7/tsc bun bd test test/cli/check/differential.test.ts
```

These run locally against `tsc` and aren't in the repo yet:

| What it generates | Programs | Different from `tsc` |
| --- | ---: | ---: |
| classes: access, overriding, initialization, heritage, `super`,
references to itself, 2 members with one name | 290,000 | 0 |
| overloads, inference, variance (21 groups) | 1,300,000 | 0 |
| JSX: 48 tags, 48 lists of attributes, 23 forms of children | 63,770 |
0 |
| where a long type is cut off in a message | 1,060,000 | 0 |
| 2 declarations of one variable or property | 123,891 | 0 |
| module augmentation through re-exports and aliases | 210 | 1 |
| JSDoc tags on every kind of host, in checked JavaScript | 68,334 | 0 |
| class expressions that refer to what holds them | 40,128 | 21 |
| type annotations inside an expression that refer to what's being
declared | 50,688 | 95 |
| symbols that `tsc` creates lazily | 15,444 | 25 |
| immediately invoked functions | 19,200 | 21 |
| type aliases that refer to themselves without end, through 36 kinds of
type | 517 | 28 |

The 1 in module augmentation is the wording of a message. In 28 of the
messages that are cut off, `tsc` doesn't agree with itself from run to
run, and those aren't counted.

The 4 rows before the last are all code that refers to itself while it's
being declared, like `class A { x = { a: null! as { p: A["x"] } } }`. In
3 of those 162, `tsc` reports an error and `bun check` reports nothing
for that declaration.

In the 28 of the last row, both report errors, but not the same list:
TS2589 is at another position or in another order, or the TS2538 next to
it is missing. `tsc` crashes with a stack overflow on 23 more of those
programs, which aren't counted. `bun check` reports an error in all 23.

#### Is it overfit to the tests?

- We searched the type checker's code for the names of TypeScript's
tests. Of the 12,271 names that are 12 characters or longer, 19 appear
in code. All 19 are also the name of a compiler option or a builtin
(`esModuleInterop`, `noUncheckedIndexedAccess`, `defineProperty` and so
on).
- It's a port. 1,791 of the 2,770 functions in typescript-go's
`internal/checker` are named in comments, so you can read the two side
by side.
- A bug found in a real project or by a generator gets reduced to a few
lines and added to `test/cli/check/check.test.ts`, with the output of
`tsc` as the expected output. A test is only added if it fails without
the fix. There are 431 tests in that file.

#### Read side by side with typescript-go

Passing tests doesn't show that a port behaves like the original.
TypeScript's tests guard `tsc` against its own regressions, and `tsc`
never had this port's bugs. So every function of typescript-go's
checker, binder, module resolver, config parser, program and build mode
was read next to its counterpart here. Wherever the two could behave
differently, the reader had to write a small program that shows it and
run both tools on it. A second reader ran each program again and tried
to refute it.

1,706 of those programs are in `test/cli/check/differential-cases.json`.
`differential-cases-1.test.ts` to `differential-cases-4.test.ts` run
`tsc` 7.0.2 and `bun check` on each, on every platform where CI runs
Bun's tests, and the output has to be the same. Windows skips 3 that
need a directory named `c:`.

| What the program is about | Programs |
| --- | ---: |
| an error that `tsc` reports | 671 |
| an error that `tsc` doesn't report | 538 |
| the words of a message | 419 |
| the position of an error | 55 |
| the order of the errors | 22 |
| a crash | 1 |

The programs on which the two disagree are under "Known problems".

#### Threads

It uses all of your cores, so the answer could depend on which thread
gets somewhere first.

One of the tests generates 60 modules that all enter the same cycles
(variance of type parameters, recursive type aliases, functions and
constants without annotations) and checks them on 1, 2, 3, 8 and 16
threads. The output has to be byte for byte the same. typescript-go
reports 518, 519, 521 and 525 errors for that program with 1, 2, 4 and 8
checkers.

#### Broken code

You run a type checker on code you're in the middle of writing. The
worst thing it can do there is give up on a file and say everything is
fine.

One of the tests takes 268 valid files from TypeScript's tests and
damages each in 7 ways (truncate, delete a token, insert a stray token,
swap 2 tokens, replace a token, cut at a random byte, delete a line),
then appends a type error. With 17 more written by hand, that's 1,893
files. Every one of them has to report something.

#### Breaking real code

Most of the code in the projects above compiles. To see what happens to
real code with errors in it, a script takes `packages/effect` from
[Effect](https://github.com/Effect-TS/effect) (496 files, and about as
hard on a type checker as TypeScript gets), breaks 4 files at a time,
runs `tsc` and `bun check`, and compares every error with its whole
message. There are 32 ways to break a file: swap 2 arguments, drop a
type argument, turn `never` into `unknown`, delete an overload, reverse
a conditional type, remove `readonly`, and so on.

| Mutations | Errors in `tsc` | Different error codes | Missing in `bun
check` | Only in `bun check` | Worded differently |
| ---: | ---: | ---: | ---: | ---: | ---: |
| 913 | 2,925 | 67 | 0 | 0 | 8 |

#### Module resolution

With `traceResolution`, `tsc` logs every step of resolving every import:
each file it looks for, each `package.json` it reads, each condition in
`exports` it tries. `bun check` prints the same log. For 69 of the
projects above that's 8,463,167 lines, and all 69 logs are identical.

typescript-go resolves on several threads, so which lookup of a
`package.json` says "cached" changes between runs. Both logs go through
the normalization that typescript-go's own test runner uses for that.

#### Overflow checks and assertions

A release build doesn't check for integer overflow, so an index that
wraps around can go unnoticed as long as the output looks right. All of
TypeScript's tests and all 637 `tsconfig.json` files from the survey
also run on a build with overflow checks and debug assertions turned on.
Nothing panics and the output is the same.

#### Big inputs

16 tests generate programs that grow in one direction: 400 `if`
statements in a `finally` block, 400 nested loops, a chain of 400 method
calls, a union of 100 object types narrowed one member at a time, 400
overloads, and so on. An exponential algorithm doesn't finish at those
sizes.

4 more are long instead of deep: 40,000 top-level lines of `export const
v = o.a.b.c + k.y.y.x`, of template expressions, and of calls with
callbacks. 16,000 lines of the first take 0.15 seconds. `tsc` 7.0.2
takes 53.

#### Everything else

Adding a type checker shouldn't make `bun run` or `bun build` slower
when you aren't type checking. Parsing 7,012 TypeScript files (37 MB)
takes 1.003x the instructions it does on `main`, and `typescript.js` (9
MB) takes 0.995x.

#### Performance

`tsc` 7.0.2 with default settings against `bun check` on 16 threads.
16-core Apple silicon, 5 rounds, alternating between the two. Wall clock
time is the fastest round, CPU time and memory are medians. Both report
exactly the same errors in every row.

Wall clock time:

| Project | Files | `tsc` 7.0.2 | `bun check` | Faster |
| --- | ---: | ---: | ---: | ---: |
| vscode `src` | 9,795 | 5.98 s | 1.24 s | 4.8x |
| mikro-orm | 2,883 | 5.97 s | 1.20 s | 5.0x |
| next.js `packages/next` | 2,881 | 1.82 s | 0.28 s | 6.4x |
| next.js, root | 3,547 | 1.28 s | 0.42 s | 3.1x |
| storybook `scripts` | 1,039 | 0.96 s | 0.27 s | 3.5x |
| nuxt | 839 | 0.80 s | 0.27 s | 3.0x |
| playwright | 706 | 0.64 s | 0.15 s | 4.2x |
| lit `packages/react` | 6 | 2.12 s | 0.46 s | 4.6x |

CPU time:

| Project | Files | `tsc` 7.0.2 | `bun check` | Less |
| --- | ---: | ---: | ---: | ---: |
| vscode `src` | 9,795 | 32.1 s | 15.2 s | 2.1x |
| mikro-orm | 2,883 | 29.4 s | 11.2 s | 2.6x |
| next.js `packages/next` | 2,881 | 6.8 s | 2.8 s | 2.5x |
| next.js, root | 3,547 | 5.2 s | 4.2 s | 1.2x |
| storybook `scripts` | 1,039 | 4.8 s | 1.8 s | 2.6x |
| nuxt | 839 | 4.2 s | 1.7 s | 2.5x |
| playwright | 706 | 3.6 s | 1.3 s | 2.8x |
| lit `packages/react` | 6 | 2.8 s | 0.5 s | 5.6x |

Peak memory:

| Project | Files | `tsc` 7.0.2 | `bun check` | Less |
| --- | ---: | ---: | ---: | ---: |
| vscode `src` | 9,795 | 7.97 GB | 2.14 GB | 3.7x |
| mikro-orm | 2,883 | 6.67 GB | 1.35 GB | 4.9x |
| next.js `packages/next` | 2,881 | 1.81 GB | 0.71 GB | 2.5x |
| next.js, root | 3,547 | 1.33 GB | 0.55 GB | 2.4x |
| storybook `scripts` | 1,039 | 1.39 GB | 0.56 GB | 2.5x |
| nuxt | 839 | 1.14 GB | 0.51 GB | 2.2x |
| playwright | 706 | 1.06 GB | 0.46 GB | 2.3x |
| lit `packages/react` | 6 | 0.63 GB | 0.24 GB | 2.6x |

Measured with other work running on the machine (load average 6 to 13),
and with a release build of the type checker on its own, not a release
build of Bun. The `tsc` 6.0.2 number at the top is the better of 2 runs
on Node 25.6, on the same machine under the same load.

#### Long-running processes

`Bun.build({ check: true })` can run many times in one process. A test
runs repeated builds and checks that memory doesn't grow.

#### A first run

`bun init`, then `bun check --skipLibCheck false`, which also checks
every declaration file in `@types/bun`, `@types/node` and React against
the built-in `lib.*.d.ts` files:

| Template | Files checked | Errors |
| --- | ---: | ---: |
| `bun init -y` | 244 | 0 |
| `bun init --react` | 254 | 0 |
| `bun init --react=tailwind` | 256 | 0 |
| `bun init --react=shadcn` | 279 | 0 |

Projects that have an older `typescript` installed. `bun check` reports
the same errors as it does with the files of the TypeScript 7.0.2
package:

| Project | Installed `typescript` | Errors |
| --- | --- | ---: |
| dayjs | 2.9.2 | 0 |
| immer | 5.0.2 | 5 |
| svelte | 5.5.4 | 62 |
| zod | 5.5.4 | 31 |
| drizzle-orm | 5.6.3 | 10,845 |
| solid | 5.7.3 | 352 |
| vercel/ai | 5.8.3 | 1,561 |
| lit | 5.9.2 | 1,891 |
| apollo-client | 5.9.3 | 461 |

A test compares all 108 built-in files with the `typescript` package,
byte for byte.

Also compared with `tsc` 7.0.2, with the same output: 150 valid,
outdated, misspelled, empty and wrongly typed values in `tsconfig.json`.
Windows line endings, a byte order mark, UTF-16 and emoji in source
files. Paths with spaces, Cyrillic, Japanese and accents. A
`tsconfig.json` that extends 1 package, 2 packages or a missing one.
Both kinds of decorators. The config files that `create vite` writes. A
workspace whose packages import each other's sources.

`bun check | head -1` with 24,000 errors exits normally. Ctrl-C in a
terminal ends it in 35 ms with exit code 130.

#### What hasn't been tested

- The comparisons against `tsc` on real projects only ran on macOS
arm64. CI builds this for Windows x64 and arm64, macOS, Linux glibc and
musl, Android and FreeBSD, and runs `test/cli/check` wherever it runs
Bun's tests.
- ThreadSanitizer hasn't run on this version.
- Most of the `unsafe` code doesn't run under Miri.
- Only TypeScript 7.0.2. We haven't tried syncing to a newer
typescript-go.

## Known problems

Where it differs from `tsc` 7.0.2:

- typescript-go's answer for a file can depend on which other files are
in the program, in what order, and on how many checkers it uses. `bun
check` gives the same answer at any thread count, and everything here is
compared against typescript-go with 1 checker. It shows in 1 of the
1,103 configs above: in vuejs/core's `packages-private/tsconfig.json`, 1
of typescript-go's 940 errors is missing, a TS2345. Reduced to 3 files,
typescript-go reports that error with `"files": ["global.ts",
"model.ts"]` and doesn't with `["model.ts", "global.ts"]` or
`["model.ts"]`. Neither file imports the other.
- In rxjs, 7 of 22,030 messages explain the error along a shorter path:
`The types returned by '[bufferTime](...)[groupBy]' are incompatible`
where typescript-go has `'[bufferTime](...)[bufferTime](...)[groupBy]'`.
The position, the code and the first line of the message are the same.
It has the same cause: the order the files are checked in.
- nuxt with `strict` off: 7 of typescript-go's 74 errors are missing,
and there are 5 that it doesn't report.
- Single-quoted strings in `package.json` are accepted. typescript-go
rejects them.
- `interface Foo extends Partial<Foo>`, or any other mapped type of
`Foo`, is an error in both (TS2310). What `tsc` says after it depends on
what's asked for first, `Foo` or `Partial<Foo>`. Within a file `bun
check` says the same. Across files it can differ, because `tsc` goes by
the order of the whole program. With `skipLibCheck` off, an interface of
the library that's extended like that can have members in `bun check`
that it doesn't have in `tsc`.
- More than 332 variables without annotations in one loop, each
initialized with the one before it: `bun check` says it couldn't finish
the file and exits with code 1.
- On 254 of the small programs written while reading the two side by
side, the output differs. 1 is ordinary code: after `if (t === null || t
=== undefined) return;` on a generic `t: T`, `for (const [k, v] of
Object.entries(t)) v.foo` gets TS18046, `'v' is of type 'unknown'`,
where `tsc` accepts it. 38 are in unusual code or unusual command lines.
For example, `function f({ a = 1, b = g(a) }) {}` gets a false TS7022,
and in `enum E { A = 1, B = next(E.A) }` with `function next(e: E)` a
false TS2502. In a directory with a `jsconfig.json` and no
`tsconfig.json`, `bun check` checks that project where `tsc` goes on to
the parent's `tsconfig.json`. The other 215 are next to another error,
without the default library, or in code nobody writes on purpose.
- The order of the members of a union of anonymous object types in a
message can be different.
- A type alias that refers to itself without end gets TS2589 from both,
but it can be at another position, and a TS2538 next to it can be
missing. That's 28 of 517 generated programs.
- The lines under an error that explain it can differ: 8 of 2,925 errors
after breaking Effect's source.
- `for (!function () { n in e; }(); ;) break;` is a syntax error in
typescript-go 7.0.2. It isn't in TypeScript 6 or here. Minified jQuery 3
has that code.

This adds 5.94 to 7.21 MB to the binary, depending on the platform.

None of this would exist without typescript-go. It's the code this is
ported from, the baselines that say what the type of every expression
is, and a binary to compare against on any project.

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-10-06 10:37:16 -07:00

62 lines
3.0 KiB
TypeScript
Raw Permalink Blame History

// Writes src/runtime/cli/typescript_libs.bin: TypeScript's `lib.*.d.ts` files, which `bun check` has built in.
//
// bun scripts/update-typescript-libs.ts [directory of the lib.*.d.ts files]
//
// Without an argument, they are those of the `typescript7` package that test/package.json installs. They have to be of
// the version of typescript-go that the type checker is a port of. test/cli/check/check.test.ts compares them.
//
// The format is described in src/runtime/cli/typescript_libs.rs.
import { readdirSync, readFileSync, writeFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { constants, zstdCompressSync } from "node:zlib";
function installed() {
const test = join(import.meta.dir, "..", "test");
const paths = [dirname(require.resolve("typescript7/package.json", { paths: [test] }))];
const name = `@typescript/typescript-${process.platform}-${process.arch}`;
return join(dirname(require.resolve(`${name}/package.json`, { paths })), "lib");
}
const from = process.argv[2] ?? installed();
const nameLength = 39;
const recordLength = 1 + nameLength + 4 + 4 + 4;
// Two of the files are the dictionaries of the others, which repeat much of them: lib.webworker.d.ts is 97 KB by
// itself and 7 KB after lib.dom.d.ts. A program that needs one of the others nearly always needs its dictionary too.
// A trained dictionary saves 4%, these save 32%. The numbers are those of `dictionary` in typescript_libs.rs.
const dictionaries = ["lib.es5.d.ts", "lib.dom.d.ts"];
const dictionaryOf = (name: string) =>
dictionaries.includes(name) ? 0 : /^lib\.(dom|webworker|scripthost)\./.test(name) ? 2 : 1;
// By bytes, which is how they are searched. The names are ASCII.
const names = readdirSync(from)
.filter(name => /^lib(\.[a-z0-9]+)*\.d\.ts$/.test(name))
.sort();
const text = (name: string) => readFileSync(join(from, name));
const frames = names.map(name => {
const dictionary = dictionaryOf(name);
return zstdCompressSync(text(name), {
dictionary: dictionary ? text(dictionaries[dictionary - 1]) : undefined,
pledgedSrcSize: text(name).length,
params: { [constants.ZSTD_c_compressionLevel]: 19, [constants.ZSTD_c_contentSizeFlag]: 1 },
});
});
const header = Buffer.alloc(4 + names.length * recordLength);
header.writeUInt32LE(names.length, 0);
let offset = header.length;
for (const [i, name] of names.entries()) {
if (name.length > nameLength) throw new Error(`${name} is longer than ${nameLength} bytes`);
const record = 4 + i * recordLength;
header.writeUInt8(name.length, record);
header.write(name, record + 1, "latin1");
header.writeUInt32LE(offset, record + 1 + nameLength);
header.writeUInt32LE(frames[i].length, record + 1 + nameLength + 4);
header.writeUInt32LE(dictionaryOf(name), record + 1 + nameLength + 8);
offset += frames[i].length;
}
const to = join(import.meta.dir, "..", "src", "runtime", "cli", "typescript_libs.bin");
writeFileSync(to, Buffer.concat([header, ...frames]));
console.log(`${names.length} files from ${from}, ${offset} bytes`);
View Git Blame Copy Permalink