## Summary
We now report a defaulted `TypeVar` anywhere after a `TypeVarTuple` in a
legacy `Generic[...]` or `Protocol[...]` base. The ambiguity persists
when a required parameter comes between them.
```python
from typing import Generic, TypeVar, TypeVarTuple
Ts = TypeVarTuple("Ts")
U = TypeVar("U")
T = TypeVar("T", default=bool)
class C(Generic[*Ts, U, T]): ... # Now an error on T
```
For `C[int, str]`, an empty `Ts` leaves `U = int` and `T = str`. Binding
`int` to `Ts` instead leaves `U = str` and uses the default `T = bool`.
The typing spec describes the adjacent case. We also diagnose the
nonadjacent case, as mypy, Pyright, and Pyrefly do. [Typing
spec](https://typing.python.org/en/latest/spec/generics.html#defaults-following-typevartuple)
Legacy bases and PEP 695 declarations now share a diagnostic that points
to the pack and each later defaulted `TypeVar`. Both allow a defaulted
`ParamSpec`, whose argument list is distinguishable from the types that
fill the pack.
## Summary
`match` is a soft keyword. It can name a variable, a function, or the
target of an annotated assignment, including after a semicolon.
```python
first(); match = value
first(); match(value)
first(); match[0]: int
```
The parser reported a compound-statement error as soon as it saw the
`match` token. We now check the preceding separator when parsing a match
statement. If the token turns out to be an identifier, speculative
parsing discards the diagnostic. A genuine `match` statement on the same
logical line still produces the original error.
Tuple class-pattern captures repeatedly normalize the same element types
and often compute identical matched and binding subject types. Reuse
those results during pattern analysis, recognize exact tuple subjects
directly, and defer normalization of expanded unions until every
alternative fits the budget.
Related to astral-sh/ty#4676.
## Test plan
I compared release builds of the direct parent `8aeb8e2ebc` and this PR
at `f57574d7bb` on Apple Silicon macOS using Rust 1.99.0, fat LTO and no
PGO. Each fresh CLI process checked the generated class-pattern
benchmark at the indicated size with Python 3.12 and a single ty worker.
The times include startup and shutdown.
Eight interleaved blocks followed an excluded warmup. Each revision ran
twice in each timing position. The table gives median wall times, the
median of the eight within-block percentage changes, and the number of
blocks in which the PR was faster. Nine generic 32-case attempts
overlapped another local checker or build and were repeated. All 256
retained measurements completed without diagnostics.
| Cases | Parent | This PR | Paired time change | Faster blocks |
| --- | ---: | ---: | ---: | ---: |
| Non-generic 8 | 101.2 ms | 81.0 ms | −21.4% | 5/8 |
| Non-generic 10 | 182.8 ms | 135.0 ms | −25.5% | 8/8 |
| Non-generic 16 | 696.9 ms | 506.3 ms | −28.7% | 8/8 |
| Non-generic 32 | 4.979 s | 3.942 s | −23.3% | 8/8 |
| Generic 8 | 69.5 ms | 60.1 ms | −17.1% | 8/8 |
| Generic 10 | 121.6 ms | 85.3 ms | −29.9% | 8/8 |
| Generic 16 | 677.8 ms | 431.7 ms | −37.2% | 8/8 |
| Generic 32 | 15.087 s | 9.228 s | −39.6% | 8/8 |
At 32 cases, reusing pattern analysis reduced the median check from
4.979 s to 3.942 s for non-generic classes and from 15.087 s to 9.228 s
for generic classes. Both improved in every block.
Checking an eager annotation such as `A | B | C` can involve looking up
the same name twice. Ty needs both the type that the annotation denotes
and the runtime value on which Python calls `|`. In longer annotations,
it can also evaluate an inner union again while checking its parent.
This change reuses the runtime values of bare names, `None`, and nested
unions within the same eager annotation. A nested union computes its
runtime value only when the enclosing `|` can use it under the same
inference flags. Each expression still records its annotation type. For
example, `bool | int` denotes `int` but evaluates to a union object,
which matters for a subsequent `| None`.
This addresses the eager-annotation case from #29312 on top of #29727.
## Test plan
Measured CPU time for fresh, single-core `ty check` processes over 12
interleaved rounds. The comparison uses main at `18c5d27c41` and the
eager operand changes from #29312 rebased onto the same commit. Times
are medians in milliseconds for each whole input file.
| Input | Main | Rebased #29312 | This change |
| --- | ---: | ---: | ---: |
| Three ordinary classes, 1,500 signatures | 58.5 | 59.3 | 56.6 |
| Twelve ordinary classes, 450 signatures | 86.7 | 72.5 | 69.4 |
| Twelve specialized generics, 350 signatures | 144.5 | 100.3 | 100.0 |
| Direct `TypeForm` arguments | 113.9 | 98.3 | 91.7 |
The source-only checks of NumPy’s `_typing` package and Pydantic showed
no clear difference. Against rebased #29312, the stored 256-member
`TypeForm` case used 1.0% more CPU time, and the
`gradual_literal_union_equality` Criterion benchmark took 0.9% longer.
Compared the inferred types of all 14 AST nodes in a pair of left- and
right-nested annotations. Both alternatives and main yielded the same
type at every node.
Constraint-set assignability constraint collection can drop a caller's
type variable when it appears opposite a parameter typed as e.g. `T |
int`. Currently this only impacts unions in a few specific positions
(callable return types, protocol members), because those use
constraint-set assignability, whereas most unions still go through
`infer_map_inner`.
The tests here currently use callable return type unions so as to
exercise constraint sets. After #29236 all unions will go through this
path, and these tests can be simplified.
Passing `Callable[[], U | str]` to a parameter annotated as
`Callable[[], T | int]` currently infers `T = str` and rejects the call.
This change infers `T = (U & ~int) | str` instead, preserving the
portion of `U` that the concrete `int` arm does not cover.
Depends on #29615 for union normalization during argument checking. This
is a prerequisite to avoid false positives in #29236.
## Ecosystem report
Analyzed head `1aa3be439aa189d201ec9dd238bfdaee6a4fa31d`. One existing
ignore in scipy-stubs becomes unnecessary because the generic
`setdefault` override is now accepted. This is an expected improvement;
no unresolved ecosystem regressions remain. [Full
report](https://github.com/astral-sh/ruff/pull/29611#issuecomment-6453277106).
Remove covered exclusions from unions so `(P & ~Q) | Q` simplifies to `P
| Q`, including when a static supertype covers a gradual exclusion and
when literal union members restore excluded literals.
Preserve unrelated exclusions and recheck widened intersections against
the remaining union members. Keep exclusions of non-final nominal
classes when the covering type is `Hashable`, since subclasses can
disable hashing.
Coverage uses strict subtyping with conservative inference-cycle
handling. Avoid unnecessary disjointness checks and allocations, and
reject dynamic covering types and shared exclusions before creating
cached relation queries.
This enables argument checking to recognize unions reconstructed from
residual inferred types in #29611.
This type-simplification improvement isn't free; it costs 5% on a few
micro-benchmarks. This seems like the lowest I'm able to get the cost,
at least without much more invasive changes. It doesn't seem to show up
as a significant cost on real-world projects.
The [current CodSpeed
comparison](https://github.com/astral-sh/ruff/pull/29615#issuecomment-6333299680)
reports a 5.22% aggregate regression across the same three
microbenchmarks: `class_or_pattern_captures_90` at 4.54%,
`large_union_narrowing` at 4.14%, and `gradual_intersection_negation` at
6.97%. CodSpeed flags different runner types between base and head, so
these measurements include that uncertainty.
## Ecosystem report
Analyzed head `9ec674243823580789419b326f2f0df75ceb090c`. Four
truthiness warnings disappear when removing a covered exclusion exposes
a direct dynamic union alternative, which that diagnostic ignores. The
other diagnostic changes simplify or reorder types. All ecosystem
changes are expected. [Full
report](https://github.com/astral-sh/ruff/pull/29615#issuecomment-6349820846).
Infer a shared `ParamSpec` from all argument constraints. Comparable
callback signatures simplify regardless of argument order, consumer
bounds no longer unnecessarily narrow returned callbacks, and callback
consumers can contribute an intersection of parameter lists.
```py
from collections.abc import Callable
from typing import reveal_type
def choose[**P](left: Callable[P, None], right: Callable[P, None]) -> Callable[P, bytes]:
raise NotImplementedError
def integer(value: int, /) -> None: ...
def anything(value: object, /) -> None: ...
reveal_type(choose(integer, anything)) # (value: int, /) -> bytes
reveal_type(choose(anything, integer)) # (value: int, /) -> bytes
```
As [Carl
explains](https://github.com/astral-sh/ruff/pull/29172#issuecomment-6310115766),
a union of parameter lists does not correctly represent one `ParamSpec`
that must satisfy several callbacks. We conservatively reject
incomparable lower bounds until we can construct a common signature.
Removing the existing union-valued expansion also temporarily rejects
valid calls with top-level union arguments; supporting those requires
separate argument union expansion.
Preserving callable intersections through `ParamSpec` wrappers is
handled in [#29623](https://github.com/astral-sh/ruff/pull/29623).
Support for intersection-valued parameter lists in `functools.partial`
is deferred to [#29479](https://github.com/astral-sh/ruff/pull/29479).
Fixesastral-sh/ty#4804.
## Summary
INP001's preview-mode check for missing `__init__.py` files in
namespace-package directories with nested packages is now documented.
Empty directories and directory trees with no Python files are not
flagged.
## Test Plan
git diff --check passing. Documentation generation stalled while syncing
the Rust 1.99 toolchain; no tests run.
---------
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
## Summary
Django does not add an automatic primary key to abstract models. Skip
synthesizing one when the model declares `abstract = True`.
closes https://github.com/astral-sh/ty/issues/4822.
## Test Plan
New Markdown tests
## Summary
A bare `raise` can end at a semicolon. For example, Python accepts this
handler and re-raises the current exception.
```python
try:
work()
except ValueError:
raise; report()
```
The parser tried to read the semicolon as the start of an exception
expression and reported a syntax error. We now recognize both a newline
and a semicolon as terminators for a bare `raise`.
## Summary
Ty could panic on `assert predicate` inside a loop where `predicate`
starts as a lambda and a later iteration rebinds it to a class. When
cycle recovery added the class to the operand type, ty looked up its
metaclass to determine its truthiness. That lookup could introduce a new
inference cycle while Salsa was recovering the first one.
Reachability now evaluates the predicate from the operand types that
expression inference has already normalized. Outside loops, it caches
the result so repeated walks of the same condition can reuse it. Inside
a loop, a condition can depend on an assignment it guards, so the caller
reads the inferred operands directly. Expression inference no longer
stores predicate truthiness or recomputes it during recovery.
The 163-project ecosystem run produced identical diagnostics. Checking
isort took 291 ms, up from 248 ms on the base revision, which gives back
some of the speedup from #29706. The average timing factor across
projects was 1.02×. CodSpeed found no significant changes among the 214
benchmarks it compared. [Ecosystem
diff](https://dqhfo45wdv0ek.cloudfront.net/ruff/pull-requests/29759/ty-ecosystem/40467940782/1/diff.html)
·
[Timings](https://dqhfo45wdv0ek.cloudfront.net/ruff/pull-requests/29759/ty-ecosystem/40467940782/1/timing.html)
·
[CodSpeed](https://github.com/astral-sh/ruff/pull/29759#issuecomment-6435864937)
Closes https://github.com/astral-sh/ty/issues/4834.
## Summary
Checking Home Assistant’s Anthropic integration took 283.1 ms before
this change and 252.6 ms after (10.8% less time, with a 95% paired
interval of 7.8–13.7% less). Each of 24 blocks ran `ty check
homeassistant/components/anthropic` twice with the baseline build and
once with the changed build, in balanced, randomized order. All runs
emitted the same 41 lines of diagnostics. The full Home Assistant
timings did not resolve this patch’s individual contribution.
The Anthropic SDK’s synchronous and asynchronous streaming functions
each test 53 event names in one `or` expression. At each operand, ty
used to reapply every condition leading up to it, recording 1,378
conditions per expression. The continuation already carries those
earlier conditions, so adding just the preceding operand’s condition
brings the count down to 52.
Checking a file that imports the two original function bodies went from
95.9 ms to 19.3 ms. Changing just those two `or` chains to tuple
membership checks made both builds take about 12 ms. Toggling prefix
replay in a single release executable also reproduced the integration
improvement (270.1 ms to 237.5 ms).
An assignment expression (`:=`) can introduce a new binding partway
through a chain. Those chains keep the earlier conditions so the
reassigned name gets the same reachability information.
## Summary
Cold CLI checks of Home Assistant’s KNX integration took 14% less time,
and checks of its Reolink integration took 4.5% less time. Each row
averages 12 paired runs of `ty check
homeassistant/components/<integration>` with the project’s dependencies
installed. Both integrations emitted the same diagnostics before and
after.
| Integration | Before | After |
| --- | ---: | ---: |
| KNX | 389.5 ms | 334.9 ms |
| Reolink | 396.0 ms | 378.1 ms |
The KNX decoder has a long sequence of branches inside a `try` block.
Reading a name like `GroupValueRead` for the first time used to trigger
another snapshot of the bindings visible to the exception handler.
Restoring an earlier snapshot already gives that name the same undefined
state. We now leave the checkpoint’s binding revision alone on that
first read and advance it when the place gets a definition.
In isolation, a cold CLI check importing the original
`xknx.APCI.from_knx` body went from 84.5 ms to 36.4 ms (57%). For
`reolink_aio.Host.map_channel_json_response`, it went from 41.9 ms to
32.4 ms (23%).
## Summary
After a truthiness check on a dictionary, ty inferred `object` for
values captured by a mapping pattern and reported an error on valid
code.
```python
def f(d: dict[str, int]) -> None:
if d:
match d:
case {"a": c}:
c.bit_length() # `c` is an `int`.
```
The truthiness check narrows `d` to an intersection. We previously
walked the receiver’s class hierarchy to recognize `dict.get` and
`Mapping.get`, but an intersection has no single class to start from. We
now use the definition of the bound `get` method that member lookup
already resolved. An inherited `dict.get` or `Mapping.get` preserves the
value type, while a custom `get` can still return a different type.
The ecosystem run added one `no-matching-overload` error in `cwltool`.
After an earlier case handles mutable sequences, ty previously inferred
`list[Unknown]` for an enum’s symbols and accepted the call to
`str.join`. A valid CWL enum has string symbols, but `{"type": "enum",
"symbols": [1]}` also satisfies the Python annotation and matches the
pattern. The new error flags a call that would raise `TypeError` for
that input. [cwltool
source](https://github.com/common-workflow-language/cwltool/blob/ad9ba5b32003b9bc6db092cd6c684265bacba199/cwltool/main.py#L236-L245)
Closes https://github.com/astral-sh/ty/issues/4831.
## Summary
Linting a file with 8,192 dictionaries nested 64 levels deep went from
44.66 ms to 10.85 ms (4.1× faster). On 755 files from the CPython
3.12.10 standard library, the difference was within measurement noise,
both with the default rules and with only F601 and F602 enabled.
F601 and F602 used to build a `ComparableExpr` for every dictionary
value. In `{"a": {"a": {"a": 0, "b": 1}, "b": 1}, "b": 1}`, every key is
unique within its own dictionary, yet Ruff walks the inner values again
at each outer level. We now record where each key first appeared and
build the set of comparable values when we report the first repeat. A
later occurrence still gets an unsafe fix if its value matches any
earlier value for the same key.
| Dictionary depth | Before | After |
| ---: | ---: | ---: |
| 1 | 15.70 ms | 14.80 ms |
| 8 | 14.90 ms | 11.67 ms |
| 32 | 27.35 ms | 11.21 ms |
| 64 | 44.66 ms | 10.85 ms |
Each row contains the same number of two-key dictionaries. Times are
medians from 30 alternating pairs of single-core runs with `--no-cache
--select F841,F601,F602`. On a separate set of dictionaries with
repeated literal and variable keys, the diagnostic output and unsafe-fix
diffs matched byte for byte.
A plain string can compare equal to a `StrEnum` member. Previously, `if
value == Color.RED` narrowed a `str` to the enum and allowed
`value.name`, even when `value` was the string `"red"`. The same problem
affected integers compared with `IntEnum` members.
Keep the matching `str` or `int` type in equality checks, tuple
membership tests, and `match` value patterns. A value already known to
belong to the enum can still narrow to a specific member.
Closesastral-sh/ty#4827
## Test plan
Ran the reproducer with Python 3.13. The plain values `"a"` and `1`
entered all three kinds of branches, and `.name` on the string raised
`AttributeError`. With both default and strict equality settings, ty now
reveals `str` and `int` in those branches and reports the invalid
`.name` access.
## Ecosystem report
Apprise loses a false warning about converting a plain string to an enum
member. pip gains an assignment error where typeshed requires
`VerifyMode` and CPython also accepts the plain integer `2`. Both
changes are expected. [Analyzed head
`e0388646c9f0d4ac09dfea732b05ca2d84839950`](https://github.com/astral-sh/ruff/commit/e0388646c9f0d4ac09dfea732b05ca2d84839950)
· [Full ecosystem
report](https://github.com/astral-sh/ruff/pull/29743#issuecomment-6428093437)
This PR implements an optimization for type relation checks of a nominal
class against a union, by doing a preliminary pass on the union to
eliminate any union arms that can't possibly match, using a cached
mapping from nominal type to the union elements that could possibly
match it.
This particularly improves tuple pattern narrowing, which can repeatedly
compare narrowed intersections against the same original union.
When multiple indexed classes match, the checker visits the union in its
original order. It also uses the full traversal when collecting
diagnostic details or when a gradual base could make the source
assignable to an unrelated class.
Related to astral-sh/ty#4676.
## Test plan
These local measurements compare the index at `cc85ba9178` with its
direct parent `e139fbd6e8`, before the rebase onto `6840db323c`. Each
build used the standard fat-LTO release profile and ran `ty check` with
one worker and Python 3.12. Timings include process startup. The build
order varied across blocks, warmups were excluded, and the percentages
are median paired wall-time changes.
| Workload | Parent | With index | Paired change |
| --- | ---: | ---: | ---: |
| Nominal tuple, 10 cases | 204.3 ms | 181.6 ms | −10.6% |
| Nominal tuple, 16 cases | 788.2 ms | 667.9 ms | −16.6% |
| Nominal tuple, 32 cases | 6.02 s | 4.75 s | −21.2% |
| Generic tuple, 32 cases | 14.14 s | 13.99 s | −1.1% |
| Assignments spread across 96 arms | 74.6 ms | 67.1 ms | −10.2% |
| Assignments matching the first arm | 65.3 ms | 67.3 ms | +2.0% |
The assignment workload makes 1,536 calls after negative `isinstance`
checks. Compared with a redundancy-only index, spreading matches across
the 96 arms reduced time from 73.8 ms to 67.1 ms, a 9.9% improvement
across 24 paired blocks with a win in every block. First-arm matches and
unnarrowed arguments were about 2% slower than the redundancy-only index
by paired median, with results varying by execution order. The generic
32-case control had only three blocks and showed little change.
Given `v = "1"`, ty used to narrow `v` to `Never` inside `if v !=
"42".upper()`. This hid the type error in `v + 1`, even though the
branch runs.
`"42".upper()` has type `LiteralString`, so its exact value is unknown.
The comparison can narrow that operand to a known value or exclude that
value. It now preserves the type of the concrete string or string-valued
enum member on the other side.
Fixes https://github.com/astral-sh/ty/issues/4826.
## Test plan
Ran the reported example under CPython 3.13 and confirmed that the
unequal branch raises `TypeError`. Checked the `!=` branch and the
`else` branch of `==` with strict equality semantics enabled and
disabled. In all four cases, ty reveals `Literal["1"]` and reports
`unsupported-operator` for `v + 1`.
When a match case captures the subject under its own name and its guard
fails, later cases now narrow that capture. Previously, those cases
could retain a wider type because narrowing targeted only the binding
read before the match.
Retain bindings that capture the entire subject under its original name,
including `as` patterns and OR alternatives. Captures of extracted
values and assignments made by guards remain distinct from the saved
subject.
Fixesastral-sh/ty#4761.
Fixes a hang when a recursive alias combines positive and negative
references, such as `type A = list[A | ~A] | list[~A]`.
Retain negated recursive aliases whose union members resolve to nominal
instance types, so mapping does not repeatedly unfold the alias into
itself. Other union members still undergo normalization to preserve
literal exclusions and positive constraints.
Fixes https://github.com/astral-sh/ty/issues/4799.
## Test plan
Checked the original reproducer and equivalent spellings using `Not[A]`
and `Intersection[object, Not[A]]` with timeout-bounded debug binaries.
The patched checker completes on all three; the original reproducer
hangs on the baseline.
Return refinement introduced in #28303 can discard valid generic
callback specializations because eager subtyping cannot infer
callback-local type variables. This incorrectly narrows wrappers around
`sum` to its bool/literal-integer overload.
Use constraint-based subtyping when revalidating the inferred
alternatives. This allows generic callbacks to match through
specialization while retaining strict subtype checks for gradual
evidence.
Fixesastral-sh/ty#4810.
## Test plan
Checked the original `sum` example at the parent of #28303 and at its
merge commit: it passes before #28303 and fails afterward. It passes
with this change.
## Summary
A full ty check of isort went from 204 to 172 ms in the ecosystem run.
Across all 163 projects, the average timing factor was 1.00×. Local
checks of two isort revisions on macOS took 16–23% less time. [Ecosystem
timings](https://dqhfo45wdv0ek.cloudfront.net/ruff/pull-requests/29706/ty-ecosystem/40342132072/1/timing.html)
In `if bool(stop()):`, the call to `bool` has type `bool`, but neither
branch can run if `stop()` returns `Never`. Ty uses the operand types to
work this out. In a loop, those types can depend on an assignment
guarded by the same condition. Computing types and truthiness in
separate queries can make ty revisit that loop repeatedly.
We now record each predicate’s evaluation context when we index it.
Expression inference computes the predicate’s truthiness alongside its
operand types and recomputes it if cycle recovery widens them. Ordinary
conditions and match guards both read that result.
The only warning that changed consistently across ecosystem runs was in
psycopg. It now lists the `int` class separately even though `type[str |
Any | int]` already includes it. [Ecosystem
diff](https://dqhfo45wdv0ek.cloudfront.net/ruff/pull-requests/29706/ty-ecosystem/40342132072/1/diff.html)
In stub files, ty required `@override` on the first overload even when
an implementation was present, while overload validation required the
decorator on the implementation. Honor `@override` on the implementation
whenever one exists, matching Pyright and Pyrefly. Continue using the
first overload when there is no implementation.
The implementation also becomes the location for
`invalid-explicit-override` diagnostics in these stubs.
Fixes https://github.com/astral-sh/ty/issues/4742.
## Test plan
- Checked the reported `.pyi` example, an ellipsis-only minimized
example, and a stub without an implementation using Python 3.13 with all
rules enabled; all passed without diagnostics.
- Compared decorator placement with mypy 2.4.0, Pyright 1.1.414,
basedpyright 1.40.2, and Pyrefly 1.4.0. Pyright, basedpyright, and
Pyrefly honor the implementation's decorator in stubs; mypy rejects stub
overload implementations. All four accept the decorator on the first
overload when no implementation exists.
## Ecosystem report
For head `5690fe75333fb6755662fda0cec8d6a192905d2b`, the ecosystem run
removes 27 false `missing-override-decorator` errors from pyochain stubs
whose overload implementations already have `@override`. All changes
were reproduced and classified as expected; CI passed and no ecosystem
concern blocks review. [Full ecosystem
report](https://github.com/astral-sh/ruff/pull/29470#issuecomment-6260122936).
## Summary
Reachability and capture inference both walk alternatives such as `case
A(value=v) | B(value=v)` from left to right. They can compute the same
type remaining after an alternative fails. For a wide union, each
subtraction distributes an exclusion over the union members and
normalizes the resulting union of intersections.
Cache `subject & ~definitely_matched` by its ordered operands and
program. The query applies the exclusion directly to the subject so the
intersection builder can discard impossible branches as it distributes.
Refs astral-sh/ty#4815. The benchmark is in #29726.
## Test plan
Compared optimized dev builds on Apple Silicon, reversing the order of
the two builds in alternating blocks. Criterion results are medians of
six process estimates with ten samples each. CLI results are medians of
five runs per build after warmup.
| Input | Parent | This change | Time change |
| --- | ---: | ---: | ---: |
| Criterion, 45 classes | 76.8 ms | 57.5 ms | −25% |
| Criterion, 90 classes | 235.4 ms | 146.4 ms | −38% |
| CLI, 180 classes, four class alternatives followed by a single-class
case | 330 ms | 252 ms | −24% |
| CLI, 180 classes, twelve groups of three alternatives | 3.82 s | 2.55
s | −33% |
The 90-class CLI input retired 25% fewer instructions, with peak memory
footprint within 1% of the parent. On the eight-case tuple inputs from
astral-sh/ty#4676, retired instructions changed by less than 1% for both
generic and non-generic classes.
Also applied the benchmark and this change to the tuple-pattern stack at
`a08de583` (#29203). Across four paired Criterion runs on that stack,
the 45-class case fell from 63.9 to 53.4 ms and the 90-class case fell
from 195.9 to 131.8 ms.
When ty checks a value assignment such as `Alias = A[int] | B[int] |
C[int]`, it currently normalizes `A[int] | B[int]` and then compares
those same members again while building the final union. Longer chains
repeat that work for every prefix, even when nothing ever uses the value
as a type.
Instead, this PR just stores the runtime operands of `|` in the
runtime-union object, and only builds and normalizes the actual type
union lazily, if or when the runtime union value needs to be understood
as a type. For example, checking a stored union against `TypeForm[Base]`
walks the operands once and caches the result. Checks such as
`isinstance` can still inspect the original operands.
This is an alternative to #29304, which keeps the type of every
intermediate union available. For `(A | B) | C`, it normalizes `A | B`,
then seeds a fresh `UnionBuilder` with those members. If the
construction policy and inference context match, the builder only needs
to compare `C` with the existing members. This PR stores the `A | B`
object as the left operand of the outer union. If only the final type is
requested, one union builder visits `A`, `B`, and `C`. A second request
for that same union uses the cached type. The two approaches take about
the same time when a 256-class union is consumed once.
- For a chain of assignments like `U2 = U1 | Literal[2]`, each
assignment provides a new type-variable binding context. In #29304 that
context is part of the cache key, so these repeated assignments defeat
the caching, and extending the union can revisit its prefixes even when
nothing ever asks for their types. This accounts for the slowness of
#29304 on the "Extend a literal union across 1024 assignments" case
below.
- If `Prefix` is a long union and `(Prefix | X, Prefix | Y, ...)` is
checked against `tuple[TypeForm[Base], ...]`, every extension needs its
type. #29304 can reuse the normalized `Prefix` across those extensions.
This PR walks all its members for each distinct result. With 32
extensions, #29304 is nearly twice as fast. If this case turns out to
occur commonly in practice, we could add intermediate prefix caching as
well.
This PR is larger than #29304, which leaves the runtime union
representation largely intact and adds a path in `UnionBuilder` for
reconstructing an already normalized prefix. This PR needs extra
complexity in `UnionTypeInstance`, where specialization, cycle recovery,
and runtime checks need to distinguish the original operands from the
denoted type. But I prefer keeping the complexity of these
runtime-unions in the runtime-union implementation, rather than allowing
it to leak into the core `UnionBuilder`, as it does in #29304.
This also fixes a bug (as seen in xarray in the ecosystem report) with a
runtime check list `isinstance(x, float | int)`. Previously we would
simplify this union, eliminating `int`, but `int` and `float` are
separate classes at runtime, so we would then miss that the `isinstance`
check catches integers as well. Now the `isinstance` narrowing can look
at the original runtime operands instead.
## Test plan
I timed optimized builds on Apple silicon using the same main revision
(`157caa55a5`) for all three. The #29304 build applies its reviewed
patch (`a0f998e2b4`) to that base. Each entry is the median elapsed time
in milliseconds for six fresh, single-worker CLI checks with warm
filesystem caches.
| Workload | Main | #29304 | This change |
| --- | ---: | ---: | ---: |
| Construct an unused union of 256 specialized classes | 326 | 56 | 34 |
| Check the stored union against `TypeForm[CommonBase]` | 328 | 57 | 60
|
| Extend a literal union across 1,024 assignments | 46 | 3,789 | 29 |
| Extend a union across 128 assignments and consume each prefix | 77 |
95 | 79 |
| Check 32 extensions of the same 128-class prefix as type forms | 112 |
57 | 105 |
## Ecosystem report
At `dcc4ce9890`, xarray loses two false positives because `isinstance`
now recognizes integer chunk sizes. Three existing xarray diagnostics
include the types that actually reach those branches. Two truthiness
warnings in aioredis and dd-trace-py display the same union members in a
different order. All seven changes are expected. [Full ecosystem
report](https://github.com/astral-sh/ruff/pull/29727#issuecomment-6422364992)
## Summary
A full isort check went from 218 ms to 184 ms in the ecosystem run, a
16% reduction. Across all 163 projects the average timing factor was
1.00x, and no diagnostics changed.
In this example, `ready` can be true for either type of `value`, so the
path that skips `isinstance` already accounts for both possibilities.
```python
from typing import reveal_type
def f(value: str | None, ready: bool) -> None:
if ready or isinstance(value, str):
reveal_type(value) # str | None
```
| isort | Before | After | Wall time saved | CPU time saved | Runs per
build |
| --- | ---: | ---: | ---: | ---: | ---: |
| This patch alone | 336.0 ms | 296.9 ms | 11.6% | 6.8% | 20 |
| With #29701 and #29706 | 227.1 ms | 215.6 ms | 5.1% | 2.9% | 60 |
## Summary
This PR adds a `shared-clock-mock` rule in preview. Patching a clock or
sleep function on a shared module can affect unrelated threads or tasks,
including when the target is qualified with the consuming module:
```python
from unittest.mock import Mock, patch
from example import worker
sleep = Mock()
with patch.object(worker.time, "sleep", sleep):
worker.retry()
```
Instead, you should replace the consuming module's binding to keep the
fake local:
```python
from types import SimpleNamespace
from unittest.mock import Mock, patch
from example import worker
sleep = Mock()
with patch.object(worker, "time", SimpleNamespace(sleep=sleep)):
worker.retry()
```
We also flag patches of wall-clock, monotonic, and performance-counter
readings, plus `asyncio.sleep`. The rule recognizes `patch`,
`patch.object`, pytest-mock fixtures, and `monkeypatch.setattr` /
`delattr`. (Nested `.time` and `.asyncio` targets use a naming
heuristic, so a fixture that already installed a private clock can
produce a false positive.)
## Case analysis
The ecosystem run adds 601 diagnostics across eight projects.
### Counting sleep calls in LangChain
This reduced version of LangChain’s device-login test records the
polling interval before and after the server asks the client to slow
down.
```python
sleeps = []
monkeypatch.setattr(oauth_module.time, "sleep", sleeps.append)
login_chatgpt_device(store_path=tmp_path / "x.json", poll_interval=2.0)
assert sleeps[:2] == [2.0, 7.0]
```
The OAuth module does `import time`, so `oauth_module.time` and `time`
are the same object. If another thread calls `time.sleep(1)` between the
two polls, the recorder sees `[2.0, 1, 7.0]`. Installing
`SimpleNamespace(sleep=sleeps.append)` on `oauth_module.time` keeps that
call out of the list.
### Running out of clock readings in Superset
Superset tests how long a screenshot request waits for a lock. It
patches `superset.dashboards.api.time.monotonic` with
`side_effect=[10.0, 10.5, 11.0]` and patches `time.sleep` on the same
module. An unrelated `time.monotonic()` can consume the next reading.
The request can then see the deadline early or exhaust the iterator and
raise `StopIteration`. A private clock for `superset.dashboards.api`
needs both `monotonic` and `sleep`.
### Retrying an async request in Prefect
Prefect simulates three failed SQS requests and expects one two-second
backoff. The test uses
`@patch("prefect_aws.observers.ecs.asyncio.sleep")`, but the observer
does `import asyncio`. A second task awaiting `asyncio.sleep(0)` would
also call the mock and break `assert_called_once_with(2)`.
### A fixture that already installed a private clock
Ruff can see the same patch in two tests even though only one reaches
the shared module. For example, an autouse fixture can replace
`worker.time` before the test starts.
```python
@pytest.fixture(autouse=True)
def private_clock(monkeypatch):
monkeypatch.setattr(worker, "time", SimpleNamespace(sleep=lambda _: None))
def test_retry(monkeypatch):
sleep = Mock()
monkeypatch.setattr(worker.time, "sleep", sleep) # shared-clock-mock
worker.retry()
```
The warning in `test_retry` is a false positive. At runtime the receiver
is the fixture’s `SimpleNamespace`. If the fixture returns that
namespace, the test can use `monkeypatch.setattr(private_clock, "sleep",
sleep)` and Ruff accepts it.
### A package with its own `time` module
Gevent ships a `gevent.time` module with its own `sleep` binding. The
dotted string is enough for Ruff to flag this reduced example.
```python
import time
from unittest.mock import patch
original_sleep = time.sleep
with patch("gevent.time.sleep"):
assert time.sleep is original_sleep
```
The assertion passes. Callers that look up `gevent.time.sleep` see the
patch. A caller of standard-library `time.sleep` still gets the original
function. Ruff infers the owner from the `.time.sleep` suffix in a
string target, so it reports the wrong module here.
Hi :)
## Summary
The S113 docs say it checks `requests` or `httpx` calls that omit
`timeout`, but that hasn't been true for httpx sincee #12213
httpx applies a default timeout, so leaving the argument out is fine
there -- only an explicit `timeout=None` gets reported
In the code the implicit case is only reported when the module is
`requests`
So this rewrites "What it does" & "Why is this bad?" to match what the
rule actually does
Doc comment only, no behaviour change -- closes#27868
## Test Plan
Ran ruff 0.16.3 over three files:
- `httpx.get(url)` -> nothing reported
- `httpx.get(url, timeout=None)` -> S113, "timeout set to `None`"
- `requests.get(url)` -> S113, "without timeout"
`docs/rules` is generated rather than committed, so there's nothing to
regenerate here
Drafted with AI assistance (Claude Fable 5)
---------
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
## Summary
Add two benchmarks for the cost of narrowing a `match` subject whose
type is a large union. Both check the same three OR patterns against a
union of 45 or 90 classes. Each of those arms returns `v` from a pattern
such as `C0(value=v) | C1(value=v) | C2(value=v)`. Using
`typing.Union[...]` for the subject keeps the separate cost of inferring
a long chain of `|` expressions out of the measurement.
Refs astral-sh/ty#4815
## Test plan
Timed both inputs with ty targeting Python 3.12, then changed only the
match subject to `cast(object, subject)` and timed them again. All four
inputs checked without diagnostics. The table shows median CLI wall time
over five runs.
| Classes in the union | `match subject` | `match cast(object, subject)`
|
| --- | ---: | ---: |
| 45 | 76.7 ms | 32.9 ms |
| 90 | 180.5 ms | 36.7 ms |
## Summary
Checking 4,800 calls to a generic function with four parameters took 108
→ 100 ms of CPU time. Each parameter uses the same set of type
variables, but ty rebuilt that set for every argument. Ty now computes
it once per generic context.
Calls with plain `int` parameters were essentially unchanged. Checking
all of
[PyTorch](https://github.com/pytorch/pytorch/tree/ab92945cea42e47ae398aead69af0872d340ea0f)
showed no consistent speedup.
## Summary
Checking
[`discord/state.py`](https://github.com/Rapptz/discord.py/blob/65232c38702be5844cf2ce865a4777eb1928b5d0/discord/state.py)
opened 100 Python files instead of 112. To decide whether `Interaction`
was abstract, we used to follow the return annotation on
[`Interaction.command`](https://github.com/Rapptz/discord.py/blob/65232c38702be5844cf2ce865a4777eb1928b5d0/discord/interactions.py#L355)
into `discord.ext.commands`.
```python
def cached_slot_property[Owner, Value](
name: str,
) -> Callable[[Callable[[Owner], Value]], CachedSlotProperty[Owner, Value]]: ...
class Interaction:
@cached_slot_property("_cs_command")
def command(self) -> Command[Any, ..., Any] | ContextMenu | None: ...
```
The decorator always returns a `CachedSlotProperty`. The return
annotation on `command` determines the type of the cached value, but it
cannot make the descriptor abstract. We now skip the method’s signature
when its sole decorator rules out an abstract method or property.
Checking all of discord.py showed no measurable speedup.
## Summary
Checking Home Assistant’s [HomeKit light
module](https://github.com/home-assistant/core/blob/a0162d2ff0ed061501405a617961be149b15bd1f/homeassistant/components/homekit_controller/light.py#L53)
opened 38 Python files instead of 45 and used 10% fewer instructions.
When a class declares `value: int = 0`, ty may still need to know
whether a method writes `self.value`. It used to infer `make()` in
`self.value = make()` just to answer that question. Ty now stops once it
finds a reachable assignment. The annotation supplies the type; checking
the method still validates the assigned value.
## Summary
With two files open in Home Assistant, skipping script discovery reduced
the language server’s mean time to first diagnostics from 465 ms to 354
ms on Linux (24%). Previously, ty indexed 10,768 files looking for PEP
723 scripts, even though uv integration was off by default and ty had no
script environments to prepare.
We now scan for scripts to prepare uv environments at startup and on
reload only when uv script environments are enabled (`TY_UV=scripts` or
`TY_UV=1`). With uv off, ty still reads an open script’s inline
settings.
The benchmark client doesn’t register file watchers. With dynamic file
watching enabled, ty still scans for script-specific import paths; five
paired runs showed no startup gain in that configuration.
## Summary
When a class inherits from multiple bases, ty resolves every inherited
attribute on the subclass before it knows whether another base defines
the same name. Binding a descriptor can specialize a method signature or
infer a property's read and write types. None of that is needed if
there’s nothing to compare.
This defers binding until we find a second attribute contract. It also
deduplicates bases that appear both in the MRO and through the explicit
bases. Distinct generic specializations are still checked in order.
## Summary
Home Assistant’s `Entity` class reads or writes 106 different `self`
attributes across 70 methods, but none of those methods annotate one.
When checking overrides, ty could search all 70 methods for each name
just to determine that there was no declaration.
Collect receiver annotations once per class and reuse that list when
checking overrides. When inferring an implicit attribute, skip names
that the class only reads: a read of `self.x` does not define `x`.
On a class with 256 methods that each read eight different attributes,
median CPU time dropped from 896 ms to 126 ms over six runs. ty still
reported the one incompatible annotation. Two full Home Assistant checks
on Linux went from 47.0 to 45.4 s and from 49.4 to 49.2 s, with about 21
MiB less peak memory and the same 13,419 diagnostics.
## Summary
In a reduced example from
[PyTorch](https://github.com/pytorch/pytorch/blob/ab92945cea42e47ae398aead69af0872d340ea0f/torch/_dynamo/variables/misc.py#L2755),
ty now opens 8 Python files instead of 68 to resolve
`tx.output.side_effects`. Across four profiling runs with a single
worker, median CPU time fell from 1.20 s to 0.25 s for that example and
from 1.99 s to 1.54 s for the full `misc.py` file. Diagnostics were
unchanged.
When looking up an instance attribute, ty inferred assignments in method
bodies before checking whether an annotation already determined the
type. Those assignments could pull in modules that the lookup never
needed. We now return the declared type first. If a subclass default
shadows an inherited declaration, augmented assignments such as `self.x
+= 1` in a base class can still affect the attribute’s type, so we
continue to collect them.
Co-authored-by: Charlie Marsh <charliemarsh@openai.com>
Passing overloaded generic functions to `Callable[P, object]` can cause
the constraint solver to enumerate combinations of signatures
specialized to their declared type-variable bounds. The captured
signatures already retain those bounds, so this adds substantial work
without new call-site evidence.
Track the replacement constraint's provenance and skip this
specialization for locally bound variables in captured ParamSpec
signatures when the replacement comes solely from a validity constraint.
Evidence and mixed constraints still specialize local variables, and
free variables retain normal substitution, including expansion of free
ParamSpecs into overloads.
Preserving the generic parameters also lets a callback constrained to
`(str, bytes)` receive either type when forwarded through a `ParamSpec`,
instead of being restricted to its first constraint.
Fixes https://github.com/astral-sh/ty/issues/4796.
## Test plan
Compared against `90bcf19c58b4` using the same Linux/aarch64 environment
and optimized dev profile (Rust 1.99.0, opt-level 1, LTO off):
- The issue's NumPy reproducer, using NumPy 2.5.4 and Python 3.12
semantics, improved from an 11.03 s mean across three runs to 0.101 s
across ten runs (about 110× faster). Every run completed without
diagnostics.
- The six-overload benchmark improved from 225 ms to 22.5 ms across ten
Criterion samples (about 10× faster), with no diagnostics before or
after.
## Ecosystem report
Analyzed head `5c07a3b83f78d45ccf9994f0bfa681ee25840b9a`: one expected
stable `invalid-argument-type` diagnostic rewrite in torchvision, now
describing both constraints of the generic callback. The invalid
argument remains rejected, and no regression was found in the reproduced
case. [Full ecosystem
report](https://github.com/astral-sh/ruff/pull/29649#issuecomment-6351757125).
Micha Reiser