mirror of
https://github.com/astral-sh/ruff.git
synced 2026-10-28 10:45:37 +08:00
[ty] Align TypeIs and isinstance narrowing (#28193)
## Summary This PR aligns `TypeIs` and `isinstance` narrowing for unspecialized generics and gradual protocols in both strict and non-strict narrowing modes: - if `analysis.strict-generic-narrowing = true`, we now intersect with the top materialization of the generic/gradual type, i.e. for a type guard returning `TypeIs[list]`, we now narrow with the (typically unspellable) `Top[list[Unknown]]`. Previously, we did not do this consistently for `isinstance` narrowing either when narrowing to a gradual protocol. - if `analysis.strict-generic-narrowing = false` (default), we now use the same "union filtering" approach that we also use for `isinstance` narrowing (note that the [`TypeIs` PEP](https://peps.python.org/pep-0742/) specifies the narrowing behavior in terms of intersections, but allows "practical approximations of these types"). Using the union filtering approach allows us to avoid overly permissive gradual unions that would result from plain intersections with a type like `list[Unknown]`. We also extend the union filtering approach here to properly handle gradual protocols. In both modes, the negative branch still uses the top materialization to exclude all specializations. The new behavior can be summarized like this: ```py class Invariant[T]: value: T def is_invariant(x: object) -> TypeIs[Invariant]: ... @final class Unrelated: ... ``` <table> <thead> <tr> <th colspan="3"><code>is_invariant(x)</code> / <code>isinstance(x, Invariant)</code></th> </tr> <tr> <th>Before narrowing</th> <th>Non-strict narrowing</th> <th>Strict narrowing</th> </tr> </thead> <tbody> <tr> <td><code>object</code></td> <td><code>Invariant[Unknown]</code></td> <td><code>Top[Invariant[Unknown]]</code></td> </tr> <tr> <td><code>Invariant[int] | Unrelated</code></td> <td><code>Invariant[int]</code></td> <td><code>Invariant[int]</code></td> </tr> </tbody> </table> ```py @runtime_checkable class Reader(Protocol): def read(self) -> Any: ... class SpecificReader: def read(self) -> str: ... def is_reader(arg: object) -> TypeIs[Reader]: ... ``` <table> <thead> <tr> <th colspan="3"><code>is_reader(x)</code> / <code>isinstance(x, Reader)</code></th> </tr> <tr> <th>Before narrowing</th> <th>Non-strict narrowing</th> <th>Strict narrowing</th> </tr> </thead> <tbody> <tr> <td><code>object</code></td> <td><code>Reader</code></td> <td><code>Top[Reader]</code></td> </tr> <tr> <td><code>SpecificReader | Unrelated</code></td> <td><code>SpecificReader</code></td> <td><code>SpecificReader</code></td> </tr> </tbody> </table> closes https://github.com/astral-sh/ty/issues/4394 (Fixed in both non-strict and strict mode; Before: reveals `(list[Unknown] & ~list[Unknown]) | int` and reports `invalid-return-type`, After: reveals `int`; no errors) ## Ecosystem impact Looks great. Lots of removed diagnostics in non-strict-mode projects, and only very few new diagnostics in strict-mode projects, all of which are expected with these changes here. ## Test strategy New and updated Markdown tests
This commit is contained in:
1 parent
cc85e5d82e
commit
efb30c1f4c
29 files changed
+613
-314
No files matched your search
Generated
+2
-2
@@ -269,7 +269,7 @@ def narrow_match(x: str) -> None:
|
||||
### `strict-generic-narrowing`
|
||||
|
||||
Whether ty should use strict narrowing for unspecialized generic classes in
|
||||
`isinstance()` and `issubclass()` checks, as well as `match` class patterns.
|
||||
`isinstance()` and `issubclass()` checks, `match` class patterns, and `TypeIs` checks.
|
||||
|
||||
When enabled, ty narrows to the top materialization of the class. For example,
|
||||
`isinstance(value, list)` narrows a value of type `object` to `Top[list[Unknown]]`,
|
||||
@@ -922,7 +922,7 @@ def narrow_match(x: str) -> None:
|
||||
#### `strict-generic-narrowing`
|
||||
|
||||
Whether ty should use strict narrowing for unspecialized generic classes in
|
||||
`isinstance()` and `issubclass()` checks, as well as `match` class patterns.
|
||||
`isinstance()` and `issubclass()` checks, `match` class patterns, and `TypeIs` checks.
|
||||
|
||||
When enabled, ty narrows to the top materialization of the class. For example,
|
||||
`isinstance(value, list)` narrows a value of type `object` to `Top[list[Unknown]]`,
|
||||
|
||||
Generated
+10
-5
@@ -329,15 +329,20 @@ types with return type `T`).
|
||||
**Why is this bad?**
|
||||
|
||||
|
||||
When an object is narrowed to `Top[Callable[..., object]]` (e.g., via `callable(x)` or
|
||||
`isinstance(x, Callable)`), we know the object is callable, but we don't know its precise signature.
|
||||
This type represents the set of all possible callable types (including, e.g., functions that take no
|
||||
arguments and functions that require arguments), so no specific set of arguments can be guaranteed
|
||||
to be valid.
|
||||
When `analysis.strict-generic-narrowing` is enabled, `callable(x)` and `isinstance(x, Callable)`
|
||||
narrow an object to `Top[Callable[..., object]]`. We know the object is callable, but we don't know
|
||||
its precise signature. This type represents the set of all possible callable types (including, e.g.,
|
||||
functions that take no arguments and functions that require arguments), so no specific set of
|
||||
arguments can be guaranteed to be valid.
|
||||
|
||||
**Examples**
|
||||
|
||||
|
||||
```toml
|
||||
[analysis]
|
||||
strict-generic-narrowing = true
|
||||
```
|
||||
|
||||
```python
|
||||
def f(x: object):
|
||||
if callable(x):
|
||||
|
||||
@@ -232,7 +232,7 @@ mod tests {
|
||||
let supertypes = test.supertypes();
|
||||
insta::assert_snapshot!(
|
||||
snapshot(&test.db, &supertypes),
|
||||
@"vendored://stdlib/builtins.pyi:3638:3644 object :: builtins",
|
||||
@"vendored://stdlib/builtins.pyi:3608:3614 object :: builtins",
|
||||
);
|
||||
}
|
||||
|
||||
@@ -446,12 +446,12 @@ mod tests {
|
||||
let item = test.prepare().unwrap();
|
||||
insta::assert_snapshot!(
|
||||
snapshot(&test.db, &[item]),
|
||||
@"vendored://stdlib/builtins.pyi:8538:8542 type :: builtins",
|
||||
@"vendored://stdlib/builtins.pyi:8508:8512 type :: builtins",
|
||||
);
|
||||
let supertypes = test.supertypes();
|
||||
insta::assert_snapshot!(
|
||||
snapshot(&test.db, &supertypes),
|
||||
@"vendored://stdlib/builtins.pyi:3638:3644 object :: builtins",
|
||||
@"vendored://stdlib/builtins.pyi:3608:3614 object :: builtins",
|
||||
);
|
||||
}
|
||||
|
||||
@@ -503,7 +503,7 @@ mod tests {
|
||||
let supertypes = test.supertypes();
|
||||
insta::assert_snapshot!(
|
||||
snapshot(&test.db, &supertypes),
|
||||
@"vendored://stdlib/builtins.pyi:104699:104704 tuple :: builtins",
|
||||
@"vendored://stdlib/builtins.pyi:104669:104674 tuple :: builtins",
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -1541,7 +1541,7 @@ pub struct TerminalOptions {
|
||||
#[cfg_attr(feature = "schemars", derive(schemars::JsonSchema))]
|
||||
pub struct AnalysisOptions {
|
||||
/// Whether ty should use strict narrowing for unspecialized generic classes in
|
||||
/// `isinstance()` and `issubclass()` checks, as well as `match` class patterns.
|
||||
/// `isinstance()` and `issubclass()` checks, `match` class patterns, and `TypeIs` checks.
|
||||
///
|
||||
/// When enabled, ty narrows to the top materialization of the class. For example,
|
||||
/// `isinstance(value, list)` narrows a value of type `object` to `Top[list[Unknown]]`,
|
||||
|
||||
@@ -5,14 +5,19 @@ types with return type `T`).
|
||||
|
||||
## Why is this bad?
|
||||
|
||||
When an object is narrowed to `Top[Callable[..., object]]` (e.g., via `callable(x)` or
|
||||
`isinstance(x, Callable)`), we know the object is callable, but we don't know its precise signature.
|
||||
This type represents the set of all possible callable types (including, e.g., functions that take no
|
||||
arguments and functions that require arguments), so no specific set of arguments can be guaranteed
|
||||
to be valid.
|
||||
When `analysis.strict-generic-narrowing` is enabled, `callable(x)` and `isinstance(x, Callable)`
|
||||
narrow an object to `Top[Callable[..., object]]`. We know the object is callable, but we don't know
|
||||
its precise signature. This type represents the set of all possible callable types (including, e.g.,
|
||||
functions that take no arguments and functions that require arguments), so no specific set of
|
||||
arguments can be guaranteed to be valid.
|
||||
|
||||
## Examples
|
||||
|
||||
```toml
|
||||
[analysis]
|
||||
strict-generic-narrowing = true
|
||||
```
|
||||
|
||||
```python
|
||||
def f(x: object):
|
||||
if callable(x):
|
||||
|
||||
@@ -150,7 +150,7 @@ def get_any() -> Any:
|
||||
async def test():
|
||||
x = get_any()
|
||||
if inspect.isawaitable(x):
|
||||
reveal_type(x) # revealed: Any & Awaitable[object]
|
||||
reveal_type(x) # revealed: Any & Awaitable[Any]
|
||||
y = await x
|
||||
reveal_type(y) # revealed: Any
|
||||
```
|
||||
|
||||
@@ -278,9 +278,9 @@ info[disjoint-cast]: Cast to a disjoint type
|
||||
| | Inferred as `list[int | str]`
|
||||
| Disjoint from the inferred type
|
||||
|
|
||||
::: stdlib/builtins.pyi:2966:7
|
||||
::: stdlib/builtins.pyi:2965:7
|
||||
|
|
||||
2966 | class list(MutableSequence[_T]):
|
||||
2965 | class list(MutableSequence[_T]):
|
||||
| ---- `list` defined here
|
||||
info: `list[int]` is disjoint from `list[int | str]`
|
||||
info: `int | str` and `int` are not mutual subtypes of each other, but must be due to invariance
|
||||
@@ -358,9 +358,9 @@ info[disjoint-cast]: Cast to a disjoint type
|
||||
| | Inferred as `list[str] | list[bytes]`
|
||||
| Disjoint from the inferred type
|
||||
|
|
||||
::: stdlib/builtins.pyi:2966:7
|
||||
::: stdlib/builtins.pyi:2965:7
|
||||
|
|
||||
2966 | class list(MutableSequence[_T]):
|
||||
2965 | class list(MutableSequence[_T]):
|
||||
| ---- `list` defined here
|
||||
info: `list[int]` is disjoint from `list[str] | list[bytes]`
|
||||
info: every element of union `list[str] | list[bytes]` is disjoint from `list[int]`
|
||||
@@ -391,9 +391,9 @@ info[disjoint-cast]: Cast to a disjoint type
|
||||
| | Inferred as `tuple[int, str]`
|
||||
| Disjoint from the inferred type
|
||||
|
|
||||
::: stdlib/builtins.pyi:2851:7
|
||||
::: stdlib/builtins.pyi:2850:7
|
||||
|
|
||||
2851 | class tuple(Sequence[_T_co]):
|
||||
2850 | class tuple(Sequence[_T_co]):
|
||||
| ----- `tuple` defined here
|
||||
info: `tuple[int, int]` is disjoint from `tuple[int, str]`
|
||||
info: tuple element 2 has disjoint types `str` and `int`
|
||||
@@ -422,9 +422,9 @@ info[disjoint-cast]: Cast to a disjoint type
|
||||
| | Inferred as `tuple[int]`
|
||||
| Disjoint from the inferred type
|
||||
|
|
||||
::: stdlib/builtins.pyi:2851:7
|
||||
::: stdlib/builtins.pyi:2850:7
|
||||
|
|
||||
2851 | class tuple(Sequence[_T_co]):
|
||||
2850 | class tuple(Sequence[_T_co]):
|
||||
| ----- `tuple` defined here
|
||||
info: `tuple[int, int]` is disjoint from `tuple[int]`
|
||||
info: the tuples have incompatible lengths: 1 and 2
|
||||
|
||||
@@ -2125,9 +2125,9 @@ error[invalid-method-override]: Invalid override of method `__eq__`
|
||||
3 | def __eq__(self, other: "Bad") -> bool: # snapshot: invalid-method-override
|
||||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Definition is incompatible with `object.__eq__`
|
||||
|
|
||||
::: stdlib/builtins.pyi:137:9
|
||||
::: stdlib/builtins.pyi:136:9
|
||||
|
|
||||
137 | def __eq__(self, value: object, /) -> bool: ...
|
||||
136 | def __eq__(self, value: object, /) -> bool: ...
|
||||
| -------------------------------------- `object.__eq__` defined here
|
||||
info: parameter `value` has an incompatible type: `object` is not assignable to `Bad`
|
||||
info: This violates the Liskov Substitution Principle
|
||||
|
||||
@@ -2,38 +2,103 @@
|
||||
|
||||
## Basic narrowing
|
||||
|
||||
The `callable()` builtin returns `TypeIs[Top[Callable[..., object]]]`, which narrows the type to the
|
||||
intersection with `Top[Callable[..., object]]`. The `Top[...]` wrapper indicates this is a fully
|
||||
static type representing the top materialization of a gradual callable.
|
||||
### Non-strict mode
|
||||
|
||||
Since all callable types are subtypes of `Top[Callable[..., object]]`, intersections with `Top[...]`
|
||||
simplify to just the original callable type.
|
||||
|
||||
```py
|
||||
from typing import Any, Callable
|
||||
|
||||
def f(x: Callable[..., Any] | None):
|
||||
if callable(x):
|
||||
# The intersection simplifies because `(...) -> Any` is a subtype of
|
||||
# `Top[(...) -> object]` - all callables are subtypes of the top materialization.
|
||||
reveal_type(x) # revealed: (...) -> Any
|
||||
else:
|
||||
# Since `(...) -> Any` is a subtype of `Top[(...) -> object]`, the intersection
|
||||
# with the negation is empty (Never), leaving just None.
|
||||
reveal_type(x) # revealed: None
|
||||
```toml
|
||||
[analysis]
|
||||
strict-generic-narrowing = false
|
||||
```
|
||||
|
||||
## Narrowing with other callable types
|
||||
Narrowing with `callable(x)` or `isinstance(x, Callable)` narrows a union to just the callable
|
||||
types, while preserving their signatures:
|
||||
|
||||
```py
|
||||
from typing import Any, Callable
|
||||
|
||||
def g(x: Callable[[int], str] | None):
|
||||
if callable(x):
|
||||
# All callables are subtypes of `Top[(...) -> object]`, so the intersection simplifies.
|
||||
reveal_type(x) # revealed: (int, /) -> str
|
||||
def _(f: Callable[[int, str], bytes] | None):
|
||||
if callable(f):
|
||||
reveal_type(f) # revealed: (int, str, /) -> bytes
|
||||
else:
|
||||
reveal_type(x) # revealed: None
|
||||
reveal_type(f) # revealed: None
|
||||
|
||||
if isinstance(f, Callable):
|
||||
reveal_type(f) # revealed: (int, str, /) -> bytes
|
||||
else:
|
||||
reveal_type(f) # revealed: None
|
||||
```
|
||||
|
||||
When narrowing from `object`, the result is a gradual callable that can be called with any
|
||||
arguments.
|
||||
|
||||
```py
|
||||
def f(f: object):
|
||||
if callable(f):
|
||||
# Note: typeshed annotates `callable` with a return type of `TypeIs[Callable[..., object]]`, which
|
||||
# is a hybrid between the fully gradual callable `Callable[..., Unknown]` and the top materialization
|
||||
# `Top[Callable[..., Unknown]]` which returns `object` and cannot be called. For consistency with
|
||||
# `isinstance` narrowing below, `(...) -> Unknown` would be better here, but we currently follow
|
||||
# the typeshed annotation.
|
||||
reveal_type(f) # revealed: (...) -> object
|
||||
f(1, keyword="value")
|
||||
else:
|
||||
reveal_type(f) # revealed: ~Top[(...) -> object]
|
||||
|
||||
if isinstance(f, Callable):
|
||||
reveal_type(f) # revealed: (...) -> Unknown
|
||||
f(1, keyword="value")
|
||||
else:
|
||||
reveal_type(f) # revealed: ~Top[(...) -> object]
|
||||
```
|
||||
|
||||
### Strict mode
|
||||
|
||||
```toml
|
||||
[analysis]
|
||||
strict-generic-narrowing = true
|
||||
```
|
||||
|
||||
Narrowing with `callable(x)` or `isinstance(x, Callable)` narrows a union to just the callable
|
||||
types, while preserving their signatures. Exactly the same as in non-strict mode.
|
||||
|
||||
```py
|
||||
from typing import Any, Callable
|
||||
|
||||
def _(f: Callable[[int, str], bytes] | None):
|
||||
if callable(f):
|
||||
reveal_type(f) # revealed: (int, str, /) -> bytes
|
||||
else:
|
||||
reveal_type(f) # revealed: None
|
||||
|
||||
if isinstance(f, Callable):
|
||||
reveal_type(f) # revealed: (int, str, /) -> bytes
|
||||
else:
|
||||
reveal_type(f) # revealed: None
|
||||
```
|
||||
|
||||
However, when narrowing from `object`, the result is the top-materialized callable type
|
||||
`Top[(...) -> object]`. This type represents the set of all possible callable types (including,
|
||||
e.g., functions that take no arguments and functions that require arguments). While such objects
|
||||
*are* callable (they pass `callable()`), no specific set of arguments can be guaranteed to be valid.
|
||||
|
||||
```py
|
||||
def f(f: object):
|
||||
if callable(f):
|
||||
reveal_type(f) # revealed: Top[(...) -> object]
|
||||
f(1, keyword="value") # error: [call-top-callable]
|
||||
else:
|
||||
reveal_type(f) # revealed: ~Top[(...) -> object]
|
||||
|
||||
if isinstance(f, Callable):
|
||||
reveal_type(f) # revealed: Top[(...) -> object]
|
||||
f(1, keyword="value") # error: [call-top-callable]
|
||||
else:
|
||||
reveal_type(f) # revealed: ~Top[(...) -> object]
|
||||
```
|
||||
|
||||
## Narrowing from gradual callable types
|
||||
|
||||
```py
|
||||
from typing import Any, Callable
|
||||
|
||||
def h(x: Callable[..., int] | None):
|
||||
if callable(x):
|
||||
@@ -42,45 +107,19 @@ def h(x: Callable[..., int] | None):
|
||||
reveal_type(x) # revealed: None
|
||||
```
|
||||
|
||||
## Narrowing from object
|
||||
|
||||
```py
|
||||
def f(x: object):
|
||||
if callable(x):
|
||||
reveal_type(x) # revealed: Top[(...) -> object]
|
||||
else:
|
||||
reveal_type(x) # revealed: ~Top[(...) -> object]
|
||||
```
|
||||
|
||||
## Calling narrowed callables
|
||||
|
||||
### Strict generic narrowing mode
|
||||
## Intersections with the top-callable
|
||||
|
||||
```toml
|
||||
[analysis]
|
||||
strict-generic-narrowing = true
|
||||
```
|
||||
|
||||
In strict generic narrowing mode, an `isinstance(.., Callable)` check intersects the type with
|
||||
`Top[Callable[..., object]]`. This type represents the set of all possible callable types
|
||||
(including, e.g., functions that take no arguments and functions that require arguments). While such
|
||||
objects *are* callable (they pass `callable()`), no specific set of arguments can be guaranteed to
|
||||
be valid.
|
||||
|
||||
```py
|
||||
import typing as t
|
||||
|
||||
def call_with_args(y: object, a: int, b: str) -> object:
|
||||
if isinstance(y, t.Callable):
|
||||
# error: [call-top-callable]
|
||||
return y(a, b)
|
||||
return None
|
||||
```
|
||||
|
||||
If a top-callable is part of an intersection, it should still contribute its return type even when
|
||||
the other intersection elements are not callable:
|
||||
|
||||
```py
|
||||
from typing import Callable
|
||||
|
||||
def resolve(value: str):
|
||||
if callable(value):
|
||||
reveal_type(value) # revealed: str & Top[(...) -> object]
|
||||
@@ -88,36 +127,6 @@ def resolve(value: str):
|
||||
reveal_type(value()) # revealed: object
|
||||
```
|
||||
|
||||
### Gradual generic narrowing mode
|
||||
|
||||
```toml
|
||||
[analysis]
|
||||
strict-generic-narrowing = false
|
||||
```
|
||||
|
||||
In gradual generic narrowing mode, an `isinstance(.., Callable)` check narrows to a gradual
|
||||
callable. Its parameters accept arbitrary arguments, and its return type is `Unknown`:
|
||||
|
||||
```py
|
||||
from typing import Callable
|
||||
|
||||
def call_with_args(y: object):
|
||||
if isinstance(y, Callable):
|
||||
reveal_type(y) # revealed: (...) -> Unknown
|
||||
|
||||
reveal_type(y()) # revealed: Unknown
|
||||
reveal_type(y(1, "foo")) # revealed: Unknown
|
||||
reveal_type(y(1, "foo", keyword_arg="bar")) # revealed: Unknown
|
||||
```
|
||||
|
||||
An already-specialized callable retains its known parameter and return types:
|
||||
|
||||
```py
|
||||
def preserve_callable_signature(fn: Callable[[int], str]) -> None:
|
||||
if isinstance(fn, Callable):
|
||||
reveal_type(fn) # revealed: (int, /) -> str
|
||||
```
|
||||
|
||||
## Narrowing with named expressions (walrus operator)
|
||||
|
||||
When `callable()` is used with a named expression, the target of the named expression should be
|
||||
@@ -132,18 +141,23 @@ class Foo:
|
||||
def f(foo: Foo):
|
||||
first = getattr(foo, "func", None)
|
||||
if callable(first):
|
||||
reveal_type(first) # revealed: Any & Top[(...) -> object]
|
||||
reveal_type(first) # revealed: Any & ((...) -> object)
|
||||
else:
|
||||
reveal_type(first) # revealed: (Any & ~Top[(...) -> object]) | None
|
||||
|
||||
if callable(second := getattr(foo, "func", None)):
|
||||
reveal_type(second) # revealed: Any & Top[(...) -> object]
|
||||
reveal_type(second) # revealed: Any & ((...) -> object)
|
||||
else:
|
||||
reveal_type(second) # revealed: (Any & ~Top[(...) -> object]) | None
|
||||
```
|
||||
|
||||
## Assignability of narrowed callables
|
||||
|
||||
```toml
|
||||
[analysis]
|
||||
strict-generic-narrowing = true
|
||||
```
|
||||
|
||||
A narrowed callable `Top[Callable[..., object]]` should be assignable to `Callable[..., Any]`. This
|
||||
is important for decorators and other patterns where we need to pass the narrowed callable to
|
||||
functions expecting gradual callables.
|
||||
|
||||
@@ -1103,7 +1103,7 @@ class IntReader:
|
||||
|
||||
def f(reader: IntReader | None):
|
||||
if isinstance(reader, Reader):
|
||||
reveal_type(reader.read()) # revealed: int & Any
|
||||
reveal_type(reader.read()) # revealed: int
|
||||
else:
|
||||
reveal_type(reader) # revealed: None
|
||||
```
|
||||
|
||||
@@ -2638,7 +2638,7 @@ class IntReader:
|
||||
def f(reader: IntReader | None):
|
||||
match reader:
|
||||
case Reader():
|
||||
reveal_type(reader.read()) # revealed: int & Any
|
||||
reveal_type(reader.read()) # revealed: int
|
||||
case _:
|
||||
reveal_type(reader) # revealed: None
|
||||
```
|
||||
|
||||
@@ -474,24 +474,76 @@ def _(x: Foo | Bar, is_bar: Callable[[object], TypeIs[Bar]]):
|
||||
reveal_type(x) # revealed: Foo & ~Bar
|
||||
```
|
||||
|
||||
A `TypeIs` function that returns a gradual specialization of a generic class narrows to that generic
|
||||
type without replacing its gradual type argument:
|
||||
A `TypeIs` function can also be used to narrow to a specific callable type:
|
||||
|
||||
```py
|
||||
class Covariant[T]:
|
||||
def is_callable_that_returns_int(arg: object) -> TypeIs[Callable[[object], int]]:
|
||||
return callable(arg)
|
||||
|
||||
def is_callable_that_takes_str(arg: object) -> TypeIs[Callable[[str], object]]:
|
||||
return callable(arg)
|
||||
|
||||
def _(arg: Callable[[object], str] | Callable[[object], int]):
|
||||
if is_callable_that_returns_int(arg):
|
||||
reveal_type(arg) # revealed: (object, /) -> int
|
||||
|
||||
def _(arg: Callable[[str], object] | Callable[[int], object]):
|
||||
if is_callable_that_takes_str(arg):
|
||||
reveal_type(arg) # revealed: (str, /) -> object
|
||||
```
|
||||
|
||||
## `TypeIs` narrowing with generic classes and gradual types
|
||||
|
||||
### Non-strict mode
|
||||
|
||||
```toml
|
||||
[environment]
|
||||
python-version = "3.12"
|
||||
|
||||
[analysis]
|
||||
strict-generic-narrowing = false
|
||||
|
||||
[rules]
|
||||
# This rule is ignored in ty's default settings. Disable it here as well to
|
||||
# allow usage of `Getter` without a type argument below.
|
||||
missing-type-argument = "ignore"
|
||||
```
|
||||
|
||||
In non-strict narrowing mode, a `TypeIs` function that returns a gradual specialization of a generic
|
||||
class narrows from `object` to that gradual specialization:
|
||||
|
||||
```py
|
||||
from typing import Any
|
||||
from typing_extensions import TypeIs
|
||||
|
||||
class Getter[T]:
|
||||
def get(self) -> T:
|
||||
raise NotImplementedError
|
||||
|
||||
def is_instance_of_covariant(arg: object) -> TypeIs[Covariant[Any]]:
|
||||
return isinstance(arg, Covariant)
|
||||
def is_getter(arg: object) -> TypeIs[Getter[Any]]:
|
||||
return isinstance(arg, Getter)
|
||||
|
||||
def _(x: object):
|
||||
if is_instance_of_covariant(x):
|
||||
reveal_type(x) # revealed: Covariant[Any]
|
||||
def is_getter_unspecialized(arg: object) -> TypeIs[Getter]:
|
||||
return isinstance(arg, Getter)
|
||||
|
||||
def _(obj: object):
|
||||
if is_getter(obj):
|
||||
reveal_type(obj) # revealed: Getter[Any]
|
||||
reveal_type(obj.get()) # revealed: Any
|
||||
|
||||
if is_getter_unspecialized(obj):
|
||||
reveal_type(obj) # revealed: Getter[Unknown]
|
||||
reveal_type(obj.get()) # revealed: Unknown
|
||||
|
||||
# For comparison, `isinstance` narrowing behaves in the same way:
|
||||
if isinstance(obj, Getter):
|
||||
reveal_type(obj) # revealed: Getter[Unknown]
|
||||
reveal_type(obj.get()) # revealed: Unknown
|
||||
```
|
||||
|
||||
However, intersecting with the declared gradual type does not necessarily exclude every other
|
||||
specialization in the negative branch:
|
||||
When narrowing from a union of a specific `Getter` specialization and another type, the positive
|
||||
branch retains the specific `Getter` specialization and the negative branch excludes `Getter`
|
||||
entirely:
|
||||
|
||||
```py
|
||||
from typing import final
|
||||
@@ -499,38 +551,331 @@ from typing import final
|
||||
@final
|
||||
class Unrelated: ...
|
||||
|
||||
def needs_instance_of_unrelated(arg: Unrelated):
|
||||
pass
|
||||
def _(obj: Unrelated | Getter[int]):
|
||||
if is_getter(obj):
|
||||
reveal_type(obj) # revealed: Getter[int]
|
||||
reveal_type(obj.get()) # revealed: int
|
||||
else:
|
||||
reveal_type(obj) # revealed: Unrelated
|
||||
|
||||
def _(x: Unrelated | Covariant[int]):
|
||||
if is_instance_of_covariant(x):
|
||||
raise RuntimeError("oh no")
|
||||
|
||||
reveal_type(x) # revealed: Unrelated | (Covariant[int] & ~Covariant[Any])
|
||||
|
||||
needs_instance_of_unrelated(x) # error: [invalid-argument-type]
|
||||
# For comparison, `isinstance` narrowing behaves in the same way:
|
||||
if isinstance(obj, Getter):
|
||||
reveal_type(obj) # revealed: Getter[int]
|
||||
reveal_type(obj.get()) # revealed: int
|
||||
else:
|
||||
reveal_type(obj) # revealed: Unrelated
|
||||
```
|
||||
|
||||
If a user wants to select *all* instances of `Covariant`, they must use `Covariant[object]`, or more
|
||||
generally, `Top[C[Any]]`, which also works for invariant generic types:
|
||||
If the other type is not final, the positive branch retains the possibility of multiple inheritance
|
||||
between `Overlapping` and `Getter`:
|
||||
|
||||
```py
|
||||
from typing import TYPE_CHECKING
|
||||
class Overlapping: ...
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ty_extensions import Top
|
||||
|
||||
class Invariant[T]:
|
||||
value: T # make it invariant in `T`
|
||||
|
||||
def is_instance_of_invariant(arg: object) -> "TypeIs[Top[Invariant[Any]]]":
|
||||
return isinstance(arg, Invariant)
|
||||
|
||||
def _(x: Unrelated | Invariant[int]):
|
||||
if is_instance_of_invariant(x):
|
||||
reveal_type(x) # revealed: Invariant[int]
|
||||
def _(obj: Overlapping | Getter[int]):
|
||||
if is_getter(obj):
|
||||
reveal_type(obj) # revealed: (Overlapping & Getter[Any]) | Getter[int]
|
||||
reveal_type(obj.get()) # revealed: Any | int
|
||||
else:
|
||||
reveal_type(x) # revealed: Unrelated
|
||||
reveal_type(obj) # revealed: Overlapping & ~Getter[object]
|
||||
|
||||
# For comparison, `isinstance` narrowing behaves in the same way (except for Any -> Unknown)
|
||||
if isinstance(obj, Getter):
|
||||
reveal_type(obj) # revealed: (Overlapping & Getter[Unknown]) | Getter[int]
|
||||
reveal_type(obj.get()) # revealed: Unknown | int
|
||||
else:
|
||||
reveal_type(obj) # revealed: Overlapping & ~Getter[object]
|
||||
```
|
||||
|
||||
The same also works for invariant generics:
|
||||
|
||||
```py
|
||||
def is_list(value: object) -> TypeIs[list]:
|
||||
return isinstance(value, list)
|
||||
|
||||
def _(obj: object):
|
||||
if is_list(obj):
|
||||
reveal_type(obj) # revealed: list[Unknown]
|
||||
else:
|
||||
reveal_type(obj) # revealed: ~Top[list[Unknown]]
|
||||
|
||||
# For comparison, `isinstance` narrowing behaves in the same way:
|
||||
if isinstance(obj, list):
|
||||
reveal_type(obj) # revealed: list[Unknown]
|
||||
else:
|
||||
reveal_type(obj) # revealed: ~Top[list[Unknown]]
|
||||
|
||||
def _(obj: Unrelated | list[int]):
|
||||
if is_list(obj):
|
||||
reveal_type(obj) # revealed: list[int]
|
||||
else:
|
||||
reveal_type(obj) # revealed: Unrelated
|
||||
|
||||
# For comparison, `isinstance` narrowing behaves in the same way:
|
||||
if isinstance(obj, list):
|
||||
reveal_type(obj) # revealed: list[int]
|
||||
else:
|
||||
reveal_type(obj) # revealed: Unrelated
|
||||
|
||||
def _(obj: Overlapping | list[int]):
|
||||
if is_list(obj):
|
||||
reveal_type(obj) # revealed: (Overlapping & list[Unknown]) | list[int]
|
||||
else:
|
||||
reveal_type(obj) # revealed: Overlapping & ~Top[list[Unknown]]
|
||||
|
||||
# For comparison, `isinstance` narrowing behaves in the same way:
|
||||
if isinstance(obj, list):
|
||||
reveal_type(obj) # revealed: (Overlapping & list[Unknown]) | list[int]
|
||||
else:
|
||||
reveal_type(obj) # revealed: Overlapping & ~Top[list[Unknown]]
|
||||
|
||||
# Regression test from https://github.com/astral-sh/ty/issues/4394
|
||||
def _(value: int | list) -> int:
|
||||
if is_list(value):
|
||||
value = 0
|
||||
return value
|
||||
```
|
||||
|
||||
Similarly, for gradual protocols, `TypeIs` narrowing retains the gradualness of protocol members in
|
||||
non-strict mode, when narrowing from object:
|
||||
|
||||
```py
|
||||
from typing import Protocol, runtime_checkable
|
||||
|
||||
@runtime_checkable
|
||||
class Reader(Protocol):
|
||||
def read(self) -> Any: ...
|
||||
|
||||
def is_reader(arg: object) -> TypeIs[Reader]:
|
||||
return isinstance(arg, Reader)
|
||||
|
||||
def _(obj: object):
|
||||
if is_reader(obj):
|
||||
reveal_type(obj) # revealed: Reader
|
||||
reveal_type(obj.read()) # revealed: Any
|
||||
else:
|
||||
reveal_type(obj) # revealed: ~Top[Reader]
|
||||
|
||||
# For comparison, `isinstance` narrowing behaves in the same way:
|
||||
if isinstance(obj, Reader):
|
||||
reveal_type(obj) # revealed: Reader
|
||||
reveal_type(obj.read()) # revealed: Any
|
||||
else:
|
||||
reveal_type(obj) # revealed: ~Top[Reader]
|
||||
```
|
||||
|
||||
When narrowing from a union of a specific `Reader` implementation and another type, the positive
|
||||
branch retains the specific `Reader` implementation, and the negative branch excludes it entirely:
|
||||
|
||||
```py
|
||||
class SpecificReader:
|
||||
def read(self) -> str:
|
||||
raise NotImplementedError
|
||||
|
||||
def _(obj: SpecificReader | Unrelated):
|
||||
if is_reader(obj):
|
||||
reveal_type(obj) # revealed: SpecificReader
|
||||
reveal_type(obj.read()) # revealed: str
|
||||
else:
|
||||
reveal_type(obj) # revealed: Unrelated
|
||||
|
||||
# For comparison, `isinstance` narrowing behaves in the same way:
|
||||
if isinstance(obj, Reader):
|
||||
reveal_type(obj) # revealed: SpecificReader
|
||||
reveal_type(obj.read()) # revealed: str
|
||||
else:
|
||||
reveal_type(obj) # revealed: Unrelated
|
||||
```
|
||||
|
||||
### Strict mode
|
||||
|
||||
```toml
|
||||
[environment]
|
||||
python-version = "3.12"
|
||||
|
||||
[analysis]
|
||||
strict-generic-narrowing = true
|
||||
|
||||
[rules]
|
||||
# see above
|
||||
missing-type-argument = "ignore"
|
||||
```
|
||||
|
||||
In strict narrowing mode, we narrow to the top-materialization of the generic class:
|
||||
|
||||
```py
|
||||
from typing import Any
|
||||
from typing_extensions import TypeIs
|
||||
|
||||
class Getter[T]:
|
||||
def get(self) -> T:
|
||||
raise NotImplementedError
|
||||
|
||||
def is_getter(arg: object) -> TypeIs[Getter[Any]]:
|
||||
return isinstance(arg, Getter)
|
||||
|
||||
def is_getter_unspecialized(arg: object) -> TypeIs[Getter]:
|
||||
return isinstance(arg, Getter)
|
||||
|
||||
def _(obj: object):
|
||||
if is_getter(obj):
|
||||
reveal_type(obj) # revealed: Getter[object]
|
||||
reveal_type(obj.get()) # revealed: object
|
||||
|
||||
if is_getter_unspecialized(obj):
|
||||
reveal_type(obj) # revealed: Getter[object]
|
||||
reveal_type(obj.get()) # revealed: object
|
||||
|
||||
# For comparison, `isinstance` narrowing behaves in the same way:
|
||||
if isinstance(obj, Getter):
|
||||
reveal_type(obj) # revealed: Getter[object]
|
||||
reveal_type(obj.get()) # revealed: object
|
||||
```
|
||||
|
||||
Like in non-strict mode, when narrowing from a union of a specific `Getter` specialization and
|
||||
another type, the positive branch retains the specific `Getter` specialization and the negative
|
||||
branch excludes `Getter` entirely:
|
||||
|
||||
```py
|
||||
from typing import final
|
||||
|
||||
@final
|
||||
class Unrelated: ...
|
||||
|
||||
def _(obj: Unrelated | Getter[int]):
|
||||
if is_getter(obj):
|
||||
reveal_type(obj) # revealed: Getter[int]
|
||||
reveal_type(obj.get()) # revealed: int
|
||||
else:
|
||||
reveal_type(obj) # revealed: Unrelated
|
||||
|
||||
# For comparison, `isinstance` narrowing behaves in the same way:
|
||||
if isinstance(obj, Getter):
|
||||
reveal_type(obj) # revealed: Getter[int]
|
||||
reveal_type(obj.get()) # revealed: int
|
||||
else:
|
||||
reveal_type(obj) # revealed: Unrelated
|
||||
```
|
||||
|
||||
Like in non-strict mode, if the other type is not final, the positive branch retains the possibility
|
||||
of multiple inheritance between `Overlapping` and `Getter`:
|
||||
|
||||
```py
|
||||
class Overlapping: ...
|
||||
|
||||
def _(obj: Overlapping | Getter[int]):
|
||||
if is_getter(obj):
|
||||
reveal_type(obj) # revealed: (Overlapping & Getter[object]) | Getter[int]
|
||||
reveal_type(obj.get()) # revealed: object
|
||||
else:
|
||||
reveal_type(obj) # revealed: Overlapping & ~Getter[object]
|
||||
|
||||
# For comparison, `isinstance` narrowing behaves in the same way:
|
||||
if isinstance(obj, Getter):
|
||||
reveal_type(obj) # revealed: (Overlapping & Getter[object]) | Getter[int]
|
||||
reveal_type(obj.get()) # revealed: object
|
||||
else:
|
||||
reveal_type(obj) # revealed: Overlapping & ~Getter[object]
|
||||
```
|
||||
|
||||
The same also works for invariant generics:
|
||||
|
||||
```py
|
||||
def is_list(value: object) -> TypeIs[list]:
|
||||
return isinstance(value, list)
|
||||
|
||||
def _(obj: object):
|
||||
if is_list(obj):
|
||||
reveal_type(obj) # revealed: Top[list[Unknown]]
|
||||
else:
|
||||
reveal_type(obj) # revealed: ~Top[list[Unknown]]
|
||||
|
||||
# For comparison, `isinstance` narrowing behaves in the same way:
|
||||
if isinstance(obj, list):
|
||||
reveal_type(obj) # revealed: Top[list[Unknown]]
|
||||
else:
|
||||
reveal_type(obj) # revealed: ~Top[list[Unknown]]
|
||||
|
||||
def _(obj: Unrelated | list[int]):
|
||||
if is_list(obj):
|
||||
reveal_type(obj) # revealed: list[int]
|
||||
else:
|
||||
reveal_type(obj) # revealed: Unrelated
|
||||
|
||||
# For comparison, `isinstance` narrowing behaves in the same way:
|
||||
if isinstance(obj, list):
|
||||
reveal_type(obj) # revealed: list[int]
|
||||
else:
|
||||
reveal_type(obj) # revealed: Unrelated
|
||||
|
||||
def _(obj: Overlapping | list[int]):
|
||||
if is_list(obj):
|
||||
reveal_type(obj) # revealed: (Overlapping & Top[list[Unknown]]) | list[int]
|
||||
else:
|
||||
reveal_type(obj) # revealed: Overlapping & ~Top[list[Unknown]]
|
||||
|
||||
# For comparison, `isinstance` narrowing behaves in the same way:
|
||||
if isinstance(obj, list):
|
||||
reveal_type(obj) # revealed: (Overlapping & Top[list[Unknown]]) | list[int]
|
||||
else:
|
||||
reveal_type(obj) # revealed: Overlapping & ~Top[list[Unknown]]
|
||||
|
||||
# Regression test from https://github.com/astral-sh/ty/issues/4394
|
||||
def _(value: int | list) -> int:
|
||||
if is_list(value):
|
||||
value = 0
|
||||
return value
|
||||
```
|
||||
|
||||
Similarly, for gradual protocols, `TypeIs` narrowing in strict mode also narrows to the
|
||||
top-materialization of the protocol:
|
||||
|
||||
```py
|
||||
from typing import Protocol, runtime_checkable
|
||||
|
||||
@runtime_checkable
|
||||
class Reader(Protocol):
|
||||
def read(self) -> Any: ...
|
||||
|
||||
def is_reader(arg: object) -> TypeIs[Reader]:
|
||||
return isinstance(arg, Reader)
|
||||
|
||||
def _(obj: object):
|
||||
if is_reader(obj):
|
||||
reveal_type(obj) # revealed: Top[Reader]
|
||||
reveal_type(obj.read()) # revealed: object
|
||||
else:
|
||||
reveal_type(obj) # revealed: ~Top[Reader]
|
||||
|
||||
# For comparison, `isinstance` narrowing behaves in the same way:
|
||||
if isinstance(obj, Reader):
|
||||
reveal_type(obj) # revealed: Top[Reader]
|
||||
reveal_type(obj.read()) # revealed: object
|
||||
else:
|
||||
reveal_type(obj) # revealed: ~Top[Reader]
|
||||
```
|
||||
|
||||
When narrowing from a union of a specific `Reader` implementation and another type, the positive
|
||||
branch retains the specific `Reader` implementation, and the negative branch excludes it entirely:
|
||||
|
||||
```py
|
||||
class SpecificReader:
|
||||
def read(self) -> str:
|
||||
raise NotImplementedError
|
||||
|
||||
def _(obj: SpecificReader | Unrelated):
|
||||
if is_reader(obj):
|
||||
reveal_type(obj) # revealed: SpecificReader
|
||||
reveal_type(obj.read()) # revealed: str
|
||||
else:
|
||||
reveal_type(obj) # revealed: Unrelated
|
||||
|
||||
# For comparison, `isinstance` narrowing behaves in the same way:
|
||||
if isinstance(obj, Reader):
|
||||
reveal_type(obj) # revealed: SpecificReader
|
||||
reveal_type(obj.read()) # revealed: str
|
||||
else:
|
||||
reveal_type(obj) # revealed: Unrelated
|
||||
```
|
||||
|
||||
## `TypeIs` narrowing of `NewType` instances
|
||||
|
||||
+2
-2
@@ -26,9 +26,9 @@ error[invalid-await]: `Literal[1]` is not awaitable
|
||||
2 | await 1 # error: [invalid-await]
|
||||
| ^
|
||||
|
|
||||
::: stdlib/builtins.pyi:350:7
|
||||
::: stdlib/builtins.pyi:349:7
|
||||
|
|
||||
350 | class int:
|
||||
349 | class int:
|
||||
| --- type defined here
|
||||
info: `__await__` is missing
|
||||
|
||||
|
||||
+2
-2
@@ -41,9 +41,9 @@ error[invalid-method-override]: Invalid override of method `__eq__`
|
||||
6 | def __eq__(self, other) -> NotBoolable:
|
||||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Definition is incompatible with `object.__eq__`
|
||||
|
|
||||
::: stdlib/builtins.pyi:137:9
|
||||
::: stdlib/builtins.pyi:136:9
|
||||
|
|
||||
137 | def __eq__(self, value: object, /) -> bool: ...
|
||||
136 | def __eq__(self, value: object, /) -> bool: ...
|
||||
| -------------------------------------- `object.__eq__` defined here
|
||||
info: incompatible return types: `NotBoolable` is not assignable to `bool`
|
||||
info: This violates the Liskov Substitution Principle
|
||||
|
||||
+4
-4
@@ -46,9 +46,9 @@ error[invalid-typed-dict-header]: TypedDict class `Foo` can only inherit from Ty
|
||||
3 | class Foo(TypedDict, int): ... # error: [invalid-typed-dict-header]
|
||||
| ^^^ `int` is not a `TypedDict` class
|
||||
|
|
||||
::: stdlib/builtins.pyi:350:7
|
||||
::: stdlib/builtins.pyi:349:7
|
||||
|
|
||||
350 | class int:
|
||||
349 | class int:
|
||||
| --- `int` defined here
|
||||
|
||||
```
|
||||
@@ -60,9 +60,9 @@ error[invalid-typed-dict-header]: TypedDict class `Foo2` can only inherit from T
|
||||
6 | class Foo2(TypedDict, object): ... # error: [invalid-typed-dict-header]
|
||||
| ^^^^^^ `object` is not a `TypedDict` class
|
||||
|
|
||||
::: stdlib/builtins.pyi:114:7
|
||||
::: stdlib/builtins.pyi:113:7
|
||||
|
|
||||
114 | class object:
|
||||
113 | class object:
|
||||
| ------ `object` defined here
|
||||
|
||||
```
|
||||
|
||||
+2
-2
@@ -41,9 +41,9 @@ info: ├── type `Literal[" "]` is not assignable to protocol `Buffer`
|
||||
info: │ └── protocol member `__buffer__` is not defined on type `Literal[" "]`
|
||||
info: └── ... omitted 1 union element without additional context
|
||||
info: Method defined here
|
||||
--> stdlib/builtins.pyi:1844:9
|
||||
--> stdlib/builtins.pyi:1843:9
|
||||
|
|
||||
1844 | def split(self, sep: ReadableBuffer | None = None, maxsplit: SupportsIndex = -1) -> list[bytes]:
|
||||
1843 | def split(self, sep: ReadableBuffer | None = None, maxsplit: SupportsIndex = -1) -> list[bytes]:
|
||||
| ^^^^^ --------------------------------- Parameter declared here
|
||||
info: Union variant `bound method bytes.split(sep: Buffer | None = None, maxsplit: SupportsIndex = -1) -> list[bytes]` is incompatible with this call site
|
||||
info: Attempted to call union type `(bound method bytes.split(sep: Buffer | None = None, maxsplit: SupportsIndex = -1) -> list[bytes]) | (bound method str.split(sep: str | None = None, maxsplit: SupportsIndex = -1) -> list[str])`
|
||||
|
||||
@@ -60,6 +60,9 @@ def f(x: Top[Callable[..., str]] | Callable[[int], int]):
|
||||
```toml
|
||||
[environment]
|
||||
python-version = "3.12"
|
||||
|
||||
[analysis]
|
||||
strict-generic-narrowing = true
|
||||
```
|
||||
|
||||
We wrap the signature of a top ParamSpec with `Top[...]`:
|
||||
|
||||
@@ -3255,7 +3255,10 @@ def get_value(value: GetValue[ValueT]) -> ValueT:
|
||||
|
||||
def _(value: StringValue | dict[str, Any]) -> None:
|
||||
if isinstance(value, GetAnyValue):
|
||||
reveal_type(get_value(value)) # revealed: Any
|
||||
reveal_type(value) # revealed: StringValue | dict[str, Any]
|
||||
# TODO: this returns `object` due to a synthesized `__getitem__` fallback overload on the `StringValue` TypedDict.
|
||||
# This is a generic solver limitation. Inferring `str | Any` would be more accurate.
|
||||
reveal_type(get_value(value)) # revealed: object
|
||||
```
|
||||
|
||||
The same `Any` result must remain valid when the mapping protocol uses a bounded type variable:
|
||||
@@ -3276,7 +3279,9 @@ def get_bounded_mapping(value: SupportsKeysAndGetItem[str, BoundedValueT]) -> Bo
|
||||
|
||||
def _(value: StringValue | dict[str, Any]) -> None:
|
||||
if isinstance(value, AnyValueMapping):
|
||||
reveal_type(get_bounded_mapping(value)) # revealed: Any
|
||||
# TODO: this should not emit an `invalid-argument-type` error (see TODO above)
|
||||
# error: [invalid-argument-type]
|
||||
reveal_type(get_bounded_mapping(value)) # revealed: Unknown
|
||||
```
|
||||
|
||||
A `TypedDict` that permits extra items of type `Any` keeps that type when copied:
|
||||
|
||||
@@ -496,11 +496,11 @@ impl ClassInfoConstraintFunction {
|
||||
}
|
||||
};
|
||||
|
||||
if is_positive {
|
||||
if use_generic_filtering {
|
||||
constraint
|
||||
} else {
|
||||
// A negative result excludes every specialization of the class. Materialize the
|
||||
// whole type so that this also covers gradual protocol members.
|
||||
// Strict positive narrowing and negative narrowing both cover every
|
||||
// materialization. Materialize the whole type, including gradual protocol members.
|
||||
constraint.top_materialization(db, env)
|
||||
}
|
||||
};
|
||||
@@ -537,7 +537,12 @@ impl ClassInfoConstraintFunction {
|
||||
// protocol-conforming objects even if its nominal instances do not conform.
|
||||
SubclassOfInner::Protocol(protocol) => match self {
|
||||
ClassInfoConstraintFunction::IsInstance => {
|
||||
Some(Type::ProtocolInstance(protocol))
|
||||
let constraint = Type::ProtocolInstance(protocol);
|
||||
Some(if use_generic_filtering {
|
||||
constraint
|
||||
} else {
|
||||
constraint.top_materialization(db, env)
|
||||
})
|
||||
}
|
||||
ClassInfoConstraintFunction::IsSubclass => Some(classinfo),
|
||||
},
|
||||
@@ -813,32 +818,55 @@ fn filter_generic_narrowing_constraint<'db>(
|
||||
(subject, Type::Union(union)) => union.map(db, env, |element| {
|
||||
filter_generic_narrowing_constraint(db, env, subject, *element)
|
||||
}),
|
||||
(subject @ Type::Callable(_), Type::Callable(_)) => subject,
|
||||
// Dynamic types are assignable to every protocol. Including them here would create `Any`
|
||||
// instead of `Any & Awaitable[Any]` when narrowing from `Any` via `isawaitable`. The latter
|
||||
// is more useful in LSP use cases, so exclude dynamic types here.
|
||||
(subject, target @ Type::ProtocolInstance(_))
|
||||
if !subject.is_dynamic() && subject.is_assignable_to(db, env, target) =>
|
||||
{
|
||||
subject
|
||||
}
|
||||
(subject, target @ Type::Callable(_))
|
||||
if subject.is_subtype_of(db, env, target.top_materialization(db, env)) =>
|
||||
{
|
||||
subject
|
||||
}
|
||||
(subject, target)
|
||||
if is_typed_dict_runtime_domain(db, env, subject)
|
||||
&& target.nominal_class(db, env).is_some_and(|class| {
|
||||
!class.is_protocol(db)
|
||||
&& typed_dict_matches_class_pattern(db, env, class.class_literal(db))
|
||||
&& target.is_equivalent_to(
|
||||
db,
|
||||
env,
|
||||
Type::instance(
|
||||
db,
|
||||
env,
|
||||
class.class_literal(db).unknown_specialization(db),
|
||||
),
|
||||
)
|
||||
}) =>
|
||||
{
|
||||
// A TypedDict is a dictionary at runtime, but intersecting it with the target would
|
||||
// expose dict's unrestricted mutations and discard its required-key guarantees.
|
||||
subject
|
||||
}
|
||||
(Type::Intersection(intersection), target) => {
|
||||
let specialized_target =
|
||||
specialize_narrowing_target_from_intersection(db, env, intersection, target)
|
||||
.or_else(|| {
|
||||
intersection.positive(db).iter().find_map(|element| {
|
||||
specialize_narrowing_target(db, env, *element, target)
|
||||
})
|
||||
})
|
||||
.unwrap_or(target);
|
||||
IntersectionType::from_two_elements(db, env, subject, specialized_target)
|
||||
}
|
||||
(subject, target) => {
|
||||
let specialized_target =
|
||||
specialize_narrowing_target(db, env, subject, target).unwrap_or(target);
|
||||
let specialized_target = match subject {
|
||||
Type::Intersection(intersection) => {
|
||||
specialize_narrowing_target_from_intersection(db, env, intersection, target)
|
||||
.or_else(|| {
|
||||
intersection.positive(db).iter().find_map(|element| {
|
||||
specialize_narrowing_target(db, env, *element, target)
|
||||
})
|
||||
})
|
||||
}
|
||||
_ => specialize_narrowing_target(db, env, subject, target),
|
||||
}
|
||||
.filter(|specialized| {
|
||||
specialized.is_subtype_of(db, env, target.top_materialization(db, env))
|
||||
})
|
||||
.unwrap_or(target);
|
||||
IntersectionType::from_two_elements(db, env, subject, specialized_target)
|
||||
}
|
||||
}
|
||||
@@ -4585,13 +4613,24 @@ impl<'db> NarrowingConstraintsBuilder<'db, '_> {
|
||||
let place_and_constraint = match return_ty {
|
||||
Type::TypeIs(type_is) => {
|
||||
let (_, place) = type_is.place_info(db)?;
|
||||
let target = type_is.return_type(db);
|
||||
let use_generic_filtering = is_positive
|
||||
&& !db
|
||||
.analysis_settings(self.scope().file(db))
|
||||
.strict_generic_narrowing;
|
||||
Some((
|
||||
place,
|
||||
NarrowingConstraint::intersection(type_is.return_type(db).negate_if(
|
||||
db,
|
||||
&self.env,
|
||||
!is_positive,
|
||||
)),
|
||||
if use_generic_filtering {
|
||||
NarrowingConstraint::generic_filtering(target)
|
||||
} else {
|
||||
NarrowingConstraint::intersection(
|
||||
target.top_materialization(db, &self.env).negate_if(
|
||||
db,
|
||||
&self.env,
|
||||
!is_positive,
|
||||
),
|
||||
)
|
||||
},
|
||||
))
|
||||
}
|
||||
// TypeGuard only narrows in the positive case
|
||||
|
||||
@@ -1,26 +0,0 @@
|
||||
--- a/stdlib/inspect.pyi
|
||||
+++ b/stdlib/inspect.pyi
|
||||
@@ -345,10 +345,10 @@ def isgenerator(object: object) -> TypeIs[GeneratorType[object, Never, object]]:
|
||||
throw() used to raise an exception inside the generator
|
||||
"""
|
||||
|
||||
-def iscoroutine(object: object) -> TypeIs[CoroutineType[Any, Any, Any]]:
|
||||
+def iscoroutine(object: object) -> TypeIs[CoroutineType[object, Never, object]]:
|
||||
"""Return true if the object is a coroutine."""
|
||||
|
||||
-def isawaitable(object: object) -> TypeIs[Awaitable[Any]]:
|
||||
+def isawaitable(object: object) -> TypeIs[Awaitable[object]]:
|
||||
"""Return true if object can be passed to an ``await`` expression."""
|
||||
|
||||
@overload
|
||||
--- a/stdlib/typing_extensions.pyi
|
||||
+++ b/stdlib/typing_extensions.pyi
|
||||
@@ -1337,7 +1337,7 @@ else:
|
||||
|
||||
For example::
|
||||
|
||||
- def is_awaitable(val: object) -> TypeIs[Awaitable[Any]]:
|
||||
+ def is_awaitable(val: object) -> TypeIs[Awaitable[object]]:
|
||||
return hasattr(val, '__await__')
|
||||
|
||||
def f(val: Union[int, Awaitable[int]]) -> int:
|
||||
@@ -1,58 +0,0 @@
|
||||
--- a/stdlib/builtins.pyi
|
||||
+++ b/stdlib/builtins.pyi
|
||||
@@ -78,6 +78,7 @@ from typing import ( # noqa: Y022,UP035
|
||||
|
||||
# we can't import `Literal` from typing or mypy crashes: see #11247
|
||||
from typing_extensions import Literal, LiteralString, Self, TypeIs, TypeVarTuple, deprecated, disjoint_base # noqa: Y023, UP035
|
||||
+from ty_extensions import Top
|
||||
|
||||
if sys.version_info >= (3, 14):
|
||||
from _typeshed import AnnotateFunc
|
||||
@@ -3667,7 +3668,7 @@ def breakpoint(*args: Any, **kws: Any) -> None:
|
||||
By default, this drops you into the pdb debugger.
|
||||
"""
|
||||
|
||||
-def callable(obj: object, /) -> TypeIs[Callable[..., object]]:
|
||||
+def callable(obj: object, /) -> TypeIs[Top[Callable[..., object]]]:
|
||||
"""Return whether the object is callable (i.e., some kind of function).
|
||||
|
||||
Note that classes are callable, as are instances of classes with a
|
||||
--- a/stdlib/asyncio/base_futures.pyi
|
||||
+++ b/stdlib/asyncio/base_futures.pyi
|
||||
@@ -3,6 +3,7 @@ from collections.abc import Callable, Sequence
|
||||
from contextvars import Context
|
||||
from typing import Any, Final
|
||||
from typing_extensions import TypeIs
|
||||
+from ty_extensions import Top
|
||||
|
||||
from . import futures
|
||||
|
||||
@@ -12,7 +13,7 @@ _PENDING: Final = "PENDING" # undocumented
|
||||
_CANCELLED: Final = "CANCELLED" # undocumented
|
||||
_FINISHED: Final = "FINISHED" # undocumented
|
||||
|
||||
-def isfuture(obj: object) -> TypeIs[Future[Any]]:
|
||||
+def isfuture(obj: object) -> TypeIs[Top[Future[Any]]]:
|
||||
"""Check for a Future.
|
||||
|
||||
This returns True when obj is a Future instance or is advertising
|
||||
--- a/stdlib/asyncio/coroutines.pyi
|
||||
+++ b/stdlib/asyncio/coroutines.pyi
|
||||
@@ -1,6 +1,6 @@
|
||||
import sys
|
||||
from collections.abc import Awaitable, Callable, Coroutine
|
||||
from typing import Any, ParamSpec, TypeGuard, TypeVar, overload
|
||||
-from typing_extensions import TypeIs, deprecated
|
||||
+from typing_extensions import Never, TypeIs, deprecated
|
||||
|
||||
# Keep asyncio.__all__ updated with any changes to __all__ here
|
||||
@@ -22,7 +22,7 @@ if sys.version_info < (3, 11):
|
||||
If the coroutine is not yielded from before it is destroyed,
|
||||
an error message is logged.
|
||||
"""
|
||||
|
||||
-def iscoroutine(obj: object) -> TypeIs[Coroutine[Any, Any, Any]]:
|
||||
+def iscoroutine(obj: object) -> TypeIs[Coroutine[object, Never, object]]:
|
||||
"""Return True if obj is a coroutine object."""
|
||||
|
||||
if sys.version_info >= (3, 11):
|
||||
@@ -1,30 +0,0 @@
|
||||
diff --git a/stdlib/dataclasses.pyi b/stdlib/dataclasses.pyi
|
||||
index d46b694a7e..1db97b1893 100644
|
||||
--- a/stdlib/dataclasses.pyi
|
||||
+++ b/stdlib/dataclasses.pyi
|
||||
@@ -7,6 +7,7 @@ from collections.abc import Callable, Iterable, Mapping
|
||||
from types import GenericAlias
|
||||
from typing import Any, Final, Generic, Literal, Protocol, TypeVar, overload, type_check_only
|
||||
from typing_extensions import Never, TypeIs
|
||||
+from ty_extensions import Top
|
||||
|
||||
_T = TypeVar("_T")
|
||||
_T_co = TypeVar("_T_co", covariant=True)
|
||||
@@ -402,14 +403,14 @@ def fields(class_or_instance: DataclassInstance | type[DataclassInstance]) -> tu
|
||||
|
||||
# HACK: `obj: Never` typing matches if object argument is using `Any` type.
|
||||
@overload
|
||||
-def is_dataclass(obj: Never) -> TypeIs[DataclassInstance | type[DataclassInstance]]: # type: ignore[narrowed-type-not-subtype] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-type-guard-definition] # pyrefly: ignore [bad-function-definition]
|
||||
+def is_dataclass(obj: Never) -> TypeIs[Top[DataclassInstance | type[DataclassInstance]]]: # type: ignore[narrowed-type-not-subtype] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-type-guard-definition] # pyrefly: ignore [bad-function-definition]
|
||||
"""Returns True if obj is a dataclass or an instance of a
|
||||
dataclass.
|
||||
"""
|
||||
@overload
|
||||
-def is_dataclass(obj: type) -> TypeIs[type[DataclassInstance]]: ...
|
||||
+def is_dataclass(obj: type) -> TypeIs[Top[type[DataclassInstance]]]: ...
|
||||
@overload
|
||||
-def is_dataclass(obj: object) -> TypeIs[DataclassInstance | type[DataclassInstance]]: ...
|
||||
+def is_dataclass(obj: object) -> TypeIs[Top[DataclassInstance | type[DataclassInstance]]]: ...
|
||||
|
||||
class FrozenInstanceError(AttributeError): ...
|
||||
|
||||
@@ -3,7 +3,6 @@ from collections.abc import Callable, Sequence
|
||||
from contextvars import Context
|
||||
from typing import Any, Final
|
||||
from typing_extensions import TypeIs
|
||||
from ty_extensions import Top
|
||||
|
||||
from . import futures
|
||||
|
||||
@@ -13,7 +12,7 @@ _PENDING: Final = "PENDING" # undocumented
|
||||
_CANCELLED: Final = "CANCELLED" # undocumented
|
||||
_FINISHED: Final = "FINISHED" # undocumented
|
||||
|
||||
def isfuture(obj: object) -> TypeIs[Top[Future[Any]]]:
|
||||
def isfuture(obj: object) -> TypeIs[Future[Any]]:
|
||||
"""Check for a Future.
|
||||
|
||||
This returns True when obj is a Future instance or is advertising
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import sys
|
||||
from collections.abc import Awaitable, Callable, Coroutine
|
||||
from typing import Any, ParamSpec, TypeGuard, TypeVar, overload
|
||||
from typing_extensions import Never, TypeIs, deprecated
|
||||
from typing_extensions import TypeIs, deprecated
|
||||
|
||||
# Keep asyncio.__all__ updated with any changes to __all__ here
|
||||
if sys.version_info >= (3, 11):
|
||||
@@ -22,7 +22,7 @@ if sys.version_info < (3, 11):
|
||||
an error message is logged.
|
||||
"""
|
||||
|
||||
def iscoroutine(obj: object) -> TypeIs[Coroutine[object, Never, object]]:
|
||||
def iscoroutine(obj: object) -> TypeIs[Coroutine[Any, Any, Any]]:
|
||||
"""Return True if obj is a coroutine object."""
|
||||
|
||||
if sys.version_info >= (3, 11):
|
||||
|
||||
+1
-2
@@ -78,7 +78,6 @@ from typing import ( # noqa: Y022,UP035
|
||||
|
||||
# we can't import `Literal` from typing or mypy crashes: see #11247
|
||||
from typing_extensions import Literal, LiteralString, Self, TypeIs, TypeVarTuple, deprecated, disjoint_base # noqa: Y023, UP035
|
||||
from ty_extensions import Top
|
||||
|
||||
if sys.version_info >= (3, 14):
|
||||
from _typeshed import AnnotateFunc
|
||||
@@ -3670,7 +3669,7 @@ def breakpoint(*args: Any, **kws: Any) -> None:
|
||||
By default, this drops you into the pdb debugger.
|
||||
"""
|
||||
|
||||
def callable(obj: object, /) -> TypeIs[Top[Callable[..., object]]]:
|
||||
def callable(obj: object, /) -> TypeIs[Callable[..., object]]:
|
||||
"""Return whether the object is callable (i.e., some kind of function).
|
||||
|
||||
Note that classes are callable, as are instances of classes with a
|
||||
|
||||
@@ -7,7 +7,6 @@ from collections.abc import Callable, Iterable, Mapping
|
||||
from types import GenericAlias
|
||||
from typing import Any, Final, Generic, Literal, Protocol, TypeVar, overload, type_check_only
|
||||
from typing_extensions import Never, TypeIs
|
||||
from ty_extensions import Top
|
||||
|
||||
_T = TypeVar("_T")
|
||||
_T_co = TypeVar("_T_co", covariant=True)
|
||||
@@ -403,14 +402,14 @@ def fields(class_or_instance: DataclassInstance | type[DataclassInstance]) -> tu
|
||||
|
||||
# HACK: `obj: Never` typing matches if object argument is using `Any` type.
|
||||
@overload
|
||||
def is_dataclass(obj: Never) -> TypeIs[Top[DataclassInstance | type[DataclassInstance]]]: # type: ignore[narrowed-type-not-subtype] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-type-guard-definition] # pyrefly: ignore [bad-function-definition]
|
||||
def is_dataclass(obj: Never) -> TypeIs[DataclassInstance | type[DataclassInstance]]: # type: ignore[narrowed-type-not-subtype] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-type-guard-definition] # pyrefly: ignore [bad-function-definition]
|
||||
"""Returns True if obj is a dataclass or an instance of a
|
||||
dataclass.
|
||||
"""
|
||||
@overload
|
||||
def is_dataclass(obj: type) -> TypeIs[Top[type[DataclassInstance]]]: ...
|
||||
def is_dataclass(obj: type) -> TypeIs[type[DataclassInstance]]: ...
|
||||
@overload
|
||||
def is_dataclass(obj: object) -> TypeIs[Top[DataclassInstance | type[DataclassInstance]]]: ...
|
||||
def is_dataclass(obj: object) -> TypeIs[DataclassInstance | type[DataclassInstance]]: ...
|
||||
|
||||
class FrozenInstanceError(AttributeError): ...
|
||||
|
||||
|
||||
+2
-2
@@ -345,10 +345,10 @@ def isgenerator(object: object) -> TypeIs[GeneratorType[object, Never, object]]:
|
||||
throw() used to raise an exception inside the generator
|
||||
"""
|
||||
|
||||
def iscoroutine(object: object) -> TypeIs[CoroutineType[object, Never, object]]:
|
||||
def iscoroutine(object: object) -> TypeIs[CoroutineType[Any, Any, Any]]:
|
||||
"""Return true if the object is a coroutine."""
|
||||
|
||||
def isawaitable(object: object) -> TypeIs[Awaitable[object]]:
|
||||
def isawaitable(object: object) -> TypeIs[Awaitable[Any]]:
|
||||
"""Return true if object can be passed to an ``await`` expression."""
|
||||
|
||||
@overload
|
||||
|
||||
@@ -1336,7 +1336,7 @@ else:
|
||||
|
||||
For example::
|
||||
|
||||
def is_awaitable(val: object) -> TypeIs[Awaitable[object]]:
|
||||
def is_awaitable(val: object) -> TypeIs[Awaitable[Any]]:
|
||||
return hasattr(val, '__await__')
|
||||
|
||||
def f(val: Union[int, Awaitable[int]]) -> int:
|
||||
|
||||
Generated
+2
-2
@@ -107,7 +107,7 @@
|
||||
]
|
||||
},
|
||||
"strict-generic-narrowing": {
|
||||
"description": "Whether ty should use strict narrowing for unspecialized generic classes in\n`isinstance()` and `issubclass()` checks, as well as `match` class patterns.\n\nWhen enabled, ty narrows to the top materialization of the class. For example,\n`isinstance(value, list)` narrows a value of type `object` to `Top[list[Unknown]]`,\nrepresenting the (infinite) union of all possible `list` specializations. Iterating\nover the list would yield values of type `object`.\n\nWhen disabled, ty uses gradual generic narrowing, preserving compatible type\narguments from the original type where possible. For example,\n`isinstance(value, list)` narrows a value of type `Sequence[int]` to `list[int]`.\nIf no specialization is available, the same check narrows a value of type `object`\nto `list[Unknown]`; items of any type can then be appended to the list. Class\npatterns such as `case list():` follow the same behavior.\n\nDefaults to `false`.",
|
||||
"description": "Whether ty should use strict narrowing for unspecialized generic classes in\n`isinstance()` and `issubclass()` checks, `match` class patterns, and `TypeIs` checks.\n\nWhen enabled, ty narrows to the top materialization of the class. For example,\n`isinstance(value, list)` narrows a value of type `object` to `Top[list[Unknown]]`,\nrepresenting the (infinite) union of all possible `list` specializations. Iterating\nover the list would yield values of type `object`.\n\nWhen disabled, ty uses gradual generic narrowing, preserving compatible type\narguments from the original type where possible. For example,\n`isinstance(value, list)` narrows a value of type `Sequence[int]` to `list[int]`.\nIf no specialization is available, the same check narrows a value of type `object`\nto `list[Unknown]`; items of any type can then be appended to the list. Class\npatterns such as `case list():` follow the same behavior.\n\nDefaults to `false`.",
|
||||
"type": [
|
||||
"boolean",
|
||||
"null"
|
||||
@@ -416,7 +416,7 @@
|
||||
},
|
||||
"call-top-callable": {
|
||||
"title": "detects calls to the top callable type",
|
||||
"description": "## What it does\n\nChecks for calls to objects typed as `Top[Callable[..., T]]` (the infinite union of all callable\ntypes with return type `T`).\n\n## Why is this bad?\n\nWhen an object is narrowed to `Top[Callable[..., object]]` (e.g., via `callable(x)` or\n`isinstance(x, Callable)`), we know the object is callable, but we don't know its precise signature.\nThis type represents the set of all possible callable types (including, e.g., functions that take no\narguments and functions that require arguments), so no specific set of arguments can be guaranteed\nto be valid.\n\n## Examples\n\n```python\ndef f(x: object):\n if callable(x):\n # We know `x` is callable, but not what arguments it accepts\n x() # error\n```",
|
||||
"description": "## What it does\n\nChecks for calls to objects typed as `Top[Callable[..., T]]` (the infinite union of all callable\ntypes with return type `T`).\n\n## Why is this bad?\n\nWhen `analysis.strict-generic-narrowing` is enabled, `callable(x)` and `isinstance(x, Callable)`\nnarrow an object to `Top[Callable[..., object]]`. We know the object is callable, but we don't know\nits precise signature. This type represents the set of all possible callable types (including, e.g.,\nfunctions that take no arguments and functions that require arguments), so no specific set of\narguments can be guaranteed to be valid.\n\n## Examples\n\n```toml\n[analysis]\nstrict-generic-narrowing = true\n```\n\n```python\ndef f(x: object):\n if callable(x):\n # We know `x` is callable, but not what arguments it accepts\n x() # error\n```",
|
||||
"default": "error",
|
||||
"oneOf": [
|
||||
{
|
||||
|
||||
Reference in new issue
Block a user