[ty] Promote tuple lengths based on the outer element types (#29147)

## Summary

We normally infer `list[tuple[int, ...]]` for `[(1,), (1, 2)]`, allowing
tuples of other lengths to be added later. This tuple-size promotion
applies to tuple literals; tuple types supplied by annotations or other
expressions retain their existing lengths.

The check that preserves those lengths currently searches for tuples
anywhere inside an element's type. This incorrectly blocks promotion
when the tuple appears inside a generic type argument, a callable's
return type, or a protocol member:

```python
class Box[T]: ...

def example(box: Box[tuple[int, str]]) -> None:
    values = [(1,), (1, 2), box]
    reveal_type(values)
    # Before: list[tuple[int] | tuple[int, int] | Box[tuple[int, str]]]
    # After:  list[Box[tuple[int, str]] | tuple[int, ...]]
```

The `Box` is not a tuple, so its type argument should not prevent us
from widening the two tuple literals. We replace this use of
`any_over_type` with a check for whether the element itself may be a
tuple. The check still follows aliases, unions, type-variable bounds,
and `NewType` bases that can describe a tuple value, while skipping
nested type arguments and members.

Co-authored-by: Ibraheem Ahmed <ibraheem@ibraheem.ca>

crates/ty_python_semantic/resources/mdtest/promotion.md

@@ -306,6 +306,162 @@ segments: Mapping[str, Any] = {"start": (1, 2), "end": (3, 4, 5)}
 reveal_type(segments)  # revealed: dict[str, tuple[int, ...]]
 ```
 
+## Non-tuple elements do not block tuple-size promotion
+
+A non-literal tuple blocks tuple-size promotion because its shape must be preserved. A tuple nested
+inside a non-tuple element's type arguments does not:
+
+```toml
+[environment]
+python-version = "3.12"
+```
+
+```py
+class Box[T]: ...
+
+def _(box: Box[tuple[int, str]]) -> None:
+    # revealed: list[Box[tuple[int, str]] | tuple[int, ...]]
+    reveal_type([(1,), (1, 2), box])
+```
+
+A tuple in a callable's return type does not prevent promotion either:
+
+```py
+from collections.abc import Callable
+
+def _(callback: Callable[[], tuple[int, str]]) -> None:
+    # revealed: list[(() -> tuple[int, str]) | tuple[int, ...]]
+    reveal_type([(1,), (1, 2), callback])
+```
+
+Protocol members can also contain tuples without blocking promotion:
+
+```py
+from typing import Protocol
+
+class Proto(Protocol):
+    pair: tuple[int, str]
+
+def _(protocol: Proto) -> None:
+    reveal_type([(1,), (1, 2), protocol])  # revealed: list[Proto | tuple[int, ...]]
+```
+
+The same applies to `TypedDict` fields:
+
+```py
+from typing import TypedDict
+
+class Payload(TypedDict):
+    pair: tuple[int, str]
+
+def _(payload: Payload) -> None:
+    reveal_type([(1,), (1, 2), payload])  # revealed: list[Payload | tuple[int, ...]]
+```
+
+## Recursive members do not block tuple-size promotion
+
+Tuple-valued and recursive protocol members do not prevent neighboring tuple literals from being
+promoted:
+
+```toml
+[environment]
+python-version = "3.12"
+```
+
+```py
+from __future__ import annotations
+
+from typing import Protocol
+
+class Recursive[T](Protocol):
+    next: Recursive[list[T]]
+    pair: tuple[int, str]
+
+def _(x: Recursive[int]) -> None:
+    reveal_type([(1,), (1, 2), x])  # revealed: list[Recursive[int] | tuple[int, ...]]
+```
+
+The same applies to recursive `TypedDict` fields:
+
+```py
+from typing import TypedDict
+
+class Payload[T](TypedDict):
+    child: Payload[list[T]]
+    pair: tuple[int, str]
+
+def _(x: Payload[int]) -> None:
+    reveal_type([(1,), (1, 2), x])  # revealed: list[Payload[int] | tuple[int, ...]]
+```
+
+## Type parameter declarations do not block tuple-size promotion
+
+A class parameter's bound or default does not describe the shape of instances of that class. A tuple
+in either declaration does not prevent promotion of neighboring tuple literals.
+
+```toml
+[environment]
+python-version = "3.13"
+```
+
+```py
+from typing import Generic, TypeVar
+
+T = TypeVar("T", bound=int | tuple[int, str])
+
+class Bounded(Generic[T]): ...
+
+def _(x: Bounded[int]) -> None:
+    # revealed: list[Bounded[int] | tuple[int, ...]]
+    reveal_type([(1,), (1, 2), x])
+```
+
+The supplied type argument also takes precedence over a default:
+
+```py
+U = TypeVar("U", default=tuple[int, str])
+
+class Defaulted(Generic[U]): ...
+
+def _(x: Defaulted[int]) -> None:
+    # revealed: list[Defaulted[int] | tuple[int, ...]]
+    reveal_type([(1,), (1, 2), x])
+```
+
+A default also does not fix the type of a value annotated with the type variable:
+
+```py
+def _(x: U) -> None:
+    reveal_type([(1,), (1, 2), x])  # revealed: list[U@_ | tuple[int, ...]]
+```
+
+## Tuple-size promotion uses alias values
+
+Tuple-size promotion inspects alias values. `Alias[Alias[str]]` resolves to `str`, so it does not
+block promotion:
+
+```toml
+[environment]
+python-version = "3.12"
+```
+
+```py
+type Alias[T] = T
+
+def _(x: Alias[Alias[str]]) -> None:
+    reveal_type([(1,), (1, 2), x])  # revealed: list[str | tuple[int, ...]]
+```
+
+Even a recursively specialized alias permits promotion when its value is a `list`:
+
+```py
+type Recursive[T] = list[Recursive[list[T]]]
+
+def _(x: Recursive[int]) -> None:
+    # revealed: list[list[Recursive[list[int]]] | tuple[int, ...]]
+    reveal_type([(1,), (1, 2), x])
+```
+
 ## Invariant and contravariant return types are promoted
 
 We promote in non-covariant position in the return type of a generic function, or constructor of a

crates/ty_python_semantic/resources/mdtest/unpacking.md

@@ -538,6 +538,23 @@ def expanded_tuples(values: list[str]):
     reveal_type(last)  # revealed: Literal[False]
 ```
 
+A protocol's tuple-valued member does not prevent promotion of tuple literals collected alongside
+the protocol instance.
+
+```py
+from typing import Protocol
+
+class P(Protocol):
+    pair: tuple[int, int]
+
+def _(value: P) -> None:
+    first, *rest = [0, (1,), (2, 3), value]
+    reveal_type(rest)  # revealed: list[P | tuple[int, ...]]
+
+    first, *rest = (0, (1,), (2, 3), value)
+    reveal_type(rest)  # revealed: list[P | tuple[int, ...]]
+```
+
 ### Starred expressions on the right-hand side
 
 A starred element can contribute an unknown number of values. The literal's AST length does not

crates/ty_python_semantic/src/types/newtype.rs

@@ -274,6 +274,18 @@ pub(crate) fn walk_newtype_instance_type<'db, V: visitor::TypeVisitor<'db> + ?Si
     }
 }
 
+/// Evaluate and visit the instance type of a newtype's base.
+///
+/// The caller must guard against recursive newtypes.
+pub(super) fn walk_newtype_base<'db, V: visitor::TypeVisitor<'db> + ?Sized>(
+    db: &'db dyn Db,
+    newtype: NewType<'db>,
+    visitor: &V,
+) {
+    let base = newtype.base(db);
+    visitor.visit_type(db, base.instance_type(db, visitor.program_environment()));
+}
+
 /// `typing.NewType` typically wraps a class type, but it can also wrap another newtype.
 #[derive(Clone, Copy, Debug, Eq, Hash, PartialEq, get_size2::GetSize, salsa::SalsaValue)]
 pub enum NewTypeBase<'db> {

crates/ty_python_semantic/src/types/tuple/promotion.rs

@@ -1,13 +1,117 @@
 use crate::Db;
 use crate::ProgramEnvironment;
 use rustc_hash::FxHashSet;
+use std::cell::Cell;
 
 use ruff_python_ast::{self as ast};
 
+use crate::types::cyclic::{ActiveRecursionDetector, TypeIdentity};
+use crate::types::newtype::{NewType, walk_newtype_base};
 use crate::types::tuple::TupleSpec;
-use crate::types::typevar::BoundTypeVarIdentity;
-use crate::types::visitor::{any_over_type, contains_growing_type};
-use crate::types::{Type, UnionBuilder};
+use crate::types::typevar::{BoundTypeVarIdentity, TypeVarInstance, walk_type_var_bounds};
+use crate::types::visitor::{TypeCollector, TypeVisitor, walk_type_with_recursion_guard};
+use crate::types::{
+    IntersectionType, KnownInstanceType, RecursiveType, Type, TypeAliasType, UnionBuilder,
+};
+
+/// Return whether the outer type may be a tuple, without searching nested arguments or members.
+/// A recursion cutoff conservatively counts as a possible tuple.
+fn may_be_tuple<'db>(db: &'db dyn Db, env: &ProgramEnvironment<'db>, ty: Type<'db>) -> bool {
+    struct TupleShapeVisitor<'a, 'db> {
+        env: &'a ProgramEnvironment<'db>,
+        recursion_guard: TypeCollector<'db>,
+        active_types: ActiveRecursionDetector<TypeIdentity<'db>>,
+        may_be_tuple: Cell<bool>,
+    }
+
+    impl<'db> TupleShapeVisitor<'_, 'db> {
+        fn visit_guarded(&self, db: &'db dyn Db, ty: Type<'db>, visit: impl FnOnce()) {
+            self.active_types.visit(
+                &ty.to_type_identity(db),
+                || self.may_be_tuple.set(true),
+                visit,
+            );
+        }
+    }
+
+    impl<'db> TypeVisitor<'db> for TupleShapeVisitor<'_, 'db> {
+        fn program_environment(&self) -> &ProgramEnvironment<'db> {
+            self.env
+        }
+
+        fn should_visit_lazy_type_attributes(&self) -> bool {
+            false
+        }
+
+        fn visit_type(&self, db: &'db dyn Db, ty: Type<'db>) {
+            if self.may_be_tuple.get() {
+                return;
+            }
+
+            if ty.tuple_instance_spec(db, self.env).is_some() {
+                self.may_be_tuple.set(true);
+                return;
+            }
+
+            if matches!(
+                ty,
+                Type::Union(_)
+                    | Type::Intersection(_)
+                    | Type::TypeAlias(_)
+                    | Type::Recursive(_)
+                    | Type::NewTypeInstance(_)
+                    | Type::TypeVar(_)
+            ) {
+                walk_type_with_recursion_guard(db, ty, self, &self.recursion_guard);
+            }
+        }
+
+        fn visit_intersection_type(&self, db: &'db dyn Db, intersection: IntersectionType<'db>) {
+            for element in intersection.iter_positive(db) {
+                self.visit_type(db, element);
+            }
+        }
+
+        fn visit_type_alias_type(&self, db: &'db dyn Db, alias: TypeAliasType<'db>) {
+            self.visit_guarded(db, Type::TypeAlias(alias), || {
+                self.visit_type(db, alias.value_type(db));
+            });
+        }
+
+        fn visit_recursive_type(&self, db: &'db dyn Db, recursive: RecursiveType<'db>) {
+            self.visit_guarded(db, Type::Recursive(recursive), || {
+                self.visit_type(db, recursive.unfold(db, self.env).into_type());
+            });
+        }
+
+        fn visit_newtype_instance_type(&self, db: &'db dyn Db, newtype: NewType<'db>) {
+            self.visit_guarded(db, Type::NewTypeInstance(newtype), || {
+                walk_newtype_base(db, newtype, self);
+            });
+        }
+
+        fn visit_type_var_type(&self, db: &'db dyn Db, typevar: TypeVarInstance<'db>) {
+            self.visit_guarded(
+                db,
+                Type::KnownInstance(KnownInstanceType::TypeVar(typevar)),
+                || {
+                    if let Some(bounds) = typevar.bound_or_constraints(db, self.env) {
+                        walk_type_var_bounds(db, bounds, self);
+                    }
+                },
+            );
+        }
+    }
+
+    let visitor = TupleShapeVisitor {
+        env,
+        recursion_guard: TypeCollector::default(),
+        active_types: ActiveRecursionDetector::default(),
+        may_be_tuple: Cell::new(false),
+    };
+    visitor.visit_type(db, ty);
+    visitor.may_be_tuple.get()
+}
 
 /// Tracks the typevars of a collection to which tuple size promotion should **not** apply.
 #[derive(Default)]
@@ -35,8 +139,7 @@ impl<'db> TupleSizePromotionConstraints<'db> {
         }
     }
 
-    /// Records that a typevar is ineligible for tuple size promotion if the given type contains
-    /// a tuple type.
+    /// Block tuple size promotion if this type variable is assigned a possible tuple type.
     pub(crate) fn record_unpromotable_type(
         &mut self,
         db: &'db dyn Db,
@@ -73,12 +176,7 @@ impl<'db> TupleSizePromotionConstraints<'db> {
     ) -> bool {
         expression
             .is_some_and(|expression| Self::is_promotable_tuple_literal(db, env, expression, ty))
-            // An unbounded unfolding cannot establish the absence of annotated tuples.
-            // Preserve their lengths unless the complete finite scan permits promotion.
-            || (!contains_growing_type(db, env, ty)
-                && !any_over_type(db, env, ty, true, |ty| {
-                    ty.tuple_instance_spec(db, env).is_some()
-                }))
+            || !may_be_tuple(db, env, ty)
     }
 
     /// Returns true if the given expression is either a non-starred homogeneous tuple literal or the

crates/ty_python_semantic/src/types/typevar.rs

@@ -2276,7 +2276,7 @@ pub enum TypeVarBoundOrConstraints<'db> {
     Constraints(TypeVarConstraints<'db>),
 }
 
-fn walk_type_var_bounds<'db, V: visitor::TypeVisitor<'db> + ?Sized>(
+pub(super) fn walk_type_var_bounds<'db, V: visitor::TypeVisitor<'db> + ?Sized>(
     db: &'db dyn Db,
     bounds: TypeVarBoundOrConstraints<'db>,
     visitor: &V,