Enable `require-build-hashes` for the Python projects and inline scripts
so isolated builds require declared hashes for their build dependencies.
Pin the build backends and their dependencies with hashes.
---------
Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
Our `py-fuzzer` script currently limits each failing seed to have a
maximum of 60 seconds for minimization, even when the CI job has time
remaining. For example, in [this fuzz job on
#29025](https://github.com/astral-sh/ruff/actions/runs/37632264340/job/112829497328),
minimization timed out for seed 57 even though there would have been
another 15 minutes before GitHub cancelled the CI job. Running the
script locally demonstrated that pysource_minimize was [easily
able](https://github.com/astral-sh/ruff/pull/29025#issuecomment-6064567417)
to minimize the panic if only given slightly more time to do so.
This PR replaces the per-seed limit in CI with a global, shared budget
for checking seeds and minimizing failures. Local runs have no overall
budget by default. CI, however, records the job start time in its first
step and passes it and the job timeout to the fuzzer, which subtracts
elapsed setup and build time and reserves an additional 90 seconds for
other job overhead and reporting. Individual calls to ty or Ruff still
have a five-second timeout, allowing us to minimize hangs or
pathological performance issues.
By default, the fuzzer reports failing seeds as they are found and
prints completed minimizations. When the deadline is reached, it stops
starting new seed and minimizer checks, reports the original source for
unfinished minimizations, and identifies unchecked seeds. Checks already
in progress can finish after the deadline. The logs and, when requested,
the GitHub issue body include a command to rerun failing and unchecked
seeds; the script exits with status 2 if any seeds were left unchecked.
Here are some mockups of what bot-opened issues could look like, using
fictional data:
<details>
<summary>Issue with minimized, unminimized, and unchecked
seeds</summary>
Run listed here: https://github.com/astral-sh/ty/actions/runs/1234567890
Each snippet caused ty to exit with a status other than 0, 1, or 2, or
time out after five seconds during fuzzing.
Build ty from commit `0123456789abcdef0123456789abcdef01234567` with
`cargo build --locked --bin=ty`. Save a snippet to `repro.py`, then run:
```
target/debug/ty check repro.py --python-version=3.10 --python-platform=linux
```
### Seed: 78
```python
class Example:
pass
```
<details>
<summary>Seed: 93 (original source; minimization did not
complete)</summary>
```python
class Original:
value = [1, 2, 3]
def method(self):
return self.value
```
</details>
### Unchecked seeds
The time budget expired before 2 seeds could be checked.
- `123`
- `124`
To check all seeds that failed or were left unchecked, check out Ruff
commit `0123456789abcdef0123456789abcdef01234567` and run from the
repository root:
```shell
uv run --project=./python/py-fuzzer fuzz --bin=ty 78 93 123 124
```
</details>
<details>
<summary>Issue with unchecked seeds and no failures</summary>
Run listed here:
https://github.com/astral-sh/ruff/actions/runs/1234567890
Test executable: ruff at commit
`0123456789abcdef0123456789abcdef01234567`.
### Unchecked seeds
The time budget expired before 3 seeds could be checked.
- `10`
- `11`
- `12`
To check all seeds that failed or were left unchecked, check out Ruff
commit `0123456789abcdef0123456789abcdef01234567` and run from the
repository root:
```shell
uv run --project=./python/py-fuzzer fuzz --bin=ruff 10 11 12
```
</details>
## Rejected alternatives
1. Keep a per-minimization budget, but just increase it (by a lot?).
We could do this, and it fixes the theoretical problem that codex raised
where a couple of seeds that take a very long time to minimize stop the
script from even discovering bugs in later seeds (because it never even
gets to those later seeds).
I decided against this approach, however, because in practice I think it
will only be hangs that take a _long_ time to minimize. We've seen in
the past that a ty hang can cause the script as a whole to timeout in
CI, but whenever it's done so in the past, it's never caused the script
to fail to get onto non-hanging seeds. It's always been the case that
the script has completed all seeds except the problematic ones that were
causing it to hang. So I decided that per-minimization budgets were a
solution looking for a problem.
2. Get rid of all budgets altogether, go back to something more similar
to what the script was like prior to
https://github.com/astral-sh/ruff/pull/29045.
This is very attractive, because it avoids a lot of the new complexity
added in this PR to do with calculating budgets, interrupting
subprocesses that have already been started, etc.
I decided against this, however, because I do think the fact that the
script now minimizes hangs substantially increases the risk that the
script might finish without having finished minimizing all seeds. In the
event that this happens, I _do_ think it's quite useful (though not
essential) for the script to report which seeds it didn't get to, before
GitHub cancels the CI job altogether.
## Summary
Enable `truthiness-test-of-none-union` in the standalone scripts' inline
configuration and in the `py-fuzzer` and `ty_benchmark` projects,
following #28889.
Python tooling can currently build third-party source distributions. Set
`no-build` in the maintained Python projects and PEP 723 scripts,
retaining named exceptions for Git-pinned tooling and the local
ecosystem runner. Pin the isolated build dependencies for those
exceptions, including their transitive requirements.
Follows merged astral-sh/ruff#27856, which gives the legacy `ruff-lsp`
suite a dedicated, locked Python 3.12 test environment.
Follow-up to astral-sh/ruff#27843. Replaces astral-sh/ruff#27840, which
closed when the merged docs branch was deleted.
Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
- Does this PR follow our AI policy
(https://github.com/astral-sh/.github/blob/main/AI_POLICY.md)?
-->
## Summary
Addresses all `uv audit` findings.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
NFC.
<!-- How was it tested? -->
---
<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
- Does this PR follow our AI policy
(https://github.com/astral-sh/.github/blob/main/AI_POLICY.md)?
-->
## Summary
Adds explicit 7d cooldowns to all of our PEP 723 scripts, as well as
similar cooldowns via `.npmrc`.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
NFC.
<!-- How was it tested? -->
---
<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
Close and reopen this PR to trigger CI
---------
Co-authored-by: typeshedbot <>
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
Summary
--
I was firing up the fuzzer tonight and hit an assertion error here. We
now build with the `profiling` profile, so we need to use that
executable too.
This hasn't affected CI because we always set the `--test-executable`.
Test Plan
--
Ran the script again with the same arguments on this branch
## Summary
Infer a type of `Self` for unannotated `self` parameters in methods of
classes.
part of https://github.com/astral-sh/ty/issues/159
closes https://github.com/astral-sh/ty/issues/1081
## Conformance tests changes
```diff
+enums_member_values.py:85:9: error[invalid-assignment] Object of type `int` is not assignable to attribute `_value_` of type `str`
```
A true positive ✔️
```diff
-generics_self_advanced.py:35:9: error[type-assertion-failure] Argument does not have asserted type `Self@method2`
-generics_self_basic.py:14:9: error[type-assertion-failure] Argument does not have asserted type `Self@set_scale
```
Two false positives going away ✔️
```diff
+generics_syntax_infer_variance.py:82:9: error[invalid-assignment] Cannot assign to final attribute `x` on type `Self@__init__`
```
This looks like a true positive to me, even if it's not marked with `#
E` ✔️
```diff
+protocols_explicit.py:56:9: error[invalid-assignment] Object of type `tuple[int, int, str]` is not assignable to attribute `rgb` of type `tuple[int, int, int]`
```
True positive ✔️
```
+protocols_explicit.py:85:9: error[invalid-attribute-access] Cannot assign to ClassVar `cm1` from an instance of type `Self@__init__`
```
This looks like a true positive to me, even if it's not marked with `#
E`. But this is consistent with our understanding of `ClassVar`, I
think. ✔️
```py
+qualifiers_final_annotation.py:52:9: error[invalid-assignment] Cannot assign to final attribute `ID4` on type `Self@__init__`
+qualifiers_final_annotation.py:65:9: error[invalid-assignment] Cannot assign to final attribute `ID7` on type `Self@method1`
```
New true positives ✔️
```py
+qualifiers_final_annotation.py:52:9: error[invalid-assignment] Cannot assign to final attribute `ID4` on type `Self@__init__`
+qualifiers_final_annotation.py:57:13: error[invalid-assignment] Cannot assign to final attribute `ID6` on type `Self@__init__`
+qualifiers_final_annotation.py:59:13: error[invalid-assignment] Cannot assign to final attribute `ID6` on type `Self@__init__`
```
This is a new false positive, but that's a pre-existing issue on main
(if you annotate with `Self`):
https://play.ty.dev/3ee1c56d-7e13-43bb-811a-7a81e236e6ab❌ => reported
as https://github.com/astral-sh/ty/issues/1409
## Ecosystem
* There are 5931 new `unresolved-attribute` and 3292 new
`possibly-missing-attribute` attribute errors, way too many to look at
all of them. I randomly sampled 15 of these errors and found:
* 13 instances where there was simply no such attribute that we could
plausibly see. Sometimes [I didn't find it
anywhere](https://github.com/internetarchive/openlibrary/blob/8644d886c6579a5f49faadc4cd1ba9992e603d7e/openlibrary/plugins/openlibrary/tests/test_listapi.py#L33).
Sometimes it was set externally on the object. Sometimes there was some
[`setattr` dynamicness going
on](https://github.com/pypa/setuptools/blob/a49f6b927d83b97630b4fb030de8035ed32436fd/setuptools/wheel.py#L88-L94).
I would consider all of them to be true positives.
* 1 instance where [attribute was set on `obj` in
`__new__`](https://github.com/sympy/sympy/blob/9e87b44fd43572b9c4cc95ec569a2f4b81d56499/sympy/tensor/array/array_comprehension.py#L45C1-L45C36),
which we don't support yet
* 1 instance [where the attribute was defined via `__slots__`
](https://github.com/spack/spack/blob/e250ec0fc81130b708a8abe1894f0cc926880210/lib/spack/spack/vendor/pyrsistent/_pdeque.py#L48C5-L48C14)
* I see 44 instances [of the false positive
above](https://github.com/astral-sh/ty/issues/1409) with `Final`
instance attributes being set in `__init__`. I don't think this should
block this PR.
## Test Plan
New Markdown tests.
---------
Co-authored-by: Shaygan Hooshyari <sh.hooshyari@gmail.com>
## Summary
This PR Removes deprecated UP038 as per instructed in #18727closes#18727
## Test Plan
I have run tests non of them failing
One Question i have is do we have to document that UP038 is removed?
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
## Summary
Fuzzer seed 208 seems to be timing out all fuzzer runs on PRs today.
This has happened on multiple unrelated PRs, as well as on an initial
version of this PR that made a comment-only change in ty and didn't skip
any seeds, so the timeout appears to be consistent in CI, on ty main
branch, as of today, but it started happening due to some change in a
factor outside ty; not sure what.
I checked the code generated for seed 208 locally, and it takes about
30s to check on current ty main branch. This is slow for a fuzzer seed,
but shouldn't be slow enough to make it time out after 20min in CI (even
accounting for GH runners being slower than my laptop.)
I tried to bisect the slowness of checking that code locally, but I
didn't go back far enough to find the change that made it slow. In fact
it seems like it became significantly faster in the last few days (on an
older checkout I had to stop it after several minutes.) So whatever the
cause of the slowness, it's not a recent change in ty.
I don't want to rabbit-hole on this right now (fuzzer-discovered issues
are lower-priority than real-world-code issues), and need a working CI,
so skip this seed for now until we can investigate it.
## Test Plan
CI. This PR contains a no-op (comment) change in ty, so that the fuzz
test is triggered in CI and we can verify it now works (as well as
verify, on the previous commit, that the fuzzer job is timing out on
that seed, even with just a no-op change in ty.)
## Summary
A [passing
comment](https://github.com/astral-sh/ruff/pull/19711#issuecomment-3169312014)
led me to explore why we didn't report a class attribute as possibly
unbound if it was a method and defined in two different conditional
branches.
I found that the reason was because of our handling of "conflicting
declarations" in `place_from_declarations`. It returned a `Result` which
would be `Err` in case of conflicting declarations.
But we only actually care about conflicting declarations when we are
actually doing type inference on that scope and might emit a diagnostic
about it. And in all cases (including that one), we want to otherwise
proceed with the union of the declared types, as if there was no
conflict.
In several cases we were failing to handle the union of declared types
in the same way as a normal declared type if there was a declared-types
conflict. The `Result` return type made this mistake really easy to
make, as we'd match on e.g. `Ok(Place::Type(...))` and do one thing,
then match on `Err(...)` and do another, even though really both of
those cases should be handled the same.
This PR refactors `place_from_declarations` to instead return a struct
which always represents the declared type we should use in the same way,
as well as carrying the conflicting declared types, if any. This struct
has a method to allow us to explicitly ignore the declared-types
conflict (which is what we want in most cases), as well as a method to
get the declared type and the conflict information, in the case where we
want to emit a diagnostic on the conflict.
## Test Plan
Existing CI; added a test showing that we now understand a
multiply-conditionally-defined method as possibly-unbound.
This does trigger issues on a couple new fuzzer seeds, but the issues
are just new instances of an already-known (and rarely occurring)
problem which I already plan to address in a future PR, so I think it's
OK to land as-is.
I happened to build this initially on top of
https://github.com/astral-sh/ruff/pull/19711, which adds invalid-await
diagnostics, so I also updated some invalid-syntax tests to not await on
an invalid type, since the purpose of those tests is to check the
syntactic location of the `await`, not the validity of the awaited type.
## Summary
I tried running `py-fuzzer` using executables in the current working
directory, but that failed with:
```
▶ uvx --from ./python/py-fuzzer --reinstall fuzz --test-executable ./ty_feature --bin=ty --baseline-executable ./ty_main --only-new-bugs 0-500
Usage: fuzz [-h] [--only-new-bugs] [--quiet] [--test-executable TEST_EXECUTABLE] [--baseline-executable BASELINE_EXECUTABLE] --bin {ruff,ty} seeds [seeds ...]
fuzz: error: Bad argument passed to `--baseline-executable`: no such file or executable PosixPath('ty_main')
"Bad argument passed to `--baseline-executable`: no such file or executable PosixPath('ty_main')"
```
Using `.absolute()` on the `Path` fixes this.
## Test Plan
Successful `py-fuzzer` run with the invocation above.
## Summary
Add cycle handling for `try_metaclass` and `pep695_generic_context`
queries, as well as adjusting the cycle handling for `try_mro` to ensure
that it short-circuits on cycles and won't grow MROs indefinitely.
This reduces the number of failing fuzzer seeds from 68 to 17. The
latter count includes fuzzer seeds 120, 160, and 335, all of which
previously panicked but now either hang or are very slow; I've
temporarily skipped those seeds in the fuzzer until I can dig into that
slowness further.
This also allows us to move some more ecosystem projects from `bad.txt`
to `good.txt`, which I've done in
https://github.com/astral-sh/ruff/pull/17903
## Test Plan
Added mdtests.
Summary
--
Updates `fuzz.py` to run with `--preview`, which should allow it to
catch semantic syntax errors.
Test Plan
--
@AlexWaygood and I temporarily made any named expression a semantic
syntax error and checked that this led to fuzzing errors. We also tested
that reverting the `--preview` addition did not show any errors.
We also ran the fuzzer on 500 seeds on `main` but didn't find any
issues, (un)fortunately.
The red-knot CLI changed since the fuzzer script was added; update it to
work with current red-knot CLI.
Also add some notes on how to ensure local changes to the fuzzer script
are picked up.
## Summary
This PR builds on the changes in #16220 to pass a target Python version
to the parser. It also adds the `Parser::unsupported_syntax_errors` field, which
collects version-related syntax errors while parsing. These syntax
errors are then turned into `Message`s in ruff (in preview mode).
This PR only detects one syntax error (`match` statement before Python
3.10), but it has been pretty quick to extend to several other simple
errors (see #16308 for example).
## Test Plan
The current tests are CLI tests in the linter crate, but these could be
supplemented with inline parser tests after #16357.
I also tested the display of these syntax errors in VS Code:
---------
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
## Summary
This PR gets rid of the `requirements.in` and `requirements.txt` files
in the `scripts/fuzz-parser` directory, and replaces them with
`pyproject.toml` and `uv.lock` files. The script is renamed from
`fuzz-parser` to `py-fuzzer` (since it can now also be used to fuzz
red-knot as well as the parser, following
https://github.com/astral-sh/ruff/pull/14566), and moved from the
`scripts/` directory to the `python/` directory, since it's now a
(uv)-pip-installable project in its own right.
I've been resisting this for a while, because conceptually this script
just doesn't feel "complicated" enough to me for it to be a full-blown
package. However, I think it's time to do this. Making it a proper
package has several advantages:
- It means we can run it from the project root using `uv run` without
having to activate a virtual environment and ensure that all required
dependencies are installed into that environment
- Using a `pyproject.toml` file means that we can express that the
project requires Python 3.12+ to run properly; this wasn't possible
before
- I've been running mypy on the project locally when I've been working
on it or reviewing other people's PRs; now I can put the mypy config for
the project in the `pyproject.toml` file
## Test Plan
I manually tested that all the commands detailed in
`python/py-fuzzer/README.md` work for me locally.
---------
Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
David Peter