## Summary
`match` is a soft keyword. It can name a variable, a function, or the
target of an annotated assignment, including after a semicolon.
```python
first(); match = value
first(); match(value)
first(); match[0]: int
```
The parser reported a compound-statement error as soon as it saw the
`match` token. We now check the preceding separator when parsing a match
statement. If the token turns out to be an identifier, speculative
parsing discards the diagnostic. A genuine `match` statement on the same
logical line still produces the original error.
## Summary
A bare `raise` can end at a semicolon. For example, Python accepts this
handler and re-raises the current exception.
```python
try:
work()
except ValueError:
raise; report()
```
The parser tried to read the semicolon as the start of an exception
expression and reported a syntax error. We now recognize both a newline
and a semicolon as terminators for a bare `raise`.
Use the vendored runtime and generator instead of the registry dependency,
providing Unicode 17 names with Python-compatible matching. Update workspace
dependencies and tool configuration, add parser regressions, and document how
to update Unicode data when supporting a new Python version.
## Summary
We reduce `Expr` from 64 to 56 bytes on 64-bit targets. The largest
variants determine the enum's size, so shrinking those variants saves
eight bytes in every `Expr` value, including expressions in the parsed
modules retained by ty.
The changes are:
- Store positional call arguments in `ThinVec`, reducing `Arguments` and
`ExprCall` by eight bytes.
- Store all comparison operands in one boxed slice. This removes the
separate left-operand allocation and reduces `ExprCompare` by eight
bytes while keeping operators in a boxed slice.
- Store dictionary-comprehension generators in a boxed slice, reducing
`ExprDictComp` by eight bytes.
- Store f-string and t-string elements in boxed slices, and represent a
single f-string directly as `FString`. This keeps single parts inline
and reduces `ExprFString` and `ExprTString` by eight bytes.
The parser, AST visitors, formatter, lint rules, and type inference use
the updated layouts. Size assertions cover the 56-byte enum and its
affected variants.
Summary
--
While reviewing #28231, Codex found a preexisting bug shared with our
unary minus handling. Namely, CPython emits an error for code like this,
where the unary expression is part of a longer expression:
```pycon
>>> match foo:
... case -1**2: ...
...
File "<python-input-0>", line 2
case -1**2: ...
^^
SyntaxError: invalid syntax
```
but Ruff does not:
```console
$ ruff check --ignore ALL - <<EOF
match foo:
case -1**2: ...
EOF
All checks passed!
```
We now verify that the parsed unary expression is a number literal and
emit an error if not.
Test Plan
--
New inline snapshots
Summary
--
As added in https://github.com/python/cpython/pull/148566, CPython now
allows unary plus in match
patterns, mirroring the support for unary minus. We already had a
special case for this for improved
error reporting and recovery, so I just swapped out the `add_error` call
for
`add_unsupported_syntax_error` with a new error kind.
Part of https://github.com/astral-sh/ruff/issues/28206
Test Plan
--
New inline parser tests
## Summary
This is a first step towards #20411. After this, we would work to either
upstream changes or adjust things within Ruff.
Performance improvements have already been upstreamed
(https://github.com/rust-lang/annotate-snippets-rs/pull/437,
https://github.com/rust-lang/annotate-snippets-rs/pull/440)
Re-implemented the following from the fork:
- Hidden snippets
(https://github.com/rust-lang/annotate-snippets-rs/issues/448)
- Renderer-level disabling of hyperlinks
- Jupyter cell indices
(https://github.com/rust-lang/annotate-snippets-rs/issues/449)
- Fixable mark
(https://github.com/rust-lang/annotate-snippets-rs/issues/450)
- `lineno_offset` (maybe not needed if #20648 is taken care of)
- anonymized lines affects origin
(https://github.com/rust-lang/annotate-snippets-rs/issues/451)
- cut indicator (maybe not needed if we can switch to unicode renderer)
Additional hacks to minimize behavior change include
- Adjust upstream's hidden severity behavior to match Ruff's
(https://github.com/rust-lang/annotate-snippets-rs/issues/452)
- How newlines are annotated
(https://github.com/rust-lang/annotate-snippets-rs/issues/454)
- To not indent for line numbers when none will be present
(https://github.com/rust-lang/annotate-snippets-rs/issues/453)
- Enabling `simd` by default
All of the above represents work that needs to be resolved to address
#20411.
`lib.name` hack is used to simplify maintenance with the assumption that
we'll continue working towards #20411.
Behavior changes that were included
- Tweak when implicit padding lines are present
- Fix Origin's line/column is now derived from the primary annotation
- Make consistent the rendering of primary/context annotations
- Fix snippets to show up for empty source (but may degrade #24528)
- Fix handling of term width
- Fix rendering of annotations between newline and first character
(granted, maybe the caller is wrong)
- Tweak multiline annotation markers
New or upcoming features annotate-snippets features that we will be able
to leverage by moving this forward
- `AnnotationKind::Visible` to improve the quality of built-in
code-folding if Ruff were to switch to it
- `DecorStyle::Unicode`
- *(upcoming)* No-graphiscs mode for screen readers (may also be useful
for AIs)
Deferred:
- Moving related `Group`s into the same `Report`
- Using `secondary_title` for any `Group` after the first in a `Report`
- Using a `Snippet`, instead of titles, for "virtual code" like
https://github.com/astral-sh/ruff/pull/27033#discussion_r3622727236
- Switching hidden snippets to `Origin`
## Test Plan
Within cpython, I ran
```console
$ hyperfine -i "../../ruff/target-baseline/release/ruff check" "../../ruff/target-update/release/ruff check" -N -m20 --output=null
Benchmark 1: ../../ruff/target-baseline/release/ruff check
Time (mean ± σ): 27.5 ms ± 1.1 ms [User: 33.0 ms, System: 108.4 ms]
Range (min … max): 25.0 ms … 31.7 ms 94 runs
Warning: Ignoring non-zero exit code.
Benchmark 2: ../../ruff/target-update/release/ruff check
Time (mean ± σ): 28.1 ms ± 2.0 ms [User: 33.4 ms, System: 105.7 ms]
Range (min … max): 24.8 ms … 38.9 ms 102 runs
Warning: Ignoring non-zero exit code.
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use t
he '--warmup' or '--prepare' options.
Summary
../../ruff/target-baseline/release/ruff check ran
1.03 ± 0.08 times faster than ../../ruff/target-update/release/ruff check
```
Before:
<img width="486" height="583" alt="image"
src="https://github.com/user-attachments/assets/2d94dabf-b28d-48c1-b17c-52aac59fd57e"
/>
After:
<img width="486" height="583" alt="image"
src="https://github.com/user-attachments/assets/b388ada3-e3d7-4ba0-b90b-1330eb81f4be"
/>
## Summary
Reject unparenthesized `yield` expressions after the first element of an
expression list:
```python
def generator():
1, yield 2
```
The parser previously reused the yield-enabled expression context for
every tuple element. Python's grammar only allows the initial
simple-statement alternative to be a `yield` expression; later
comma-separated elements use the ordinary starred-expression grammar.
This clears the yield allowance for subsequent elements while preserving
valid parenthesized nested yields.
## Summary
Reject bare starred targets in comprehensions:
```python
[item for *items in source]
```
The semantic syntax checker already rejects a bare starred target in an
ordinary `for` statement. This applies the same validation to
comprehension clauses while preserving valid tuple and list unpacking
targets such as `for *items, in source`.
## Summary
Reject `__debug__` as a lambda parameter name:
```python
lambda __debug__: 0
```
Function parameters already use the semantic syntax check that prevents
assignments to `__debug__`, but lambda parameters only checked for
duplicate names. This applies the existing identifier validation
consistently to both forms.
## Summary
Reject parenthesized star imports such as:
```python
from module import (*)
```
Python only permits a bare star in `from` imports. The parser previously
accepted the parenthesized form because star aliases were parsed
independently of the surrounding import-list delimiters.
This adds the missing parser validation while preserving ordinary
parenthesized imports and bare star imports.
## Summary
Mapping-pattern duplicate detection and F601 compare keys through
`HashableExpr`, but numeric normalization previously only recognized
values equivalent to `True` or `False`. This meant that other values
Python treats as the same dictionary key were considered distinct:
```python
match value:
case {2: first, 2.0: second}: ...
```
This adds a dedicated numeric representation for integer, float,
boolean, and complex literals so that exactly integral floats compare
with equivalent integers, including signed forms and zero-imaginary
complex values. Tuple normalization is limited to statically known
values so dynamic tuple elements do not introduce false-positive F601
diagnostics.
Numeric normalization uses the existing `u64` integer representation.
Larger integers retain structural comparison, avoiding
arbitrary-precision conversion in parser and linter hot paths.
## Summary
Python reserves `_` for wildcard patterns and does not allow it as the
binding target in either of these forms:
```python
match value:
case 1 as _:
pass
case {**_}:
pass
```
We currently recover an AST for both forms without reporting a parser
error. Both are grammar errors in CPython, so this change validates the
capture target while parsing `as` and mapping patterns and reports
`cannot use '_' as a target` on the underscore.
We previously tried to address the mapping-pattern case in
[#6838](https://github.com/astral-sh/ruff/pull/6838) by representing the
rest target as a general `PatternMatchAs` node. This keeps the existing
`Identifier` representation and recovered AST, including statements
after the invalid match, while rejecting the invalid target at the
parser boundary.
## Summary
Class definitions do not allow a bare generator expression as a base,
even though the same syntax is valid as the sole argument in a function
call. We currently parse the class form without an error:
```python
class C(base for base in bases):
pass
```
This change reuses the existing unparenthesized-generator diagnostic for
class arguments while continuing to accept explicitly parenthesized or
unpacked generator expressions:
```python
class C((base for base in bases)): ...
class C(*(base for base in bases)): ...
```
## Summary
Mapping patterns reject duplicate literal keys, but duplicate detection
compared AST shapes instead of Python hash equality. As a result, values
that Python treats as the same key were accepted:
```python
match value:
case {1.0: first, True: second}: ...
```
This switches mapping-key detection to `HashableExpr` and extends
boolean-equivalent normalization through signed numeric expressions and
complex values with a zero imaginary part. We now report duplicate-key
syntax errors for `0`/`False`, `1.0`/`True`, `-0`/`False`, and `1 +
0j`/`True`, matching CPython.
## Summary
Function type parameter lists did not run the same default-order
validation as class and type-alias parameter lists. As a result, we
accepted functions with a required type parameter after one with a
default:
```python
def f[T = int, U](): ...
```
This reuses the existing type-parameter ordering check for function
definitions, so we now report the same semantic syntax error that we
already emit for equivalent classes and type aliases.
## Summary
Python permits at most one starred name in a sequence pattern, but our
parser currently accepts multiple:
```python
match value:
case [*head, middle, *tail]:
pass
```
This tracks whether a sequence pattern already contains a star and
reports `invalid-syntax` on each later starred name. The diagnostic
points to the additional starred pattern and uses CPython's error
message, while keeping the recovered AST available to downstream tools.
## Summary
Stacks on #25726 and enables Clippy's `iter_over_hash_type` restriction
lint across the workspace, so new hash-map and hash-set iterations must
make their ordering assumptions explicit.
Most existing loops are order-independent and use a site-local
`#[expect(clippy::iter_over_hash_type)]`. At the order-sensitive sites,
this sorts before iteration. (In practice, that should stabilize some
parser fixtures, configuration diagnostics, union normalization, etc.).
This lint is kind of annoying, but given how much we want to crack down
on non-determinism in ty, it seems worthwhile to enable IMO.
## Summary
Python rejects assignment expressions in several comprehension contexts,
but we previously only reported the case in which a walrus rebound a
comprehension variable.
As a result, we accepted invalid walruses in the iterable and class-body
portions clause of a comprehension, and missed some rebinding cases in
filter clauses and nested comprehensions. It turns out the rules are
really complicated!
For example, we now report the two newly modeled syntax errors:
```py
# error: [invalid-syntax] "assignment expression cannot be used in a comprehension iterable expression"
[x for x in (values := [1])]
class C:
# error: [invalid-syntax] "assignment expression within a comprehension cannot be used in a class body"
[(value := item) for item in [1]]
```
We also catch previously missed instances of the existing rebinding
error:
```py
# error: [invalid-syntax] "assignment expression cannot rebind comprehension variable"
[x for x in [1] if (x := 0)]
```
In general, I decided not to worry about emitting multiple diagnostics
for a single error. These are rare, they're fatal, and it added a lot of
complexity (so in some cases we may emit two errors for a walrus that is
invalid for two reasons).
Fixes#25098.
This adds full support for PEP 798 through the parser, linter,
formatter, and type checker.
There are a lot of changes but they are mostly straightforward. Let me
know though if it's better to split this up.
## Summary
Before Python 3.12, a replacement field in an f-string can span multiple
lines only if the outer f-string is triple-quoted. This was relaxed in
Python 3.12, but we weren't rejecting these as syntactically invalid on
earlier versions.
Closes https://github.com/astral-sh/ruff/issues/24348.
## Summary
We now reject cases like:
- `lazy import ...` inside functions, including `async def`
- `lazy from ... import ...` inside functions, including `async def`
- `lazy import ...` and `lazy from ... import ...` inside class bodies
- `lazy import ...` and `lazy from ... import ...` anywhere inside a
`try` statement, including `except` / `except*`
- `lazy from ... import *` anywhere
- `lazy from __future__ import ...` anywhere
A follow-up to #23755.
See: https://github.com/astral-sh/ruff/issues/21305.
## Summary
Previously, `Parameters::visit_source_order` used `ParametersIterator`
which visits in type-based order. For well-formed Python code, this
matches source order; but error recovery can produce ASTs like `def
foo(**kwargs, a):`.
## Summary
Fixes#22528.
The parser incorrectly treats `match[0]: int` and `match [x, y, z]:
{dict}` as the start of a `match` statement, producing a false syntax
error. The issue is that the parser checks only for a colon after the
subject expression to confirm a match statement, but annotated
assignments with `match` as a variable name also have a colon.
This PR adds a `self.peek() == TokenKind::Newline` guard to the colon
check, so that `match` is only treated as a keyword when the colon is
followed by a newline (which is required for a match statement's body).
When the colon is followed by something else (like a type annotation
value), it falls through to the soft-keyword-as-identifier path.
## Test Plan
All 42 existing match-related parser tests pass. No snapshot changes
needed. Clippy clean.
Reproduction from the issue:
```python
# These should parse without error:
match[0]: int
match [x, y, z]: {0: "a", 1: "b"}
```
## Summary
Fixes#21750 and a related bug in `PLE1142`. We were not properly
considering generators to be valid `await` contexts, which caused the
`F704` issue. One of the tests I added for this also uncovered an issue
in `PLE1142` for comprehensions nested within async generators because
we were only checking the current scope rather than traversing the
nested context.
## Test Plan
Both of these rules are implemented as semantic syntax errors, so I
added tests (and fixes) in both Ruff and ty.
## Summary
Fixes a panic when parsing IPython escape commands with `Help` kind
(`?`) in expression contexts. The parser now reports an error instead of
panicking.
Fixes#21465.
## Problem
The parser panicked with `unreachable!()` in
`parse_ipython_escape_command_expression` when encountering escape
commands with `Help` kind (`?`) in expression contexts, where only
`Magic` (`%`) and `Shell` (`!`) are allowed.
## Approach
Replaced the `unreachable!()` panic with error handling that adds a
`ParseErrorType::OtherError` and continues parsing, returning a valid
AST node with the error attached.
## Test Plan
Added `test_ipython_escape_command_in_with_statement` and
`test_ipython_help_escape_command_as_expression` to verify the fix.
---------
Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
Summary
--
Fixes#21360 by using the union of names instead of overwriting them, as
Micha suggested originally on #21104.
This avoids overwriting the `n` name in the `Subscript` by the empty set
of names visited in the nested OR pattern before visiting the other arm
of the outer OR pattern.
Test Plan
--
A new inline test case taken from the issue
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
This PR ports PLE0117 as a semantic syntax error.
## Test Plan
<!-- How was it tested? -->
Tests previously written
---------
Signed-off-by: 11happy <soni5happy@gmail.com>
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
The parser currently uses single quotes to wrap tokens. This is
inconsistent with the rest of ruff/ty, which use backticks.
For example, see the inconsistent diagnostics produced in this simple
example: https://play.ty.dev/0a9d6eab-6599-4a1d-8e40-032091f7f50f
Consistently wrapping tokens in backticks produces uniform diagnostics.
Following the style decision of #723, in #2889 some quotes were already
switched into backticks.
This is also in line with Rust's guide on diagnostics
(https://rustc-dev-guide.rust-lang.org/diagnostics.html#diagnostic-structure):
> When code or an identifier must appear in a message or label, it
should be surrounded with backticks
## Summary
Fixes#21101 by storing the child visitor's names in the parent visitor.
This makes sure that `visitor.names` on line 1818 isn't empty after we
visit a nested OR pattern.
## Test Plan
New inline test cases derived from the issue,
[playground](https://play.ruff.rs/7b6439ac-ee8f-4593-9a3e-c2aa34a595d0)
Charlie Marsh