[ty] Check PEP 723 scripts in isolation (#27462)

## Summary
This PR extends the PEP 723 configuration support to `ty.environment`
settings. That is:

* Scripts are checked with a separate virtual environment and search
paths
* Scripts can configure a different Python version
* Scripts can configure a different Python platform

Our long-term plan is to use uv to create the script's virtual
environment. We don't do this in this PR. Instead, we:

* Respect `VIRTUAL_ENV`, LSP fallback environment (the selected
environment), global available Python installation. Unlike for projects,
ty does not search for a local `.venv` folder
* Unlike for projects, scripts default to an empty `environment.root`
because scripts are considered to work in isolation. Users can customize
`environment.root`.


Out of scope for this PR but things we should look into next:

* Report a diagnostic if a script's metadata fails to parse instead of
falling back to the project's configuration.
* Add support for the Python environment extension's file-specific
virtual env (so that you can select a virtual env for a specific file)
* Preserve source ranges for configuration options from script metadata
(shouldn't be too hard but the PR is already large as it is)
* Integrating with uv. I expect that this will require significant
changes to the implementation to provide good UX, but I like to tackle
problems one step at a time (it also makes it easier to explain why we
changed something).
* We currently don't watch for changes in script search paths. That
means, results in the LSP or when using `ty check --watch` could get
outdated

## Test Plan
Testing: Added integration and language-server coverage and verified the
relevant unit, CLI, server, and Markdown suites.
This commit is contained in:
Micha Reiser authored and GitHub committed 2026-08-14 13:24:27 +02:00
1 parent bf15c8cfcc
commit a103832c28
30 files changed
+1882 -296

No files matched your search

+1
View File
@@ -104,6 +104,7 @@ impl std::fmt::Display for RuleResolutionError {
};
let source = match &source {
ValueSource::File(path) => format_args!("`{}`", path.as_path()),
ValueSource::ScriptMetadata(_) => format_args!("script metadata"),
ValueSource::Cli => format_args!("the CLI"),
ValueSource::Editor => format_args!("the editor configuration"),
ValueSource::UvWorkspace => format_args!("uv workspace metadata"),
+20 -6
View File
@@ -8,7 +8,9 @@ use std::sync::Arc;
use serde::{Deserialize, Deserializer};
use toml::Spanned;
use ruff_db::system::{SystemPath, SystemPathBuf};
use ruff_db::Db;
use ruff_db::files::{File, system_path_to_file};
use ruff_db::system::SystemPathBuf;
use ruff_text_size::{TextRange, TextSize};
#[derive(Clone, Debug, PartialEq)]
@@ -20,6 +22,12 @@ pub enum ValueSource {
/// created when loading the configuration.
File(Arc<SystemPathBuf>),
/// Value loaded from inline metadata in a standalone script.
///
/// Unlike project configuration, scripts are parsed after the database exists, so their
/// existing Salsa file can be retained directly, including for virtual files.
ScriptMetadata(File),
/// The value comes from a CLI argument, while it's left open if specified using a short argument,
/// long argument (`--extra-paths`) or `--config key=value`.
Cli,
@@ -35,9 +43,11 @@ pub enum ValueSource {
}
impl ValueSource {
pub fn file(&self) -> Option<&SystemPath> {
/// Resolves the file containing this setting, if its source is file-backed.
pub fn file(&self, db: &dyn Db) -> Option<File> {
match self {
ValueSource::File(path) => Some(&**path),
ValueSource::File(path) => system_path_to_file(db, &**path).ok(),
ValueSource::ScriptMetadata(file) => Some(*file),
ValueSource::Cli => None,
ValueSource::Editor => None,
ValueSource::UvWorkspace => None,
@@ -140,15 +150,19 @@ where
impl<T> RangedValue<T> {
pub fn new(value: T, source: ValueSource) -> Self {
Self::with_range(value, source, TextRange::default())
Self {
value,
source,
range: None,
}
}
pub fn cli(value: T) -> Self {
Self::with_range(value, ValueSource::Cli, TextRange::default())
Self::new(value, ValueSource::Cli)
}
pub fn python_extension(value: T) -> Self {
Self::with_range(value, ValueSource::Editor, TextRange::default())
Self::new(value, ValueSource::Editor)
}
fn with_range(value: T, source: ValueSource, range: TextRange) -> Self {
+10
View File
@@ -364,6 +364,10 @@ your environment from an activated Conda environment, and will look for a `.venv
in the project root if none of the above apply. Failing that, ty will look for a `python3`
or `python` binary available in `PATH`.
Scripts with inline metadata use their own Python environment. They can use an explicitly
configured environment, an activated environment, or an environment selected by the editor.
Unlike projects, they do not automatically use a `.venv` directory.
[`sys.prefix`]: https://docs.python.org/3/library/sys.html#sys.prefix
**Default value**: `null`
@@ -446,6 +450,9 @@ to determine a value:
and attempt to infer the Python version of that environment
3. Fall back to the default value (see below)
Scripts with inline metadata use their `requires-python` field instead of
`project.requires-python`. They do not inherit the Python version of the enclosing project.
For some language features, ty can also understand conditionals based on comparisons
with `sys.version_info`. These are commonly found in typeshed, for example,
to reflect the differing contents of the standard library across Python versions.
@@ -486,6 +493,9 @@ if they exist and are not packages (i.e. they do not contain `__init__.py` or `_
* `./<project-name>` (if a `./<project-name>/<project-name>` directory exists)
* `./python`
Scripts with inline metadata have no first-party roots by default because they are
single-file programs. Set `root = ["."]` to allow importing local modules.
**Default value**: `null`
**Type**: `list[str]`
+797 -26
View File
@@ -49,6 +49,82 @@ fn project_settings_and_overrides_do_not_apply() -> anyhow::Result<()> {
Ok(())
}
#[test]
fn verbose_rule_diagnostics_identify_script_metadata() -> anyhow::Result<()> {
let case = CliTest::with_file(
"script.py",
r#"
# /// script
# [tool.ty.rules]
# unresolved-reference = "warn"
# ///
print(missing)
"#,
)?;
assert_cmd_snapshot!(case.command().arg("--verbose"), @"
success: false
exit_code: 1
----- stdout -----
warning[unresolved-reference]: Name `missing` used when not defined
--> script.py:7:7
|
7 | print(missing)
| ^^^^^^^
info: rule `unresolved-reference` was selected in script metadata
Found 1 diagnostic
----- stderr -----
INFO Indexed 1 file(s) in 0.000s
");
Ok(())
}
#[test]
fn python_version_diagnostics_identify_script_metadata() -> anyhow::Result<()> {
let case = CliTest::with_file(
"script.py",
r#"
# /// script
# requires-python = ">=3.12"
# ///
PythonFinalizationError
"#,
)?;
assert_cmd_snapshot!(case.command(), @"
success: false
exit_code: 1
----- stdout -----
error[unresolved-reference]: Name `PythonFinalizationError` used when not defined
--> script.py:6:1
|
6 | PythonFinalizationError
| ^^^^^^^^^^^^^^^^^^^^^^^
info: `PythonFinalizationError` was added as a builtin in Python 3.13
info: Python 3.12 was assumed when resolving types because it was specified in script metadata
Found 1 diagnostic
----- stderr -----
");
assert_cmd_snapshot!(case.command().arg("--output-format").arg("concise"), @"
success: false
exit_code: 1
----- stdout -----
script.py:6:1: error[unresolved-reference] Name `PythonFinalizationError` used when not defined
Found 1 diagnostic
----- stderr -----
");
Ok(())
}
#[test]
fn metadata_without_tool_ty_uses_default_settings() -> anyhow::Result<()> {
let case = CliTest::with_files([
@@ -97,32 +173,22 @@ fn metadata_without_tool_ty_uses_default_settings() -> anyhow::Result<()> {
#[test]
fn environment_options() -> anyhow::Result<()> {
// TODO: This is not yet supported, but we should support this.
let case = CliTest::with_files([
(
"pyproject.toml",
r#"
[tool.ty.environment]
python-version = "3.12"
"#,
),
(
"script.py",
r#"
# /// script
# requires-python = ">=3.7"
#
# [tool.ty.environment]
# python-version = "3.7"
# ///
let case = CliTest::with_file(
"script.py",
r#"
# /// script
# requires-python = ">=3.13"
#
# [tool.ty.environment]
# python-version = "3.11"
# ///
import sys
from typing import reveal_type
import sys
from typing import reveal_type
reveal_type(sys.version_info[:2] == (3, 12))
"#,
),
])?;
reveal_type(sys.version_info[:2])
"#,
)?;
assert_cmd_snapshot!(case.command(), @"
success: true
@@ -131,8 +197,8 @@ fn environment_options() -> anyhow::Result<()> {
info[revealed-type]: Revealed type
--> script.py:12:13
|
12 | reveal_type(sys.version_info[:2] == (3, 12))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `Literal[True]`
12 | reveal_type(sys.version_info[:2])
| ^^^^^^^^^^^^^^^^^^^^ `tuple[Literal[3], Literal[11]]`
Found 1 diagnostic
@@ -403,3 +469,708 @@ fn explicit_config_replaces_inline_metadata() -> anyhow::Result<()> {
Ok(())
}
#[test]
fn explicit_config_replaces_the_script_environment() -> anyhow::Result<()> {
let case = CliTest::with_files([
(
"explicit.toml",
r#"
[environment]
python-version = "3.12"
python-platform = "linux"
"#,
),
(
"script.py",
r#"
# /// script
# requires-python = ">=3.13"
# [tool.ty.environment]
# python-platform = "win32"
# ///
import sys
from typing import reveal_type
reveal_type(sys.version_info[:2])
reveal_type(sys.platform)
"#,
),
])?;
assert_cmd_snapshot!(case.command().arg("--config-file").arg("explicit.toml"), @r#"
success: true
exit_code: 0
----- stdout -----
info[revealed-type]: Revealed type
--> script.py:11:13
|
11 | reveal_type(sys.version_info[:2])
| ^^^^^^^^^^^^^^^^^^^^ `tuple[Literal[3], Literal[12]]`
info[revealed-type]: Revealed type
--> script.py:12:13
|
12 | reveal_type(sys.platform)
| ^^^^^^^^^^^^ `Literal["linux"]`
Found 2 diagnostics
----- stderr -----
"#);
Ok(())
}
#[test]
fn cli_arguments_override_script_environment() -> anyhow::Result<()> {
let case = CliTest::with_file(
"script.py",
r#"
# /// script
# requires-python = ">=3.13"
# [tool.ty.environment]
# python-platform = "win32"
# ///
import sys
from typing import reveal_type
reveal_type(sys.version_info[:2])
reveal_type(sys.platform)
"#,
)?;
assert_cmd_snapshot!(
case.command()
.arg("--python-version")
.arg("3.12")
.arg("--python-platform")
.arg("linux"),
@r#"
success: true
exit_code: 0
----- stdout -----
info[revealed-type]: Revealed type
--> script.py:11:13
|
11 | reveal_type(sys.version_info[:2])
| ^^^^^^^^^^^^^^^^^^^^ `tuple[Literal[3], Literal[12]]`
info[revealed-type]: Revealed type
--> script.py:12:13
|
12 | reveal_type(sys.platform)
| ^^^^^^^^^^^^ `Literal["linux"]`
Found 2 diagnostics
----- stderr -----
"#
);
Ok(())
}
#[test]
fn script_version_and_platform_are_isolated_from_project_configuration() -> anyhow::Result<()> {
let case = CliTest::with_files([
(
"pyproject.toml",
r#"
[tool.ty.environment]
python-version = "3.12"
python-platform = "linux"
"#,
),
(
"script.py",
r#"
# /// script
# requires-python = ">=3.13"
# [tool.ty.environment]
# python-version = "3.11"
# python-platform = "win32"
# ///
import sys
from typing import reveal_type
reveal_type(sys.version_info[:2])
reveal_type(sys.platform)
"#,
),
])?;
assert_cmd_snapshot!(case.command(), @r#"
success: true
exit_code: 0
----- stdout -----
info[revealed-type]: Revealed type
--> script.py:12:13
|
12 | reveal_type(sys.version_info[:2])
| ^^^^^^^^^^^^^^^^^^^^ `tuple[Literal[3], Literal[11]]`
info[revealed-type]: Revealed type
--> script.py:13:13
|
13 | reveal_type(sys.platform)
| ^^^^^^^^^^^^ `Literal["win32"]`
Found 2 diagnostics
----- stderr -----
"#);
Ok(())
}
#[test]
fn python_requirement_overrides_user_configuration() -> anyhow::Result<()> {
let case = CliTest::with_file(
"script.py",
r#"
# /// script
# requires-python = ">=3.13"
# ///
import sys
from typing import reveal_type
reveal_type(sys.version_info[:2])
"#,
)?;
case.write_file(
case.user_config_directory().join("ty/ty.toml"),
r#"
[environment]
python-version = "3.12"
"#,
)?;
assert_cmd_snapshot!(case.command(), @"
success: true
exit_code: 0
----- stdout -----
info[revealed-type]: Revealed type
--> script.py:9:13
|
9 | reveal_type(sys.version_info[:2])
| ^^^^^^^^^^^^^^^^^^^^ `tuple[Literal[3], Literal[13]]`
Found 1 diagnostic
----- stderr -----
");
Ok(())
}
#[test]
fn scripts_have_no_implicit_first_party_roots() -> anyhow::Result<()> {
let case = CliTest::with_files([
("shared.py", "value = 1\n"),
("src/layout_dependency.py", "value = 1\n"),
("scripts/local_dependency.py", "value = 1\n"),
(
"scripts/script.py",
r#"
# /// script
# dependencies = []
# ///
from layout_dependency import value as layout_value
from local_dependency import value as local_value
from shared import value
"#,
),
])?;
assert_cmd_snapshot!(case.command(), @"
success: false
exit_code: 1
----- stdout -----
error[unresolved-import]: Cannot resolve imported module `layout_dependency`
--> scripts/script.py:6:6
|
6 | from layout_dependency import value as layout_value
| ^^^^^^^^^^^^^^^^^
info: Searched in the following paths during module resolution:
info: 1. vendored://stdlib (stdlib typeshed stubs vendored by ty)
info: make sure your Python environment is properly configured: https://docs.astral.sh/ty/modules/#python-environment
error[unresolved-import]: Cannot resolve imported module `local_dependency`
--> scripts/script.py:7:6
|
7 | from local_dependency import value as local_value
| ^^^^^^^^^^^^^^^^
info: Searched in the following paths during module resolution:
info: 1. vendored://stdlib (stdlib typeshed stubs vendored by ty)
info: make sure your Python environment is properly configured: https://docs.astral.sh/ty/modules/#python-environment
error[unresolved-import]: Cannot resolve imported module `shared`
--> scripts/script.py:8:6
|
8 | from shared import value
| ^^^^^^
info: Searched in the following paths during module resolution:
info: 1. vendored://stdlib (stdlib typeshed stubs vendored by ty)
info: make sure your Python environment is properly configured: https://docs.astral.sh/ty/modules/#python-environment
Found 3 diagnostics
----- stderr -----
");
Ok(())
}
#[test]
fn configured_source_roots_and_extra_paths_are_relative_to_the_script() -> anyhow::Result<()> {
let case = CliTest::with_files([
("scripts/source/first_party.py", "value = 1\n"),
("scripts/extra/dependency.py", "value = 1\n"),
(
"scripts/script.py",
r#"
# /// script
# [tool.ty.environment]
# root = ["./source"]
# extra-paths = ["./extra"]
# ///
from dependency import value as dependency
from first_party import value as first_party
"#,
),
])?;
assert_cmd_snapshot!(case.command(), @"
success: true
exit_code: 0
----- stdout -----
All checks passed!
----- stderr -----
");
Ok(())
}
#[test]
fn project_search_paths_do_not_apply_to_scripts() -> anyhow::Result<()> {
let case = CliTest::with_files([
(
"pyproject.toml",
r#"
[tool.ty.environment]
root = ["./project-source"]
extra-paths = ["./project-extra"]
"#,
),
("project-source/project_only.py", "value = 1\n"),
("project-extra/extra_only.py", "value = 1\n"),
(
"ordinary.py",
"from extra_only import value as extra\nfrom project_only import value as project\n",
),
(
"scripts/script.py",
r#"
# /// script
# dependencies = []
# ///
from extra_only import value as extra
from project_only import value as project
"#,
),
])?;
assert_cmd_snapshot!(case.command(), @"
success: false
exit_code: 1
----- stdout -----
error[unresolved-import]: Cannot resolve imported module `extra_only`
--> scripts/script.py:6:6
|
6 | from extra_only import value as extra
| ^^^^^^^^^^
info: Searched in the following paths during module resolution:
info: 1. vendored://stdlib (stdlib typeshed stubs vendored by ty)
info: make sure your Python environment is properly configured: https://docs.astral.sh/ty/modules/#python-environment
error[unresolved-import]: Cannot resolve imported module `project_only`
--> scripts/script.py:7:6
|
7 | from project_only import value as project
| ^^^^^^^^^^^^
info: Searched in the following paths during module resolution:
info: 1. vendored://stdlib (stdlib typeshed stubs vendored by ty)
info: make sure your Python environment is properly configured: https://docs.astral.sh/ty/modules/#python-environment
Found 2 diagnostics
----- stderr -----
");
Ok(())
}
#[test]
fn shared_imports_use_each_scripts_platform() -> anyhow::Result<()> {
let case = CliTest::with_files([
(
"shared.py",
r#"
import sys
if sys.platform == "win32":
value = "windows"
else:
value = "other"
"#,
),
(
"windows.py",
r#"
# /// script
# [tool.ty.environment]
# extra-paths = ["."]
# python-platform = "win32"
# ///
from shared import value
from typing import reveal_type
reveal_type(value)
"#,
),
(
"linux.py",
r#"
# /// script
# [tool.ty.environment]
# extra-paths = ["."]
# python-platform = "linux"
# ///
from shared import value
from typing import reveal_type
reveal_type(value)
"#,
),
])?;
assert_cmd_snapshot!(case.command(), @r#"
success: true
exit_code: 0
----- stdout -----
info[revealed-type]: Revealed type
--> linux.py:11:13
|
11 | reveal_type(value)
| ^^^^^ `Literal["other"]`
info[revealed-type]: Revealed type
--> windows.py:11:13
|
11 | reveal_type(value)
| ^^^^^ `Literal["windows"]`
Found 2 diagnostics
----- stderr -----
"#);
Ok(())
}
#[test]
fn inherited_file_settings_are_relative_to_the_script() -> anyhow::Result<()> {
let case = CliTest::with_files([
("user-extra/project_dependency.py", "value = 1\n"),
("scripts/user-extra/user_dependency.py", "value = 1\n"),
("cli-extra/cli_dependency.py", "value = 1\n"),
(
"scripts/script.py",
r#"
# /// script
# dependencies = []
# ///
from user_dependency import value as user_value
from cli_dependency import value as cli_value
"#,
),
])?;
case.write_file(
case.user_config_directory().join("ty/ty.toml"),
r#"
[environment]
extra-paths = ["./user-extra"]
"#,
)?;
assert_cmd_snapshot!(case.command().arg("--extra-search-path").arg("./cli-extra"), @"
success: true
exit_code: 0
----- stdout -----
All checks passed!
----- stderr -----
");
Ok(())
}
#[test]
fn scripts_do_not_use_an_inactive_project_environment() -> anyhow::Result<()> {
let dependency = if cfg!(windows) {
".venv/Lib/site-packages/project_dependency.py"
} else {
".venv/lib/python3.13/site-packages/project_dependency.py"
};
let case = CliTest::with_files([
(".venv/pyvenv.cfg", "home = ./\nversion = 3.13\n"),
(dependency, "value = 1\n"),
(
"scripts/script.py",
r#"
# /// script
# dependencies = []
# ///
from project_dependency import value
"#,
),
])?;
assert_cmd_snapshot!(case.command(), @"
success: false
exit_code: 1
----- stdout -----
error[unresolved-import]: Cannot resolve imported module `project_dependency`
--> scripts/script.py:6:6
|
6 | from project_dependency import value
| ^^^^^^^^^^^^^^^^^^
info: Searched in the following paths during module resolution:
info: 1. vendored://stdlib (stdlib typeshed stubs vendored by ty)
info: make sure your Python environment is properly configured: https://docs.astral.sh/ty/modules/#python-environment
Found 1 diagnostic
----- stderr -----
");
Ok(())
}
#[test]
fn scripts_use_an_activated_virtual_environment() -> anyhow::Result<()> {
let dependency = if cfg!(windows) {
".venv/Lib/site-packages/project_dependency.py"
} else {
".venv/lib/python3.13/site-packages/project_dependency.py"
};
let case = CliTest::with_files([
(".venv/pyvenv.cfg", "home = ./\nversion = 3.13\n"),
(dependency, "value = 1\n"),
(
"scripts/script.py",
r#"
# /// script
# dependencies = []
# ///
from project_dependency import value
"#,
),
])?;
assert_cmd_snapshot!(case.command().env("VIRTUAL_ENV", case.root().join(".venv")), @"
success: true
exit_code: 0
----- stdout -----
All checks passed!
----- stderr -----
");
Ok(())
}
#[test]
fn invalid_python_requirement_falls_back_to_project_configuration() -> anyhow::Result<()> {
let case = CliTest::with_files([
(
"pyproject.toml",
r#"
[tool.ty.environment]
python-platform = "linux"
[tool.ty.rules]
unresolved-reference = "error"
"#,
),
(
"script.py",
r#"
# /// script
# requires-python = "<3.12"
# [tool.ty.environment]
# python-platform = "win32"
# [tool.ty.rules]
# unresolved-reference = "warn"
# ///
import sys
from typing import reveal_type
reveal_type(sys.platform)
print(missing)
"#,
),
])?;
assert_cmd_snapshot!(case.command(), @r#"
success: false
exit_code: 1
----- stdout -----
info[revealed-type]: Revealed type
--> script.py:13:13
|
13 | reveal_type(sys.platform)
| ^^^^^^^^^^^^ `Literal["linux"]`
error[unresolved-reference]: Name `missing` used when not defined
--> script.py:14:7
|
14 | print(missing)
| ^^^^^^^
Found 2 diagnostics
----- stderr -----
"#);
assert_cmd_snapshot!(case.command().arg("--output-format").arg("concise"), @r#"
success: false
exit_code: 1
----- stdout -----
script.py:13:13: info[revealed-type] Revealed type: `Literal["linux"]`
script.py:14:7: error[unresolved-reference] Name `missing` used when not defined
Found 2 diagnostics
----- stderr -----
"#);
Ok(())
}
#[test]
fn invalid_script_settings_fall_back_to_project_configuration() -> anyhow::Result<()> {
// FIXME: Scripts with invalid settings should not be checked.
let case = CliTest::with_files([
(
"pyproject.toml",
r#"
[tool.ty.environment]
python-platform = "linux"
"#,
),
(
"script.py",
r#"
# /// script
# [tool.ty.src]
# include = ["src/**test/"]
# [tool.ty.environment]
# python-platform = "win32"
# ///
import sys
from typing import reveal_type
reveal_type(sys.platform)
"#,
),
])?;
assert_cmd_snapshot!(case.command(), @r#"
success: true
exit_code: 0
----- stdout -----
info[revealed-type]: Revealed type
--> script.py:12:13
|
12 | reveal_type(sys.platform)
| ^^^^^^^^^^^^ `Literal["linux"]`
Found 1 diagnostic
----- stderr -----
"#);
Ok(())
}
#[test]
fn invalid_script_environment_falls_back_to_project_configuration() -> anyhow::Result<()> {
// FIXME: Scripts with invalid environments should not be checked.
let case = CliTest::with_files([
(
"pyproject.toml",
r#"
[tool.ty.environment]
python-version = "3.13"
python-platform = "linux"
"#,
),
(
"script.py",
r#"
# /// script
# [tool.ty.environment]
# python = "./missing-environment"
# python-version = "3.12"
# python-platform = "win32"
# ///
import sys
from typing import reveal_type
reveal_type(sys.version_info[:2])
reveal_type(sys.platform)
"#,
),
])?;
assert_cmd_snapshot!(case.command(), @r#"
success: true
exit_code: 0
----- stdout -----
info[revealed-type]: Revealed type
--> script.py:12:13
|
12 | reveal_type(sys.version_info[:2])
| ^^^^^^^^^^^^^^^^^^^^ `tuple[Literal[3], Literal[13]]`
info[revealed-type]: Revealed type
--> script.py:13:13
|
13 | reveal_type(sys.platform)
| ^^^^^^^^^^^^ `Literal["linux"]`
Found 2 diagnostics
----- stderr -----
"#);
Ok(())
}
+29 -10
View File
@@ -6,6 +6,7 @@ use std::{cmp, fmt};
pub use self::changes::ChangeResult;
use crate::CollectReporter;
use crate::metadata::settings::file_settings;
use crate::script::Script;
use crate::{ProgressReporter, Project, ProjectMetadata};
use get_size2::StandardTracker;
use ruff_db::Db as SourceDb;
@@ -527,11 +528,19 @@ impl SemanticDb for ProjectDatabase {
}
fn program_file(&self, file: File) -> ProgramFile<'_> {
self.project().program(self).program_file(self, file)
let program = match Script::for_file(self, file) {
None => self.project().program(self),
Some(script) => script.program(self),
};
program.program_file(self, file)
}
fn python_version_with_source(&self, _file: File) -> &PythonVersionWithSource {
&self.project().program_settings(self).python_version
fn python_version_with_source(&self, file: File) -> &PythonVersionWithSource {
match Script::for_file(self, file) {
None => &self.project().program_settings(self).python_version,
Some(script) => script.python_version_with_source(self),
}
}
fn rule_selection(&self, file: File) -> &RuleSelection {
@@ -642,6 +651,8 @@ pub(crate) mod testing {
use ty_python_semantic::{AnalysisSettings, PythonVersionWithSource};
use crate::db::Db;
use crate::metadata::settings::file_settings;
use crate::script::Script;
use crate::{Project, ProjectMetadata};
type Events = Arc<Mutex<Vec<salsa::Event>>>;
@@ -773,11 +784,19 @@ pub(crate) mod testing {
#[salsa::db]
impl ty_python_semantic::Db for TestDb {
fn program_file(&self, file: File) -> ProgramFile<'_> {
self.project().program(self).program_file(self, file)
let program = match Script::for_file(self, file) {
None => self.project().program(self),
Some(script) => script.program(self),
};
program.program_file(self, file)
}
fn python_version_with_source(&self, _file: File) -> &PythonVersionWithSource {
&self.project().program_settings(self).python_version
fn python_version_with_source(&self, file: File) -> &PythonVersionWithSource {
match Script::for_file(self, file) {
None => &self.project().program_settings(self).python_version,
Some(script) => script.python_version_with_source(self),
}
}
#[inline]
@@ -785,16 +804,16 @@ pub(crate) mod testing {
crate::check_file(self, file)
}
fn rule_selection(&self, _file: ruff_db::files::File) -> &RuleSelection {
self.project().rules(self)
fn rule_selection(&self, file: ruff_db::files::File) -> &RuleSelection {
file_settings(self, file).rules(self)
}
fn lint_registry(&self) -> &LintRegistry {
ty_python_semantic::default_lint_registry()
}
fn analysis_settings(&self, _file: ruff_db::files::File) -> &AnalysisSettings {
self.project().settings(self).analysis()
fn analysis_settings(&self, file: ruff_db::files::File) -> &AnalysisSettings {
file_settings(self, file).analysis(self)
}
fn verbose(&self) -> bool {
+15 -2
View File
@@ -5,6 +5,7 @@
use crate::glob::{GlobFilterCheckMode, IncludeResult};
use crate::metadata::options::OptionDiagnostic;
use crate::parallel::ParallelIteratorExt;
use crate::script::Script;
use crate::walk::{ProjectFilesFilter, ProjectFilesWalker};
#[cfg(feature = "testing")]
pub use db::testing::TestDb;
@@ -243,7 +244,7 @@ impl Project {
#[salsa::tracked(returns(copy), heap_size=ruff_memory_usage::heap_size)]
pub fn program(self, db: &dyn Db) -> Program<'_> {
Program::from_settings(db, self.program_settings(db).clone())
Program::from_settings(db, self.program_settings(db))
}
pub fn update_program(self, db: &mut dyn Db, settings: ProgramSettings) {
@@ -742,7 +743,19 @@ pub(crate) fn check_file_impl(
{
let db = AssertUnwindSafe(db);
match catch(&**db, source_file, || {
ty_python_semantic::check_file(*db, file)
let diagnostics = ty_python_semantic::check_file(*db, file)?;
let Some(script) = Script::for_file(*db, source_file) else {
return Ok(diagnostics);
};
let script_diagnostics = script.diagnostics(*db);
if script_diagnostics.is_empty() {
return Ok(diagnostics);
}
let mut diagnostics = diagnostics.into_vec();
diagnostics.extend(script_diagnostics.iter().cloned());
Ok(diagnostics.into_boxed_slice())
}) {
Ok(result) => result,
Err(diagnostic) => Ok(Box::new([diagnostic])),
+32 -23
View File
@@ -12,7 +12,8 @@ use ty_static::EnvVars;
use crate::Db;
use crate::metadata::options::{
EnvironmentOptions, OptionDiagnostic, ProgramSettingsDiagnostic, ToSettingsError,
EnvironmentOptions, OptionDiagnostic, OptionsContext, ProgramSettingsDiagnostic,
ToSettingsError,
};
use crate::metadata::pyproject::{Project, PyProject, PyProjectError, ResolveRequiresPythonError};
use crate::metadata::settings::Settings;
@@ -143,26 +144,13 @@ impl ProjectMetadata {
.map(|name| ProjectName::new(&**name))
.unwrap_or_else(|| ProjectName::new(root.file_name().unwrap_or("root")));
// If the `options` don't specify a python version but the `project.requires-python` field is set,
// use that as a lower bound instead.
if let Some(project) = project {
if options
.environment
.as_ref()
.is_none_or(|env| env.python_version.is_none())
{
let requires_python = strategy.fallback_opt(
project.resolve_requires_python_lower_bound(),
|err| {
tracing::debug!("skipping invalid requires_python lower bound: {err}");
},
)?;
if let Some(requires_python) = requires_python.flatten() {
let mut environment = options.environment.unwrap_or_default();
environment.python_version = Some(requires_python);
options.environment = Some(environment);
}
}
// If the `options` don't specify a python version but the `project.requires-python` field is set,
// use that as a lower bound instead.
strategy.fallback(
options.apply_requires_python(project.requires_python.as_ref()),
|error| tracing::debug!("skipping invalid requires_python lower bound: {error}"),
)?;
}
Ok(Self {
@@ -412,7 +400,7 @@ impl ProjectMetadata {
self.name.as_str()
}
fn options(&self) -> &Options {
pub(crate) fn options(&self) -> &Options {
&self.options
}
@@ -496,6 +484,26 @@ impl ProjectMetadata {
.chain(self.fallback_options.as_deref())
}
/// Returns the option layers applicable to a standalone script.
///
/// Scripts inherit invocation and user settings, but not the enclosing project's options or
/// Python-version settings derived from its uv workspace.
pub(crate) fn script_options_in_precedence_order<'a>(
&'a self,
options: &'a Options,
) -> impl Iterator<Item = &'a Options> {
self.override_options
.as_deref()
.into_iter()
.chain(std::iter::once(options))
.chain(
self.user_configuration
.as_deref()
.map(|(_, options)| options),
)
.chain(self.fallback_options.as_deref())
}
/// Loads the lower-precedence options from configuration files.
///
/// This includes:
@@ -567,7 +575,7 @@ impl MergedOptions<'_> {
) -> Result<(ProgramSettings, Vec<ProgramSettingsDiagnostic>), Strategy::Error<anyhow::Error>>
{
self.options.to_program_settings(
self.metadata.root(),
OptionsContext::Project(self.metadata.root()),
self.metadata.name(),
system,
vendored,
@@ -580,7 +588,8 @@ impl MergedOptions<'_> {
db: &dyn Db,
strategy: &Strategy,
) -> Result<(Settings, Vec<OptionDiagnostic>), Strategy::Error<ToSettingsError>> {
self.options.to_settings(db, self.metadata.root(), strategy)
self.options
.to_settings(db, OptionsContext::Project(self.metadata.root()), strategy)
}
}
+108 -60
View File
@@ -1,5 +1,6 @@
use crate::Db;
use crate::glob::{ExcludeFilter, IncludeExcludeFilter, IncludeFilter, PortableGlobKind};
use crate::metadata::pyproject::{ResolveRequiresPythonError, resolve_requires_python_lower_bound};
use crate::metadata::python_version::SupportedPythonVersion;
use crate::metadata::settings::{OverrideSettings, SrcSettings};
@@ -7,12 +8,12 @@ use super::settings::{Override, Settings, TerminalSettings};
use crate::metadata::value::{RelativeGlobPattern, RelativePathBuf};
use anyhow::Context;
use ordermap::OrderMap;
use pep440_rs::VersionSpecifiers;
use ruff_db::RustDoc;
use ruff_db::diagnostic::{
Annotation, Diagnostic, DiagnosticFormat, DiagnosticId, DisplayDiagnosticConfig, Severity,
Span, SubDiagnostic, SubDiagnosticSeverity,
};
use ruff_db::files::system_path_to_file;
use ruff_db::system::{System, SystemPath, SystemPathBuf};
use ruff_db::vendored::VendoredFileSystem;
use ruff_macros::{Combine, OptionsMetadata, RustDoc};
@@ -111,13 +112,6 @@ pub struct Options {
}
impl Options {
pub(super) fn file_options(&self) -> FileOptions {
FileOptions {
rules: self.rules.clone(),
analysis: self.analysis.clone(),
}
}
pub fn from_toml_str(content: &str, source: ValueSource) -> Result<Self, TyTomlError> {
let _guard = ValueSourceGuard::new(source, true);
let mut options: Self = toml::from_str(content)?;
@@ -125,6 +119,28 @@ impl Options {
Ok(options)
}
/// Infers the Python version from `requires-python` unless it was configured explicitly.
pub(crate) fn apply_requires_python(
&mut self,
requires_python: Option<&RangedValue<VersionSpecifiers>>,
) -> Result<(), ResolveRequiresPythonError> {
if self
.environment
.as_ref()
.is_some_and(|environment| environment.python_version.is_some())
{
return Ok(());
}
if let Some(requires_python) = requires_python
&& let Some(python_version) = resolve_requires_python_lower_bound(requires_python)?
{
self.environment.get_or_insert_default().python_version = Some(python_version);
}
Ok(())
}
/// Ensures that the `all` selector is applied before per-rule selectors
/// in all rule tables (top-level and overrides).
///
@@ -164,9 +180,10 @@ impl Options {
Self::deserialize(deserializer)
}
/// Resolve configured paths and discover defaults according to the project or script context.
pub(crate) fn to_program_settings<Strategy: MisconfigurationStrategy>(
&self,
project_root: &SystemPath,
context: OptionsContext<'_>,
project_name: &str,
system: &dyn System,
vendored: &VendoredFileSystem,
@@ -196,15 +213,20 @@ impl Options {
ValueSource::File(path) => {
SysPrefixPathOrigin::ConfigFileSetting(path.clone(), python_path.range())
}
ValueSource::ScriptMetadata(_) => SysPrefixPathOrigin::ScriptMetadataSetting,
ValueSource::Editor => SysPrefixPathOrigin::Editor,
ValueSource::UvWorkspace => SysPrefixPathOrigin::UvWorkspace,
};
PythonEnvironment::new(python_path.absolute(project_root, system), origin, system)
.map_err(anyhow::Error::from)
.map(Some)
PythonEnvironment::new(
python_path.absolute(context.configuration_root(), system),
origin,
system,
)
.map_err(anyhow::Error::from)
.map(Some)
} else {
PythonEnvironment::discover(project_root, system)
PythonEnvironment::discover(context.project_root(), system)
.context("Failed to discover local Python environment")
};
@@ -277,7 +299,7 @@ impl Options {
// Safe mode is handled inside this function, so we just assume this can't fail
let search_paths = strategy.to_anyhow(self.to_search_paths(
project_root,
context,
project_name,
site_packages_paths,
real_stdlib_path,
@@ -304,7 +326,7 @@ impl Options {
#[expect(clippy::too_many_arguments)]
fn to_search_paths<Strategy: MisconfigurationStrategy>(
&self,
project_root: &SystemPath,
context: OptionsContext<'_>,
project_name: &str,
site_packages_paths: SitePackagesPaths,
real_stdlib_path: Option<SystemPathBuf>,
@@ -317,9 +339,10 @@ impl Options {
let environment_roots = if let Some(roots) = environment.root.as_deref() {
roots
.iter()
.map(|root| root.absolute(project_root, system))
.map(|root| root.absolute(context.configuration_root(), system))
.collect()
} else {
let project_root = context.configuration_root();
let mut roots = vec![];
let is_package = |dir: &SystemPath| {
system.is_file(&dir.join("__init__.py"))
@@ -375,7 +398,7 @@ impl Options {
.as_deref()
.unwrap_or_default()
.iter()
.map(|path| path.absolute(project_root, system))
.map(|path| path.absolute(context.configuration_root(), system))
.collect();
// read all the paths off the PYTHONPATH environment variable, check
@@ -421,7 +444,7 @@ impl Options {
custom_typeshed: environment
.typeshed
.as_ref()
.map(|path| path.absolute(project_root, system)),
.map(|path| path.absolute(context.configuration_root(), system)),
site_packages_paths: site_packages_paths.into_vec(),
real_stdlib_path,
};
@@ -432,7 +455,7 @@ impl Options {
pub(crate) fn to_settings<Strategy: MisconfigurationStrategy>(
&self,
db: &dyn Db,
project_root: &SystemPath,
context: OptionsContext<'_>,
strategy: &Strategy,
) -> Result<(Settings, Vec<OptionDiagnostic>), Strategy::Error<ToSettingsError>> {
let mut diagnostics = Vec::new();
@@ -451,7 +474,7 @@ impl Options {
let src_options = self.src.or_default();
let src = src_options
.to_settings(db, project_root, &mut diagnostics)
.to_settings(db, context.configuration_root(), &mut diagnostics)
.map_err(|err| ToSettingsError {
diagnostic: err,
output_format: terminal.output_format,
@@ -478,7 +501,7 @@ impl Options {
let analysis = strategy.fallback(analysis_result, |_| AnalysisSettings::default())?;
let overrides = self
.to_overrides_settings(db, project_root, &mut diagnostics)
.to_overrides_settings(db, context.configuration_root(), &mut diagnostics)
.map_err(|err| ToSettingsError {
diagnostic: err,
output_format: terminal.output_format,
@@ -534,6 +557,29 @@ impl Options {
}
}
/// The project or standalone script whose options are being resolved.
#[derive(Clone, Copy, Debug)]
pub(crate) enum OptionsContext<'a> {
Project(&'a SystemPath),
/// The directory containing a standalone script, or the working directory for a virtual script.
Script(&'a SystemPath),
}
impl<'a> OptionsContext<'a> {
fn configuration_root(self) -> &'a SystemPath {
match self {
Self::Project(root) | Self::Script(root) => root,
}
}
fn project_root(self) -> Option<&'a SystemPath> {
match self {
Self::Project(root) => Some(root),
Self::Script(_) => None,
}
}
}
fn python_version_from_config(
ranged_version: &RangedValue<SupportedPythonVersion>,
) -> PythonVersionWithSource {
@@ -544,6 +590,9 @@ fn python_version_from_config(
ValueSource::File(path) => PythonVersionSource::ConfigFile(
PythonVersionFileSource::new(path.clone(), ranged_version.range()),
),
ValueSource::ScriptMetadata(file) => PythonVersionSource::ScriptMetadata(
Span::from(*file).with_optional_range(ranged_version.range()),
),
ValueSource::Editor => PythonVersionSource::Editor,
ValueSource::UvWorkspace => PythonVersionSource::UvWorkspace,
},
@@ -637,6 +686,12 @@ fn unsupported_inferred_python_version_diagnostic(
SubDiagnosticSeverity::Info,
"The version was inferred from a configuration file.",
)),
source @ PythonVersionSource::ScriptMetadata(_) => diagnostic
.with_annotation(inferred_python_version_source_annotation(db, source))
.sub(SubDiagnostic::new(
SubDiagnosticSeverity::Info,
"The version was inferred from script metadata.",
)),
source @ PythonVersionSource::PyvenvCfgFile(_) => diagnostic
.with_annotation(inferred_python_version_source_annotation(db, source))
.sub(SubDiagnostic::new(
@@ -751,6 +806,9 @@ pub struct EnvironmentOptions {
/// * `./src`
/// * `./<project-name>` (if a `./<project-name>/<project-name>` directory exists)
/// * `./python`
///
/// Scripts with inline metadata have no first-party roots by default because they are
/// single-file programs. Set `root = ["."]` to allow importing local modules.
#[serde(skip_serializing_if = "Option::is_none")]
#[option(
default = r#"null"#,
@@ -780,6 +838,9 @@ pub struct EnvironmentOptions {
/// and attempt to infer the Python version of that environment
/// 3. Fall back to the default value (see below)
///
/// Scripts with inline metadata use their `requires-python` field instead of
/// `project.requires-python`. They do not inherit the Python version of the enclosing project.
///
/// For some language features, ty can also understand conditionals based on comparisons
/// with `sys.version_info`. These are commonly found in typeshed, for example,
/// to reflect the differing contents of the standard library across Python versions.
@@ -864,6 +925,10 @@ pub struct EnvironmentOptions {
/// in the project root if none of the above apply. Failing that, ty will look for a `python3`
/// or `python` binary available in `PATH`.
///
/// Scripts with inline metadata use their own Python environment. They can use an explicitly
/// configured environment, an activated environment, or an environment selected by the editor.
/// Unlike projects, they do not automatically use a `.venv` directory.
///
/// [`sys.prefix`]: https://docs.python.org/3/library/sys.html#sys.prefix
#[serde(skip_serializing_if = "Option::is_none")]
#[option(
@@ -1079,6 +1144,7 @@ impl Rules {
let source = rule_name.source();
let lint_source = match source {
ValueSource::File(_) => LintSource::File,
ValueSource::ScriptMetadata(_) => LintSource::ScriptMetadata,
ValueSource::Cli => LintSource::Cli,
ValueSource::Editor => LintSource::Editor,
ValueSource::UvWorkspace => LintSource::UvWorkspace,
@@ -1105,12 +1171,9 @@ impl Rules {
set_lint_level(lint);
}
Err(error) => {
// `system_path_to_file` can return `Err` if the file was deleted since the configuration
// was read. This should be rare and it should be okay to default to not showing a configuration
// file in that case.
let file = source
.file()
.and_then(|path| system_path_to_file(db, path).ok());
// The file may have been deleted since its configuration was read. In that
// case, report the diagnostic without a configuration-file annotation.
let file = source.file(db);
// TODO: Add a note if the value was configured on the CLI
let diagnostic = OptionDiagnostic::new(
@@ -1187,14 +1250,12 @@ fn build_include_filter(
));
// Add source annotation if we have source information
if let Some(source_file) = include_patterns.source().file() {
if let Ok(file) = system_path_to_file(db, source_file) {
let annotation = Annotation::primary(
Span::from(file).with_optional_range(include_patterns.range()),
)
.message("This `include` list is empty");
diagnostic = diagnostic.with_annotation(Some(annotation));
}
if let Some(file) = include_patterns.source().file(db) {
let annotation = Annotation::primary(
Span::from(file).with_optional_range(include_patterns.range()),
)
.message("This `include` list is empty");
diagnostic = diagnostic.with_annotation(Some(annotation));
}
diagnostics.push(diagnostic);
@@ -1951,13 +2012,11 @@ impl ToOverride for RangedValue<OverrideOptions> {
));
// Add source annotation if we have source information
if let Some(source_file) = self.source().file() {
if let Ok(file) = system_path_to_file(db, source_file) {
let annotation =
Annotation::primary(Span::from(file).with_optional_range(self.range()))
.message("This overrides section overrides no settings");
diagnostic = diagnostic.with_annotation(Some(annotation));
}
if let Some(file) = self.source().file(db) {
let annotation =
Annotation::primary(Span::from(file).with_optional_range(self.range()))
.message("This overrides section overrides no settings");
diagnostic = diagnostic.with_annotation(Some(annotation));
}
diagnostics.push(diagnostic);
@@ -2004,13 +2063,11 @@ impl ToOverride for RangedValue<OverrideOptions> {
));
// Add source annotation if we have source information
if let Some(source_file) = self.source().file() {
if let Ok(file) = system_path_to_file(db, source_file) {
let annotation =
Annotation::primary(Span::from(file).with_optional_range(self.range()))
.message("This overrides section applies to all files");
diagnostic = diagnostic.with_annotation(Some(annotation));
}
if let Some(file) = self.source().file(db) {
let annotation =
Annotation::primary(Span::from(file).with_optional_range(self.range()))
.message("This overrides section applies to all files");
diagnostic = diagnostic.with_annotation(Some(annotation));
}
diagnostics.push(diagnostic);
@@ -2080,15 +2137,6 @@ pub(super) struct InnerOverrideOptions {
pub(super) analysis: Option<AnalysisOptions>,
}
/// The settings that can vary between individual files.
#[derive(Debug, Default, Clone, PartialEq, Eq, Hash, Combine, get_size2::GetSize)]
pub(super) struct FileOptions {
/// Raw rule options, preserved so multiple configuration layers can be merged.
pub(super) rules: Option<Rules>,
pub(super) analysis: Option<AnalysisOptions>,
}
/// Error returned when the settings can't be resolved because of a hard error.
#[derive(Debug)]
pub struct ToSettingsError {
@@ -2263,8 +2311,8 @@ impl OptionDiagnostic {
err: impl Display,
) -> Self {
match value.source() {
ValueSource::File(file_path) => {
if let Ok(file) = system_path_to_file(db, &**file_path) {
ValueSource::File(_) | ValueSource::ScriptMetadata(_) => {
if let Some(file) = value.source().file(db) {
let concise_message = std::mem::take(&mut self.message);
self.with_concise_message(concise_message)
.with_message(format_args!("Invalid {value_label}"))
+50 -64
View File
@@ -40,14 +40,6 @@ impl PyProject {
Self::deserialize_toml(content)
}
pub(crate) fn from_toml_str_without_spans(
content: &str,
source: ValueSource,
) -> Result<Self, PyProjectError> {
let _guard = ValueSourceGuard::new(source, false);
Self::deserialize_toml(content)
}
fn deserialize_toml(content: &str) -> Result<Self, PyProjectError> {
let mut pyproject: Self = toml::from_str(content).map_err(PyProjectError::TomlSyntax)?;
// TOML tables are unordered and the `toml` crate sorts keys
@@ -79,69 +71,63 @@ pub struct Project {
pub(crate) requires_python: Option<RangedValue<VersionSpecifiers>>,
}
impl Project {
pub(super) fn resolve_requires_python_lower_bound(
&self,
) -> Result<Option<RangedValue<SupportedPythonVersion>>, ResolveRequiresPythonError> {
let Some(requires_python) = self.requires_python.as_ref() else {
return Ok(None);
};
pub(super) fn resolve_requires_python_lower_bound(
requires_python: &RangedValue<VersionSpecifiers>,
) -> Result<Option<RangedValue<SupportedPythonVersion>>, ResolveRequiresPythonError> {
tracing::debug!("Resolving requires-python constraint: `{requires_python}`");
tracing::debug!("Resolving requires-python constraint: `{requires_python}`");
let ranges = release_specifiers_to_ranges((**requires_python).clone());
let Some((lower, _)) = ranges.bounding_range() else {
return Ok(None);
};
let ranges = release_specifiers_to_ranges((**requires_python).clone());
let Some((lower, _)) = ranges.bounding_range() else {
return Ok(None);
};
let version = match lower {
// Ex) `>=3.10.1` -> `>=3.10`
Bound::Included(version) => version,
let version = match lower {
// Ex) `>=3.10.1` -> `>=3.10`
Bound::Included(version) => version,
// Ex) `>3.10.1` -> `>=3.10` or `>3.10` -> `>=3.10`
// The second example looks obscure at first but it is required because
// `3.10.1 > 3.10` is true but we only have two digits here. So including 3.10 is the
// right move. Overall, using `>` without a patch release is most likely bogus.
Bound::Excluded(version) => version,
// Ex) `>3.10.1` -> `>=3.10` or `>3.10` -> `>=3.10`
// The second example looks obscure at first but it is required because
// `3.10.1 > 3.10` is true but we only have two digits here. So including 3.10 is the
// right move. Overall, using `>` without a patch release is most likely bogus.
Bound::Excluded(version) => version,
// Ex) `<3.10` or ``
Bound::Unbounded => {
return Err(ResolveRequiresPythonError::NoLowerBound(
requires_python.to_string(),
));
}
};
// Take the major and minor version
let mut versions = version.release().iter().take(2);
let Some(major) = versions.next().copied() else {
return Ok(None);
};
let minor = versions.next().copied().unwrap_or_default();
tracing::debug!("Resolved requires-python constraint to: {major}.{minor}");
let major =
u8::try_from(major).map_err(|_| ResolveRequiresPythonError::TooLargeMajor(major))?;
let minor =
u8::try_from(minor).map_err(|_| ResolveRequiresPythonError::TooLargeMinor(minor))?;
let lower_bound = PythonVersion::from((major, minor));
let supported_version = SupportedPythonVersion::iter()
.find(|supported_version| supported_version.to_python_version() >= lower_bound);
let Some(supported_version) = supported_version else {
return Err(ResolveRequiresPythonError::NoSupportedVersion(
// Ex) `<3.10` or ``
Bound::Unbounded => {
return Err(ResolveRequiresPythonError::NoLowerBound(
requires_python.to_string(),
));
};
}
};
Ok(Some(
requires_python.clone().map_value(|_| supported_version),
))
}
// Take the major and minor version
let mut versions = version.release().iter().take(2);
let Some(major) = versions.next().copied() else {
return Ok(None);
};
let minor = versions.next().copied().unwrap_or_default();
tracing::debug!("Resolved requires-python constraint to: {major}.{minor}");
let major =
u8::try_from(major).map_err(|_| ResolveRequiresPythonError::TooLargeMajor(major))?;
let minor =
u8::try_from(minor).map_err(|_| ResolveRequiresPythonError::TooLargeMinor(minor))?;
let lower_bound = PythonVersion::from((major, minor));
let supported_version = SupportedPythonVersion::iter()
.find(|supported_version| supported_version.to_python_version() >= lower_bound);
let Some(supported_version) = supported_version else {
return Err(ResolveRequiresPythonError::NoSupportedVersion(
requires_python.to_string(),
));
};
Ok(Some(
requires_python.clone().map_value(|_| supported_version),
))
}
#[derive(Debug, Error)]
+11 -33
View File
@@ -5,8 +5,8 @@ use ty_combine::Combine;
use ty_python_semantic::AnalysisSettings;
use ty_python_semantic::lint::RuleSelection;
use crate::metadata::options::{InnerOverrideOptions, Options, OutputFormat};
use crate::script::script_metadata;
use crate::metadata::options::{InnerOverrideOptions, OutputFormat};
use crate::script::Script;
use crate::{Db, glob::IncludeExcludeFilter};
/// The resolved [`super::Options`] for the project.
@@ -125,38 +125,16 @@ impl Override {
/// Resolves the settings for a given file.
#[salsa::tracked(returns(ref), heap_size=ruff_memory_usage::heap_size)]
pub(crate) fn file_settings(db: &dyn Db, file: File) -> FileSettings {
let project = db.project();
// Ignore script settings for files that aren't checked as part of the project. Check for
// metadata first so files without metadata don't depend on the low-durability open-file set.
if let Some(script) = script_metadata(db, file)
&& crate::should_check_file(db, file)
{
let inline = script.ty().cloned().unwrap_or_default();
let metadata = project.metadata(db);
let primary = if metadata.config_file_override().is_some() {
metadata.options()
} else {
&inline
};
let mut options = metadata
.options_in_precedence_order(primary)
.map(Options::file_options);
let mut merged = options.next().unwrap_or_default();
for option in options {
merged.combine_with(option);
}
let rules = merged.rules.unwrap_or_default();
let analysis = merged.analysis.unwrap_or_default();
let rules = rules.to_rule_selection(db, &mut Vec::new());
let analysis = analysis.to_settings(db, &mut Vec::new());
return FileSettings::File(Arc::new(OverrideSettings { rules, analysis }));
if let Some(script) = Script::for_file(db, file) {
let settings = script.settings(db);
return FileSettings::File(Arc::new(OverrideSettings {
rules: settings.rules().clone(),
analysis: settings.analysis().clone(),
}));
}
let project = db.project();
let settings = project.settings(db);
let path = match file.path(db) {
@@ -249,7 +227,7 @@ fn merge_overrides(db: &dyn Db, overrides: Vec<Arc<InnerOverrideOptions>>, _: ()
/// The resolved settings for a file.
#[derive(Debug, Eq, PartialEq, Clone, get_size2::GetSize)]
pub enum FileSettings {
pub(crate) enum FileSettings {
/// The file uses the global settings.
Global,
+4 -4
View File
@@ -17,7 +17,7 @@ use crate::glob::{
/// require different anchoring:
///
/// * CLI: The path is relative to the current working directory
/// * Configuration file: The path is relative to the project's root.
/// * Configuration file: The path is relative to the project's or script's configuration root.
#[derive(
Debug,
Clone,
@@ -62,9 +62,9 @@ impl RelativePathBuf {
}
/// Resolves the absolute path for `self` based on its origin.
pub fn absolute(&self, project_root: &SystemPath, system: &dyn System) -> SystemPathBuf {
pub fn absolute(&self, configuration_root: &SystemPath, system: &dyn System) -> SystemPathBuf {
let relative_to = match self.0.source() {
ValueSource::File(_) => project_root,
ValueSource::File(_) | ValueSource::ScriptMetadata(_) => configuration_root,
ValueSource::Cli | ValueSource::Editor | ValueSource::UvWorkspace => {
system.current_directory()
}
@@ -136,7 +136,7 @@ impl RelativeGlobPattern {
kind: PortableGlobKind,
) -> Result<AbsolutePortableGlobPattern, PortableGlobError> {
let relative_to = match self.0.source() {
ValueSource::File(_) => project_root,
ValueSource::File(_) | ValueSource::ScriptMetadata(_) => project_root,
ValueSource::Cli | ValueSource::Editor | ValueSource::UvWorkspace => {
system.current_directory()
}
+280 -12
View File
@@ -1,17 +1,175 @@
use std::sync::Arc;
use ruff_db::Db;
use pep440_rs::VersionSpecifiers;
use ruff_db::Db as SourceDb;
use ruff_db::diagnostic::Diagnostic;
use ruff_db::files::File;
use ruff_db::source::source_text;
use ruff_db::system::SystemPathBuf;
use ruff_python_ast::script::ScriptTag;
use ruff_ranged_value::ValueSource;
use ruff_ranged_value::{RangedValue, ValueSource, ValueSourceGuard};
use serde::Deserialize;
use ty_combine::Combine;
use ty_python_core::program::{FallibleStrategy, Program, ProgramSettings};
use ty_python_semantic::PythonVersionWithSource;
use crate::metadata::pyproject::PyProject;
use crate::metadata::options::{Options, OptionsContext};
use crate::metadata::pyproject::Tool;
use crate::metadata::settings::Settings;
use crate::{Db, ProjectMetadata};
/// A standalone PEP 723 script and its resolved settings.
#[salsa::tracked(debug, heap_size=ruff_memory_usage::heap_size)]
pub(crate) struct Script<'db> {
#[returns(copy)]
pub(crate) file: File,
#[tracked]
#[returns(ref)]
pub(crate) settings: Settings,
#[tracked]
#[returns(copy)]
pub(crate) program: Program<'db>,
#[tracked]
#[returns(ref)]
pub(crate) python_version_with_source: PythonVersionWithSource,
#[tracked]
#[returns(deref)]
pub(crate) diagnostics: Box<[Diagnostic]>,
}
impl<'db> Script<'db> {
/// Returns the script for `file` without creating a second Salsa memo for ordinary files.
pub(crate) fn for_file(db: &'db dyn Db, file: File) -> Option<Self> {
// Most files are not scripts. Check the existing metadata query first so ordinary files
// do not also allocate a tracked `script` memo just to cache another `None`.
script_metadata(db, file)?;
script(db, file)
}
}
impl get_size2::GetSize for Script<'_> {}
/// Resolve the `Script` for `file` if it has a PEP 723 metadata block or `None` otherwise.
#[salsa::tracked(returns(copy), heap_size=ruff_memory_usage::heap_size)]
pub(crate) fn script(db: &dyn Db, file: File) -> Option<Script<'_>> {
// Files without script metadata must not depend on the low-durability open-file set.
let metadata = script_metadata(db, file)?;
// Never treat third-party files as scripts.
if !crate::should_check_file(db, file) {
return None;
}
let configuration_root = file
.path(db)
.as_system_path()
.and_then(|path| path.parent())
.unwrap_or_else(|| db.system().current_directory());
let context = OptionsContext::Script(configuration_root);
let project_metadata = db.project().metadata(db);
let mut diagnostics = Vec::new();
// FIXME: Report configuration errors as diagnostics and skip checking the script entirely so
// that fixes cannot be applied using the enclosing project's configuration.
let options = resolve_script_options(project_metadata, metadata)?;
let settings = resolve_script_settings(db, &options, context, &mut diagnostics)?;
let program_settings = resolve_script_program_settings(
db,
&options,
context,
project_metadata.name(),
&mut diagnostics,
)?;
program_settings.search_paths.try_register_static_roots(db);
let program = Program::from_settings(db, &program_settings);
Some(Script::new(
db,
file,
settings,
program,
program_settings.python_version,
diagnostics.into_boxed_slice(),
))
}
fn resolve_script_options(
project_metadata: &ProjectMetadata,
metadata: &ScriptMetadata,
) -> Option<Options> {
// When using `--config-file <FILE>`, use the settings from `<FILE>`
let inline = if project_metadata.config_file_override().is_some() {
project_metadata.options().clone()
} else {
// Otherwise use the script's settings.
metadata.to_options()?
};
let mut options = Options::default();
// Merge the options with CLI, LSP, user configuration, and fallback options
for layer in project_metadata.script_options_in_precedence_order(&inline) {
options.combine_with(layer.clone());
}
// Unlike Project's, default to `[]` for scripts (unless explicitly specified).
options
.environment
.get_or_insert_default()
.root
.get_or_insert_default();
Some(options)
}
fn resolve_script_settings(
db: &dyn Db,
options: &Options,
context: OptionsContext<'_>,
diagnostics: &mut Vec<Diagnostic>,
) -> Option<Settings> {
let (settings, settings_diagnostics) =
options.to_settings(db, context, &FallibleStrategy).ok()?;
diagnostics.extend(
settings_diagnostics
.into_iter()
.map(|diagnostic| diagnostic.to_diagnostic()),
);
Some(settings)
}
fn resolve_script_program_settings(
db: &dyn Db,
options: &Options,
context: OptionsContext<'_>,
project_name: &str,
diagnostics: &mut Vec<Diagnostic>,
) -> Option<ProgramSettings> {
let (settings, settings_diagnostics) = options
.to_program_settings(
context,
project_name,
db.system(),
db.vendored(),
&FallibleStrategy,
)
.ok()?;
diagnostics.extend(
settings_diagnostics
.into_iter()
.map(|diagnostic| diagnostic.into_diagnostic(db).to_diagnostic()),
);
Some(settings)
}
/// Returns the PEP 723 metadata embedded in `file`.
#[salsa::tracked(returns(ref))]
pub(crate) fn script_metadata(db: &dyn Db, file: File) -> Option<Box<PyProject>> {
///
/// Most files have no script metadata. Boxing keeps the cached result compact when it is `None`.
#[salsa::tracked(returns(as_deref))]
pub(crate) fn script_metadata(db: &dyn SourceDb, file: File) -> Option<Box<ScriptMetadata>> {
let path = file.path(db);
if path.is_vendored_path() {
return None;
@@ -23,9 +181,119 @@ pub(crate) fn script_metadata(db: &dyn Db, file: File) -> Option<Box<PyProject>>
}
let tag = ScriptTag::parse(source.as_bytes())?;
let value_source = ValueSource::File(Arc::new(SystemPathBuf::from(path.as_str())));
let _guard = ValueSourceGuard::new(ValueSource::ScriptMetadata(file), false);
// FIXME: Report invalid TOML in script metadata instead of silently ignoring it.
let mut metadata: ScriptMetadata = toml::from_str(tag.metadata()).ok()?;
PyProject::from_toml_str_without_spans(tag.metadata(), value_source)
.map(Box::new)
.ok()
if let Some(options) = metadata.tool.as_mut().and_then(|tool| tool.ty.as_mut()) {
options.prioritize_all_selectors();
}
Some(Box::new(metadata))
}
/// PEP 723 metadata, whose Python requirement belongs at the top level rather than in `project`.
#[derive(Debug, Clone, PartialEq, Eq, Deserialize)]
#[serde(rename_all = "kebab-case")]
pub(crate) struct ScriptMetadata {
requires_python: Option<RangedValue<VersionSpecifiers>>,
tool: Option<Tool>,
}
impl ScriptMetadata {
fn to_options(&self) -> Option<Options> {
let mut options = self.ty().cloned().unwrap_or_default();
options
.apply_requires_python(self.requires_python.as_ref())
.ok()?;
Some(options)
}
fn ty(&self) -> Option<&Options> {
self.tool.as_ref().and_then(|tool| tool.ty.as_ref())
}
}
#[cfg(test)]
mod tests {
use ruff_db::files::system_path_to_file;
use ruff_db::system::{DbWithWritableSystem as _, SystemPath, SystemPathBuf};
use ruff_db::testing::assert_function_query_was_not_run;
use ty_python_semantic::Db as _;
use crate::db::testing::TestDb;
use crate::{Db as _, ProjectMetadata};
use super::{Script, script};
#[test]
fn ordinary_files_do_not_depend_on_open_files() -> anyhow::Result<()> {
let mut db = TestDb::new(ProjectMetadata::new(
"test",
SystemPathBuf::from("/project"),
));
db.write_files([
("/project/ordinary.py", "value = 1\n"),
("/project/opened.py", "value = 2\n"),
])?;
let ordinary = system_path_to_file(&db, SystemPath::new("/project/ordinary.py"))?;
let opened = system_path_to_file(&db, SystemPath::new("/project/opened.py"))?;
assert!(Script::for_file(&db, ordinary).is_none());
let events = db.take_salsa_events();
assert_function_query_was_not_run(&db, script, ordinary, &events);
assert!(script(&db, ordinary).is_none());
db.take_salsa_events();
db.project().open_file(&mut db, opened);
db.take_salsa_events();
assert!(script(&db, ordinary).is_none());
let events = db.take_salsa_events();
assert_function_query_was_not_run(&db, crate::should_check_file, ordinary, &events);
assert_function_query_was_not_run(&db, script, ordinary, &events);
Ok(())
}
#[test]
fn equivalent_script_settings_share_programs() -> anyhow::Result<()> {
let mut db = TestDb::new(ProjectMetadata::new(
"test",
SystemPathBuf::from("/project"),
));
db.write_dedented(
"/project/requirement.py",
r#"
# /// script
# requires-python = ">=3.12"
# ///
"#,
)?;
db.write_dedented(
"/project/nested/configured.py",
r#"
# /// script
# [tool.ty.environment]
# python-version = "3.12"
# ///
"#,
)?;
let requirement = system_path_to_file(&db, SystemPath::new("/project/requirement.py"))?;
let configured =
system_path_to_file(&db, SystemPath::new("/project/nested/configured.py"))?;
assert_eq!(
db.program_file(requirement).program(&db),
db.program_file(configured).program(&db)
);
assert_ne!(
db.python_version_with_source(requirement),
db.python_version_with_source(configured)
);
Ok(())
}
}
+1 -1
View File
@@ -23,7 +23,7 @@ pub trait TestProgramDb: Db {
{
#[salsa::tracked(returns(copy), heap_size=ruff_memory_usage::heap_size)]
fn program_inner(db: &dyn TestProgramDb) -> Program<'_> {
Program::from_settings(db, db.program_settings().clone())
Program::from_settings(db, db.program_settings())
}
program_inner(self)
+2 -2
View File
@@ -25,7 +25,7 @@ impl get_size2::GetSize for Program<'_> {}
impl<'db> Program<'db> {
/// Creates a program from settings whose search roots have already been registered.
pub fn from_settings(db: &'db dyn Db, settings: ProgramSettings) -> Self {
pub fn from_settings(db: &'db dyn Db, settings: &ProgramSettings) -> Self {
let ProgramSettings {
python_version,
python_platform,
@@ -33,7 +33,7 @@ impl<'db> Program<'db> {
} = settings;
let resolver_environment =
ResolverEnvironment::new(db, python_version.version, &search_paths);
ResolverEnvironment::new(db, python_version.version, search_paths);
Program::new(db, python_platform, resolver_environment)
}
@@ -1,6 +1,6 @@
Scripts with PEP 723 metadata are considered single-file projects. For now, they can configure
`rules` and `analysis`, but we plan to also support dependencies and changing `environment`
settings.
Scripts with PEP 723 metadata are considered single-file projects. They can configure `rules`,
`analysis`, and their Python environment independently of the enclosing project. Dependencies are
resolved from existing Python environments; ty does not install them.
```toml
[environment]
@@ -15,10 +15,10 @@ respect-type-ignore-comments = false
# Inline settings
A script can change its `rules` and `analysis` settings. In the future, it can also change its
`environment` settings. A script is standalone, it does not inherit any settings from the project
(that's not entirely true today, because scripts still inherit `environment` settings but it's our
end goal).
A script can change its `rules`, `analysis`, and `environment` settings. A script does not inherit
the enclosing project's configuration or Python environment, but can use an activated or explicitly
configured environment. First-party imports require explicitly configured source roots or extra
search paths.
```py
# /// script
@@ -27,8 +27,12 @@ pub fn inferred_python_version_source_annotation(
source: &PythonVersionSource,
) -> Option<Annotation> {
match source {
PythonVersionSource::ConfigFile(source) => source.span(db).map(Annotation::primary),
PythonVersionSource::PyvenvCfgFile(source) => source.span(db).map(Annotation::primary),
PythonVersionSource::ConfigFile(source) | PythonVersionSource::PyvenvCfgFile(source) => {
source.span(db).map(Annotation::primary)
}
PythonVersionSource::ScriptMetadata(span) => {
span.range().map(|_| Annotation::primary(span.clone()))
}
PythonVersionSource::InstallationDirectoryLayout { source, .. } => source
.as_ref()
.and_then(|source| source.span(db))
@@ -72,6 +76,18 @@ pub(crate) fn add_inferred_python_version_hint_to_diagnostic(
));
}
}
source @ crate::PythonVersionSource::ScriptMetadata(_) => {
let mut sub_diagnostic = SubDiagnostic::new(
SubDiagnosticSeverity::Info,
format_args!(
"Python {version} was assumed when {action} because it was specified in script metadata"
),
);
if let Some(annotation) = inferred_python_version_source_annotation(db, source) {
sub_diagnostic.annotate(annotation.message("Python version configured here"));
}
diagnostic.sub(sub_diagnostic);
}
source @ crate::PythonVersionSource::PyvenvCfgFile(_) => {
if let Some(annotation) = inferred_python_version_source_annotation(db, source) {
let mut sub_diagnostic = SubDiagnostic::new(
+3
View File
@@ -660,6 +660,9 @@ pub enum LintSource {
/// The rule was enabled in a configuration file.
File,
/// The rule was enabled in a standalone script's inline metadata.
ScriptMetadata,
/// The rule was enabled from the configuration in the editor.
Editor,
@@ -2148,7 +2148,7 @@ mod tests {
python_platform: python_platform.clone(),
search_paths: search_paths.clone(),
};
program = Program::from_settings(&db, settings);
program = Program::from_settings(&db, &settings);
current_version = version_added;
}
@@ -527,6 +527,9 @@ impl Drop for LintDiagnosticGuard<'_, '_> {
LintSource::File => {
format!("rule `{rule}` was selected in the configuration file")
}
LintSource::ScriptMetadata => {
format!("rule `{rule}` was selected in script metadata")
}
LintSource::Editor => {
format!("rule `{rule}` was selected in the editor settings")
}
@@ -122,7 +122,7 @@ fn same_file_at_different_python_versions() -> anyhow::Result<()> {
file,
Program::from_settings(
&db,
ProgramSettings {
&ProgramSettings {
python_version: PythonVersionWithSource {
version: PythonVersion::PY311,
source: PythonVersionSource::Default,
@@ -137,7 +137,7 @@ fn same_file_at_different_python_versions() -> anyhow::Result<()> {
file,
Program::from_settings(
&db,
ProgramSettings {
&ProgramSettings {
python_version: PythonVersionWithSource {
version: PythonVersion::PY312,
source: PythonVersionSource::Default,
@@ -201,7 +201,7 @@ fn program_file_changes_with_python_version() -> anyhow::Result<()> {
let equivalent_program = Program::from_settings(
&db,
ProgramSettings {
&ProgramSettings {
python_version: db.program_settings().python_version.clone(),
python_platform: program.python_platform(&db).clone(),
search_paths: program.search_paths(&db).clone(),
@@ -215,7 +215,7 @@ fn program_file_changes_with_python_version() -> anyhow::Result<()> {
let py312_program = Program::from_settings(
&db,
ProgramSettings {
&ProgramSettings {
python_version: PythonVersionWithSource {
version: PythonVersion::PY312,
source: PythonVersionSource::Default,
@@ -0,0 +1,58 @@
use anyhow::Result;
use lsp_types::Position;
use ruff_db::system::SystemPath;
use crate::TestServerBuilder;
#[test]
fn script_search_paths_resolve_imported_symbols() -> Result<()> {
let workspace_root = SystemPath::new("src");
let dependency = SystemPath::new("src/dependencies/dependency.py");
let script = SystemPath::new("src/script.py");
let script_content = r#"# /// script
# [tool.ty.environment]
# extra-paths = ["./dependencies"]
# ///
from dependency import script_only
"#;
let ordinary = SystemPath::new("src/ordinary.py");
let ordinary_content = "from dependency import script_only\n";
let mut server = TestServerBuilder::new()?
.with_workspace(workspace_root, None)?
.with_file(dependency, "def script_only() -> None: ...\n")?
.with_file(script, script_content)?
.with_file(ordinary, ordinary_content)?
.build()
.wait_until_workspaces_are_initialized();
server.open_text_document(script, script_content, 1);
server.open_text_document(ordinary, ordinary_content, 1);
let script_definition = server.goto_definition_request(script, Position::new(5, 24));
insta::assert_json_snapshot!(script_definition, @r#"
[
{
"uri": "file://<temp_dir>/src/dependencies/dependency.py",
"range": {
"start": {
"line": 0,
"character": 4
},
"end": {
"line": 0,
"character": 15
}
}
}
]
"#);
let ordinary_definition = server.goto_definition_request(ordinary, Position::new(0, 24));
insta::assert_json_snapshot!(ordinary_definition, @"null");
Ok(())
}
+93
View File
@@ -40,6 +40,99 @@ fn supports_only_plain_text() -> Result<()> {
Ok(())
}
#[test]
fn shared_import_hover_uses_each_script_python_version() -> Result<()> {
let workspace_root = SystemPath::new("src");
let shared = SystemPath::new("src/shared.py");
let older = SystemPath::new("src/older.py");
let newer = SystemPath::new("src/newer.py");
let shared_content = "\
import sys
if sys.version_info >= (3, 13):
value = 13
else:
value = 12
";
let older_content = r#"# /// script
# requires-python = ">=3.12"
# [tool.ty.environment]
# extra-paths = ["."]
# ///
from shared import value
value
"#;
let newer_content = r#"# /// script
# requires-python = ">=3.13"
# [tool.ty.environment]
# extra-paths = ["."]
# ///
from shared import value
value
"#;
let mut server = TestServerBuilder::new()?
.with_workspace(workspace_root, None)?
.with_file(
"src/pyproject.toml",
r#"[tool.ty.environment]
python-version = "3.12"
"#,
)?
.with_file(shared, shared_content)?
.with_file(older, older_content)?
.with_file(newer, newer_content)?
.build()
.wait_until_workspaces_are_initialized();
server.open_text_document(older, older_content, 1);
server.open_text_document(newer, newer_content, 1);
let older_hover = server.hover_request(older, Position::new(7, 1));
insta::assert_json_snapshot!(older_hover, @r#"
{
"contents": {
"kind": "plaintext",
"value": "Literal[12]"
},
"range": {
"start": {
"line": 7,
"character": 0
},
"end": {
"line": 7,
"character": 5
}
}
}
"#);
let newer_hover = server.hover_request(newer, Position::new(7, 1));
insta::assert_json_snapshot!(newer_hover, @r#"
{
"contents": {
"kind": "plaintext",
"value": "Literal[13]"
},
"range": {
"start": {
"line": 7,
"character": 0
},
"end": {
"line": 7,
"character": 5
}
}
}
"#);
Ok(())
}
fn hover_content_format(formats: Vec<MarkupKind>) -> Result<MarkupKind> {
let workspace_root = SystemPath::new("src");
let document_path = SystemPath::new("src/foo.py");
+41 -3
View File
@@ -33,6 +33,7 @@ mod commands;
mod completions;
mod configuration;
mod folding_range;
mod goto_definition;
mod hover;
mod implementation;
mod initialize;
@@ -59,7 +60,8 @@ use insta::internals::SettingsBindDropGuard;
use lsp_server::{Connection, Message, RequestId, Response, ResponseError};
use lsp_types::{
ClientCapabilities, CompletionItem, CompletionParams, CompletionRequest, CompletionResponse,
CompletionTriggerKind, ConfigurationParams, ConfigurationRequest, DiagnosticClientCapabilities,
CompletionTriggerKind, ConfigurationParams, ConfigurationRequest, DefinitionParams,
DefinitionRequest, DefinitionResponse, DiagnosticClientCapabilities,
DidChangeTextDocumentNotification, DidChangeTextDocumentParams,
DidChangeWatchedFilesClientCapabilities, DidChangeWatchedFilesNotification,
DidChangeWatchedFilesParams, DidChangeWorkspaceFoldersNotification,
@@ -78,7 +80,7 @@ use lsp_types::{
WorkspaceDiagnosticParams, WorkspaceDiagnosticReport, WorkspaceDiagnosticRequest,
WorkspaceEdit, WorkspaceFolder, WorkspaceFoldersChangeEvent, WorkspaceFoldersInitializeParams,
};
use ruff_db::system::{OsSystem, SystemPath, SystemPathBuf, TestSystem};
use ruff_db::system::{OsSystem, SystemPath, SystemPathBuf, SystemVirtualPath, TestSystem};
use rustc_hash::FxHashMap;
use tempfile::TempDir;
use ty_server::{ClientOptions, LogLevel, Server, init_logging};
@@ -798,9 +800,25 @@ impl TestServer {
content: impl AsRef<str>,
version: i32,
) {
self.open_text_document_with_uri(self.file_uri(path), content, version);
}
/// Send a `textDocument/didOpen` notification for an unsaved virtual document.
pub(crate) fn open_virtual_text_document(
&mut self,
path: impl AsRef<SystemVirtualPath>,
content: impl AsRef<str>,
version: i32,
) -> Result<()> {
let uri = Uri::parse(path.as_ref().as_str())?;
self.open_text_document_with_uri(uri, content, version);
Ok(())
}
fn open_text_document_with_uri(&mut self, uri: Uri, content: impl AsRef<str>, version: i32) {
let params = DidOpenTextDocumentParams {
text_document: TextDocumentItem {
uri: self.file_uri(path),
uri,
language_id: LanguageKind::Python,
version,
text: content.as_ref().to_string(),
@@ -955,6 +973,26 @@ impl TestServer {
self.await_response::<WorkspaceDiagnosticRequest>(&id)
}
/// Send a `textDocument/definition` request for the document at the given path and position.
pub(crate) fn goto_definition_request(
&mut self,
path: impl AsRef<SystemPath>,
position: Position,
) -> Option<DefinitionResponse> {
let params = DefinitionParams {
text_document_position_params: TextDocumentPositionParams {
text_document: TextDocumentIdentifier {
uri: self.file_uri(path),
},
position,
},
work_done_progress_params: WorkDoneProgressParams::default(),
partial_result_params: PartialResultParams::default(),
};
let id = self.send_request::<DefinitionRequest>(params);
self.await_response::<DefinitionRequest>(&id)
}
/// Send a `textDocument/hover` request for the document at the given path and position.
pub(crate) fn hover_request(
&mut self,
+134 -13
View File
@@ -8,7 +8,7 @@ use lsp_types::{
TextDocumentContentChangePartial, TextDocumentContentChangeWholeDocument, TextDocumentItem,
Uri,
};
use ruff_db::system::SystemPath;
use ruff_db::system::{SystemPath, SystemVirtualPath};
use ty_server::ClientOptions;
use crate::notebook::NotebookBuilder;
@@ -142,7 +142,7 @@ def foo() -> str:
#[test]
fn on_did_open_non_existing_file_workspace_with_untitled_uri() -> Result<()> {
let workspace_root = SystemPath::new("src");
let foo = SystemPath::new("src/foo.py");
let foo = SystemVirtualPath::new("untitled:foo.py");
let foo_content = "\
def foo() -> str:
return 42
@@ -159,17 +159,7 @@ def foo() -> str:
.build()
.wait_until_workspaces_are_initialized();
server.send_notification::<DidOpenTextDocumentNotification>(DidOpenTextDocumentParams {
text_document: TextDocumentItem {
uri: {
let uri = server.file_uri(foo);
Uri::parse(&format!("untitled://{}", uri.path())).unwrap()
},
language_id: LanguageKind::Python,
version: 1,
text: foo_content.to_string(),
},
});
server.open_virtual_text_document(foo, foo_content, 1)?;
let diagnostics = server.await_notification::<PublishDiagnosticsNotification>();
insta::assert_debug_snapshot!(diagnostics);
@@ -245,6 +235,137 @@ def foo() -> str:
Ok(())
}
#[test]
fn on_did_change_script_python_requirement() -> Result<()> {
let workspace_root = SystemPath::new("src");
let script = SystemPath::new("src/script.py");
let initial = r#"# /// script
# requires-python = ">=3.12"
# ///
PythonFinalizationError
"#;
let updated = r#"# /// script
# requires-python = ">=3.13"
# ///
PythonFinalizationError
"#;
let mut server = TestServerBuilder::new()?
.with_workspace(workspace_root, None)?
.with_file(script, initial)?
.enable_pull_diagnostics(false)
.build()
.wait_until_workspaces_are_initialized();
server.open_text_document(script, initial, 1);
let initial_diagnostics = server.await_notification::<PublishDiagnosticsNotification>();
insta::assert_debug_snapshot!(initial_diagnostics);
server.change_text_document(
script,
vec![
lsp_types::TextDocumentContentChangeEvent::TextDocumentContentChangeWholeDocument(
TextDocumentContentChangeWholeDocument {
text: updated.to_string(),
},
),
],
2,
);
let updated_diagnostics = server.await_notification::<PublishDiagnosticsNotification>();
insta::assert_debug_snapshot!(updated_diagnostics, @r#"
PublishDiagnosticsParams {
uri: Url {
scheme: "file",
cannot_be_a_base: false,
username: "",
password: None,
host: None,
port: None,
path: "<temp_dir>/src/script.py",
query: None,
fragment: None,
},
version: Some(
2,
),
diagnostics: [],
}
"#);
Ok(())
}
#[test]
fn on_did_open_virtual_script_uses_its_python_requirement() -> Result<()> {
let workspace_root = SystemPath::new("src");
let script = SystemVirtualPath::new("untitled:script.py");
let content = r#"# /// script
# requires-python = ">=3.13"
# ///
PythonFinalizationError
"#;
let mut server = TestServerBuilder::new()?
.with_workspace(workspace_root, None)?
.enable_pull_diagnostics(false)
.build()
.wait_until_workspaces_are_initialized();
server.open_virtual_text_document(script, content, 1)?;
let diagnostics = server.await_notification::<PublishDiagnosticsNotification>();
insta::assert_debug_snapshot!(diagnostics, @r#"
PublishDiagnosticsParams {
uri: Url {
scheme: "untitled",
cannot_be_a_base: true,
username: "",
password: None,
host: None,
port: None,
path: "script.py",
query: None,
fragment: None,
},
version: Some(
1,
),
diagnostics: [],
}
"#);
Ok(())
}
#[test]
fn on_did_open_virtual_script_reports_inline_configuration_diagnostics() -> Result<()> {
let workspace_root = SystemPath::new("src");
let script = SystemVirtualPath::new("untitled:script.py");
let content = r#"# /// script
# [tool.ty.rules]
# unknown-rule = "warn"
# ///
"#;
let mut server = TestServerBuilder::new()?
.with_workspace(workspace_root, None)?
.enable_pull_diagnostics(false)
.build()
.wait_until_workspaces_are_initialized();
server.open_virtual_text_document(script, content, 1)?;
let diagnostics = server.await_notification::<PublishDiagnosticsNotification>();
insta::assert_debug_snapshot!(diagnostics);
Ok(())
}
#[test]
fn on_did_save_publishes_open_file_documents() -> Result<()> {
let workspace_root = SystemPath::new("src");
@@ -0,0 +1,72 @@
---
source: crates/ty_server/tests/e2e/publish_diagnostics.rs
expression: initial_diagnostics
---
PublishDiagnosticsParams {
uri: Url {
scheme: "file",
cannot_be_a_base: false,
username: "",
password: None,
host: None,
port: None,
path: "<temp_dir>/src/script.py",
query: None,
fragment: None,
},
version: Some(
1,
),
diagnostics: [
Diagnostic {
range: Range {
start: Position {
line: 4,
character: 0,
},
end: Position {
line: 4,
character: 23,
},
},
severity: Some(
Error,
),
code: Some(
String(
"unresolved-reference",
),
),
code_description: Some(
CodeDescription {
href: Url {
scheme: "https",
cannot_be_a_base: false,
username: "",
password: None,
host: Some(
Domain(
"ty.dev",
),
),
port: None,
path: "/rules",
query: None,
fragment: Some(
"unresolved-reference",
),
},
},
),
source: Some(
"ty",
),
message: String(
"Name `PythonFinalizationError` used when not defined\n\ninfo: `PythonFinalizationError` was added as a builtin in Python 3.13\ninfo: Python 3.12 was assumed when resolving types because it was specified in script metadata",
),
tags: None,
related_information: None,
data: None,
},
],
}
@@ -5,12 +5,12 @@ expression: diagnostics
PublishDiagnosticsParams {
uri: Url {
scheme: "untitled",
cannot_be_a_base: false,
cannot_be_a_base: true,
username: "",
password: None,
host: None,
port: None,
path: "<temp_dir>/src/foo.py",
path: "foo.py",
query: None,
fragment: None,
},
@@ -0,0 +1,52 @@
---
source: crates/ty_server/tests/e2e/publish_diagnostics.rs
expression: diagnostics
---
PublishDiagnosticsParams {
uri: Url {
scheme: "untitled",
cannot_be_a_base: true,
username: "",
password: None,
host: None,
port: None,
path: "script.py",
query: None,
fragment: None,
},
version: Some(
1,
),
diagnostics: [
Diagnostic {
range: Range {
start: Position {
line: 0,
character: 0,
},
end: Position {
line: 0,
character: 0,
},
},
severity: Some(
Warning,
),
code: Some(
String(
"unknown-rule",
),
),
code_description: None,
source: Some(
"ty",
),
message: String(
"Unknown rule `unknown-rule`. Did you mean `unknown-argument`?",
),
tags: None,
related_information: None,
data: None,
},
],
}
+27 -17
View File
@@ -279,10 +279,10 @@ impl PythonEnvironment {
///
/// 1. activated virtual environment
/// 2. conda (child)
/// 3. working dir virtual environment
/// 3. project virtual environment, when a project root is provided
/// 4. conda (base)
pub fn discover(
project_root: &SystemPath,
project_root: Option<&SystemPath>,
system: &dyn System,
) -> Result<Option<Self>, SitePackagesDiscoveryError> {
fn resolve_environment(
@@ -308,22 +308,24 @@ impl PythonEnvironment {
.map(Some);
}
tracing::debug!("Discovering virtual environment in `{project_root}`");
let virtual_env_directory = project_root.join(".venv");
if let Some(project_root) = project_root {
tracing::debug!("Discovering virtual environment in `{project_root}`");
let virtual_env_directory = project_root.join(".venv");
match PythonEnvironment::new(
&virtual_env_directory,
SysPrefixPathOrigin::LocalVenv,
system,
) {
Ok(environment) => return Ok(Some(environment)),
Err(err) => {
if system.is_directory(&virtual_env_directory) {
tracing::debug!(
"Ignoring automatically detected virtual environment at `{}`: {}",
&virtual_env_directory,
err
);
match PythonEnvironment::new(
&virtual_env_directory,
SysPrefixPathOrigin::LocalVenv,
system,
) {
Ok(environment) => return Ok(Some(environment)),
Err(err) => {
if system.is_directory(&virtual_env_directory) {
tracing::debug!(
"Ignoring automatically detected virtual environment at `{}`: {}",
&virtual_env_directory,
err
);
}
}
}
}
@@ -2119,6 +2121,8 @@ impl Deref for SysPrefixPath {
pub enum SysPrefixPathOrigin {
/// The `sys.prefix` path came from a configuration file setting: `pyproject.toml` or `ty.toml`
ConfigFileSetting(Arc<SystemPathBuf>, Option<TextRange>),
/// The `sys.prefix` path came from a standalone script's inline metadata.
ScriptMetadataSetting,
/// The `sys.prefix` path came from a `--python` CLI flag
PythonCliFlag,
/// The selected interpreter in the user's editor.
@@ -2149,6 +2153,7 @@ impl SysPrefixPathOrigin {
match self {
Self::LocalVenv | Self::VirtualEnvVar => true,
Self::ConfigFileSetting(..)
| Self::ScriptMetadataSetting
| Self::PythonCliFlag
| Self::Editor
| Self::DerivedFromPyvenvCfg
@@ -2167,6 +2172,7 @@ impl SysPrefixPathOrigin {
match self {
Self::PythonCliFlag
| Self::ConfigFileSetting(..)
| Self::ScriptMetadataSetting
| Self::Editor
| Self::SelfEnvironment
| Self::PythonBinary => false,
@@ -2188,6 +2194,7 @@ impl SysPrefixPathOrigin {
| Self::Editor
| Self::DerivedFromPyvenvCfg
| Self::ConfigFileSetting(..)
| Self::ScriptMetadataSetting
| Self::PythonCliFlag
| Self::PythonBinary
| Self::UvWorkspace => false,
@@ -2201,6 +2208,9 @@ impl std::fmt::Display for SysPrefixPathOrigin {
match self {
Self::PythonCliFlag => f.write_str("`--python` argument"),
Self::ConfigFileSetting(_, _) => f.write_str("`environment.python` setting"),
Self::ScriptMetadataSetting => {
f.write_str("`environment.python` setting in script metadata")
}
Self::VirtualEnvVar => f.write_str("`VIRTUAL_ENV` environment variable"),
Self::CondaPrefixVar => f.write_str("`CONDA_PREFIX` environment variable"),
Self::DerivedFromPyvenvCfg => f.write_str("derived `sys.prefix` path"),
+3
View File
@@ -15,6 +15,9 @@ pub enum PythonVersionSource {
/// Value loaded from a project's configuration file.
ConfigFile(PythonVersionFileSource),
/// Value configured in a standalone script's inline metadata.
ScriptMetadata(Span),
/// Value loaded from the `pyvenv.cfg` file of the virtual environment.
/// The virtual environment might have been configured, activated or inferred.
PyvenvCfgFile(PythonVersionFileSource),
+4 -4
View File
@@ -136,7 +136,7 @@
}
},
"python": {
"description": "Path to your project's Python environment or interpreter.\n\nty uses the `site-packages` directory of your project's Python environment\nto resolve third-party (and, in some cases, first-party) imports in your code.\n\nThis can be a path to:\n\n- A Python interpreter, e.g. `.venv/bin/python3`\n- A virtual environment directory, e.g. `.venv`\n- A system Python [`sys.prefix`] directory, e.g. `/usr`\n\nIf you're using a project management tool such as uv, you should not generally need to\nspecify this option, as commands such as `uv run` will set the `VIRTUAL_ENV` environment\nvariable to point to your project's virtual environment. ty can also infer the location of\nyour environment from an activated Conda environment, and will look for a `.venv` directory\nin the project root if none of the above apply. Failing that, ty will look for a `python3`\nor `python` binary available in `PATH`.\n\n[`sys.prefix`]: https://docs.python.org/3/library/sys.html#sys.prefix",
"description": "Path to your project's Python environment or interpreter.\n\nty uses the `site-packages` directory of your project's Python environment\nto resolve third-party (and, in some cases, first-party) imports in your code.\n\nThis can be a path to:\n\n- A Python interpreter, e.g. `.venv/bin/python3`\n- A virtual environment directory, e.g. `.venv`\n- A system Python [`sys.prefix`] directory, e.g. `/usr`\n\nIf you're using a project management tool such as uv, you should not generally need to\nspecify this option, as commands such as `uv run` will set the `VIRTUAL_ENV` environment\nvariable to point to your project's virtual environment. ty can also infer the location of\nyour environment from an activated Conda environment, and will look for a `.venv` directory\nin the project root if none of the above apply. Failing that, ty will look for a `python3`\nor `python` binary available in `PATH`.\n\nScripts with inline metadata use their own Python environment. They can use an explicitly\nconfigured environment, an activated environment, or an environment selected by the editor.\nUnlike projects, they do not automatically use a `.venv` directory.\n\n[`sys.prefix`]: https://docs.python.org/3/library/sys.html#sys.prefix",
"anyOf": [
{
"$ref": "#/definitions/RelativePathBuf"
@@ -158,7 +158,7 @@
]
},
"python-version": {
"description": "Specifies the version of Python that will be used to analyze the source code.\nThe version should be specified as a string in the format `M.m` where `M` is the major version\nand `m` is the minor (e.g. `\"3.7\"` or `\"3.12\"`).\nIf a version is provided, ty will generate errors if the source code makes use of language features\nthat are not supported in that version.\n\nty officially supports type checking code that targets Python 3.10 and later. Python 3.7\nthrough 3.9 can still be selected, but ty may produce false positives or false negatives for\nstandard-library APIs because its bundled stubs do not fully describe those versions.\n\nIf a version is not specified, ty will try the following techniques in order of preference\nto determine a value:\n1. Check for the `project.requires-python` setting in a `pyproject.toml` file\n and use the minimum version from the specified range\n2. Check for an activated or configured Python environment\n and attempt to infer the Python version of that environment\n3. Fall back to the default value (see below)\n\nFor some language features, ty can also understand conditionals based on comparisons\nwith `sys.version_info`. These are commonly found in typeshed, for example,\nto reflect the differing contents of the standard library across Python versions.",
"description": "Specifies the version of Python that will be used to analyze the source code.\nThe version should be specified as a string in the format `M.m` where `M` is the major version\nand `m` is the minor (e.g. `\"3.7\"` or `\"3.12\"`).\nIf a version is provided, ty will generate errors if the source code makes use of language features\nthat are not supported in that version.\n\nty officially supports type checking code that targets Python 3.10 and later. Python 3.7\nthrough 3.9 can still be selected, but ty may produce false positives or false negatives for\nstandard-library APIs because its bundled stubs do not fully describe those versions.\n\nIf a version is not specified, ty will try the following techniques in order of preference\nto determine a value:\n1. Check for the `project.requires-python` setting in a `pyproject.toml` file\n and use the minimum version from the specified range\n2. Check for an activated or configured Python environment\n and attempt to infer the Python version of that environment\n3. Fall back to the default value (see below)\n\nScripts with inline metadata use their `requires-python` field instead of\n`project.requires-python`. They do not inherit the Python version of the enclosing project.\n\nFor some language features, ty can also understand conditionals based on comparisons\nwith `sys.version_info`. These are commonly found in typeshed, for example,\nto reflect the differing contents of the standard library across Python versions.",
"anyOf": [
{
"$ref": "#/definitions/SupportedPythonVersion"
@@ -169,7 +169,7 @@
]
},
"root": {
"description": "The root paths of the project, used for finding first-party modules.\n\nAccepts a list of directory paths searched in priority order (first has highest priority).\n\nIf left unspecified, ty will try to detect common project layouts and initialize `root` accordingly.\nThe project root (`.`) is always included. Additionally, the following directories are included\nif they exist and are not packages (i.e. they do not contain `__init__.py` or `__init__.pyi` files):\n\n* `./src`\n* `./<project-name>` (if a `./<project-name>/<project-name>` directory exists)\n* `./python`",
"description": "The root paths of the project, used for finding first-party modules.\n\nAccepts a list of directory paths searched in priority order (first has highest priority).\n\nIf left unspecified, ty will try to detect common project layouts and initialize `root` accordingly.\nThe project root (`.`) is always included. Additionally, the following directories are included\nif they exist and are not packages (i.e. they do not contain `__init__.py` or `__init__.pyi` files):\n\n* `./src`\n* `./<project-name>` (if a `./<project-name>/<project-name>` directory exists)\n* `./python`\n\nScripts with inline metadata have no first-party roots by default because they are\nsingle-file programs. Set `root = [\".\"]` to allow importing local modules.",
"type": [
"array",
"null"
@@ -325,7 +325,7 @@
]
},
"RelativePathBuf": {
"description": "A possibly relative path in a configuration file.\n\nRelative paths in configuration files or from CLI options\nrequire different anchoring:\n\n* CLI: The path is relative to the current working directory\n* Configuration file: The path is relative to the project's root.",
"description": "A possibly relative path in a configuration file.\n\nRelative paths in configuration files or from CLI options\nrequire different anchoring:\n\n* CLI: The path is relative to the current working directory\n* Configuration file: The path is relative to the project's or script's configuration root.",
"allOf": [
{
"$ref": "#/definitions/SystemPathBuf"