From 06007c81fa99e813cbf4583dab326764ba0ef017 Mon Sep 17 00:00:00 2001 From: youknowdot Date: Sat, 10 Oct 2026 20:57:49 -0700 Subject: [PATCH] Begin the Python 3.15 target and essential-library migration (#8950) * Target Python 3.15 using the final upstream reference Update active version and comparison configuration to Python 3.15.0. Preserve the alpha implementation status, actual bytecode format, and existing libraries pending separate migration. Align generated library-upgrade workflow pins without changing permissions or actions. Assisted-by: OpenAI Codex:model identifier unavailable * Adapt essential snippet contracts to Python 3.15 final Preserve existing coverage while following verified final behavior for script metadata, builtin signatures and loading, AST fixture construction, memoryview diagnostics, mmap search bounds, and struct format errors. The unchanged final comparator exposes the earlier expectations; focused native baseline failures establish the required implementation changes. Assisted-by: OpenAI Codex:model identifier unavailable * Keep final-reference regression paths and remove expired sysconfig argument Preserve reverse multiplication syntax and native exact overflow diagnostics. Accept the two source-proven CPython 3.15 specialization messages only in affected reverse str/bytes/tuple cases. Use an actually out-of-range Windows datetime errno fixture. Remove the sysconfig check_home argument at its existing canonical 3.15 removal test boundary. Assisted-by: OpenAI Codex:model identifier unavailable * Keep README wording unchanged in Python 3.15 transition Assisted-by: OpenAI Codex:model identifier unavailable * Match Python 3.15 repetition specialization and keep universal tests Use exact sequence-type guards and preserve generic fallback diagnostics. Validate both cold and warmed expressions without implementation-specific assertions. Reject stale CPython source checkouts and cross-version library caches. Assisted-by: OpenAI Codex:model identifier unavailable * Limit in-place specialization additions to exact sequence repetition Assisted-by: OpenAI Codex:model identifier unavailable * Keep subclass regression scoped to repetition dispatch Assisted-by: OpenAI Codex:model identifier unavailable * Provision final CPython 3.15 with pinned uv builds Assisted-by: OpenAI Codex:model identifier unavailable * Enforce Python 3.15 AST constructor requirements Reject retired missing-field and arbitrary-keyword forms while preserving defaults and explicit placeholders. Sync only matching final CPython constructor test hunks. Assisted-by: OpenAI Codex:model identifier unavailable * Keep seeded Python ahead of its Windows DLL directory Assisted-by: OpenAI Codex:model identifier unavailable * Keep opcode metadata tied to the implemented bytecode format The interpreter target and reference tests use Python 3.15.0, but the current Rust opcode enum retains the 3.14 bytecode format. Generate metadata from its matching source rather than importing unsupported instruction families. Assisted-by: OpenAI Codex:model identifier unavailable * Align CSV, logarithm and decimal formatting with Python 3.15 Propagate dialect lookup errors except AttributeError, retry integer logarithms after float overflow, and update the decimal conversion diagnostic. Exercise the final reference contracts without implementation-specific branches. Assisted-by: OpenAI Codex:model identifier unavailable * Bind the logarithm index result before borrowing its integer Assisted-by: OpenAI Codex:model identifier unavailable * Align required canonical test expectations with Python 3.15 Take only demonstrated contract changes from CPython v3.15.0, d220409a13b77de7a6c30abe16ea90db01528edc: removed module __cached__, Struct format exception classes, and decimal-format diagnostic. Preserve unrelated tests and expectations. Assisted-by: OpenAI Codex:model identifier unavailable * Track pending frozendict pickle support during the 3.15 transition Keep the two existing frozendict pickle tests running as expected failures until #8951 supplies the builtin. Unexpected success will signal removal of these temporary markers. Assisted-by: OpenAI Codex:model identifier unavailable * Support final Python 3.15 Windows negative timestamps Use FILETIME for pre-epoch conversions and preserve precise Windows timestamp errors. Assisted-by: OpenAI Codex:model identifier unavailable * Derive historical Windows timezone offsets from wall time FILETIME reports historical DST as unknown, so calculate the actual offset rather than selecting standard-time bias. Assisted-by: OpenAI Codex:model identifier unavailable * Identify decimal formatting arguments in Python 3.15 errors Track consumed positional arguments, including star width and precision, and preserve mapping key representations in decimal conversion diagnostics. Assisted-by: OpenAI Codex:model identifier unavailable * Reuse mapping format keys for lookup and error context Assisted-by: OpenAI Codex:model identifier unavailable * Borrow mapping diagnostic keys as Python objects Assisted-by: OpenAI Codex:model identifier unavailable * Avoid empty components in the reference Python library path Assisted-by: OpenAI Codex:model identifier unavailable * Isolate RustPython bytecode caches during the 3.15 transition Assisted-by: OpenAI Codex:model identifier unavailable * Sync final Windows timestamp test expectations Import only the final CPython platform-limit guards and mktime roundtrip expectation required by the existing Windows CI suite. Assisted-by: OpenAI Codex:model identifier unavailable * Restore official Python setup for 3.15 final Use the existing pinned setup-python action now that its official catalog supplies CPython 3.15.0. Remove the temporary standalone provisioning action and its sparse-checkout entry while retaining interpreter outputs and requested versions. Assisted-by: OpenAI Codex:model identifier unavailable * Align merged runtime contracts with Python 3.15 final Restrict mappingproxy ordering, use final codec registry normalization, and prefer index conversion before float conversion for sleep. Retain universal Python assertions and import the matching final codec/time canonical test hunks plus the non-ASCII codec-name deprecation warning. Upstream source: CPython d220409a13b77de7a6c30abe16ea90db01528edc. The existing codec normalization implementation and bytecode contract remain unchanged. Assisted-by: OpenAI Codex:model identifier unavailable * Limit Windows launcher version check to its interpreter Exclude only the CPython py.exe version-identity case from RustPython CI. The installed launcher reports its own final version, while RustPython correctly remains alpha. Preserve the other 45 launcher cases and all RustPython version assertions without modifying Lib. Assisted-by: OpenAI Codex:model identifier unavailable --------- Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com> --- .claude/scripts/setup-env.sh | 18 ++- .github/workflows/ci.yaml | 15 +-- .github/workflows/pyperformance.yaml | 2 +- .github/workflows/update-doc-db.yml | 2 +- .github/workflows/upgrade-pylib.lock.yml | 8 +- .github/workflows/upgrade-pylib.md | 17 ++- .python-version | 2 +- AGENTS.md | 2 +- CONTRIBUTING.md | 2 +- Lib/encodings/__init__.py | 6 + Lib/sysconfig/__init__.py | 13 +- Lib/test/datetimetester.py | 2 + Lib/test/pickletester.py | 4 + Lib/test/test_ast/test_ast.py | 79 +++++------- Lib/test/test_cmd_line_script.py | 3 - Lib/test/test_codecs.py | 35 ++--- Lib/test/test_peepholer.py | 3 +- Lib/test/test_struct.py | 4 +- Lib/test/test_time.py | 7 +- README.md | 2 +- crates/host_env/src/time.rs | 73 ++++++++++- crates/stdlib/src/csv.rs | 5 +- crates/stdlib/src/math.rs | 89 +++++++------ crates/stdlib/src/mmap.rs | 13 +- crates/stdlib/src/pystruct.rs | 31 ++--- crates/vm/build.rs | 4 +- crates/vm/src/builtins/mappingproxy.rs | 6 +- crates/vm/src/builtins/memory.rs | 2 +- crates/vm/src/cformat.rs | 102 ++++++++++----- crates/vm/src/codecs.rs | 26 +++- crates/vm/src/exceptions.rs | 12 ++ crates/vm/src/frame.rs | 129 +++++++++++++++++-- crates/vm/src/stdlib/_ast.rs | 6 +- crates/vm/src/stdlib/_ast/python.rs | 87 ++++++------- crates/vm/src/stdlib/_imp.rs | 46 ++++++- crates/vm/src/stdlib/builtins.rs | 8 +- crates/vm/src/stdlib/sys.rs | 4 +- crates/vm/src/stdlib/time.rs | 82 ++++++------ crates/vm/src/vm/mod.rs | 6 +- extra_tests/snippets/builtin___main__.py | 2 +- extra_tests/snippets/builtin_mappingproxy.py | 22 +++- extra_tests/snippets/builtin_memoryview.py | 2 +- extra_tests/snippets/builtin_signature.py | 4 +- extra_tests/snippets/builtin_str.py | 29 ++++- extra_tests/snippets/operator_arithmetic.py | 85 +++++++++++- extra_tests/snippets/stdlib_ast.py | 77 ++++++++++- extra_tests/snippets/stdlib_codecs_wide.py | 6 +- extra_tests/snippets/stdlib_csv.py | 12 +- extra_tests/snippets/stdlib_datetime.py | 27 ++-- extra_tests/snippets/stdlib_imp.py | 3 +- extra_tests/snippets/stdlib_math.py | 65 +++++++++- extra_tests/snippets/stdlib_mmap.py | 6 +- extra_tests/snippets/stdlib_struct.py | 18 +-- extra_tests/snippets/stdlib_sys.py | 7 + extra_tests/snippets/stdlib_time.py | 62 ++++++++- scripts/libc_posix.py | 2 +- scripts/whats_left.py | 6 +- 57 files changed, 977 insertions(+), 415 deletions(-) diff --git a/.claude/scripts/setup-env.sh b/.claude/scripts/setup-env.sh index f5d28a3f1..97e38f229 100755 --- a/.claude/scripts/setup-env.sh +++ b/.claude/scripts/setup-env.sh @@ -16,7 +16,7 @@ if [ "$(printf '%s\n' "3.13" "$CURRENT_PY" | sort -V | head -1)" != "3.13" ]; th echo "Upgrading python3 default to 3.13..." # Find best available Python >= 3.13 TARGET="" - for ver in python3.14 python3.13; do + for ver in python3.15 python3.14 python3.13; do if command -v "$ver" &>/dev/null; then TARGET=$(command -v "$ver") break @@ -38,13 +38,17 @@ else echo "python3 already >= 3.13: $(python3 --version)" fi -# 2. Clone CPython source if not present (needed for scripts/update_lib) -if [ ! -d "cpython" ]; then - echo "Cloning CPython v3.14.3 (shallow)..." - git clone --depth 1 --branch v3.14.3 https://github.com/python/cpython.git cpython - echo "CPython source ready." +# 2. Ensure the CPython source matches the target (needed for scripts/update_lib) +PYTHON_VERSION="v3.15.0" +if [ ! -e "cpython" ]; then + git clone --depth 1 --branch "$PYTHON_VERSION" https://github.com/python/cpython.git cpython +elif [ ! -d "cpython/.git" ] || [ ! -d "cpython/Lib" ] || + [ "$(git -C cpython rev-parse HEAD)" != "$(git -C cpython rev-parse --verify "$PYTHON_VERSION^{commit}" 2>/dev/null)" ] || + [ -n "$(git -C cpython status --porcelain)" ]; then + echo "ERROR: cpython must be a clean checkout of $PYTHON_VERSION. Move the existing directory aside and rerun." >&2 + exit 1 else - echo "CPython source already present." + echo "CPython $PYTHON_VERSION source already present." fi echo "=== Setup complete ===" diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 2f62d9199..c48bd99a1 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -502,6 +502,9 @@ jobs: - '-u all' - '--timeout 600' - '--dont-add-python-opts' + # CPython's installed py.exe reports its own version, not RustPython's. + # Keep the remaining launcher integration tests and runtime version checks. + - '--ignore test.test_launcher.TestLauncher.test_version' env_polluting_tests: [] skips: [] timeout: 50 @@ -781,18 +784,14 @@ jobs: - name: prek run run: prek run --show-diff-on-failure --color=always --all-files - - name: Get target CPython version - id: cpython-version - run: | - version=$(cat .python-version) - echo "version=${version}" >> "$GITHUB_OUTPUT" - - - name: Clone CPython + - name: Clone CPython opcode metadata source uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: repository: python/cpython path: cpython - ref: "v${{ steps.cpython-version.outputs.version }}" + # RustPython still uses the 3.14 bytecode format; this source is only + # for opcode metadata, independent of the 3.15 runtime target. + ref: "v3.14.7" persist-credentials: false - name: prek run (manual stage) diff --git a/.github/workflows/pyperformance.yaml b/.github/workflows/pyperformance.yaml index 1d304582c..05f76266e 100644 --- a/.github/workflows/pyperformance.yaml +++ b/.github/workflows/pyperformance.yaml @@ -63,7 +63,7 @@ jobs: - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 id: cpython with: - python-version: "3.14" + python-version: "3.15.0" - name: Restore cache uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 diff --git a/.github/workflows/update-doc-db.yml b/.github/workflows/update-doc-db.yml index 8874c3aea..3abdf27de 100644 --- a/.github/workflows/update-doc-db.yml +++ b/.github/workflows/update-doc-db.yml @@ -8,7 +8,7 @@ on: python-version: description: Target python version to generate doc db for type: string - default: "3.14.7" + default: "3.15.0" base-ref: description: Base branch to create the update branch from type: string diff --git a/.github/workflows/upgrade-pylib.lock.yml b/.github/workflows/upgrade-pylib.lock.yml index c08d19058..6fa651d50 100644 --- a/.github/workflows/upgrade-pylib.lock.yml +++ b/.github/workflows/upgrade-pylib.lock.yml @@ -24,7 +24,7 @@ # Pick an out-of-sync Python library from the todo list and upgrade it # by running `scripts/update_lib quick`, then open a pull request. # -# frontmatter-hash: 3129480d6628afe028911bb8b31b6bb3b5eb251e395f00ed0677922cd21727cb +# frontmatter-hash: bf5b6c1bf4e49141df567a8ff69cce1fc96faa991e1cc353ed6a77aa7c2ba719 name: "Upgrade Python Library" "on": @@ -44,7 +44,7 @@ run-name: "Upgrade Python Library" env: ISSUE_ID: "6839" - PYTHON_VERSION: v3.14.3 + PYTHON_VERSION: v3.15.0 # Cache configuration from frontmatter was processed and added to the main job steps @@ -109,7 +109,7 @@ jobs: - name: Setup Python uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 with: - python-version: '3.14' + python-version: '3.15' - name: Create gh-aw temp directory run: bash /opt/gh-aw/actions/create_gh_aw_tmp_dir.sh # Cache configuration from frontmatter processed below @@ -118,8 +118,6 @@ jobs: with: key: cpython-lib-${{ env.PYTHON_VERSION }} path: cpython - restore-keys: | - cpython-lib- - name: Configure Git credentials env: REPO_NAME: ${{ github.repository }} diff --git a/.github/workflows/upgrade-pylib.md b/.github/workflows/upgrade-pylib.md index cd05eb777..238c0d43b 100644 --- a/.github/workflows/upgrade-pylib.md +++ b/.github/workflows/upgrade-pylib.md @@ -28,7 +28,7 @@ engine: copilot runtimes: python: - version: "3.14" + version: "3.15" tools: bash: @@ -48,11 +48,9 @@ safe-outputs: cache: key: cpython-lib-${{ env.PYTHON_VERSION }} path: cpython - restore-keys: - - cpython-lib- env: - PYTHON_VERSION: "v3.14.7" + PYTHON_VERSION: "v3.15.0" ISSUE_ID: "6839" --- @@ -65,10 +63,15 @@ You are an automated maintenance agent for RustPython, a Python 3 interpreter wr The CPython source may already be cached. Check if the `cpython` directory exists and has the correct version: ```bash -if [ -d "cpython/Lib" ]; then - echo "CPython cache hit, skipping clone" -else +if [ ! -e "cpython" ]; then git clone --depth 1 --branch "$PYTHON_VERSION" https://github.com/python/cpython.git cpython +elif [ ! -d "cpython/.git" ] || [ ! -d "cpython/Lib" ] || + [ "$(git -C cpython rev-parse HEAD)" != "$(git -C cpython rev-parse --verify "$PYTHON_VERSION^{commit}" 2>/dev/null)" ] || + [ -n "$(git -C cpython status --porcelain)" ]; then + echo "ERROR: cpython must be a clean checkout of $PYTHON_VERSION. Move the existing directory aside and rerun." >&2 + exit 1 +else + echo "CPython $PYTHON_VERSION source already present." fi ``` diff --git a/.python-version b/.python-version index 9b8fe2fbb..f02113fe8 100644 --- a/.python-version +++ b/.python-version @@ -1 +1 @@ -3.14.8 +3.15.0 diff --git a/AGENTS.md b/AGENTS.md index ae24d4167..9aa4cc677 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,7 +4,7 @@ This document provides guidelines for AI coding agents (GitHub Copilot, Claude C ## Project Overview -RustPython is a Python 3 interpreter written in Rust, implementing Python 3.14.0+ compatibility. The project aims to provide: +RustPython is a Python 3 interpreter written in Rust, targeting Python 3.15 with compatibility work in progress. The project aims to provide: - A complete Python-3 environment entirely in Rust (not CPython bindings) - A clean implementation without compatibility hacks diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 637ecf099..710bb2caf 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -49,7 +49,7 @@ RustPython requires the following: stable version: `rustup update stable` - If you do not have Rust installed, use [rustup](https://rustup.rs/) to do so. -- CPython version 3.14 or higher +- CPython version 3.15 or higher - CPython can be installed by your operating system's package manager, from the [Python website](https://www.python.org/downloads/), or using a third-party distribution, such as diff --git a/Lib/encodings/__init__.py b/Lib/encodings/__init__.py index 8548bbe04..18f5da70c 100644 --- a/Lib/encodings/__init__.py +++ b/Lib/encodings/__init__.py @@ -56,6 +56,12 @@ def normalize_encoding(encoding): if isinstance(encoding, bytes): encoding = str(encoding, "ascii") + if not encoding.isascii(): + import warnings + warnings.warn( + "Support for non-ascii encoding names will be removed in 3.17", + DeprecationWarning, stacklevel=2) + chars = [] punct = False for c in encoding: diff --git a/Lib/sysconfig/__init__.py b/Lib/sysconfig/__init__.py index 029d5bcfc..45f74bc2d 100644 --- a/Lib/sysconfig/__init__.py +++ b/Lib/sysconfig/__init__.py @@ -219,18 +219,7 @@ if os.name == 'nt': if "_PYTHON_PROJECT_BASE" in os.environ: _PROJECT_BASE = _safe_realpath(os.environ["_PYTHON_PROJECT_BASE"]) -def is_python_build(check_home=None): - if check_home is not None: - import warnings - warnings.warn( - ( - 'The check_home argument of sysconfig.is_python_build is ' - 'deprecated and its value is ignored. ' - 'It will be removed in Python 3.15.' - ), - DeprecationWarning, - stacklevel=2, - ) +def is_python_build(): for fn in ("Setup", "Setup.local"): if os.path.isfile(os.path.join(_PROJECT_BASE, "Modules", fn)): return True diff --git a/Lib/test/datetimetester.py b/Lib/test/datetimetester.py index 1b71b71c5..3d3ac2dbc 100644 --- a/Lib/test/datetimetester.py +++ b/Lib/test/datetimetester.py @@ -2745,6 +2745,7 @@ class TestDateTime(TestDate): # If that assumption changes, this value can change as well self.assertEqual(max_ts, 253402300799.0) + @unittest.skipIf(sys.platform == "win32", "Windows doesn't support min timestamp") def test_fromtimestamp_limits(self): try: self.theclass.fromtimestamp(-2**32 - 1) @@ -2784,6 +2785,7 @@ class TestDateTime(TestDate): # OverflowError, especially on 32-bit platforms. self.theclass.fromtimestamp(ts) + @unittest.skipIf(sys.platform == "win32", "Windows doesn't support min timestamp") def test_utcfromtimestamp_limits(self): with self.assertWarns(DeprecationWarning): try: diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py index 75e2afa7a..41e4087ba 100644 --- a/Lib/test/pickletester.py +++ b/Lib/test/pickletester.py @@ -4394,6 +4394,8 @@ class AbstractPickleTests: def test_fast_save_enter_set(self): self.fast_save_enter(lambda i: set([i])) + # TODO: RUSTPYTHON; frozendict support: https://github.com/RustPython/RustPython/pull/8951 + @unittest.expectedFailure def test_fast_save_enter_frozendict(self): if self.py_version < (3, 15): self.skipTest('need frozendict') @@ -4441,6 +4443,8 @@ class AbstractPickleTests: depth=FAST_NESTING_LIMIT+1, compare_equal=False) + # TODO: RUSTPYTHON; frozendict support: https://github.com/RustPython/RustPython/pull/8951 + @unittest.expectedFailure def test_deep_nested_struct_frozendict(self): if self.py_version < (3, 15): self.skipTest('need frozendict') diff --git a/Lib/test/test_ast/test_ast.py b/Lib/test/test_ast/test_ast.py index aeaa4872e..b006e513f 100644 --- a/Lib/test/test_ast/test_ast.py +++ b/Lib/test/test_ast/test_ast.py @@ -460,14 +460,9 @@ class AST_Tests(unittest.TestCase): self.assertEqual(x._fields, 666) def test_classattrs(self): - with self.assertWarns(DeprecationWarning): - x = ast.Constant() + x = ast.Constant(42) self.assertEqual(x._fields, ('value', 'kind')) - with self.assertRaises(AttributeError): - x.value - - x = ast.Constant(42) self.assertEqual(x.value, 42) with self.assertRaises(AttributeError): @@ -487,11 +482,8 @@ class AST_Tests(unittest.TestCase): self.assertRaises(TypeError, ast.Constant, 1, None, 2) self.assertRaises(TypeError, ast.Constant, 1, None, 2, lineno=0) - # Arbitrary keyword arguments are supported (but deprecated) - with self.assertWarns(DeprecationWarning): - self.assertEqual(ast.Constant(1, foo='bar').foo, 'bar') - - with self.assertRaisesRegex(TypeError, "Constant got multiple values for argument 'value'"): + msg = "ast.Constant got multiple values for argument 'value'" + with self.assertRaisesRegex(TypeError, re.escape(msg)): ast.Constant(1, value=2) self.assertEqual(ast.Constant(42).value, 42) @@ -530,23 +522,24 @@ class AST_Tests(unittest.TestCase): self.assertEqual(x.body, body) def test_nodeclasses(self): - # Zero arguments constructor explicitly allowed (but deprecated) - with self.assertWarns(DeprecationWarning): - x = ast.BinOp() - self.assertEqual(x._fields, ('left', 'op', 'right')) - - # Random attribute allowed too - x.foobarbaz = 5 - self.assertEqual(x.foobarbaz, 5) + # Zero arguments constructor is not allowed + msg = "ast.BinOp.__init__ missing 3 required positional arguments: 'left', 'op', and 'right'" + self.assertRaisesRegex(TypeError, re.escape(msg), ast.BinOp) n1 = ast.Constant(1) n3 = ast.Constant(3) addop = ast.Add() x = ast.BinOp(n1, addop, n3) + self.assertEqual(x._fields, ('left', 'op', 'right')) self.assertEqual(x.left, n1) self.assertEqual(x.op, addop) self.assertEqual(x.right, n3) + # Arbitrary attributes are allowed + x.foobarbaz = 5 + self.assertEqual(x.foobarbaz, 5) + self.assertEqual(x._fields, ('left', 'op', 'right')) + x = ast.BinOp(1, 2, 3) self.assertEqual(x.left, 1) self.assertEqual(x.op, 2) @@ -570,10 +563,10 @@ class AST_Tests(unittest.TestCase): self.assertEqual(x.right, 3) self.assertEqual(x.lineno, 0) - # Random kwargs also allowed (but deprecated) - with self.assertWarns(DeprecationWarning): - x = ast.BinOp(1, 2, 3, foobarbaz=42) - self.assertEqual(x.foobarbaz, 42) + # Arbitrary keyword arguments are not allowed + msg = "ast.BinOp.__init__ got an unexpected keyword argument 'foobarbaz'" + with self.assertRaisesRegex(TypeError, re.escape(msg)): + ast.BinOp(1, 2, 3, foobarbaz=42) def test_no_fields(self): # this used to fail because Sub._fields was None @@ -3183,11 +3176,10 @@ class ASTConstructorTests(unittest.TestCase): args = ast.arguments() self.assertEqual(args.args, []) self.assertEqual(args.posonlyargs, []) - with self.assertWarnsRegex(DeprecationWarning, - r"FunctionDef\.__init__ missing 1 required positional argument: 'name'"): - node = ast.FunctionDef(args=args) - self.assertNotHasAttr(node, "name") - self.assertEqual(node.decorator_list, []) + msg = "ast.FunctionDef.__init__ missing 1 required positional argument: 'name'" + with self.assertRaisesRegex(TypeError, re.escape(msg)): + ast.FunctionDef(args=args) + node = ast.FunctionDef(name='foo', args=args) self.assertEqual(node.name, 'foo') self.assertEqual(node.decorator_list, []) @@ -3205,9 +3197,8 @@ class ASTConstructorTests(unittest.TestCase): self.assertEqual(name3.id, "x") self.assertIsInstance(name3.ctx, ast.Del) - with self.assertWarnsRegex(DeprecationWarning, - r"Name\.__init__ missing 1 required positional argument: 'id'"): - name3 = ast.Name() + msg = "ast.Name.__init__ missing 1 required positional argument: 'id'" + self.assertRaisesRegex(TypeError, re.escape(msg), ast.Name) def test_custom_subclass_with_no_fields(self): class NoInit(ast.AST): @@ -3246,20 +3237,18 @@ class ASTConstructorTests(unittest.TestCase): self.assertEqual(obj.a, 1) self.assertEqual(obj.b, 2) - with self.assertWarnsRegex(DeprecationWarning, - r"MyAttrs.__init__ got an unexpected keyword argument 'c'."): - obj = MyAttrs(c=3) + msg = "MyAttrs.__init__ got an unexpected keyword argument 'c'" + with self.assertRaisesRegex(TypeError, re.escape(msg)): + MyAttrs(c=3) def test_fields_and_types_no_default(self): class FieldsAndTypesNoDefault(ast.AST): _fields = ('a',) _field_types = {'a': int} - with self.assertWarnsRegex(DeprecationWarning, - r"FieldsAndTypesNoDefault\.__init__ missing 1 required positional argument: 'a'\."): - obj = FieldsAndTypesNoDefault() - with self.assertRaises(AttributeError): - obj.a + msg = "FieldsAndTypesNoDefault.__init__ missing 1 required positional argument: 'a'" + self.assertRaisesRegex(TypeError, re.escape(msg), FieldsAndTypesNoDefault) + obj = FieldsAndTypesNoDefault(a=1) self.assertEqual(obj.a, 1) @@ -3270,13 +3259,8 @@ class ASTConstructorTests(unittest.TestCase): a: int | None = None b: int | None = None - with self.assertWarnsRegex( - DeprecationWarning, - r"Field 'b' is missing from MoreFieldsThanTypes\._field_types" - ): - obj = MoreFieldsThanTypes() - self.assertIs(obj.a, None) - self.assertIs(obj.b, None) + msg = r"Field 'b' is missing from .*\.MoreFieldsThanTypes\._field_types" + self.assertRaisesRegex(TypeError, msg, MoreFieldsThanTypes) obj = MoreFieldsThanTypes(a=1, b=2) self.assertEqual(obj.a, 1) @@ -3288,8 +3272,7 @@ class ASTConstructorTests(unittest.TestCase): _field_types = {'a': int} # This should not crash - with self.assertWarnsRegex(DeprecationWarning, r"Field b'\\xff\\xff.*' .*"): - obj = BadFields() + self.assertRaisesRegex(TypeError, r"Field b'\\xff\\xff.*' .*", BadFields) def test_complete_field_types(self): class _AllFieldTypes(ast.AST): diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py index 8dd64afcc..a2ea05353 100644 --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -44,7 +44,6 @@ from importlib.machinery import BuiltinImporter _loader = __loader__ if __loader__ is BuiltinImporter else type(__loader__) print('__loader__==%a' % _loader) print('__file__==%a' % __file__) -print('__cached__==%a' % __cached__) print('__package__==%r' % __package__) # Check PEP 451 details import os.path @@ -58,8 +57,6 @@ if __package__ is not None: assertEqual(__spec__.parent, __package__) assertIdentical(__spec__.submodule_search_locations, None) assertEqual(__spec__.origin, __file__) - if __spec__.cached is not None: - assertEqual(__spec__.cached, __cached__) # Check the sys module import sys assertIdentical(globals(), sys.modules[__name__].__dict__) diff --git a/Lib/test/test_codecs.py b/Lib/test/test_codecs.py index fd21c7b05..bcaffb79c 100644 --- a/Lib/test/test_codecs.py +++ b/Lib/test/test_codecs.py @@ -13,6 +13,7 @@ import warnings from test import support from test.support import os_helper +from test.support import warnings_helper try: import _testlimitedcapi @@ -3875,32 +3876,27 @@ class Rot13UtilTest(unittest.TestCase): class CodecNameNormalizationTest(unittest.TestCase): """Test codec name normalization""" def test_codecs_lookup(self): - FOUND = (1, 2, 3, 4) - NOT_FOUND = (None, None, None, None) def search_function(encoding): - if encoding == "aaa_8": - return FOUND + if encoding.startswith("test."): + return (encoding, 2, 3, 4) else: - return NOT_FOUND + return None codecs.register(search_function) self.addCleanup(codecs.unregister, search_function) - self.assertEqual(FOUND, codecs.lookup('aaa_8')) - self.assertEqual(FOUND, codecs.lookup('AAA-8')) - self.assertEqual(FOUND, codecs.lookup('AAA---8')) - self.assertEqual(FOUND, codecs.lookup('AAA 8')) - self.assertEqual(FOUND, codecs.lookup('aaa\xe9\u20ac-8')) - self.assertEqual(NOT_FOUND, codecs.lookup('AAA.8')) - self.assertEqual(NOT_FOUND, codecs.lookup('AAA...8')) - self.assertEqual(NOT_FOUND, codecs.lookup('BBB-8')) - self.assertEqual(NOT_FOUND, codecs.lookup('BBB.8')) - self.assertEqual(NOT_FOUND, codecs.lookup('a\xe9\u20ac-8')) + self.assertEqual(codecs.lookup('test.aaa_8'), ('test.aaa_8', 2, 3, 4)) + self.assertEqual(codecs.lookup('TEST.AAA-8'), ('test.aaa-8', 2, 3, 4)) + self.assertEqual(codecs.lookup('TEST.AAA 8'), ('test.aaa-8', 2, 3, 4)) + self.assertEqual(codecs.lookup('TEST.AAA---8'), ('test.aaa---8', 2, 3, 4)) + self.assertEqual(codecs.lookup('TEST.AAA 8'), ('test.aaa---8', 2, 3, 4)) + self.assertEqual(codecs.lookup('TEST.AAA.8'), ('test.aaa.8', 2, 3, 4)) + self.assertEqual(codecs.lookup('TEST.AAA...8'), ('test.aaa...8', 2, 3, 4)) + with self.assertWarns(DeprecationWarning): + self.assertEqual(codecs.lookup('TEST.AAA\xe9\u20ac-8'), ('test.aaa\xe9\u20ac-8', 2, 3, 4)) def test_encodings_normalize_encoding(self): - # encodings.normalize_encoding() ignores non-ASCII characters. normalize = encodings.normalize_encoding self.assertEqual(normalize('utf_8'), 'utf_8') - self.assertEqual(normalize('utf\xE9\u20AC\U0010ffff-8'), 'utf_8') self.assertEqual(normalize('utf 8'), 'utf_8') # encodings.normalize_encoding() doesn't convert # characters to lower case. @@ -3908,6 +3904,11 @@ class CodecNameNormalizationTest(unittest.TestCase): self.assertEqual(normalize('utf.8'), 'utf.8') self.assertEqual(normalize('utf...8'), 'utf...8') + # Non-ASCII *encoding* is deprecated. + msg = "Support for non-ascii encoding names will be removed in 3.17" + with warnings_helper.check_warnings((msg, DeprecationWarning)): + self.assertEqual(normalize('utf\xE9\u20AC\U0010ffff-8'), 'utf_8') + class CodecCacheTest(unittest.TestCase): def test_cache_bounded(self): diff --git a/Lib/test/test_peepholer.py b/Lib/test/test_peepholer.py index d06b9a4b6..66af5ae7c 100644 --- a/Lib/test/test_peepholer.py +++ b/Lib/test/test_peepholer.py @@ -743,7 +743,8 @@ class TestTranforms(BytecodeTestCase): eval("'%s%z' % (x,)", {'x': 1234}) with self.assertRaisesRegex(ValueError, 'unsupported format character'): eval("'%s%z' % (x, 5)", {'x': 1234}) - with self.assertRaisesRegex(TypeError, 'a real number is required, not str'): + with self.assertRaisesRegex(TypeError, + 'format argument 1: %d requires a real number, not str'): eval("'%d' % (x,)", {'x': '1234'}) with self.assertRaisesRegex(TypeError, 'an integer is required, not float'): eval("'%x' % (x,)", {'x': 1234.56}) diff --git a/Lib/test/test_struct.py b/Lib/test/test_struct.py index e18bad9c9..13e323de9 100644 --- a/Lib/test/test_struct.py +++ b/Lib/test/test_struct.py @@ -622,7 +622,7 @@ class StructTest(ComplexesAreIdenticalMixin, unittest.TestCase): self.assertEqual(s.pack(1, 2), packed) self.assertEqual(s.unpack(packed), (1, 2)) - with self.assertRaises(UnicodeEncodeError): + with self.assertRaises(ValueError): s.__init__('\udc00') self.assertEqual(s.format, '>hh') self.assertEqual(s.pack(1, 2), packed) @@ -882,7 +882,7 @@ class StructTest(ComplexesAreIdenticalMixin, unittest.TestCase): self.assertRaises(RuntimeError, S.pack_into, spam, 1) self.assertRaises(RuntimeError, S.unpack, spam) self.assertRaises(RuntimeError, S.unpack_from, spam) - self.assertRaises(RuntimeError, getattr, S, 'format') + self.assertRaises(AttributeError, getattr, S, 'format') self.assertRaises(RuntimeError, S.__sizeof__) self.assertRaises(RuntimeError, repr, S) self.assertEqual(S.size, -1) diff --git a/Lib/test/test_time.py b/Lib/test/test_time.py index f66a37edb..298c45faf 100644 --- a/Lib/test/test_time.py +++ b/Lib/test/test_time.py @@ -170,7 +170,7 @@ class TimeTestCase(unittest.TestCase): # Improved exception #81267 with self.assertRaises(TypeError) as errmsg: time.sleep([]) - self.assertIn("integer or float", str(errmsg.exception)) + self.assertIn("real number", str(errmsg.exception)) def test_sleep(self): for value in [-0.0, 0, 0.0, 1e-100, 1e-9, 1e-6, 1, 1.2]: @@ -498,12 +498,13 @@ class TimeTestCase(unittest.TestCase): def test_mktime(self): # Issue #1726687 for t in (-2, -1, 0, 1): + t_struct = time.localtime(t) try: - tt = time.localtime(t) + t1 = time.mktime(t_struct) except (OverflowError, OSError): pass else: - self.assertEqual(time.mktime(tt), t) + self.assertEqual(t1, t) # Issue #13309: passing extreme values to mktime() or localtime() # borks the glibc's internal timezone data. diff --git a/README.md b/README.md index 1fc891b79..c3aaa00c9 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ # [RustPython](https://rustpython.github.io/) -A Python-3 (CPython >= 3.14.0) Interpreter written in Rust :snake: :scream: +A Python-3 (CPython >= 3.15.0) Interpreter written in Rust :snake: :scream: :metal:. [![Build Status](https://github.com/RustPython/RustPython/workflows/CI/badge.svg)](https://github.com/RustPython/RustPython/actions?query=workflow%3ACI) diff --git a/crates/host_env/src/time.rs b/crates/host_env/src/time.rs index 85427bb9a..94448a8ac 100644 --- a/crates/host_env/src/time.rs +++ b/crates/host_env/src/time.rs @@ -145,23 +145,84 @@ pub fn current_time_t() -> TimeT { } #[cfg(windows)] -#[cfg_attr(target_env = "musl", allow(deprecated))] -pub fn gmtime_from_timestamp(when: TimeT) -> std::io::Result { +#[derive(Debug)] +pub enum TimestampError { + FiletimeOutOfRange, + Os(std::io::Error), +} + +#[cfg(windows)] +fn windows_filetime(when: TimeT, local: bool) -> Result { + use windows_sys::Win32::{ + Foundation::{FILETIME, SYSTEMTIME}, + System::Time::{FileTimeToSystemTime, SystemTimeToTzSpecificLocalTime}, + }; + + const SECS_BETWEEN_EPOCHS: i64 = 11_644_473_600; + if when < -SECS_BETWEEN_EPOCHS { + return Err(TimestampError::FiletimeOutOfRange); + } + // Only negative timestamps reach this fallback, so the FILETIME cannot overflow. + debug_assert!(when < 0); + let ticks = ((when + SECS_BETWEEN_EPOCHS) as u64) * 10_000_000; + let filetime = FILETIME { + dwLowDateTime: ticks as u32, + dwHighDateTime: (ticks >> 32) as u32, + }; + let mut utc = SYSTEMTIME::default(); + if unsafe { FileTimeToSystemTime(&filetime, &mut utc) } == 0 { + return Err(TimestampError::Os(std::io::Error::last_os_error())); + } + let time = if local { + let mut result = SYSTEMTIME::default(); + // Unlike FileTimeToLocalFileTime, this applies DST rules for the timestamp. + if unsafe { SystemTimeToTzSpecificLocalTime(core::ptr::null(), &utc, &mut result) } == 0 { + return Err(TimestampError::Os(std::io::Error::last_os_error())); + } + result + } else { + utc + }; + const DAYS_BEFORE_MONTH: [i32; 12] = [0, 31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334]; + let year = i32::from(time.wYear); + let leap = year % 4 == 0 && (year % 100 != 0 || year % 400 == 0); + let yday = DAYS_BEFORE_MONTH[usize::from(time.wMonth - 1)] + i32::from(time.wDay) - 1 + + i32::from(time.wMonth > 2 && leap); + Ok(libc::tm { + tm_year: year - 1900, + tm_mon: i32::from(time.wMonth) - 1, + tm_mday: i32::from(time.wDay), + tm_hour: i32::from(time.wHour), + tm_min: i32::from(time.wMinute), + tm_sec: i32::from(time.wSecond), + tm_wday: i32::from(time.wDayOfWeek), + tm_yday: yday, + tm_isdst: if local { -1 } else { 0 }, + }) +} + +#[cfg(windows)] +pub fn gmtime_from_timestamp(when: TimeT) -> Result { + if when < 0 { + return windows_filetime(when, false); + } let mut out = core::mem::MaybeUninit::::uninit(); let err = unsafe { _gmtime64_s(out.as_mut_ptr(), &when) }; if err != 0 { - return Err(crate::os::io_error_from_errno(err)); + return Err(TimestampError::Os(crate::os::io_error_from_errno(err))); } Ok(unsafe { out.assume_init() }) } #[cfg(windows)] -#[cfg_attr(target_env = "musl", allow(deprecated))] -pub fn localtime_from_timestamp(when: TimeT) -> std::io::Result { +pub fn localtime_from_timestamp(when: TimeT) -> Result { + if when < 0 { + return windows_filetime(when, true); + } let mut out = core::mem::MaybeUninit::::uninit(); let err = unsafe { _localtime64_s(out.as_mut_ptr(), &when) }; if err != 0 { - return Err(crate::os::io_error_from_errno(err)); + return Err(TimestampError::Os(crate::os::io_error_from_errno(err))); } Ok(unsafe { out.assume_init() }) } diff --git a/crates/stdlib/src/csv.rs b/crates/stdlib/src/csv.rs index 51bba58c9..792546a6d 100644 --- a/crates/stdlib/src/csv.rs +++ b/crates/stdlib/src/csv.rs @@ -473,12 +473,11 @@ mod _csv { } else { obj }; - // CPython ignores attribute lookup errors, but not errors converting - // the retrieved values. Read all non-overridden attributes first. + // Read all non-overridden attributes before converting their values. macro_rules! fill_from_dialect { ($($name:ident),* $(,)?) => { $(if self.$name.is_none() { - self.$name = obj.get_attr(stringify!($name), vm).ok(); + self.$name = vm.get_attribute_opt(&obj, stringify!($name))?; })* }; } diff --git a/crates/stdlib/src/math.rs b/crates/stdlib/src/math.rs index 3f5dab6b9..97e34efcc 100644 --- a/crates/stdlib/src/math.rs +++ b/crates/stdlib/src/math.rs @@ -93,24 +93,51 @@ mod math { pymath::math::expm1(x.into_float()).map_err(|err| pymath_exception(err, vm)) } + fn logarithm( + x: &PyObject, + float_log: impl Fn(f64) -> Result, + int_log: impl Fn(&BigInt) -> Result, + vm: &VirtualMachine, + ) -> PyResult { + let integer_log = |i: &PyInt| { + int_log(i.as_bigint()).map_err(|err| match err { + pymath::Error::EDOM => vm.new_value_error("expected a positive input"), + _ => pymath_exception(err, vm), + }) + }; + if let Some(i) = x.downcast_ref::() { + return integer_log(i); + } + let val = match x.try_float(vm) { + Ok(value) => value.to_f64(), + Err(error) => { + if error.fast_isinstance(vm.ctx.exceptions.overflow_error) + && let Some(index) = x.try_index_opt(vm) + { + let index = index?; + return integer_log(&index); + } + return Err(error); + } + }; + float_log(val).map_err(|err| match err { + pymath::Error::EDOM => vm.new_value_error(format!( + "expected a positive input, got {}", + super::float_repr(val) + )), + _ => pymath_exception(err, vm), + }) + } + #[pyfunction] fn log(x: PyObjectRef, base: OptionalArg, vm: &VirtualMachine) -> PyResult { let natural_log = |x: &PyObject| { - // Handle BigInt specially for large values (only for actual int type, not float) - if let Some(i) = x.downcast_ref::() { - return pymath::math::log_bigint(i.as_bigint(), None).map_err(|err| match err { - pymath::Error::EDOM => vm.new_value_error("expected a positive input"), - _ => pymath_exception(err, vm), - }); - } - let val = x.try_float(vm)?.to_f64(); - pymath::math::log(val, None).map_err(|err| match err { - pymath::Error::EDOM => vm.new_value_error(format!( - "expected a positive input, got {}", - super::float_repr(val) - )), - _ => pymath_exception(err, vm), - }) + logarithm( + x, + |value| pymath::math::log(value, None), + |value| pymath::math::log_bigint(value, None), + vm, + ) }; let numerator = natural_log(&x)?; match base { @@ -133,40 +160,12 @@ mod math { #[pyfunction] fn log2(x: PyObjectRef, vm: &VirtualMachine) -> PyResult { - // Handle BigInt specially for large values (only for actual int type, not float) - if let Some(i) = x.downcast_ref::() { - return pymath::math::log2_bigint(i.as_bigint()).map_err(|err| match err { - pymath::Error::EDOM => vm.new_value_error("expected a positive input"), - _ => pymath_exception(err, vm), - }); - } - let val = x.try_float(vm)?.to_f64(); - pymath::math::log2(val).map_err(|err| match err { - pymath::Error::EDOM => vm.new_value_error(format!( - "expected a positive input, got {}", - super::float_repr(val) - )), - _ => pymath_exception(err, vm), - }) + logarithm(&x, pymath::math::log2, pymath::math::log2_bigint, vm) } #[pyfunction] fn log10(x: PyObjectRef, vm: &VirtualMachine) -> PyResult { - // Handle BigInt specially for large values (only for actual int type, not float) - if let Some(i) = x.downcast_ref::() { - return pymath::math::log10_bigint(i.as_bigint()).map_err(|err| match err { - pymath::Error::EDOM => vm.new_value_error("expected a positive input"), - _ => pymath_exception(err, vm), - }); - } - let val = x.try_float(vm)?.to_f64(); - pymath::math::log10(val).map_err(|err| match err { - pymath::Error::EDOM => vm.new_value_error(format!( - "expected a positive input, got {}", - super::float_repr(val) - )), - _ => pymath_exception(err, vm), - }) + logarithm(&x, pymath::math::log10, pymath::math::log10_bigint, vm) } #[pyfunction] diff --git a/crates/stdlib/src/mmap.rs b/crates/stdlib/src/mmap.rs index 371a800c4..790a7bc0f 100644 --- a/crates/stdlib/src/mmap.rs +++ b/crates/stdlib/src/mmap.rs @@ -854,9 +854,16 @@ mod mmap { fn get_find_range(&self, options: &FindOptions) -> (usize, usize) { let size = self.__len__(); - let start = options - .start - .map_or_else(|| self.pos(), |start| start.saturated_at(size)); + let start = options.start.map_or_else( + || self.pos(), + |start| { + if start < 0 { + start.saturated_at(size) + } else { + start as usize + } + }, + ); let end = options.end.map_or(size, |end| end.saturated_at(size)); (start, end) } diff --git a/crates/stdlib/src/pystruct.rs b/crates/stdlib/src/pystruct.rs index f586c1448..d71452f6b 100644 --- a/crates/stdlib/src/pystruct.rs +++ b/crates/stdlib/src/pystruct.rs @@ -30,33 +30,18 @@ pub(crate) mod _struct { impl TryFromObject for IntoStructFormatBytes { fn try_from_object(vm: &VirtualMachine, obj: PyObjectRef) -> PyResult { - // CPython turns str to bytes (via str.encode('ascii')) but we keep str. - // The error reporting for non-ASCII input still matches CPython: - // - str input with non-ASCII char: UnicodeEncodeError, the same exception - // str.encode('ascii') would produce. - // - bytes input with non-ASCII byte: struct.error("bad char in struct format"), - // matching CPython where bytes are passed through to the format parser. + // Keep ASCII formats as str and reject non-ASCII str or bytes with + // the same ValueError as CPython's format parser. let fmt = match_class!(match obj { s @ PyStr => { if !s.isascii() { - let start = s - .as_wtf8() - .code_points() - .position(|cp| !cp.to_char().is_some_and(|c| c.is_ascii())) - .unwrap_or(0); - return Err(vm.new_unicode_encode_error( - vm.ctx.new_str("ascii"), - s, - start, - start + 1, - vm.ctx.new_str("ordinal not in range(128)"), - )); + return Err(vm.new_value_error("non-ASCII character in struct format")); } s } b @ PyBytes => { let ascii_str = ascii::AsciiStr::from_ascii(&b) - .map_err(|_| new_struct_error(vm, "bad char in struct format"))?; + .map_err(|_| vm.new_value_error("non-ASCII character in struct format"))?; vm.ctx.new_str(ascii_str) } other => @@ -322,7 +307,13 @@ pub(crate) mod _struct { #[pygetset] fn format(zelf: &Py, vm: &VirtualMachine) -> PyResult { - Ok(zelf.ready(vm)?.format.clone()) + zelf.inner + .read() + .as_ref() + .map(|inner| inner.format.clone()) + .ok_or_else(|| { + vm.new_no_attribute_error(zelf.as_object().to_owned(), vm.ctx.new_str("format")) + }) } // The size an uninitialized `Struct` reports, which no format has diff --git a/crates/vm/build.rs b/crates/vm/build.rs index 867cf5775..098cf05aa 100644 --- a/crates/vm/build.rs +++ b/crates/vm/build.rs @@ -22,8 +22,8 @@ fn main() { } println!("cargo:rerun-if-changed=../../Lib/importlib/_bootstrap.py"); - // = 3.14.0alpha - python_version(3, 14, 0, "alpha", 0); + // = 3.15.0alpha + python_version(3, 15, 0, "alpha", 0); println!("cargo:rustc-env=RUSTPYTHON_GIT_HASH={}", git_hash()); println!( diff --git a/crates/vm/src/builtins/mappingproxy.rs b/crates/vm/src/builtins/mappingproxy.rs index 7342fb3c0..3e53f1729 100644 --- a/crates/vm/src/builtins/mappingproxy.rs +++ b/crates/vm/src/builtins/mappingproxy.rs @@ -193,9 +193,11 @@ impl Py { op.method_name(&vm.ctx).as_str() )) })?; + if !matches!(op, PyComparisonOp::Eq | PyComparisonOp::Ne) { + return Ok(Either::B(PyComparisonValue::NotImplemented)); + } let obj = zelf.to_object(vm)?; - // CPython parity (Objects/descrobject.c::mappingproxy_richcompare): - // delegate to PyObject_RichCompare on the underlying mapping. + // Equality comparisons preserve the underlying mapping's result. obj.rich_compare(other.to_owned(), op, vm).map(Either::A) } diff --git a/crates/vm/src/builtins/memory.rs b/crates/vm/src/builtins/memory.rs index 33481ce22..6dd4d71fa 100644 --- a/crates/vm/src/builtins/memory.rs +++ b/crates/vm/src/builtins/memory.rs @@ -747,7 +747,7 @@ impl PyMemoryView { let format_str = format.as_str(); let Some(dest_char) = Self::native_fmtchar(format_str) else { return Err(vm.new_value_error( - "memoryview: destination format must be a native single character format prefixed with an optional '@'", + "memoryview: destination format must be a native format prefixed with an optional '@'", )); }; // One side has to be bytes. Casting between two item types would diff --git a/crates/vm/src/cformat.rs b/crates/vm/src/cformat.rs index 1b281880c..abb845f9d 100644 --- a/crates/vm/src/cformat.rs +++ b/crates/vm/src/cformat.rs @@ -26,31 +26,44 @@ use crate::{ stdlib::builtins, }; +enum FormatArgument<'a> { + Scalar, + Positional(usize), + Mapping(&'a PyObject), +} + +impl FormatArgument<'_> { + fn description(&self, vm: &VirtualMachine) -> PyResult { + Ok(match self { + Self::Scalar => "format argument".to_owned(), + Self::Positional(index) => format!("format argument {index}"), + Self::Mapping(key) => format!("format argument {}", key.repr(vm)?), + }) + } +} + fn format_decimal_object( vm: &VirtualMachine, spec: &CFormatSpec, obj: &PyObject, + argument: FormatArgument<'_>, ) -> PyResult { - let type_error = || { - vm.new_type_error(format!( - "%{} format: a real number is required, not {}", - spec.format_type.to_char(), - obj.class().slot_name() - )) - }; // Decimal conversions prefer __int__; __index__ is only a fallback when absent. - let i = obj - .number() - .int(vm) - .or_else(|| obj.try_index_opt(vm)) - .ok_or_else(type_error)? - .map_err(|error| { - if error.fast_isinstance(vm.ctx.exceptions.type_error) { - type_error() - } else { - error - } - })?; + let conversion = obj.number().int(vm).or_else(|| obj.try_index_opt(vm)); + let i = match conversion { + Some(Ok(value)) => value, + Some(Err(error)) if !error.fast_isinstance(vm.ctx.exceptions.type_error) => { + return Err(error); + } + _ => { + return Err(vm.new_type_error(format!( + "{}: %{} requires a real number, not {}", + argument.description(vm)?, + spec.format_type.to_char(), + obj.class().slot_name() + ))); + } + }; check_int_to_str_digits(i.as_bigint(), vm)?; Ok(spec.format_number(i.as_bigint())) } @@ -59,6 +72,7 @@ fn spec_format_bytes( vm: &VirtualMachine, spec: &CFormatSpec, obj: PyObjectRef, + argument: FormatArgument<'_>, ) -> PyResult> { match &spec.format_type { CFormatType::Unsupported { ch, index } => Err(vm.new_value_error(format!( @@ -106,7 +120,7 @@ fn spec_format_bytes( check_int_to_str_digits(&bigint, vm)?; Ok(spec.format_number(&bigint).into_bytes()) } else { - format_decimal_object(vm, spec, &obj).map(String::into_bytes) + format_decimal_object(vm, spec, &obj, argument).map(String::into_bytes) } } _ => { @@ -188,6 +202,7 @@ fn spec_format_string( vm: &VirtualMachine, spec: &CFormatSpec, obj: PyObjectRef, + argument: FormatArgument<'_>, ) -> PyResult { match &spec.format_type { CFormatType::Unsupported { ch, index } => Err(vm.new_value_error(format!( @@ -217,7 +232,7 @@ fn spec_format_string( check_int_to_str_digits(&bigint, vm)?; Ok(spec.format_number(&bigint).into()) } else { - format_decimal_object(vm, spec, &obj).map(Into::into) + format_decimal_object(vm, spec, &obj, argument).map(Into::into) } } _ => { @@ -379,8 +394,14 @@ pub(crate) fn cformat_bytes( CFormatPart::Literal(literal) => result.extend(literal), CFormatPart::Spec(CFormatSpecKeyed { mapping_key, spec }) => { let key = mapping_key.unwrap(); - let value = values_obj.get_item(&key, vm)?; - let part_result = spec_format_bytes(vm, &spec, value)?; + let key = vm.ctx.new_bytes(key); + let value = values_obj.get_item(key.as_object(), vm)?; + let part_result = spec_format_bytes( + vm, + &spec, + value, + FormatArgument::Mapping(key.as_object()), + )?; result.extend(part_result); } } @@ -392,7 +413,7 @@ pub(crate) fn cformat_bytes( // tuple let mut slice_iter; let mut once_iter; - let mut value_iter: &mut dyn Iterator = + let value_iter: &mut dyn Iterator = if let Some(tup) = values_obj.downcast_ref::() { slice_iter = tup.as_slice().iter().map(|v| &**v); &mut slice_iter @@ -400,6 +421,9 @@ pub(crate) fn cformat_bytes( once_iter = core::iter::once(values_obj); &mut once_iter }; + let argument_index = core::cell::Cell::new(0); + let mut value_iter = value_iter.inspect(|_| argument_index.set(argument_index.get() + 1)); + let is_tuple = values_obj.downcast_ref::().is_some(); for (_, part) in format { match part { @@ -417,7 +441,12 @@ pub(crate) fn cformat_bytes( return Err(vm.new_type_error("not enough arguments for format string")); }; - let part_result = spec_format_bytes(vm, &spec, value.to_owned())?; + let argument = if is_tuple { + FormatArgument::Positional(argument_index.get()) + } else { + FormatArgument::Scalar + }; + let part_result = spec_format_bytes(vm, &spec, value.to_owned(), argument)?; result.extend(part_result); } } @@ -479,8 +508,15 @@ pub(crate) fn cformat_string( match part { CFormatPart::Literal(literal) => result.push_wtf8(&literal), CFormatPart::Spec(CFormatSpecKeyed { mapping_key, spec }) => { - let value = values_obj.get_item(&mapping_key.unwrap(), vm)?; - let part_result = spec_format_string(vm, &spec, value)?; + let key = mapping_key.unwrap(); + let key = vm.ctx.new_str(key); + let value = values_obj.get_item(key.as_object(), vm)?; + let part_result = spec_format_string( + vm, + &spec, + value, + FormatArgument::Mapping(key.as_object()), + )?; result.push_wtf8(&part_result); } } @@ -492,7 +528,7 @@ pub(crate) fn cformat_string( // tuple let mut slice_iter; let mut once_iter; - let mut value_iter: &mut dyn Iterator = + let value_iter: &mut dyn Iterator = if let Some(tup) = values_obj.downcast_ref::() { slice_iter = tup.as_slice().iter().map(|v| &**v); &mut slice_iter @@ -500,6 +536,9 @@ pub(crate) fn cformat_string( once_iter = core::iter::once(values_obj); &mut once_iter }; + let argument_index = core::cell::Cell::new(0); + let mut value_iter = value_iter.inspect(|_| argument_index.set(argument_index.get() + 1)); + let is_tuple = values_obj.downcast_ref::().is_some(); for (_, part) in format { match part { @@ -517,7 +556,12 @@ pub(crate) fn cformat_string( return Err(vm.new_type_error("not enough arguments for format string")); }; - let part_result = spec_format_string(vm, &spec, value.to_owned())?; + let argument = if is_tuple { + FormatArgument::Positional(argument_index.get()) + } else { + FormatArgument::Scalar + }; + let part_result = spec_format_string(vm, &spec, value.to_owned(), argument)?; result.push_wtf8(&part_result); } } diff --git a/crates/vm/src/codecs.rs b/crates/vm/src/codecs.rs index f3b6b1334..32724984e 100644 --- a/crates/vm/src/codecs.rs +++ b/crates/vm/src/codecs.rs @@ -253,7 +253,7 @@ impl CodecsRegistry { } pub(crate) fn register_manual(&self, name: &str, codec: PyCodec) { - let name = normalize_encoding_name(name); + let name = normalize_registry_name(name); self.inner .write() .search_cache @@ -262,7 +262,7 @@ impl CodecsRegistry { pub fn lookup(&self, encoding: &str, vm: &VirtualMachine) -> PyResult { let original_encoding = encoding; - let encoding = normalize_encoding_name(encoding); + let encoding = normalize_registry_name(encoding); let search_path = { let inner = self.inner.read(); if let Some(codec) = inner.search_cache.get(encoding.as_ref()) { @@ -308,7 +308,7 @@ impl CodecsRegistry { } pub fn forget(&self, encoding: &str) -> Option { - let encoding = normalize_encoding_name(encoding); + let encoding = normalize_registry_name(encoding); self.inner.write().search_cache.remove(encoding.as_ref()) } @@ -623,6 +623,26 @@ impl FastCodec { } } +fn normalize_registry_name(encoding: &str) -> Cow<'_, str> { + if !encoding + .bytes() + .any(|b| b.is_ascii_uppercase() || b == b' ') + { + return encoding.into(); + } + encoding + .chars() + .map(|c| { + if c == ' ' { + '-' + } else { + c.to_ascii_lowercase() + } + }) + .collect::() + .into() +} + fn normalize_encoding_name(encoding: &str) -> Cow<'_, str> { // _Py_normalize_encoding: collapse non-alphanumeric/non-dot chars into // single underscore, strip non-ASCII, lowercase ASCII letters. diff --git a/crates/vm/src/exceptions.rs b/crates/vm/src/exceptions.rs index dbbdb9204..f7869fccd 100644 --- a/crates/vm/src/exceptions.rs +++ b/crates/vm/src/exceptions.rs @@ -1576,6 +1576,18 @@ impl ToPyException for rustpython_host_env::socket::AncillaryPackError { } } +#[cfg(windows)] +impl ToPyException for rustpython_host_env::time::TimestampError { + fn to_pyexception(&self, vm: &VirtualMachine) -> PyBaseExceptionRef { + match self { + Self::FiletimeOutOfRange => { + vm.new_overflow_error("timestamp out of range for Windows FILETIME") + } + Self::Os(error) => error.to_pyexception(vm), + } + } +} + #[cfg(any(unix, windows))] impl ToPyException for rustpython_host_env::time::CheckedTmError { fn to_pyexception(&self, vm: &VirtualMachine) -> PyBaseExceptionRef { diff --git a/crates/vm/src/frame.rs b/crates/vm/src/frame.rs index b56dbc9e9..2c9d60751 100644 --- a/crates/vm/src/frame.rs +++ b/crates/vm/src/frame.rs @@ -6,9 +6,10 @@ use crate::{ AsObject, Py, PyExact, PyObject, PyObjectRef, PyPayload, PyRef, PyResult, PyStackRef, TryFromObject, VirtualMachine, builtins::{ - PyBaseException, PyBaseExceptionRef, PyBaseObject, PyCode, PyCoroutine, PyDict, PyDictRef, - PyFloat, PyFrozenSet, PyGenerator, PyInt, PyInterpolation, PyList, PyModule, PyProperty, - PySet, PySlice, PyStr, PyStrInterned, PyTemplate, PyTraceback, PyType, PyUtf8Str, + PyBaseException, PyBaseExceptionRef, PyBaseObject, PyBytes, PyCode, PyCoroutine, PyDict, + PyDictRef, PyFloat, PyFrozenSet, PyGenerator, PyInt, PyInterpolation, PyList, PyModule, + PyProperty, PySet, PySlice, PyStr, PyStrInterned, PyTemplate, PyTraceback, PyType, + PyUtf8Str, builtin_func::PyNativeFunction, descriptor::{PyMemberDescriptor, PyMethodDescriptor}, frame::stack_analysis, @@ -3098,7 +3099,7 @@ pub(crate) fn release_datastack_frame(frame: &Py, vm: &VirtualMachi } type BinaryOpExtendGuard = fn(&PyObject, &PyObject, &VirtualMachine) -> bool; -type BinaryOpExtendAction = fn(&PyObject, &PyObject, &VirtualMachine) -> Option; +type BinaryOpExtendAction = fn(&PyObject, &PyObject, &VirtualMachine) -> Option; struct BinaryOpExtendSpecializationDescr { oparg: bytecode::BinaryOperator, @@ -3159,10 +3160,10 @@ struct FusedBoolJump { macro_rules! bitwise_longs_action { ($name:ident, $op:tt) => { #[inline] - fn $name(lhs: &PyObject, rhs: &PyObject, vm: &VirtualMachine) -> Option { + fn $name(lhs: &PyObject, rhs: &PyObject, vm: &VirtualMachine) -> Option { let lhs_val = compact_int_from_obj(lhs, vm)?; let rhs_val = compact_int_from_obj(rhs, vm)?; - Some(vm.ctx.new_int(lhs_val $op rhs_val).into()) + Some(Ok(vm.ctx.new_int(lhs_val $op rhs_val).into())) } }; } @@ -3184,10 +3185,10 @@ fn nonzero_float_compactlong_guard(lhs: &PyObject, rhs: &PyObject, vm: &VirtualM macro_rules! float_long_action { ($name:ident, $op:tt) => { #[inline] - fn $name(lhs: &PyObject, rhs: &PyObject, vm: &VirtualMachine) -> Option { + fn $name(lhs: &PyObject, rhs: &PyObject, vm: &VirtualMachine) -> Option { let lhs_val = exact_float_from_obj(lhs, vm)?; let rhs_val = compact_int_from_obj(rhs, vm)?; - Some(vm.ctx.new_float(lhs_val $op rhs_val as f64).into()) + Some(Ok(vm.ctx.new_float(lhs_val $op rhs_val as f64).into())) } }; } @@ -3210,10 +3211,10 @@ fn nonzero_compactlong_float_guard(lhs: &PyObject, rhs: &PyObject, vm: &VirtualM macro_rules! long_float_action { ($name:ident, $op:tt) => { #[inline] - fn $name(lhs: &PyObject, rhs: &PyObject, vm: &VirtualMachine) -> Option { + fn $name(lhs: &PyObject, rhs: &PyObject, vm: &VirtualMachine) -> Option { let lhs_val = compact_int_from_obj(lhs, vm)?; let rhs_val = exact_float_from_obj(rhs, vm)?; - Some(vm.ctx.new_float(lhs_val as f64 $op rhs_val).into()) + Some(Ok(vm.ctx.new_float(lhs_val as f64 $op rhs_val).into())) } }; } @@ -3222,7 +3223,108 @@ long_float_action!(compactlong_float_subtract, -); long_float_action!(compactlong_float_multiply, *); long_float_action!(compactlong_float_true_div, /); +fn sequence_int_guard(lhs: &PyObject, rhs: &PyObject, vm: &VirtualMachine) -> bool { + (lhs.downcast_ref_if_exact::(vm).is_some() + || lhs.downcast_ref_if_exact::(vm).is_some() + || lhs.downcast_ref_if_exact::(vm).is_some()) + && rhs.downcast_ref_if_exact::(vm).is_some() +} + +fn int_sequence_guard(lhs: &PyObject, rhs: &PyObject, vm: &VirtualMachine) -> bool { + sequence_int_guard(rhs, lhs, vm) +} + +fn exact_sequence_int_guard( + lhs: &PyObject, + rhs: &PyObject, + vm: &VirtualMachine, +) -> bool { + lhs.downcast_ref_if_exact::(vm).is_some() && rhs.downcast_ref_if_exact::(vm).is_some() +} + +fn exact_int_sequence_guard( + lhs: &PyObject, + rhs: &PyObject, + vm: &VirtualMachine, +) -> bool { + exact_sequence_int_guard::(rhs, lhs, vm) +} + +fn sequence_int_multiply(lhs: &PyObject, rhs: &PyObject, vm: &VirtualMachine) -> Option { + let count = rhs.downcast_ref_if_exact::(vm)?; + // CPython 3.15's specialized repetition uses PyLong_AsSsize_t, unlike + // the generic sequence dispatch's PyNumber_AsSsize_t conversion. + Some(match count.as_bigint().to_isize() { + Some(count) => lhs.sequence_unchecked().repeat(count, vm), + None => Err(vm.new_overflow_error("Python int too large to convert to C ssize_t")), + }) +} + +fn int_sequence_multiply(lhs: &PyObject, rhs: &PyObject, vm: &VirtualMachine) -> Option { + sequence_int_multiply(rhs, lhs, vm) +} + static BINARY_OP_EXTEND_DESCRIPTORS: &[BinaryOpExtendSpecializationDescr] = &[ + BinaryOpExtendSpecializationDescr { + oparg: bytecode::BinaryOperator::Multiply, + guard: exact_sequence_int_guard::, + action: sequence_int_multiply, + }, + BinaryOpExtendSpecializationDescr { + oparg: bytecode::BinaryOperator::Multiply, + guard: exact_int_sequence_guard::, + action: int_sequence_multiply, + }, + BinaryOpExtendSpecializationDescr { + oparg: bytecode::BinaryOperator::InplaceMultiply, + guard: exact_sequence_int_guard::, + action: sequence_int_multiply, + }, + BinaryOpExtendSpecializationDescr { + oparg: bytecode::BinaryOperator::InplaceMultiply, + guard: exact_int_sequence_guard::, + action: int_sequence_multiply, + }, + BinaryOpExtendSpecializationDescr { + oparg: bytecode::BinaryOperator::Multiply, + guard: exact_sequence_int_guard::, + action: sequence_int_multiply, + }, + BinaryOpExtendSpecializationDescr { + oparg: bytecode::BinaryOperator::Multiply, + guard: exact_int_sequence_guard::, + action: int_sequence_multiply, + }, + BinaryOpExtendSpecializationDescr { + oparg: bytecode::BinaryOperator::InplaceMultiply, + guard: exact_sequence_int_guard::, + action: sequence_int_multiply, + }, + BinaryOpExtendSpecializationDescr { + oparg: bytecode::BinaryOperator::InplaceMultiply, + guard: exact_int_sequence_guard::, + action: int_sequence_multiply, + }, + BinaryOpExtendSpecializationDescr { + oparg: bytecode::BinaryOperator::Multiply, + guard: exact_sequence_int_guard::, + action: sequence_int_multiply, + }, + BinaryOpExtendSpecializationDescr { + oparg: bytecode::BinaryOperator::Multiply, + guard: exact_int_sequence_guard::, + action: int_sequence_multiply, + }, + BinaryOpExtendSpecializationDescr { + oparg: bytecode::BinaryOperator::InplaceMultiply, + guard: exact_sequence_int_guard::, + action: sequence_int_multiply, + }, + BinaryOpExtendSpecializationDescr { + oparg: bytecode::BinaryOperator::InplaceMultiply, + guard: exact_int_sequence_guard::, + action: int_sequence_multiply, + }, // long-long arithmetic BinaryOpExtendSpecializationDescr { oparg: bytecode::BinaryOperator::Or, @@ -6522,7 +6624,7 @@ impl ExecutingFrame<'_> { { self.pop_stackref(); self.pop_stackref(); - self.push_value(result); + self.push_value(result?); Ok(None) } else { self.execute_bin_op(vm, op) @@ -10661,6 +10763,11 @@ impl ExecutingFrame<'_> { && b.downcast_ref_if_exact::(vm).is_some() { Some(Instruction::BinaryOpMultiplyFloat) + } else if (sequence_int_guard(a, b, vm) || int_sequence_guard(a, b, vm)) + && let Some(descr) = self.binary_op_extended_specialization(op, a, b, vm) + { + cached_extend_descr = Some(descr); + Some(Instruction::BinaryOpExtend) } else { None } diff --git a/crates/vm/src/stdlib/_ast.rs b/crates/vm/src/stdlib/_ast.rs index 497dec48b..6560d7ee1 100644 --- a/crates/vm/src/stdlib/_ast.rs +++ b/crates/vm/src/stdlib/_ast.rs @@ -1804,7 +1804,7 @@ pub(crate) fn parse>( ) -> Result { let source_file = SourceFileBuilder::new(filename.to_owned(), source.to_owned()).finish(); let mut options = parser::ParseOptions::from(mode); - let target_version = target_version.unwrap_or(ast::PythonVersion::PY314); + let target_version = target_version.unwrap_or(ast::PythonVersion::PY315); if let Some(error) = feature_version_syntax_error(source, &source_file, target_version) { return Err(error.into()); } @@ -2045,7 +2045,7 @@ pub(crate) fn parse_func_type>( let parse_expr = |expr_src: &str| -> Result { let source_file = SourceFileBuilder::new(filename.to_owned(), expr_src.to_owned()).finish(); let options = parser::ParseOptions::from(parser::Mode::Expression) - .with_target_version(target_version.unwrap_or(ast::PythonVersion::PY314)); + .with_target_version(target_version.unwrap_or(ast::PythonVersion::PY315)); let parsed = parser::parse(expr_src, options).map_err(|parse_error| { let range = text_range_to_source_range(&source_file, parse_error.location); CompileError::from(ParseError { @@ -2080,7 +2080,7 @@ pub(crate) fn parse_func_type>( let call_source = format!("{ARG_PREFIX}{inner})"); let source_file = SourceFileBuilder::new(filename.to_owned(), call_source.clone()).finish(); let options = parser::ParseOptions::from(parser::Mode::Expression) - .with_target_version(target_version.unwrap_or(ast::PythonVersion::PY314)); + .with_target_version(target_version.unwrap_or(ast::PythonVersion::PY315)); let parsed = parser::parse(&call_source, options).map_err(|parse_error| { let range = text_range_to_source_range(&source_file, parse_error.location); CompileError::from(ParseError { diff --git a/crates/vm/src/stdlib/_ast/python.rs b/crates/vm/src/stdlib/_ast/python.rs index cb9027aeb..297ffbc67 100644 --- a/crates/vm/src/stdlib/_ast/python.rs +++ b/crates/vm/src/stdlib/_ast/python.rs @@ -13,7 +13,6 @@ pub(crate) mod _ast { function::{ArgIterable, FuncArgs, KwArgs, PyMethodDef, PyMethodFlags}, stdlib::_ast::repr, types::{Constructor, Initializer}, - warn, }; #[pyattr] #[pyclass(module = "ast", name = "AST")] @@ -388,9 +387,9 @@ pub(crate) mod _ast { if contains { if !ast_replace_set_discard(&remaining_fields, &key_obj, vm)? { return Err(vm.new_type_error(format!( - "{} got multiple values for argument '{}'", - zelf.class().name(), - key + "{} got multiple values for argument {}", + zelf.class().fully_qualified_name(vm)?, + key_obj.repr(vm)? ))); } } else { @@ -410,19 +409,11 @@ pub(crate) mod _ast { attributes.as_deref().unwrap() }; if !attrs.sequence_unchecked().contains(&key_obj, vm)? { - let message = vm.ctx.new_str(format!( - "{}.__init__ got an unexpected keyword argument '{}'. \ -Support for arbitrary keyword arguments is deprecated and will be removed in Python 3.15.", - zelf.class().name(), - key - )); - warn::warn( - message.into(), - Some(vm.ctx.exceptions.deprecation_warning.to_owned()), - 1, - None, - vm, - )?; + return Err(vm.new_type_error(format!( + "{}.__init__ got an unexpected keyword argument {}", + zelf.class().fully_qualified_name(vm)?, + key_obj.repr(vm)? + ))); } } @@ -438,7 +429,13 @@ Support for arbitrary keyword arguments is deprecated and will be removed in Pyt let expr_ctx_type: PyObjectRef = super::super::pyast::NodeExprContext::make_static_type().into(); - for field in remaining_fields.elements() { + let mut missing = Vec::new(); + // Render missing arguments in their declared positional order. + for i in 0..numfields { + let field = fields_seq.get_item(i as isize, vm)?; + if !ast_replace_set_discard(&remaining_fields, &field, vm)? { + continue; + } if let Some(ftype) = ft_dict.get_item_opt(&*field, vm)? { if ftype.fast_isinstance(vm.ctx.types.union_type) { // Optional field (T | None) — no default @@ -457,39 +454,35 @@ Support for arbitrary keyword arguments is deprecated and will be removed in Pyt }); ast_set_attr(zelf, &field, load_instance, vm)?; } else { - // Required field missing: emit DeprecationWarning. - let field_repr = field.repr(vm)?; - let message = vm.ctx.new_str(format!( - "{}.__init__ missing 1 required positional argument: {}. \ -This will become an error in Python 3.15.", - zelf.class().name(), - field_repr - )); - warn::warn( - message.into(), - Some(vm.ctx.exceptions.deprecation_warning.to_owned()), - 1, - None, - vm, - )?; + missing.push(format!("'{}'", field.str(vm)?)); } } else { - let field_repr = field.repr(vm)?; - let message = vm.ctx.new_str(format!( - "Field {} is missing from {}._field_types. \ -This will become an error in Python 3.15.", - field_repr, - zelf.class().name() - )); - warn::warn( - message.into(), - Some(vm.ctx.exceptions.deprecation_warning.to_owned()), - 1, - None, - vm, - )?; + return Err(vm.new_type_error(format!( + "Field {} is missing from {}._field_types", + field.repr(vm)?, + zelf.class().fully_qualified_name(vm)? + ))); } } + if !missing.is_empty() { + let count = missing.len(); + let names = match missing.as_slice() { + [name] => name.clone(), + [first, last] => format!("{first} and {last}"), + names => format!( + "{}, and {}", + names[..count - 1].join(", "), + names[count - 1] + ), + }; + return Err(vm.new_type_error(format!( + "{}.__init__ missing {} required positional argument{}: {}", + zelf.class().fully_qualified_name(vm)?, + count, + if count == 1 { "" } else { "s" }, + names + ))); + } } Ok(()) diff --git a/crates/vm/src/stdlib/_imp.rs b/crates/vm/src/stdlib/_imp.rs index 07503ea91..d9ee8d1bb 100644 --- a/crates/vm/src/stdlib/_imp.rs +++ b/crates/vm/src/stdlib/_imp.rs @@ -194,7 +194,10 @@ fn find_frozen(name: &str, vm: &VirtualMachine) -> Result PyResult { - let sys_modules = vm.sys_module.get_attr("modules", vm).unwrap(); - let name: PyUtf8StrRef = spec.get_attr("name", vm)?.try_into_value(vm)?; + let name_obj = spec.get_attr("name", vm)?; + let name: PyStrRef = name_obj.downcast().map_err(|obj| { + vm.new_type_error(format!( + "name must be string, not {:.200}", + obj.class().name() + )) + })?; + if name.is_empty() { + return Err(vm.new_value_error("name must not be empty")); + } + // Builtin module names use ASCII even when an unknown name is supplied. + if !name.as_bytes().is_ascii() { + vm.state + .codec_registry + .encode_text(name.clone(), "ascii", None, vm)?; + } + let name_str = name.as_utf8().expect("ASCII module name").as_str(); + if name_str.contains('\0') { + return Err(vm.new_value_error("embedded null character")); + } + if !matches!(name_str, "sys" | "builtins") && !vm.state.module_defs.contains_key(name_str) { + let exc = vm.new_payload_exception::( + vm.ctx.exceptions.module_not_found_error.to_owned(), + vec![ + vm.ctx + .new_str(format!("{name_str} module not found")) + .into(), + ] + .into(), + )?; + exc.as_object().set_attr("name", name.clone(), vm)?; + return Err(exc.upcast()); + } - // Check sys.modules first + let sys_modules = vm.sys_module.get_attr("modules", vm)?; + // Only registered builtins (or the two core modules) may use this cache. if let Ok(module) = sys_modules.get_item(&*name, vm) { return Ok(module); } - let name_str = name.as_str(); if let Some(&def) = vm.state.module_defs.get(name_str) { // Phase 1: Create module (use create slot if provided, else default creation) let module = if let Some(create) = def.slots.create { @@ -274,7 +308,7 @@ mod _imp { module.__init_methods(vm)?; // Add to sys.modules BEFORE exec (critical for circular import handling) - sys_modules.set_item(name.as_pystr(), module.clone().into(), vm)?; + sys_modules.set_item(&*name, module.clone().into(), vm)?; // Phase 2: Call exec slot (can safely import other modules now) if let Some(exec) = def.slots.exec { diff --git a/crates/vm/src/stdlib/builtins.rs b/crates/vm/src/stdlib/builtins.rs index 23e4c7ea7..655f30f87 100644 --- a/crates/vm/src/stdlib/builtins.rs +++ b/crates/vm/src/stdlib/builtins.rs @@ -40,8 +40,8 @@ mod builtins { "can't compile() to bytecode when the `codegen` feature of rustpython is disabled"; #[pyfunction] - fn abs(x: PyObjectRef, vm: &VirtualMachine) -> PyResult { - vm._abs(&x) + fn abs(number: PyObjectRef, vm: &VirtualMachine) -> PyResult { + vm._abs(&number) } #[pyfunction] @@ -70,8 +70,8 @@ mod builtins { } #[pyfunction] - fn bin(number: ArgIndex) -> String { - let number = number.into_int_ref(); + fn bin(integer: ArgIndex) -> String { + let number = integer.into_int_ref(); let x = number.as_bigint(); if x.is_negative() { format!("-0b{:b}", x.abs()) diff --git a/crates/vm/src/stdlib/sys.rs b/crates/vm/src/stdlib/sys.rs index a77fbb10f..28f107862 100644 --- a/crates/vm/src/stdlib/sys.rs +++ b/crates/vm/src/stdlib/sys.rs @@ -667,8 +667,8 @@ pub mod sys { fn implementation(vm: &VirtualMachine) -> PyRef { const NAME: &str = "rustpython"; - // cache tag uses 'cpython' because our compiler is cpython compatible - let cache_tag = format!("cpython-{}{}", version::MAJOR, version::MINOR); + // Keep implementation-specific bytecode caches separate. + let cache_tag = format!("{NAME}-{}{}", version::MAJOR, version::MINOR); let ctx = &vm.ctx; py_namespace!(vm, { "name" => ctx.new_str(NAME), diff --git a/crates/vm/src/stdlib/time.rs b/crates/vm/src/stdlib/time.rs index 9ea375e13..867891eff 100644 --- a/crates/vm/src/stdlib/time.rs +++ b/crates/vm/src/stdlib/time.rs @@ -18,9 +18,9 @@ mod decl { use crate::convert::{ToPyException, ToPyObject}; use crate::{ AsObject, Py, PyObjectRef, PyResult, VirtualMachine, - builtins::{PyFloat, PyStr, PyStrRef, PyTypeRef}, + builtins::{PyStr, PyStrRef, PyTypeRef}, class::PyClassDef, - common::wtf8::{Wtf8Buf, wtf8_concat}, + common::wtf8::Wtf8Buf, function::{Either, FuncArgs, OptionalArg, OptionalOption}, types::{PyStructSequence, PyStructSequenceData, struct_sequence_new}, }; @@ -128,8 +128,21 @@ mod decl { vm.audit("time.sleep", || (object.clone(),))?; let overflow = || vm.new_overflow_error("timestamp out of range for C PyTime_t"); - let nanoseconds = if let Some(float) = object.downcast_ref::() { - let seconds = float.to_f64(); + let nanoseconds = if object.number().is_index() { + let seconds = object.try_index(vm).map_err(|e| { + if e.fast_isinstance(vm.ctx.exceptions.overflow_error) { + overflow() + } else { + e + } + })?; + seconds + .as_bigint() + .to_i64() + .and_then(|seconds| seconds.checked_mul(SEC_TO_NS)) + .ok_or_else(overflow)? + } else { + let seconds = object.try_float(vm)?.to_f64(); if seconds.is_nan() { return Err(vm.new_value_error("Invalid value NaN (not a number)")); } @@ -146,47 +159,6 @@ mod decl { return Err(overflow()); } nanoseconds as i64 - } else { - let seconds = object.try_index(vm).or_else(|e| { - if e.fast_isinstance(vm.ctx.exceptions.overflow_error) { - Err(overflow()) - } else if e.fast_isinstance(vm.ctx.exceptions.type_error) { - // CPython's %T reads stored type metadata, preserving surrogates. - let class = object.class(); - let name = if let Some(heap_type) = &class.heaptype_ext { - let qualname = heap_type.qualname.read().clone(); - let module = if let Some(dict) = class.attributes.as_dict() { - dict.get_item_opt(identifier!(vm, __module__), vm)? - .ok_or_else(|| vm.new_attribute_error("__module__"))? - } else { - class.__module__(vm)? - }; - match module.downcast_ref::() { - Some(module) - if module.as_wtf8() != "builtins" - && module.as_wtf8() != "__main__" => - { - wtf8_concat!(module.as_wtf8(), ".", qualname.as_wtf8()) - } - _ => qualname.as_wtf8().to_owned(), - } - } else { - Wtf8Buf::from(class.slot_name().to_owned()) - }; - Err(vm.new_type_error(wtf8_concat!( - "'", - name, - "' object cannot be interpreted as an integer or float" - ))) - } else { - Err(e) - } - })?; - seconds - .as_bigint() - .to_i64() - .and_then(|seconds| seconds.checked_mul(SEC_TO_NS)) - .ok_or_else(overflow)? }; if nanoseconds < 0 { return Err(vm.new_value_error("sleep length must be non-negative")); @@ -1353,7 +1325,25 @@ mod platform { (info.standard_bias, &info.standard_name) }; - let gmtoff = -(info.bias + bias) * 60; + let gmtoff = if when < 0 { + // FILETIME marks historical DST as unknown. Derive the offset from + // the converted wall time rather than treating unknown as standard. + let wall_time = jiff::civil::DateTime::new( + (tm.tm_year + 1900) as i16, + (tm.tm_mon + 1) as i8, + tm.tm_mday as i8, + tm.tm_hour as i8, + tm.tm_min as i8, + tm.tm_sec as i8, + 0, + ) + .and_then(|dt| dt.to_zoned(jiff::tz::TimeZone::UTC)) + .map_err(|_| vm.new_overflow_error("timestamp out of range for Windows FILETIME"))?; + i32::try_from(wall_time.timestamp().as_second() - when) + .map_err(|_| vm.new_overflow_error("timezone offset out of range"))? + } else { + -(info.bias + bias) * 60 + }; Ok(struct_time_from_tm(vm, tm, name, gmtoff)) } diff --git a/crates/vm/src/vm/mod.rs b/crates/vm/src/vm/mod.rs index 238b19a2b..304991f95 100644 --- a/crates/vm/src/vm/mod.rs +++ b/crates/vm/src/vm/mod.rs @@ -1607,7 +1607,7 @@ impl VirtualMachine { /// ```no_run /// use rustpython_vm::Interpreter; /// Interpreter::without_stdlib(Default::default()).enter(|vm| { - /// let bytes = std::fs::read("__pycache__/.rustpython-314.pyc").unwrap(); + /// let bytes = std::fs::read("__pycache__/.rustpython-315.pyc").unwrap(); /// let main_scope = vm.new_scope_with_main().unwrap(); /// vm.run_pyc_bytes(&bytes, main_scope); /// }); @@ -2258,17 +2258,15 @@ impl VirtualMachine { self.ctx.new_str(path).into(), self, )?; - module_dict.set_item(identifier!(self, __cached__), self.ctx.none(), self)?; } let result = run(&module_dict); self.flush_io(); - // Cleanup __file__ and __cached__ after execution + // Cleanup __file__ after execution if set_file_name { let _ = module_dict.del_item(identifier!(self, __file__), self); - let _ = module_dict.del_item(identifier!(self, __cached__), self); } result diff --git a/extra_tests/snippets/builtin___main__.py b/extra_tests/snippets/builtin___main__.py index 97e76ce32..14cd3f3b9 100644 --- a/extra_tests/snippets/builtin___main__.py +++ b/extra_tests/snippets/builtin___main__.py @@ -2,4 +2,4 @@ import sys main_module = sys.modules["__main__"] assert main_module.__file__.endswith("builtin___main__.py") -assert main_module.__cached__ is None +assert not hasattr(main_module, "__cached__") diff --git a/extra_tests/snippets/builtin_mappingproxy.py b/extra_tests/snippets/builtin_mappingproxy.py index c4d302136..8adf4d7e9 100644 --- a/extra_tests/snippets/builtin_mappingproxy.py +++ b/extra_tests/snippets/builtin_mappingproxy.py @@ -25,11 +25,13 @@ assert A.__dict__.get("a", "default") is A.a assert A.__dict__.get("not here") is None -# A mappingproxy must preserve arbitrary rich-comparison results unchanged. +# A mappingproxy preserves arbitrary equality-comparison results unchanged. def check_mappingproxy_comparison_results(): import operator from types import MappingProxyType + from testutils import assert_raises + class Result: def __bool__(self): raise AssertionError("comparison result must not be coerced to bool") @@ -58,10 +60,6 @@ def check_mappingproxy_comparison_results(): for operation, reflected in ( (operator.eq, "eq"), (operator.ne, "ne"), - (operator.lt, "gt"), - (operator.le, "ge"), - (operator.gt, "lt"), - (operator.ge, "le"), ): calls.clear() assert operation(MappingProxyType(CustomMapping()), {}) is result @@ -70,5 +68,19 @@ def check_mappingproxy_comparison_results(): assert operation(MappingProxyType({}), Compared()) is result assert calls == [reflected], calls + for operation, reflected in ( + (operator.lt, "gt"), + (operator.le, "ge"), + (operator.gt, "lt"), + (operator.ge, "le"), + ): + calls.clear() + with assert_raises(TypeError): + operation(MappingProxyType(CustomMapping()), {}) + assert calls == [], calls + calls.clear() + assert operation(MappingProxyType({}), Compared()) is result + assert calls == [reflected], calls + check_mappingproxy_comparison_results() diff --git a/extra_tests/snippets/builtin_memoryview.py b/extra_tests/snippets/builtin_memoryview.py index ef28ffc3d..ea0475d7c 100644 --- a/extra_tests/snippets/builtin_memoryview.py +++ b/extra_tests/snippets/builtin_memoryview.py @@ -399,7 +399,7 @@ def test_cast_rejects_non_native_format(): memoryview(b"").cast(fmt) raise AssertionError(f"cast accepted {fmt!r}") except ValueError as e: - assert "native single character format" in str(e), e + assert "native format" in str(e), e assert memoryview(bytes(8)).cast("@i").itemsize == 4 diff --git a/extra_tests/snippets/builtin_signature.py b/extra_tests/snippets/builtin_signature.py index 68998566a..06d5cf92b 100644 --- a/extra_tests/snippets/builtin_signature.py +++ b/extra_tests/snippets/builtin_signature.py @@ -19,7 +19,7 @@ else: raise AssertionError("len() should not accept keyword arguments") assert str(inspect.signature(len)) == "(obj, /)" -assert str(inspect.signature(abs)) == "(x, /)" +assert str(inspect.signature(abs)) == "(number, /)" assert str(inspect.signature(hash)) == "(obj, /)" assert str(inspect.signature(chr)) == "(i, /)" assert str(inspect.signature(callable)) == "(obj, /)" @@ -37,7 +37,7 @@ assert breakpoint_kinds == [ ], breakpoint_kinds # Parameter names follow CPython, so signatures are directly comparable. -assert str(inspect.signature(bin)) == "(number, /)" +assert str(inspect.signature(bin)) == "(integer, /)" assert str(inspect.signature(ord)) == "(character, /)" assert str(inspect.signature(divmod)) == "(x, y, /)" assert str(inspect.signature(hasattr)) == "(obj, name, /)" diff --git a/extra_tests/snippets/builtin_str.py b/extra_tests/snippets/builtin_str.py index f54082325..579976fd4 100644 --- a/extra_tests/snippets/builtin_str.py +++ b/extra_tests/snippets/builtin_str.py @@ -411,7 +411,10 @@ for template in ("%d", "%i", "%u", b"%d", b"%i", b"%u"): for value in (None, TypeError("conversion failed")): with assert_raises(TypeError) as cm: template % PercentInt(value) - assert str(cm.exception).endswith("a real number is required, not PercentInt") + conversion = chr(template[-1]) if isinstance(template, bytes) else template[-1] + assert str(cm.exception) == ( + f"format argument: %{conversion} requires a real number, not PercentInt" + ) assert_raises( RuntimeError, template.__mod__, PercentInt(RuntimeError("conversion failed")) ) @@ -1224,3 +1227,27 @@ def test_subscript_typeerror_message(): test_subscript_typeerror_message() + + +# Decimal conversion errors identify the consumed argument, including star arguments. +for is_bytes in (False, True): + for conversion in "diu": + for template, values, context in ( + ("%" + conversion, "bad", ""), + ("%" + conversion, ("bad",), " 1"), + ("%s %" + conversion, ("ok", "bad"), " 2"), + ("%*.*" + conversion, (5, 2, "bad"), " 3"), + ("%(value)" + conversion, {"value": "bad"}, " 'value'"), + ): + if is_bytes: + template = template.encode() + if isinstance(values, dict): + values = {b"value": "bad"} + context = " b'value'" + elif isinstance(values, tuple) and len(values) == 2: + values = (b"ok", "bad") + with assert_raises(TypeError) as caught: + template % values + assert str(caught.exception) == ( + f"format argument{context}: %{conversion} requires a real number, not str" + ) diff --git a/extra_tests/snippets/operator_arithmetic.py b/extra_tests/snippets/operator_arithmetic.py index 14763107b..559ad7a4f 100644 --- a/extra_tests/snippets/operator_arithmetic.py +++ b/extra_tests/snippets/operator_arithmetic.py @@ -89,11 +89,21 @@ for value in (range(3), GetItemOnly()): # Native sequence slots use the same count conversion in both directions. for value in ([1], (1,), "a", b"a", bytearray(b"a")): - for operation in (mul, lambda seq, count: count * seq, imul): + namespace = {} + exec("def reverse(seq, count):\n return count * seq", namespace) + for operation_index, operation in enumerate((mul, namespace["reverse"], imul)): + for _ in range(64): + operation(value, 1) for count, error, message in ( (1.5, TypeError, "can't multiply sequence by non-int of type 'float'"), (2**100, OverflowError, "cannot fit 'int' into an index-sized integer"), ): + if ( + type(value) in (str, bytes, tuple) + and operation_index == 1 + and error is OverflowError + ): + message = "Python int too large to convert to C ssize_t" check_repeat_error(error, message, operation, value, count) @@ -123,6 +133,79 @@ check_repeat_error( ) +# Each expression has fresh bytecode so cold and warmed paths are both checked. +for value in ("a", b"a", (1,), "", b"", ()): + for expression in ( + "return seq * count", + "return count * seq", + "seq *= count; return seq", + "count *= seq; return count", + ): + namespace = {} + exec("def repeat(seq, count):\n " + expression, namespace) + repeat = namespace["repeat"] + check_repeat_error( + OverflowError, + "cannot fit 'int' into an index-sized integer", + repeat, + value, + 2**100, + ) + for _ in range(64): + assert repeat(value, 2) == mul(value, 2) + assert repeat(value, -2) == value[:0] + assert repeat(value, 0) == value[:0] + assert repeat(value, 1) == value + for count in (2**100, -(2**100)): + check_repeat_error( + OverflowError, + "Python int too large to convert to C ssize_t", + repeat, + value, + count, + ) + for other in ("a", b"a", (1,)): + if type(other) is not type(value): + check_repeat_error( + OverflowError, + "cannot fit 'int' into an index-sized integer", + repeat, + other, + 2**100, + ) + if expression == "count *= seq; return count": + check_repeat_error( + TypeError, + f"unsupported operand type(s) for *=: 'RepeatIndex' and '{type(value).__name__}'", + repeat, + value, + RepeatIndex(2), + ) + else: + assert repeat(value, RepeatIndex(2)) == mul(value, 2) + check_repeat_error( + OverflowError, + "cannot fit 'RepeatIndex' into an index-sized integer", + repeat, + value, + RepeatIndex(2**100), + ) + subclass = type("RepeatSubclass", (type(value),), {})(value) + assert repeat(subclass, 2) == mul(value, 2) + overridden = type( + "OverrideRepeat", + (type(value),), + { + "__mul__": lambda self, count: "forward", + "__rmul__": lambda self, count: "reverse", + }, + )(value) + expected = ( + "forward" if expression.startswith(("return seq", "seq *=")) else "reverse" + ) + assert repeat(overridden, 2) == expected + + class DeclineRepeat(GetItemOnly): calls = [] diff --git a/extra_tests/snippets/stdlib_ast.py b/extra_tests/snippets/stdlib_ast.py index 2dd227672..c9a3ae13d 100644 --- a/extra_tests/snippets/stdlib_ast.py +++ b/extra_tests/snippets/stdlib_ast.py @@ -69,12 +69,13 @@ import warnings def _cyclic_cases(): # Box descents - u = ast.UnaryOp(op=ast.Not(), lineno=0, col_offset=0) + u = ast.UnaryOp(op=ast.Not(), operand=None, lineno=0, col_offset=0) u.operand = u yield "UnaryOp", u b = ast.BinOp( op=ast.Add(), + left=None, right=ast.Constant(value=0, lineno=0, col_offset=0), lineno=0, col_offset=0, @@ -82,11 +83,11 @@ def _cyclic_cases(): b.left = b yield "BinOp", b - c = ast.Call(args=[], keywords=[], lineno=0, col_offset=0) + c = ast.Call(func=None, args=[], keywords=[], lineno=0, col_offset=0) c.func = c yield "Call", c - a = ast.Attribute(attr="x", ctx=ast.Load(), lineno=0, col_offset=0) + a = ast.Attribute(value=None, attr="x", ctx=ast.Load(), lineno=0, col_offset=0) a.value = a yield "Attribute", a @@ -108,3 +109,73 @@ with warnings.catch_warnings(): raise AssertionError(f"cyclic {name} should raise RecursionError") except RecursionError: pass # expected; matches CPython + + +# Python 3.15 rejects omitted required fields and unknown constructor keywords. +from testutils import assert_raises + +with assert_raises(TypeError) as exc: + ast.Name() +assert ( + str(exc.exception) + == "ast.Name.__init__ missing 1 required positional argument: 'id'" +) + +with assert_raises(TypeError) as exc: + ast.BinOp() +assert str(exc.exception) == ( + "ast.BinOp.__init__ missing 3 required positional arguments: 'left', 'op', and 'right'" +) + +with assert_raises(TypeError) as exc: + ast.BinOp(op=ast.Add()) +assert str(exc.exception) == ( + "ast.BinOp.__init__ missing 2 required positional arguments: 'left' and 'right'" +) + +with assert_raises(TypeError) as exc: + ast.Name(id="x", extra=True) +assert ( + str(exc.exception) == "ast.Name.__init__ got an unexpected keyword argument 'extra'" +) + +with assert_raises(TypeError) as exc: + ast.Name("x", id="y") +assert str(exc.exception) == "ast.Name got multiple values for argument 'id'" + +# Constructor defaults and explicit placeholders still work. +assert ast.Name("x").ctx is ast.Name("y").ctx +assert isinstance(ast.Name("x").ctx, ast.Load) +assert ast.FunctionDef(name="f", args=ast.arguments()).returns is None +assert ast.Module().body == [] +assert ast.Module().body is not ast.Module().body +assert ast.BinOp(left=None, op=None, right=None).left is None + + +class FieldsWithoutTypes(ast.AST): + _fields = ("value",) + + +assert not hasattr(FieldsWithoutTypes(), "value") +assert FieldsWithoutTypes(value=1).value == 1 + + +class IncompleteFieldTypes(ast.AST): + _fields = ("value",) + _field_types = {} + + +with assert_raises(TypeError) as exc: + IncompleteFieldTypes() +assert ( + str(exc.exception) + == "Field 'value' is missing from IncompleteFieldTypes._field_types" +) + + +for key in ("a'b", "line\nbreak"): + with assert_raises(TypeError) as exc: + ast.Name(id="x", **{key: 1}) + assert str(exc.exception) == ( + f"ast.Name.__init__ got an unexpected keyword argument {key!r}" + ) diff --git a/extra_tests/snippets/stdlib_codecs_wide.py b/extra_tests/snippets/stdlib_codecs_wide.py index 4b950ba70..0c45cc91c 100644 --- a/extra_tests/snippets/stdlib_codecs_wide.py +++ b/extra_tests/snippets/stdlib_codecs_wide.py @@ -31,7 +31,7 @@ assert ( # Lookup errors preserve the requested spelling, while search hooks receive -# the Python 3.14 normalized registry key. +# the Python 3.15 normalized registry key. import codecs seen_encoding_names = [] @@ -45,8 +45,8 @@ def record_unknown_encoding(name): codecs.register(record_unknown_encoding) try: for requested, normalized in ( - ("NO Such Codec!", "no_such_codec"), - ("x_ weird--Ω .Ab", "x_weird_.ab"), + ("NO Such Codec!", "no-such--codec!"), + ("x_ weird--Ω .Ab", "x_-weird--Ω-.ab"), ): for lookup in ( codecs.lookup, diff --git a/extra_tests/snippets/stdlib_csv.py b/extra_tests/snippets/stdlib_csv.py index 08adeaf52..5911b0a1c 100644 --- a/extra_tests/snippets/stdlib_csv.py +++ b/extra_tests/snippets/stdlib_csv.py @@ -493,9 +493,15 @@ def test_dialect_lookup_and_conversion_order(): events.append(name) raise error_type(name) - result = construct(Missing()) - assert tuple(getattr(result, field) for field in fields) == defaults - assert events == list(fields) + if error_type is AttributeError: + result = construct(Missing()) + assert tuple(getattr(result, field) for field in fields) == defaults + assert events == list(fields) + else: + with assert_raises(error_type) as caught: + construct(Missing()) + assert caught.exception.args == ("delimiter",) + assert events == ["delimiter"] for dialect in (None, object(), _csv.Dialect(csv.excel)): result = construct(dialect) assert tuple(getattr(result, field) for field in fields) == defaults diff --git a/extra_tests/snippets/stdlib_datetime.py b/extra_tests/snippets/stdlib_datetime.py index 41efd03a8..817f92371 100644 --- a/extra_tests/snippets/stdlib_datetime.py +++ b/extra_tests/snippets/stdlib_datetime.py @@ -3944,17 +3944,28 @@ def test_timezone_cycles(): def test_timestamp_errno(): if sys.platform == "win32": try: - datetime_module.datetime.fromtimestamp(-1) + datetime_module.datetime.fromtimestamp(2**40) except OSError as error: assert error.errno == errno.EINVAL else: - raise AssertionError("Windows localtime accepted a negative timestamp") - try: - datetime_module.datetime.fromtimestamp(-86400, datetime_module.timezone.utc) - except OSError as error: - assert error.errno == errno.EINVAL - else: - raise AssertionError("Windows gmtime accepted an out-of-range timestamp") + raise AssertionError( + "Windows localtime accepted a timestamp beyond the CRT range" + ) + assert datetime_module.datetime.fromtimestamp( + -86400, datetime_module.timezone.utc + ) == datetime_module.datetime(1969, 12, 31, tzinfo=datetime_module.timezone.utc) + assert datetime_module.datetime.fromtimestamp( + -11_644_473_600, datetime_module.timezone.utc + ) == datetime_module.datetime(1601, 1, 1, tzinfo=datetime_module.timezone.utc) + for zone in (None, datetime_module.timezone.utc): + try: + datetime_module.datetime.fromtimestamp(-11_644_473_601, zone) + except OverflowError as error: + assert str(error) == "timestamp out of range for Windows FILETIME" + else: + raise AssertionError( + "pre-FILETIME datetime did not raise OverflowError" + ) test_astimezone_subclass_state() diff --git a/extra_tests/snippets/stdlib_imp.py b/extra_tests/snippets/stdlib_imp.py index 8153790c3..b5184e89a 100644 --- a/extra_tests/snippets/stdlib_imp.py +++ b/extra_tests/snippets/stdlib_imp.py @@ -23,7 +23,8 @@ imp_time = _imp.create_builtin(A) # assert imp_time.sleep == import_time.sleep B = FakeSpec("not existing module") -assert _imp.create_builtin(B) == None +with assert_raises(ModuleNotFoundError): + _imp.create_builtin(B) _imp.exec_builtin(imp_time) == 0 diff --git a/extra_tests/snippets/stdlib_math.py b/extra_tests/snippets/stdlib_math.py index 29824043a..6e4e74b04 100644 --- a/extra_tests/snippets/stdlib_math.py +++ b/extra_tests/snippets/stdlib_math.py @@ -48,14 +48,12 @@ assert log_conversions == [] class LogFloatOverflow: def __float__(self): + log_conversions.append("float") raise OverflowError("float conversion failed") def __index__(self): - raise AssertionError("must not retry __index__ after __float__ fails") - - -assert_raises(OverflowError, math.log, LogFloatOverflow()) -assert_raises(OverflowError, math.log, 8, LogFloatOverflow()) + log_conversions.append("index") + return 2**2000 class LogIndexOnly: @@ -63,8 +61,61 @@ class LogIndexOnly: return 2**2000 -assert_raises(OverflowError, math.log, LogIndexOnly()) -assert_raises(OverflowError, math.log, 8, LogIndexOnly()) +for logarithm in (math.log, math.log2, math.log10): + log_conversions.clear() + assert math.isclose(logarithm(LogFloatOverflow()), logarithm(2**2000)) + assert log_conversions == ["float", "index"] + assert math.isclose(logarithm(LogIndexOnly()), logarithm(2**2000)) + + for error_type in (TypeError, ValueError, RuntimeError): + error = error_type("float conversion failed") + + class FailedFloat(LogFloatOverflow): + def __float__(self): + raise error + + with assert_raises(error_type) as caught: + logarithm(FailedFloat()) + assert caught.exception is error + + error = RuntimeError("index conversion failed") + + class FailedIndex(LogFloatOverflow): + def __index__(self): + raise error + + with assert_raises(RuntimeError) as caught: + logarithm(FailedIndex()) + assert caught.exception is error + + class NegativeIndex(LogFloatOverflow): + def __index__(self): + return -1 + + assert_raises(ValueError, logarithm, NegativeIndex()) + + class InvalidIndex(LogFloatOverflow): + def __index__(self): + return 1.0 + + assert_raises(TypeError, logarithm, InvalidIndex()) + + class NoIndex: + __float__ = LogFloatOverflow.__float__ + + assert_raises(OverflowError, logarithm, NoIndex()) + + class SuccessfulFloat: + def __float__(self): + return 8.0 + + def __index__(self): + raise AssertionError("successful float conversion must not retry index") + + assert logarithm(SuccessfulFloat()) == logarithm(8.0) + +assert math.isclose(math.log(8, LogFloatOverflow()), 3 / 2000) +assert math.isclose(math.log(8, LogIndexOnly()), 3 / 2000) class LogInt(int): diff --git a/extra_tests/snippets/stdlib_mmap.py b/extra_tests/snippets/stdlib_mmap.py index 1dc89e29d..094c1aa9d 100644 --- a/extra_tests/snippets/stdlib_mmap.py +++ b/extra_tests/snippets/stdlib_mmap.py @@ -25,9 +25,9 @@ assert mapped.find(b"") == 0 assert mapped.rfind(b"") == 3 assert mapped.find(b"", 1, 2) == 1 assert mapped.rfind(b"", 1, 2) == 2 -# offsets past the end are clamped and negative ones count from the end -assert mapped.find(b"", 5) == 3 -assert mapped.rfind(b"", 5) == 3 +# starts past the end find nothing and negative offsets count from the end +assert mapped.find(b"", 5) == -1 +assert mapped.rfind(b"", 5) == -1 assert mapped.find(b"", -1) == 2 assert mapped.rfind(b"", 0, -1) == 2 # an inverted range holds nothing, not even the empty subsequence diff --git a/extra_tests/snippets/stdlib_struct.py b/extra_tests/snippets/stdlib_struct.py index 0f379df34..5184493b1 100644 --- a/extra_tests/snippets/stdlib_struct.py +++ b/extra_tests/snippets/stdlib_struct.py @@ -78,19 +78,18 @@ assert struct.error.__module__ == "struct" assert struct.error.__name__ == "error" # Non-ASCII format string: error type matches CPython. -# str → UnicodeEncodeError (encoding='ascii') -# bytes → struct.error +# str and bytes → ValueError try: struct.Struct("\udc00") -except UnicodeEncodeError as e: - assert e.encoding == "ascii" +except ValueError as e: + assert str(e) == "non-ASCII character in struct format" else: - raise AssertionError("expected UnicodeEncodeError") + raise AssertionError("expected ValueError") -with assert_raises(UnicodeEncodeError): +with assert_raises(ValueError): struct.Struct("한") -with assert_raises(struct.error): +with assert_raises(ValueError): struct.Struct(b"\xff") @@ -128,7 +127,7 @@ assert s.unpack(b"\x00\x01\x00\x02") == (1, 2) # A format that cannot be read leaves the Struct as it was. for bad in ("\udc00", "$"): - with assert_raises((UnicodeEncodeError, struct.error)): + with assert_raises((ValueError, struct.error)): s.__init__(bad) assert s.format == ">hh" assert s.pack(1, 2) == b"\x00\x01\x00\x02" @@ -145,8 +144,9 @@ assert BigShort().pack(12345) == b"\x30\x39" # Until __init__ runs there is no format to answer with. blank = struct.Struct.__new__(struct.Struct) assert blank.size == -1 +with assert_raises(AttributeError): + blank.format for call in ( - lambda: blank.format, lambda: blank.pack(1), lambda: blank.unpack(b"aa"), lambda: blank.unpack_from(b"aaaa"), diff --git a/extra_tests/snippets/stdlib_sys.py b/extra_tests/snippets/stdlib_sys.py index ba2fcef4c..185e89f8b 100644 --- a/extra_tests/snippets/stdlib_sys.py +++ b/extra_tests/snippets/stdlib_sys.py @@ -25,6 +25,13 @@ assert "sys" in sys.builtin_module_names assert isinstance(sys.implementation.name, str) assert isinstance(sys.implementation.cache_tag, str) +assert sys.version_info[:3] == (3, 15, 0) +assert sys.hexversion >> 8 == 0x030F00 +assert sys.version.startswith("3.15.0") +assert sys.implementation.cache_tag == ( + f"{sys.implementation.name}-{sys.version_info.major}{sys.version_info.minor}" +) + assert sys.getfilesystemencoding() == "utf-8" assert sys.getfilesystemencodeerrors().startswith("surrogate") diff --git a/extra_tests/snippets/stdlib_time.py b/extra_tests/snippets/stdlib_time.py index 33b356c23..614d96105 100644 --- a/extra_tests/snippets/stdlib_time.py +++ b/extra_tests/snippets/stdlib_time.py @@ -75,9 +75,8 @@ if sys.platform == "win32": import errno for converter in (time.gmtime, time.localtime, time.ctime): - # Python 3.14 preserves UCRT conversion failures as OSError. Use a - # negative timestamp below gmtime's near-epoch allowance in UCRT. - for timestamp in (-86400, 2**40, 2**63 - 1, -(2**63), -float(2**63)): + # Non-negative timestamps still use UCRT and preserve errno. + for timestamp in (2**40, 2**63 - 1): try: converter(timestamp) except OSError as error: @@ -85,6 +84,29 @@ if sys.platform == "win32": else: raise AssertionError("UCRT timestamp failure did not raise OSError") + # Negative timestamps use FILETIME, whose epoch is 1601-01-01. + for timestamp in (-11_644_473_601, -(2**63), -float(2**63)): + try: + converter(timestamp) + except OverflowError as error: + assert str(error) == "timestamp out of range for Windows FILETIME" + else: + raise AssertionError( + "pre-FILETIME timestamp did not raise OverflowError" + ) + + assert time.gmtime(-11_644_473_600) == (1601, 1, 1, 0, 0, 0, 0, 1, 0) + assert time.gmtime(-86400) == (1969, 12, 31, 0, 0, 0, 2, 365, 0) + assert time.gmtime(-1) == (1969, 12, 31, 23, 59, 59, 2, 365, 0) + # Leap-year accounting uses the historical date, including century rules. + assert time.gmtime(-2_203_891_200).tm_yday == 60 # 1900-03-01 + assert time.gmtime(-2_330_035_200).tm_yday == 61 # 1896-03-01 + for timestamp in (-15_897_600, -86400, -43201, -1): + local = time.localtime(timestamp) + assert local.tm_isdst == -1 + assert local[:6] == time.gmtime(timestamp + local.tm_gmtoff)[:6] + assert time.ctime(timestamp) == time.asctime(local) + x = time.gmtime(1000) assert x.tm_year == 1970 @@ -212,13 +234,39 @@ class SleepTypeNamespace: pass -for module, prefix in (("__main__", ""), ("sleep_test", "sleep_test.")): +for module in ("__main__", "sleep_test"): SleepTypeNamespace.Invalid.__module__ = module with assert_raises(TypeError) as error: time.sleep(SleepTypeNamespace.Invalid()) - assert str(error.exception) == ( - f"'{prefix}SleepTypeNamespace.Invalid' object cannot be interpreted as an integer or float" - ) + assert str(error.exception) == "must be real number, not Invalid" + + +class FloatSleepDuration: + def __float__(self): + return -0.5 + + +class IndexedSleepDuration(FloatSleepDuration): + def __index__(self): + return -1 + + def __float__(self): + raise AssertionError("sleep must prefer __index__") + + +class InvalidIndexedSleepDuration(FloatSleepDuration): + def __index__(self): + raise TypeError("index conversion failed") + + +for duration in (FloatSleepDuration(), IndexedSleepDuration()): + with assert_raises(ValueError) as error: + time.sleep(duration) + assert str(error.exception) == "sleep length must be non-negative" + +with assert_raises(TypeError) as error: + time.sleep(InvalidIndexedSleepDuration()) +assert str(error.exception) == "index conversion failed" # The optional second argument fills the fields that are not part of the # sequence. diff --git a/scripts/libc_posix.py b/scripts/libc_posix.py index 82f3eb96d..139235244 100644 --- a/scripts/libc_posix.py +++ b/scripts/libc_posix.py @@ -8,7 +8,7 @@ import urllib.request import tomllib -CPYTHON_VERSION = "3.14" +CPYTHON_VERSION = "3.15" CONSTS_PATTERN = re.compile(r"\b_*[A-Z]+(?:_+[A-Z]+)*_*\b") diff --git a/scripts/whats_left.py b/scripts/whats_left.py index 40b7b5be9..9adae4c9c 100755 --- a/scripts/whats_left.py +++ b/scripts/whats_left.py @@ -1,6 +1,6 @@ #!/usr/bin/env -S python3 -I # /// script -# requires-python = ">=3.14" +# requires-python = ">=3.15.0" # /// # This script generates Lib/snippets/whats_left_data.py with these variables defined: @@ -39,9 +39,9 @@ GENERATED_FILE = "extra_tests/not_impl.py" implementation = platform.python_implementation() if implementation != "CPython": sys.exit(f"whats_left.py must be run under CPython, got {implementation} instead") -if sys.version_info[:2] < (3, 14): +if sys.version_info[:2] < (3, 15): sys.exit( - f"whats_left.py must be run under CPython 3.14 or newer, got {implementation} {sys.version} instead. If you have uv, try `uv run python -I scripts/whats_left.py` to select a proper Python interpreter easier." + f"whats_left.py must be run under CPython 3.15 or newer, got {implementation} {sys.version} instead. If you have uv, try `uv run python -I scripts/whats_left.py` to select a proper Python interpreter easier." )