a403341791
* Match CPython's SyntaxError range for unparenthesized `except` types
CPython's `invalid_except_stmt` rule raises the error only once the whole
clause has matched, and reports a range that starts at the first exception
type and ends at the `:` closing the clause, so it covers the `as NAME`
part as well:
invalid_except_stmt:
| 'except' a=expression ',' expressions 'as' NAME ':' {
RAISE_SYNTAX_ERROR_STARTING_FROM(a, "multiple exception types
must be parenthesized when using 'as'") }
The parser reports the exception types alone, so look up that `:` in the
source and widen the range to match. Only `as NAME` may follow the
exception types, which is why the first `:` after them is the one closing
the clause.
try:
pass
except A, B, C as e:
pass
CPython 3.14: ('x.py', 3, 8, 'except A, B, C as e:\n', 3, 20)
before: ('x.py', 3, 8, 'except A, B, C as e:\n', 3, 15)
Fixes #8496
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Clarify that the `except` error range stops before the closing colon
`invalid_except_stmt_end` returns the exclusive end of the range, so the
column it yields is the one the `:` sits on even though the `:` is not part
of the range. The previous "ending at the `:`" wording read as if the colon
were included, which invites an off-by-one "fix".
Comments only; no behavior change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Fix the `except ... as` error column for non-ASCII names and line joins
`invalid_except_stmt_end` recovers the end of CPython's range by scanning the
source, which has to redo by hand what CPython gets from its tokenizer:
- The column was the byte distance from the line start, but `end_offset` is a
character column. `except Ä, B as e:` reported 18 instead of 17. CPython
converts explicitly, in `_PyPegen_byte_offset_to_character_offset`.
- The colon scan already spans explicit line joins, but the `as` check ran on
the raw slice, so a `\` before `as` left the range unextended.
`except A, B \` + `as exc:` reported (3, 12) instead of (4, 7).
Add regression cases to `syntax_invalid.py` covering both, plus the whitespace,
`except*` and line-join placements that already worked. All values are verified
against CPython 3.14.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Update the Ruff fork to 0.16.10
Point the rustpython-ruff_* dependencies at the 0.16.10-rustpython tag
of the RustPython Ruff fork and port to its AST changes:
`ExprCompare` stores all operands in `operands`, call arguments use
`ThinVec`, dict comprehension generators and f-string elements are boxed
slices, `FStringValue` iterates `FStringPartRef`, and `TokenKind::Name`
is now `TokenKind::Identifier`.
Assisted-by: Grok:grok-4.6
Assisted-by: Claude Code:claude-opus-5-5
* Drop the source scan for the `except ... as` error range
The parser now reports the unparenthesized exception types error with a
range ending at the `:` that closes the clause, so
`invalid_except_stmt_end` and the message match that widened the range
are removed.
Assisted-by: Claude Code:claude-opus-5-5
* Take parse error messages from the parser as they are
The Ruff fork now emits the final message texts, so `vm_new.rs` no
longer rewrites parse error messages by matching their text and no
longer lowercases the first character of every message.
`analyze_compile_error` keeps only the caret narrowing for starred
expressions and the line number in the unterminated string message.
The compiler matches the new lowercase texts for indented block and
mixed bytes literal errors.
The fork is referenced by revision until it is published.
Assisted-by: Claude Code:claude-opus-5-5
* Take starred and bytes literal error ranges from the parser
The Ruff fork now reports misplaced starred expressions at the `*` and
non-ASCII bytes literal errors over the whole literal. `vm_new.rs` drops
`narrow_caret` together with `SyntaxErrorInfo`, which only carried the
message after that, and the compiler drops `bytes_literal_span`.
Assisted-by: Claude Code:claude-opus-5-5
* Take indentation error kinds and positions from the parser
The Ruff fork now reports `TabError` and `TooDeepIndentation` as their
own lexer errors and places indentation and line continuation errors
itself. `vm_new.rs` picks TabError or IndentationError from the error
kind instead of scanning the source for mixed indentation, drops the
TabError message override, and sets the end offset of tokenizer errors
to 0 or -1 by kind. The compiler drops the line-end relocation of
unindent errors and the line continuation rewrite. `_tokenize` raises
TabError and depth errors for the token they replace, also for sources
containing tabs.
Remove expectedFailure from test_tokenize.test_max_indent and four
test_tabnanny tests.
Assisted-by: Claude Code:claude-opus-5-5
* Take unterminated string messages from the parser
The Ruff fork's unterminated string errors now include the detected
line and are reported at the start of the string. `vm_new.rs` drops its
message override for `UnclosedStringError`, and the shell continues a
line on an unterminated triple-quoted string by the error's
`triple_quoted` flag instead of reading the quotes from the source.
Assisted-by: Claude Code:claude-opus-5-5
* Take bracket errors and tokenizer error priority from the parser
Bump the ruff fork, which now reports unmatched, mismatched, unclosed
and too deeply nested brackets and decides whether a tokenizer error
replaces the parser's first error.
Remove the bracket, unterminated string and nesting depth source
scanners and `pre_parse_source_error`. A tokenizer error from the
parser now wins over source scanner diagnostics found after it. An
unclosed bracket is incomplete input only when the parser reports it
as such, and other tokenizer errors end where they start.
Remove the expected failure markers from test_unicode_identifiers
test_invalid and the `import ä £` doctest in test_syntax.
Assisted-by: Claude Code:claude-opus-5-5
* Take number literal, string prefix and non-printable errors from the parser
Bump the ruff fork, which now reports malformed number literals,
incompatible string prefixes and non-printable characters itself.
Remove the number literal, string prefix and non-printable character
source scanners and the override classes that ranked them. Leading zero
and string prefix errors keep their range instead of ending where they
start.
Assisted-by: Claude Code:claude-opus-5-5
* Take the missing indented block error from the parser
Bump the ruff fork, which now reports `ExpectedIndentedBlock` with the
clause and the line of its header keyword.
Remove the message rewrite that derived the clause and line from the
source. The VM chooses IndentationError or incomplete input by the
error variant instead of its message.
Assisted-by: Claude Code:claude-opus-5-5
* Take binding target errors from the parser
Bump the ruff fork, which now names the invalid expression in
assignment, augmented assignment, delete, `for`, `with`, import,
pattern and `except` target errors.
Remove the source scanners for those targets and the message rewrites
of `InvalidAssignmentTarget` and `InvalidNamedAssignmentTarget`.
Remove the expected failure markers from test_syntax doctests that now
pass.
Assisted-by: Claude Code:claude-opus-5-5
* Take parameter, argument, star, comma and dictionary errors from the parser
Bump the ruff fork to 36919b62fa. Remove the source scanners for type
parameters, parameter lists, star annotations, call arguments, def type
parameters, parenthesized groups, missing commas and dictionary items.
Set end_offset 0 for ExpectedColonAfterDictionaryKey. Remove 14
EXPECTED_FAILURE markers in test_syntax and test_unpack_ex.
Assisted-by: Claude Code:claude-opus-5-5
* Take expression errors from the parser
Bump the ruff fork to 4f922d2124. Remove the source scanners for
comprehensions, collection and condition assignments, expressions
between strings, named expression targets, missing `in` after for-loop
variables and statements in `if` expressions. Remove the now passing
expected failure markers in test_syntax and test_named_expressions.
Assisted-by: Claude Code:claude-opus-5-5
* Take statement errors from the parser
Bump the ruff fork to c7f6594ed6. Remove the source scanners for
standalone except, `import ... from`, `**_` mapping rests, `elif` after
`else` and mixed except handlers, and the post-parse scanners for call
arguments, match targets, yield after a comma and parenthesized star
imports.
Assisted-by: Claude Code:claude-opus-5-5
* Take string and f-string errors from the parser
Bump the ruff fork to 6b46d0dfc8. Remove the source scanners for
malformed `\N` escapes, f-string and t-string replacement fields, mixed
t-string literals and `print`/`exec` statements, and the override
plumbing they used. Drop the `f'{x'; '` case from the compiler test,
whose expected message differs from CPython 3.14.
Assisted-by: Claude Code:claude-opus-5-5
* Restore the bracket depth check and fix incomplete unclosed strings
Restore `pre_parse_source_error`, which rejects bracket nesting past 200
levels before parsing. The parser reports the same error but still
builds the full tree, which overflows the native stack when dropped.
Remove the VM's unclosed string scanner, which ignored the mode and
treated `'abc` in exec mode as incomplete input. The compiler's check
now skips escaped newlines.
Assisted-by: Claude Code:claude-opus-5-5
* Report multiple statements before later errors in single mode
In single mode, when the first statement is a simple statement, report
code after its line as multiple statements before any error found
later, at the newline token or the comment before it. This replaces the
multiple statements check on the parsed body and also applies to
`ast.parse(mode='single')`.
An unclosed string that follows an earlier syntax error is no longer
incomplete input. The bracket depth error range is now empty.
Assisted-by: Claude Code:claude-opus-5-5
* Update the Ruff fork for parser speed and starred annotations
Bump the ruff fork to 8b59b85344, which skips the tokenizer error pass
when parsing finds no errors, stops allocating for every identifier's
string prefix check, and reports `def f(*args: *b = ...)` as invalid
syntax.
Remove expected failure markers from test_code_module
test_indentation_error and test_sysexcepthook_indentation_error, and
from test_dictcomps test_illegal_assignment.
Assisted-by: Claude Code:claude-opus-5-5
* Use the published Ruff fork 0.16.10 crates
Switch the rustpython-ruff_* dependencies from the git revision to the
0.16.10 crates.io release, published from the same commit (8b59b85344).
Assisted-by: Claude Code:claude-opus-5-5
---------
Co-authored-by: zzarbttoo <qodbwls70@naver.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
RustPython
A Python-3 (CPython >= 3.14.0) Interpreter written in Rust 🐍 😱 🤘.
Usage
Check out our online demo running on WebAssembly.
RustPython requires Rust latest stable version (e.g 1.67.1 at February 7th 2023). If you don't currently have Rust installed on your system you can do so by following the instructions at rustup.rs.
To check the version of Rust you're currently running, use rustc --version. If you wish to update,
rustup update stable will update your Rust installation to the most recent stable release.
To build RustPython locally, first, clone the source code:
git clone https://github.com/RustPython/RustPython
RustPython uses symlinks to manage python libraries in Lib/. If on windows, running the following helps:
git config core.symlinks true
Then you can change into the RustPython directory and run the demo (Note: --release is
needed to prevent stack overflow on Windows):
$ cd RustPython
$ cargo run --release -- -c 'print("Hello, RustPython!")'
Hello, RustPython!
Or use the interactive shell:
$ cargo run --release
Welcome to rustpython
>>>>> 2+2
4
NOTE: For windows users, please set RUSTPYTHONPATH environment variable as Lib path in project directory.
(e.g. When RustPython directory is C:\RustPython, set RUSTPYTHONPATH as C:\RustPython\Lib)
You can also install and run RustPython with the following:
$ cargo install --git https://github.com/RustPython/RustPython rustpython
$ rustpython
Welcome to the magnificent Rust Python interpreter
>>>>>
venv
Because RustPython currently doesn't provide a well-packaged installation, using venv helps to use pip easier.
$ rustpython -m venv <your_env_name>
$ . <your_env_name>/bin/activate
$ python # now `python` is the alias of the RustPython for the new env
PIP
If you'd like to make https requests, you can enable the ssl feature, which
also lets you install the pip package manager. Note that on Windows, you may
need to install OpenSSL, or you can enable the ssl-openssl-vendor feature instead,
which compiles OpenSSL for you but requires a C compiler, perl, and make.
OpenSSL version 3 is expected and tested in CI. Older versions may not work.
Once you've installed rustpython with SSL support, you can install pip by running:
cargo install --git https://github.com/RustPython/RustPython
rustpython --install-pip
You can also install RustPython through the conda package manager, though
this isn't officially supported and may be out of date:
conda install rustpython -c conda-forge
rustpython
SSL provider
For HTTPS requests, ssl-rustls-aws-lc is enabled by default for the RustPython binary. Embedders can use rustpython-stdlib's provider-agnostic ssl-rustls feature and install their own rustls crypto provider, or replace rustls with ssl-openssl if their environment requires OpenSSL.
Note that to use OpenSSL on Windows, you may need to install OpenSSL, or you can enable the ssl-openssl-vendor feature instead,
which compiles OpenSSL for you but requires a C compiler, perl, and make.
OpenSSL version 3 is expected and tested in CI. Older versions may not work.
WASI
You can compile RustPython to a standalone WebAssembly WASI module so it can run anywhere.
Build
cargo build --target wasm32-wasip1 --no-default-features --features freeze-stdlib,stdlib --release
Run by wasmer
wasmer run --volume `pwd` -- target/wasm32-wasip1/release/rustpython.wasm `pwd`/extra_tests/snippets/stdlib_random.py
Run by wapm
$ wapm install rustpython
$ wapm run rustpython
>>>>> 2+2
4
Building the WASI file
You can build the WebAssembly WASI file with:
cargo build --release --target wasm32-wasip1 --features="freeze-stdlib"
Note: we use the
freeze-stdlibto include the standard library inside the binary. You also have to run oncerustup target add wasm32-wasip1.
JIT (Just in time) compiler
RustPython has a very experimental JIT compiler that compile python functions into native code.
Building
By default the JIT compiler isn't enabled, it's enabled with the jit cargo feature.
cargo run --features jit
This requires autoconf, automake, libtool, and clang to be installed.
Using
To compile a function, call __jit__() on it.
def foo():
a = 5
return 10 + a
foo.__jit__() # this will compile foo to native code and subsequent calls will execute that native code
assert foo() == 15
Embedding RustPython into your Rust Applications
Interested in exposing Python scripting in an application written in Rust,
perhaps to allow quickly tweaking logic where Rust's compile times would be inhibitive?
Then examples/hello_embed.rs and examples/mini_repl.rs may be of some assistance.
Disclaimer
RustPython is in development, and while the interpreter certainly can be used in interesting use cases like running Python in WASM and embedding into a Rust project, do note that RustPython is not totally production-ready.
Contribution is more than welcome! See our contribution section for more information on this.
Conference videos
Checkout those talks on conferences:
Use cases
Although RustPython is a fairly young project, a few people have used it to make cool projects:
- GreptimeDB: an open-source, cloud-native, distributed time-series database. Using RustPython for embedded scripting.
- pyckitup: a game engine written in rust.
- Robot Rumble: an arena-based AI competition platform
- Ruff: an extremely fast Python linter, written in Rust
Goals
- Full Python-3 environment entirely in Rust (not CPython bindings)
- A clean implementation without compatibility hacks
Documentation
Currently along with other areas of the project, documentation is still in an early phase.
You can read the online documentation for the latest release, or the user guide.
You can also generate documentation locally by running:
cargo doc # Including documentation for all dependencies
cargo doc --no-deps --all # Excluding all dependencies
Documentation HTML files can then be found in the target/doc directory or you can append --open to the previous commands to
have the documentation open automatically on your default browser.
For a high level overview of the components, see the architecture document.
Contributing
Contributions are welcome and highly appreciated. To get started, check out the contributing guidelines.
You can also join us on Discord.
Compiling to WebAssembly
Community
Chat with us on Discord.
Code of conduct
Our code of conduct can be found here.
Credit
The initial work was based on windelbouwman/rspython and shinglyu/RustPython
Links
These are some useful links to related projects:
- https://github.com/ProgVal/pythonvm-rust
- https://github.com/shinglyu/RustPython
- https://github.com/windelbouwman/rspython
License
This project is licensed under the MIT license. Please see the LICENSE file for more details.
The project logo is licensed under the CC-BY-4.0 license. Please see the LICENSE-logo file for more details.