Hanjeong Lee
e189ca7923
* Record a module's docstring mismatches on their own
They were only kept when the module also had missing items or signature
mismatches, so annotationlib, base64, ntpath and _ctypes reported none.
Whether a module counts as implemented still depends on missing items
and signatures alone.
Assisted-by: Claude Code:claude-opus-5
* Report a builtin type method that differs as a mismatch
A method whose signature or docstring differed was listed under
"builtin items" as if it were missing, with both extra_info dicts as the
reason. That section feeds the website's builtin_items.csv. Differences
now go to mismatched_items and mismatched_doc_items, keyed by type name,
like module items. Nothing reaches this path yet because builtin
signatures are still skipped; without this change, lifting that skip
turns the one "builtin items" line into 455.
Assisted-by: Claude Code:claude-opus-5
* Compare signatures and docstrings of native callables
extra_info() skipped anything inspect treats as builtin since cc2c46b1b
(#2410), when RustPython had no __text_signature__ for builtins. It has
one now, so --signature and --doc cover functions and methods written in
Rust. Modules with a mismatched signature go from 45 to 134, and 160 of
the 710 mismatched items are ones CPython itself cannot parse, which the
--signature listing already skips.
__builtins__ is no longer compared. inspect.signature() evals defaults in
the owning module's namespace and eval() adds __builtins__ there, so
CPython's builtins module gained one after list.index was inspected and
it showed up as missing.
Assisted-by: Claude Code:claude-opus-5
* Scan the members of native classes in each module
A module was only compared one level deep, so a native class counted as
present whatever its methods were. Classes a module owns and that are
not Python-defined are now opened too, and their own members are
compared as "Class.member": 438 classes and 4,833 members, 1,033 of
which have a signature --signature reports.
A class is native when it has no __firstlineno__, which a class
statement records, and no Python functions of its own, which rules out
namedtuple-style classes made by type(). BUILTIN_TYPES moves to module
level so the builtin types are not reported twice, an alias like
array.ArrayType is skipped, and CPython's __annotations_cache__ and
__vectorcalloffset__ are left out.
RustPython follows the CPython list instead of deciding again. A member
only inherited is listed as missing with "(inherited)", as builtin type
methods are. A class missing here, or implemented as something else such
as _sha3's hash functions, is reported by its own line only.
Assisted-by: Claude Code:claude-opus-5
* Report a module that exists but cannot be imported
dir_of_mod_or_error() ran dir() on the exception import_module() returns,
so a failed import looked like a module whose every item is missing:
sqlite3 alone filled 198 of those lines, and CPython's own _ios_support
was compared as a module holding ImportError's attributes. Returning the
exception reaches the "exists but not importable" line that was already
written for it, and drops 529 lines from "stdlib items".
Assisted-by: Claude Code:claude-opus-5
* Compare a module's own docstring
extra_info() reports a docstring for callables, and a module is not one,
so the __doc__ item both sides record was None and a module docstring
never counted as a mismatch. Reading it with getdoc(), the way a
callable's is read, brings 221 module docstrings into the comparison.
sys is the one that differs; base64 and imaplib differ here only
because their Lib copies track 3.14.7 while this run used 3.14.6.
Non-callables keep reporting None. getdoc() on a plain value answers
with its type's docstring, which would turn every string constant into
a mismatch.
Assisted-by: Claude Code:claude-opus-5
* Decide a member's ownership by the class dictionary
attr_is_not_inherited() compared the attribute's value with each base's.
That took an inherited classmethod for the class's own, since a bound
method compares unequal across classes, and dropped an own attribute
whose value matched a base's default None: 394 of CPython's in all,
ast.FunctionDef.returns among them. On the RustPython side it called
AttributeError.name, a class attribute set to None, inherited.
Ownership now means the class dictionary holds the name. An inherited
member is reported only when what it resolves to differs from CPython's
own, so dict_items.__doc__, a slot CPython fills with None that
RustPython resolves to None without one, stays quiet. "(inherited)"
lines drop from 196 to 54, and ast.alias.end_lineno is among the real
gaps that appear.
Assisted-by: Claude Code:claude-fable-5-1
RustPython
A Python-3 (CPython >= 3.14.0) Interpreter written in Rust 🐍 😱 🤘.
Usage
Check out our online demo running on WebAssembly.
RustPython requires Rust latest stable version (e.g 1.67.1 at February 7th 2023). If you don't currently have Rust installed on your system you can do so by following the instructions at rustup.rs.
To check the version of Rust you're currently running, use rustc --version. If you wish to update,
rustup update stable will update your Rust installation to the most recent stable release.
To build RustPython locally, first, clone the source code:
git clone https://github.com/RustPython/RustPython
RustPython uses symlinks to manage python libraries in Lib/. If on windows, running the following helps:
git config core.symlinks true
Then you can change into the RustPython directory and run the demo (Note: --release is
needed to prevent stack overflow on Windows):
$ cd RustPython
$ cargo run --release -- -c 'print("Hello, RustPython!")'
Hello, RustPython!
Or use the interactive shell:
$ cargo run --release
Welcome to rustpython
>>>>> 2+2
4
NOTE: For windows users, please set RUSTPYTHONPATH environment variable as Lib path in project directory.
(e.g. When RustPython directory is C:\RustPython, set RUSTPYTHONPATH as C:\RustPython\Lib)
You can also install and run RustPython with the following:
$ cargo install --git https://github.com/RustPython/RustPython rustpython
$ rustpython
Welcome to the magnificent Rust Python interpreter
>>>>>
venv
Because RustPython currently doesn't provide a well-packaged installation, using venv helps to use pip easier.
$ rustpython -m venv <your_env_name>
$ . <your_env_name>/bin/activate
$ python # now `python` is the alias of the RustPython for the new env
PIP
If you'd like to make https requests, you can enable the ssl feature, which
also lets you install the pip package manager. Note that on Windows, you may
need to install OpenSSL, or you can enable the ssl-openssl-vendor feature instead,
which compiles OpenSSL for you but requires a C compiler, perl, and make.
OpenSSL version 3 is expected and tested in CI. Older versions may not work.
Once you've installed rustpython with SSL support, you can install pip by running:
cargo install --git https://github.com/RustPython/RustPython
rustpython --install-pip
You can also install RustPython through the conda package manager, though
this isn't officially supported and may be out of date:
conda install rustpython -c conda-forge
rustpython
SSL provider
For HTTPS requests, ssl-rustls-aws-lc is enabled by default for the RustPython binary. Embedders can use rustpython-stdlib's provider-agnostic ssl-rustls feature and install their own rustls crypto provider, or replace rustls with ssl-openssl if their environment requires OpenSSL.
Note that to use OpenSSL on Windows, you may need to install OpenSSL, or you can enable the ssl-openssl-vendor feature instead,
which compiles OpenSSL for you but requires a C compiler, perl, and make.
OpenSSL version 3 is expected and tested in CI. Older versions may not work.
WASI
You can compile RustPython to a standalone WebAssembly WASI module so it can run anywhere.
Build
cargo build --target wasm32-wasip1 --no-default-features --features freeze-stdlib,stdlib --release
Run by wasmer
wasmer run --volume `pwd` -- target/wasm32-wasip1/release/rustpython.wasm `pwd`/extra_tests/snippets/stdlib_random.py
Run by wapm
$ wapm install rustpython
$ wapm run rustpython
>>>>> 2+2
4
Building the WASI file
You can build the WebAssembly WASI file with:
cargo build --release --target wasm32-wasip1 --features="freeze-stdlib"
Note: we use the
freeze-stdlibto include the standard library inside the binary. You also have to run oncerustup target add wasm32-wasip1.
JIT (Just in time) compiler
RustPython has a very experimental JIT compiler that compile python functions into native code.
Building
By default the JIT compiler isn't enabled, it's enabled with the jit cargo feature.
cargo run --features jit
This requires autoconf, automake, libtool, and clang to be installed.
Using
To compile a function, call __jit__() on it.
def foo():
a = 5
return 10 + a
foo.__jit__() # this will compile foo to native code and subsequent calls will execute that native code
assert foo() == 15
Embedding RustPython into your Rust Applications
Interested in exposing Python scripting in an application written in Rust,
perhaps to allow quickly tweaking logic where Rust's compile times would be inhibitive?
Then examples/hello_embed.rs and examples/mini_repl.rs may be of some assistance.
Disclaimer
RustPython is in development, and while the interpreter certainly can be used in interesting use cases like running Python in WASM and embedding into a Rust project, do note that RustPython is not totally production-ready.
Contribution is more than welcome! See our contribution section for more information on this.
Conference videos
Checkout those talks on conferences:
Use cases
Although RustPython is a fairly young project, a few people have used it to make cool projects:
- GreptimeDB: an open-source, cloud-native, distributed time-series database. Using RustPython for embedded scripting.
- pyckitup: a game engine written in rust.
- Robot Rumble: an arena-based AI competition platform
- Ruff: an extremely fast Python linter, written in Rust
Goals
- Full Python-3 environment entirely in Rust (not CPython bindings)
- A clean implementation without compatibility hacks
Documentation
Currently along with other areas of the project, documentation is still in an early phase.
You can read the online documentation for the latest release, or the user guide.
You can also generate documentation locally by running:
cargo doc # Including documentation for all dependencies
cargo doc --no-deps --all # Excluding all dependencies
Documentation HTML files can then be found in the target/doc directory or you can append --open to the previous commands to
have the documentation open automatically on your default browser.
For a high level overview of the components, see the architecture document.
Contributing
Contributions are welcome and highly appreciated. To get started, check out the contributing guidelines.
You can also join us on Discord.
Compiling to WebAssembly
Community
Chat with us on Discord.
Code of conduct
Our code of conduct can be found here.
Credit
The initial work was based on windelbouwman/rspython and shinglyu/RustPython
Links
These are some useful links to related projects:
- https://github.com/ProgVal/pythonvm-rust
- https://github.com/shinglyu/RustPython
- https://github.com/windelbouwman/rspython
License
This project is licensed under the MIT license. Please see the LICENSE file for more details.
The project logo is licensed under the CC-BY-4.0 license. Please see the LICENSE-logo file for more details.