Jeong, YunWon
bef2e41295
* Emit TO_BOOL before conditional jumps, fix class/module prologue - Emit TO_BOOL before POP_JUMP_IF_TRUE/FALSE in the general case of compile_jump_if (Compare expressions excluded since they already produce a bool) - Module-level __doc__: use STORE_NAME instead of STORE_GLOBAL - Class body __module__: use LOAD_NAME instead of LOAD_GLOBAL - Class body: store __firstlineno__ before __doc__ * Emit MAKE_CELL and COPY_FREE_VARS before RESUME Emit MAKE_CELL for each cell variable and COPY_FREE_VARS N for free variables at the start of each code object, before RESUME. These instructions are no-ops in the VM but align the bytecode with CPython 3.14's output. * Emit __static_attributes__ at end of class bodies Store a tuple of attribute names (currently always empty) as __static_attributes__ in the class namespace, matching CPython 3.14's class body epilogue. Attribute name collection from self.xxx accesses is a follow-up task. * Remove expectedFailure from DictProxyTests iter tests test_iter_keys, test_iter_values, test_iter_items now pass because class bodies emit __static_attributes__ and __firstlineno__, matching the expected dict key set. * Use 1-based stack indexing for LIST_EXTEND, SET_UPDATE, etc. Switch LIST_APPEND, LIST_EXTEND, SET_ADD, SET_UPDATE, MAP_ADD from 0-based to 1-based stack depth argument, matching CPython's PEEK(oparg) convention. Adjust the VM to subtract 1 before calling nth_value. * Use plain LOAD_ATTR + PUSH_NULL for calls on imported names When the call target is an attribute of an imported name (e.g., logging.getLogger()), use plain LOAD_ATTR (method_flag=0) with a separate PUSH_NULL instead of method-mode LOAD_ATTR. This matches CPython 3.14's behavior which avoids the method call optimization for module attribute access. * Duplicate return-None epilogue for fall-through blocks When the last block in a code object is exactly LOAD_CONST None + RETURN_VALUE (the implicit return), duplicate these instructions into blocks that would otherwise fall through to it. This matches CPython 3.14's behavior of giving each code path its own explicit return instruction. * Run cargo fmt on ir.rs * Remove expectedFailure from test_intrinsic_1 in test_dis * Emit TO_BOOL before conditional jumps for all expressions including Compare * Add __classdict__ cell for classes with function definitions Set needs_classdict=true for class scopes that contain function definitions (def/async def), matching CPython 3.14's behavior for PEP 649 deferred annotation support. Also restore the Compare expression check in compile_jump_if to skip TO_BOOL for comparison operations. * Emit __classdictcell__ store in class body epilogue Store the __classdict__ cell reference as __classdictcell__ in the class namespace when the class has __classdict__ as a cell variable. Uses LOAD_DEREF (RustPython separates cell vars from fast locals unlike CPython's unified array). * Always run DCE to remove dead code after terminal instructions Run basic dead code elimination (truncating instructions after RETURN_VALUE/RAISE/JUMP within blocks) at all optimization levels, not just optimize > 0. CPython always removes this dead code during assembly. * Restrict LOAD_ATTR plain mode to module/class scope imports Only use plain LOAD_ATTR + PUSH_NULL for imports at module or class scope. Function-local imports use method call mode LOAD_ATTR, matching CPython 3.14's behavior. * Eliminate unreachable blocks after jump normalization Split DCE into two phases: (1) within-block truncation after terminal instructions (always runs), (2) whole-block elimination for blocks only reachable via fall-through from terminal blocks (runs after normalize_jumps when dead jump instructions exist). * Fold BUILD_TUPLE 0 into LOAD_CONST empty tuple Convert BUILD_TUPLE with size 0 to LOAD_CONST () during constant folding, matching CPython's optimization for empty tuple literals. * Handle __classcell__ and __classdictcell__ in type.__new__ - Remove __classcell__ from class dict after setting the cell value - Add __classdictcell__ handling: set cell to class namespace dict, then remove from class dict - Register __classdictcell__ identifier - Use LoadClosure instead of LoadDeref for __classdictcell__ emission - Reorder MakeFunctionFlag bits to match CPython - Run ruff format on scripts * Revert __classdict__ cell and __classdictcell__ changes The __classdict__ cell addition (for classes with function defs) and __classdictcell__ store caused cell initialization failures in importlib. These require deeper VM changes to properly support the cell variable lifecycle. Reverted for stability. * Fix unreachable block elimination with fixpoint reachability Use fixpoint iteration to properly determine block reachability: only mark jump targets of already-reachable blocks, preventing orphaned blocks from falsely marking their targets as reachable. Also add a final DCE pass after assembly NOP removal to catch dead code created by normalize_jumps. * Check enclosing scopes for IMPORTED flag in LOAD_ATTR mode When deciding whether to use plain LOAD_ATTR for attribute calls, check if the name is imported in any enclosing scope (not just the current scope). This handles the common pattern where a module is imported at module level but used inside functions. * Add __classdict__ cell for classes with function definitions Set needs_classdict=true when a class scope contains function definitions (def/async def), matching CPython 3.14 which always creates a __classdict__ cell for PEP 649 support in such classes. * Store __classdictcell__ in class body epilogue Store the __classdict__ cell reference as __classdictcell__ in the class namespace using LoadClosure (which loads the cell object itself, not the value inside). This matches CPython 3.14's class body epilogue. * Fix clippy collapsible_if warnings and cargo fmt * Revert __classdict__ and __classdictcell__ changes (cause import failures) * Revert type.__new__ __classcell__ removal and __classdictcell__ handling Revert the class cell cleanup changes from e6975f973 that cause import failures when frozen module bytecode is stale. The original behavior (not removing __classcell__ from class dict) is restored. * Re-add __classdict__ cell and __classdictcell__ store Restore the __classdict__ cell for classes with function definitions and __classdictcell__ store in class body epilogue. Previous failure was caused by stale .pyc cache files containing bytecode from an intermediate MakeFunctionFlag reorder attempt, not by these changes themselves. * Reorder MakeFunctionFlag to match CPython's SET_FUNCTION_ATTRIBUTE Reorder discriminants: Defaults=0, KwOnlyDefaults=1, Annotations=2, Closure=3, Annotate=4, TypeParams=5. This aligns the oparg values with CPython 3.14's convention. Note: after this change, stale .pyc cache files must be deleted (find . -name '*.pyc' -delete) to avoid bytecode mismatch errors. * Use CPython-compatible power-of-two encoding for SET_FUNCTION_ATTRIBUTE Override From/TryFrom for MakeFunctionFlag to use power-of-two values (1,2,4,8,16,32) matching CPython's SET_FUNCTION_ATTRIBUTE oparg encoding, instead of sequential discriminants (0,1,2,3,4,5). * Remove expectedFailure from test_elim_jump_after_return1 and test_no_jump_over_return_out_of_finally_block * Remove __classcell__ and __classdictcell__ from class dict in type.__new__ * Remove expectedFailure from test___classcell___expected_behaviour, cargo fmt * Handle MakeCell and CopyFreeVars as no-ops in JIT These prologue instructions are handled at frame creation time by the VM. The JIT operates on already-initialized frames, so these can be safely skipped during compilation. * Remove expectedFailure from test_load_fast_known_simple * Restore expectedFailure for test_load_fast_known_simple The test expects LOAD_FAST_BORROW_LOAD_FAST_BORROW superinstruction which RustPython does not emit yet.
RustPython
A Python-3 (CPython >= 3.14.0) Interpreter written in Rust 🐍 😱 🤘.
Usage
Check out our online demo running on WebAssembly.
RustPython requires Rust latest stable version (e.g 1.67.1 at February 7th 2023). If you don't currently have Rust installed on your system you can do so by following the instructions at rustup.rs.
To check the version of Rust you're currently running, use rustc --version. If you wish to update,
rustup update stable will update your Rust installation to the most recent stable release.
To build RustPython locally, first, clone the source code:
git clone https://github.com/RustPython/RustPython
RustPython uses symlinks to manage python libraries in Lib/. If on windows, running the following helps:
git config core.symlinks true
Then you can change into the RustPython directory and run the demo (Note: --release is
needed to prevent stack overflow on Windows):
$ cd RustPython
$ cargo run --release demo_closures.py
Hello, RustPython!
Or use the interactive shell:
$ cargo run --release
Welcome to rustpython
>>>>> 2+2
4
NOTE: For windows users, please set RUSTPYTHONPATH environment variable as Lib path in project directory.
(e.g. When RustPython directory is C:\RustPython, set RUSTPYTHONPATH as C:\RustPython\Lib)
You can also install and run RustPython with the following:
$ cargo install --git https://github.com/RustPython/RustPython rustpython
$ rustpython
Welcome to the magnificent Rust Python interpreter
>>>>>
venv
Because RustPython currently doesn't provide a well-packaged installation, using venv helps to use pip easier.
$ rustpython -m venv <your_env_name>
$ . <your_env_name>/bin/activate
$ python # now `python` is the alias of the RustPython for the new env
PIP
If you'd like to make https requests, you can enable the ssl feature, which
also lets you install the pip package manager. Note that on Windows, you may
need to install OpenSSL, or you can enable the ssl-vendor feature instead,
which compiles OpenSSL for you but requires a C compiler, perl, and make.
OpenSSL version 3 is expected and tested in CI. Older versions may not work.
Once you've installed rustpython with SSL support, you can install pip by running:
cargo install --git https://github.com/RustPython/RustPython
rustpython --install-pip
You can also install RustPython through the conda package manager, though
this isn't officially supported and may be out of date:
conda install rustpython -c conda-forge
rustpython
SSL provider
For HTTPS requests, ssl-rustls feature is enabled by default. You can replace it with ssl-openssl feature if your environment requires OpenSSL.
Note that to use OpenSSL on Windows, you may need to install OpenSSL, or you can enable the ssl-vendor feature instead,
which compiles OpenSSL for you but requires a C compiler, perl, and make.
OpenSSL version 3 is expected and tested in CI. Older versions may not work.
WASI
You can compile RustPython to a standalone WebAssembly WASI module so it can run anywhere.
Build
cargo build --target wasm32-wasip1 --no-default-features --features freeze-stdlib,stdlib --release
Run by wasmer
wasmer run --dir `pwd` -- target/wasm32-wasip1/release/rustpython.wasm `pwd`/extra_tests/snippets/stdlib_random.py
Run by wapm
$ wapm install rustpython
$ wapm run rustpython
>>>>> 2+2
4
Building the WASI file
You can build the WebAssembly WASI file with:
cargo build --release --target wasm32-wasip1 --features="freeze-stdlib"
Note: we use the
freeze-stdlibto include the standard library inside the binary. You also have to run oncerustup target add wasm32-wasip1.
JIT (Just in time) compiler
RustPython has a very experimental JIT compiler that compile python functions into native code.
Building
By default the JIT compiler isn't enabled, it's enabled with the jit cargo feature.
cargo run --features jit
This requires autoconf, automake, libtool, and clang to be installed.
Using
To compile a function, call __jit__() on it.
def foo():
a = 5
return 10 + a
foo.__jit__() # this will compile foo to native code and subsequent calls will execute that native code
assert foo() == 15
Embedding RustPython into your Rust Applications
Interested in exposing Python scripting in an application written in Rust,
perhaps to allow quickly tweaking logic where Rust's compile times would be inhibitive?
Then examples/hello_embed.rs and examples/mini_repl.rs may be of some assistance.
Disclaimer
RustPython is in development, and while the interpreter certainly can be used in interesting use cases like running Python in WASM and embedding into a Rust project, do note that RustPython is not totally production-ready.
Contribution is more than welcome! See our contribution section for more information on this.
Conference videos
Checkout those talks on conferences:
Use cases
Although RustPython is a fairly young project, a few people have used it to make cool projects:
- GreptimeDB: an open-source, cloud-native, distributed time-series database. Using RustPython for embedded scripting.
- pyckitup: a game engine written in rust.
- Robot Rumble: an arena-based AI competition platform
- Ruff: an extremely fast Python linter, written in Rust
Goals
- Full Python-3 environment entirely in Rust (not CPython bindings)
- A clean implementation without compatibility hacks
Documentation
Currently along with other areas of the project, documentation is still in an early phase.
You can read the online documentation for the latest release, or the user guide.
You can also generate documentation locally by running:
cargo doc # Including documentation for all dependencies
cargo doc --no-deps --all # Excluding all dependencies
Documentation HTML files can then be found in the target/doc directory or you can append --open to the previous commands to
have the documentation open automatically on your default browser.
For a high level overview of the components, see the architecture document.
Contributing
Contributions are more than welcome, and in many cases we are happy to guide contributors through PRs or on Discord. Please refer to the development guide as well for tips on developments.
With that in mind, please note this project is maintained by volunteers, some of the best ways to get started are below:
Most tasks are listed in the issue tracker. Check issues labeled with good first issue if you wish to start coding.
To enhance CPython compatibility, try to increase unittest coverage by checking this article: How to contribute to RustPython by CPython unittest
Another approach is to checkout the source code: builtin functions and object methods are often the simplest and easiest way to contribute.
You can also simply run python -I scripts/whats_left.py to assist in finding any unimplemented
method.
Compiling to WebAssembly
Community
Chat with us on Discord.
Code of conduct
Our code of conduct can be found here.
Credit
The initial work was based on windelbouwman/rspython and shinglyu/RustPython
Links
These are some useful links to related projects:
- https://github.com/ProgVal/pythonvm-rust
- https://github.com/shinglyu/RustPython
- https://github.com/windelbouwman/rspython
License
This project is licensed under the MIT license. Please see the LICENSE file for more details.
The project logo is licensed under the CC-BY-4.0 license. Please see the LICENSE-logo file for more details.