Jeong, YunWon
a28ff42345
* Factor with_iframe into enter_iframe/exit_iframe helpers Extract the frame entry (recursion check, TLS link, exception save) and exit (materialization sync, TLS restore, GC tracking) logic from with_iframe into standalone enter_iframe/exit_iframe methods. with_iframe now calls them, with no behavioral change. This prepares for the trampoline loop where enter/exit are called individually rather than wrapped around a closure. Assisted-by: Claude * Allocate InterpreterFrame and LocalsPlus together on the datastack Add InterpreterFrame::new_on_datastack() that bump-allocates both the InterpreterFrame struct and its LocalsPlus data array in a single datastack push, eliminating one allocation per function call. Update datastack_frame_size_bytes_for_code() to include InterpreterFrame size. Convert invoke_prepared_exact_args() and the invoke() fast path to use the combined allocation. Add release_datastack_frame() method on InterpreterFrame that drops all localsplus values, runs field destructors (trace, temporary_refs, retained_back, etc.), and returns the datastack base pointer for pop. Assisted-by: Claude * Implement trampoline loop for Python-to-Python calls Add ExecutionResult::TailCall variant and a trampoline in run_frame_fast that flattens Python-to-Python calls into a single Rust stack frame instead of recursing through the eval loop. CallPyExactArgs now prepares the callee frame on the datastack and returns TailCall when tailcall_enabled is set (run_iframe path only). The trampoline dispatches via a state machine (EnterCallee / ReturnValue / Unwind) in a single loop, avoiding mutual recursion between helper functions that would exhaust the C stack. Exception propagation through suspended frames uses trampoline_handle_exception which adds traceback entries and calls unwind_blocks on each caller. Assisted-by: Claude * Optimize trampoline: skip recursion check, avoid temporary_refs mutex, add bound method TailCall - Add enter_iframe_unchecked for trampoline callee entry (recursion already checked by specialization_call_recursion_guard) - Move callable ownership from per-frame temporary_refs mutex to trampoline-local SuspendedFrame.owned_refs via VM side channel - Add TailCall support for CallBoundMethodExactArgs - Move args directly from caller stack to callee fastlocals - Read materialized pointer once in exit_iframe Incremental call overhead: ~55 ns -> ~35 ns Assisted-by: Claude * Use UnsafeCell for pending_tailcall_refs The VM is per-thread so RefCell's runtime borrow checking is unnecessary overhead. Replace with UnsafeCell for direct access. Assisted-by: Claude * Remove .claude/settings.json from tracking Assisted-by: Claude * Replace SendPtr with Option<SendNonNull> for type-safe pending tailcall Use NonNull + Option instead of raw *mut T with manual null checks. The compiler enforces non-null via the type system, and Option<NonNull> has the same size as a raw pointer thanks to niche optimization. Also extract take_pending_tailcall helper to deduplicate the pattern. Assisted-by: Claude * Restrict PendingFrame to private, expose only set/take methods Rename SendNonNull to PendingFrame and make it fully private: the struct, its field, and the pending_tailcall_frame Cell are all non-pub. External code accesses the side channel only through set_pending_tailcall (pub(crate)) and take_pending_tailcall (private). This ensures the unsafe Send+Sync impl cannot be reused elsewhere without justifying a new safety argument. Assisted-by: Claude * Restore C-stack overflow check in trampoline enter_iframe_unchecked enter_iframe_unchecked was skipping C-stack checks under the assumption that the trampoline stays in one Rust stack frame. But each run_iframe call still consumes Rust stack, so deep Python recursion through the trampoline can exhaust the C stack (observed as STATUS_STACK_OVERFLOW on Windows CI). Keep the C-stack check (every 8th call) while still skipping the Python recursion depth check (already done by specialization_call_recursion_guard). Assisted-by: Claude * Address code review: entry frame double-free, panic safety, cleanup - Fix double-free: mark entry frame with is_entry flag in SuspendedFrame so the trampoline skips its datastack release (the caller owns that cleanup) - Clear iframe.previous in exit_iframe before unlinking the chain, matching with_frame and resume_gen_frame behavior - Add scopeguard in with_iframe for panic safety - Use saturating_sub(1) for lasti in trampoline_handle_exception - Extract datastack_iframe_localsplus_offset helper to avoid duplicated alignment computation Assisted-by: Claude
RustPython
A Python-3 (CPython >= 3.14.0) Interpreter written in Rust 🐍 😱 🤘.
Usage
Check out our online demo running on WebAssembly.
RustPython requires Rust latest stable version (e.g 1.67.1 at February 7th 2023). If you don't currently have Rust installed on your system you can do so by following the instructions at rustup.rs.
To check the version of Rust you're currently running, use rustc --version. If you wish to update,
rustup update stable will update your Rust installation to the most recent stable release.
To build RustPython locally, first, clone the source code:
git clone https://github.com/RustPython/RustPython
RustPython uses symlinks to manage python libraries in Lib/. If on windows, running the following helps:
git config core.symlinks true
Then you can change into the RustPython directory and run the demo (Note: --release is
needed to prevent stack overflow on Windows):
$ cd RustPython
$ cargo run --release demo_closures.py
Hello, RustPython!
Or use the interactive shell:
$ cargo run --release
Welcome to rustpython
>>>>> 2+2
4
NOTE: For windows users, please set RUSTPYTHONPATH environment variable as Lib path in project directory.
(e.g. When RustPython directory is C:\RustPython, set RUSTPYTHONPATH as C:\RustPython\Lib)
You can also install and run RustPython with the following:
$ cargo install --git https://github.com/RustPython/RustPython rustpython
$ rustpython
Welcome to the magnificent Rust Python interpreter
>>>>>
venv
Because RustPython currently doesn't provide a well-packaged installation, using venv helps to use pip easier.
$ rustpython -m venv <your_env_name>
$ . <your_env_name>/bin/activate
$ python # now `python` is the alias of the RustPython for the new env
PIP
If you'd like to make https requests, you can enable the ssl feature, which
also lets you install the pip package manager. Note that on Windows, you may
need to install OpenSSL, or you can enable the ssl-openssl-vendor feature instead,
which compiles OpenSSL for you but requires a C compiler, perl, and make.
OpenSSL version 3 is expected and tested in CI. Older versions may not work.
Once you've installed rustpython with SSL support, you can install pip by running:
cargo install --git https://github.com/RustPython/RustPython
rustpython --install-pip
You can also install RustPython through the conda package manager, though
this isn't officially supported and may be out of date:
conda install rustpython -c conda-forge
rustpython
SSL provider
For HTTPS requests, ssl-rustls-aws-lc is enabled by default for the RustPython binary. Embedders can use rustpython-stdlib's provider-agnostic ssl-rustls feature and install their own rustls crypto provider, or replace rustls with ssl-openssl if their environment requires OpenSSL.
Note that to use OpenSSL on Windows, you may need to install OpenSSL, or you can enable the ssl-openssl-vendor feature instead,
which compiles OpenSSL for you but requires a C compiler, perl, and make.
OpenSSL version 3 is expected and tested in CI. Older versions may not work.
WASI
You can compile RustPython to a standalone WebAssembly WASI module so it can run anywhere.
Build
cargo build --target wasm32-wasip1 --no-default-features --features freeze-stdlib,stdlib --release
Run by wasmer
wasmer run --dir `pwd` -- target/wasm32-wasip1/release/rustpython.wasm `pwd`/extra_tests/snippets/stdlib_random.py
Run by wapm
$ wapm install rustpython
$ wapm run rustpython
>>>>> 2+2
4
Building the WASI file
You can build the WebAssembly WASI file with:
cargo build --release --target wasm32-wasip1 --features="freeze-stdlib"
Note: we use the
freeze-stdlibto include the standard library inside the binary. You also have to run oncerustup target add wasm32-wasip1.
JIT (Just in time) compiler
RustPython has a very experimental JIT compiler that compile python functions into native code.
Building
By default the JIT compiler isn't enabled, it's enabled with the jit cargo feature.
cargo run --features jit
This requires autoconf, automake, libtool, and clang to be installed.
Using
To compile a function, call __jit__() on it.
def foo():
a = 5
return 10 + a
foo.__jit__() # this will compile foo to native code and subsequent calls will execute that native code
assert foo() == 15
Embedding RustPython into your Rust Applications
Interested in exposing Python scripting in an application written in Rust,
perhaps to allow quickly tweaking logic where Rust's compile times would be inhibitive?
Then examples/hello_embed.rs and examples/mini_repl.rs may be of some assistance.
Disclaimer
RustPython is in development, and while the interpreter certainly can be used in interesting use cases like running Python in WASM and embedding into a Rust project, do note that RustPython is not totally production-ready.
Contribution is more than welcome! See our contribution section for more information on this.
Conference videos
Checkout those talks on conferences:
Use cases
Although RustPython is a fairly young project, a few people have used it to make cool projects:
- GreptimeDB: an open-source, cloud-native, distributed time-series database. Using RustPython for embedded scripting.
- pyckitup: a game engine written in rust.
- Robot Rumble: an arena-based AI competition platform
- Ruff: an extremely fast Python linter, written in Rust
Goals
- Full Python-3 environment entirely in Rust (not CPython bindings)
- A clean implementation without compatibility hacks
Documentation
Currently along with other areas of the project, documentation is still in an early phase.
You can read the online documentation for the latest release, or the user guide.
You can also generate documentation locally by running:
cargo doc # Including documentation for all dependencies
cargo doc --no-deps --all # Excluding all dependencies
Documentation HTML files can then be found in the target/doc directory or you can append --open to the previous commands to
have the documentation open automatically on your default browser.
For a high level overview of the components, see the architecture document.
Contributing
Contributions are welcome and highly appreciated. To get started, check out the contributing guidelines.
You can also join us on Discord.
Compiling to WebAssembly
Community
Chat with us on Discord.
Code of conduct
Our code of conduct can be found here.
Credit
The initial work was based on windelbouwman/rspython and shinglyu/RustPython
Links
These are some useful links to related projects:
- https://github.com/ProgVal/pythonvm-rust
- https://github.com/shinglyu/RustPython
- https://github.com/windelbouwman/rspython
License
This project is licensed under the MIT license. Please see the LICENSE file for more details.
The project logo is licensed under the CC-BY-4.0 license. Please see the LICENSE-logo file for more details.