Jeong, YunWon 55c012cffd unicode crate (#8211)
* Extract unicodedata core into rustpython-unicode crate

Move the vm-free Unicode character database access — the generated UCD 3.2.0 /
latest tables, their build-time generation, and the icu4x/unicode_names2 lookups
for category/bidirectional/combining/east_asian_width/mirrored/decomposition/
normalize/is_normalized/digit/decimal/numeric/name/lookup — into a new leaf crate
over char/CodePoint/&Wtf8. stdlib/unicodedata.rs keeps the UCD pyclass binding: it
extracts the code point, boxes results, and maps errors. The unicode data files,
build.rs table generation, and the icu_properties/icu_normalizer/unicode_names2
dependencies move out of stdlib.

Assisted-by: Claude

* Route str predicates, casefold, and printable through rustpython-unicode

Add classify/case/identifier modules to the shared crate over char/&str/&Wtf8:
isalpha/isalnum/isdecimal/isdigit/isnumeric/isspace/isprintable classification,
XID identifier predicates, and full-mapping casefold. vm/builtins/str.rs and
literal/char.rs now call these instead of icu_properties/icu_casemap directly.
String-level iteration with final-sigma handling (lower/upper/title/capitalize,
islower/isupper) stays in the runtime.

Assisted-by: Claude

* Route sre_engine char classes through rustpython-unicode

Add the regex module with the SRE character-class and case predicates
(is_word/is_space/is_digit, is_uni_* Unicode variants, ascii/locale/unicode case
folding). sre_engine/string.rs now forwards to it, keeping its public API and
behavior identical — including the ASCII-only is_uni_digit and the hardcoded
is_uni_space table.

Assisted-by: Claude

* Route \N{} name lookups through rustpython-unicode

codegen/string_parser.rs (\N{...} escapes) and common/encodings.rs (the
namereplace error handler) now resolve character names via rustpython-unicode
instead of depending on unicode_names2 directly, leaving the crate as the sole
owner of the name database.

Assisted-by: Claude

* Add differential Unicode sweep and shared-crate snippet test

tests/differential.rs sweeps the full 0..0x110000 range and compares each str
classification predicate against a committed CPython 3.14 reference dataset
(tests/data/cpython3.14_predicates.txt, produced by generate_reference.py). Code
points that differ only because the Rust std / icu4x build ships a later Unicode
release than CPython 3.14's 16.0.0 are recorded in
tests/data/version_skew_cpython3.14.txt, regenerable via
RUSTPYTHON_UNICODE_REGEN_SKEW=1; the regen refuses to record any
cpython=true/crate=false divergence, so only newly-assigned code points are
allowed. Any other divergence fails. Both data files use a run-length
`predicate start:end,...` encoding.

extra_tests/snippets/stdlib_unicode_shared.py exercises the routed surface end to
end (str predicates, casefold, identifiers, unicodedata, normalize, \N{}, and re
character classes) and passes identically on CPython 3.14 and RustPython.

Assisted-by: Claude

* Match regex \d to Unicode decimal digits

is_uni_digit previously matched only ASCII 0-9, so re's \d in Unicode mode
missed decimal digits like ٥ and ५. SRE_UNI_IS_DIGIT matches Py_UNICODE_ISDECIMAL
(category Nd), so route it through classify::is_decimal. Unmasks test_bug_6561 in
test_re and extends the shared-crate snippet with the Nd/Nl/No cases.

Assisted-by: Claude

* Replace shallow Unicode wrappers with re-exports

Address review feedback on the extraction.

Functions that only forwarded to the shared crate become `use ... as`
re-exports rather than hand-written wrappers:

- literal: is_printable is dropped; escape.rs calls
  rustpython_unicode::classify::is_repr_printable directly.
- unicode::identifier::is_continue re-exports is_xid_continue.
- unicode::data::lookup_character re-exports unicode_names2::character.
- Drop the unused repr(u8) on DecompositionType.

The SRE character-class and case predicates move back into
sre_engine::string (their pre-extraction home) instead of living in a
unicode::regex module that only sre_engine and the vm _sre binding used;
is_uni_digit/is_uni_alnum still delegate to rustpython_unicode::classify.
engine.rs and _sre consume them from sre_engine::string again.

The unicode crate's tests move from a single root tests module in lib.rs
into per-module test submodules (case, classify, data, identifier,
normalize).

Assisted-by: Claude

* no-std

* Address review feedback: test module, dead variant, WTF-8 normalization

- differential.rs: move the sweep tests into a `mod tests` block and drop
  the file-level allow of clippy::tests_outside_test_module and
  std_instead_of_alloc; the test uses alloc collections instead.
- data.rs: remove the never-constructed DecompositionType::Canonical variant
  and its allow(unused); compatibility decomposition never produces it and
  canonical decomposition is handled through icu4x.
- normalize.rs: is_normalized now takes &Wtf8 and checks each UTF-8 run,
  skipping lone surrogates, matching normalize's run-wise behavior.
- unicodedata.rs: pass as_wtf8() to is_normalized.

Assisted-by: Claude
2026-07-07 09:19:25 +09:00
2025-07-04 23:47:05 +09:00
2020-07-06 18:25:10 +00:00
2026-07-07 09:19:25 +09:00
2026-07-07 09:19:25 +09:00
2026-07-07 09:19:25 +09:00
2025-07-01 04:40:58 +09:00
2020-09-13 06:58:57 +09:00
2021-03-14 12:50:00 -05:00
2022-08-23 01:50:04 -05:00
2026-07-07 09:19:25 +09:00
2026-07-07 09:19:25 +09:00
2019-03-07 20:00:02 +01:00
2026-05-28 12:43:43 +09:00
2020-03-13 08:04:33 -05:00
2020-03-13 08:04:33 -05:00
2026-05-16 19:01:40 +09:00
2026-04-13 12:49:32 +09:00
2019-03-22 18:09:05 -05:00
2021-10-02 21:13:14 +09:00
2026-05-28 09:37:44 +03:00
2023-03-10 02:05:52 +09:00
2025-02-26 11:48:22 -08:00
2026-03-16 11:32:27 +09:00

RustPython

A Python-3 (CPython >= 3.14.0) Interpreter written in Rust 🐍 😱 🤘.

Build Status codecov License: MIT Contributors Discord Shield docs.rs Crates.io dependency status Open in Gitpod

Usage

Check out our online demo running on WebAssembly.

RustPython requires Rust latest stable version (e.g 1.67.1 at February 7th 2023). If you don't currently have Rust installed on your system you can do so by following the instructions at rustup.rs.

To check the version of Rust you're currently running, use rustc --version. If you wish to update, rustup update stable will update your Rust installation to the most recent stable release.

To build RustPython locally, first, clone the source code:

git clone https://github.com/RustPython/RustPython

RustPython uses symlinks to manage python libraries in Lib/. If on windows, running the following helps:

git config core.symlinks true

Then you can change into the RustPython directory and run the demo (Note: --release is needed to prevent stack overflow on Windows):

$ cd RustPython
$ cargo run --release demo_closures.py
Hello, RustPython!

Or use the interactive shell:

$ cargo run --release
Welcome to rustpython
>>>>> 2+2
4

NOTE: For windows users, please set RUSTPYTHONPATH environment variable as Lib path in project directory. (e.g. When RustPython directory is C:\RustPython, set RUSTPYTHONPATH as C:\RustPython\Lib)

You can also install and run RustPython with the following:

$ cargo install --git https://github.com/RustPython/RustPython rustpython
$ rustpython
Welcome to the magnificent Rust Python interpreter
>>>>>

venv

Because RustPython currently doesn't provide a well-packaged installation, using venv helps to use pip easier.

$ rustpython -m venv <your_env_name>
$ . <your_env_name>/bin/activate
$ python # now `python` is the alias of the RustPython for the new env

PIP

If you'd like to make https requests, you can enable the ssl feature, which also lets you install the pip package manager. Note that on Windows, you may need to install OpenSSL, or you can enable the ssl-openssl-vendor feature instead, which compiles OpenSSL for you but requires a C compiler, perl, and make. OpenSSL version 3 is expected and tested in CI. Older versions may not work.

Once you've installed rustpython with SSL support, you can install pip by running:

cargo install --git https://github.com/RustPython/RustPython
rustpython --install-pip

You can also install RustPython through the conda package manager, though this isn't officially supported and may be out of date:

conda install rustpython -c conda-forge
rustpython

SSL provider

For HTTPS requests, ssl-rustls-aws-lc is enabled by default for the RustPython binary. Embedders can use rustpython-stdlib's provider-agnostic ssl-rustls feature and install their own rustls crypto provider, or replace rustls with ssl-openssl if their environment requires OpenSSL. Note that to use OpenSSL on Windows, you may need to install OpenSSL, or you can enable the ssl-openssl-vendor feature instead, which compiles OpenSSL for you but requires a C compiler, perl, and make. OpenSSL version 3 is expected and tested in CI. Older versions may not work.

WASI

You can compile RustPython to a standalone WebAssembly WASI module so it can run anywhere.

Build

cargo build --target wasm32-wasip1 --no-default-features --features freeze-stdlib,stdlib --release

Run by wasmer

wasmer run --dir `pwd` -- target/wasm32-wasip1/release/rustpython.wasm `pwd`/extra_tests/snippets/stdlib_random.py

Run by wapm

$ wapm install rustpython
$ wapm run rustpython
>>>>> 2+2
4

Building the WASI file

You can build the WebAssembly WASI file with:

cargo build --release --target wasm32-wasip1 --features="freeze-stdlib"

Note: we use the freeze-stdlib to include the standard library inside the binary. You also have to run once rustup target add wasm32-wasip1.

JIT (Just in time) compiler

RustPython has a very experimental JIT compiler that compile python functions into native code.

Building

By default the JIT compiler isn't enabled, it's enabled with the jit cargo feature.

cargo run --features jit

This requires autoconf, automake, libtool, and clang to be installed.

Using

To compile a function, call __jit__() on it.

def foo():
    a = 5
    return 10 + a

foo.__jit__()  # this will compile foo to native code and subsequent calls will execute that native code
assert foo() == 15

Embedding RustPython into your Rust Applications

Interested in exposing Python scripting in an application written in Rust, perhaps to allow quickly tweaking logic where Rust's compile times would be inhibitive? Then examples/hello_embed.rs and examples/mini_repl.rs may be of some assistance.

Disclaimer

RustPython is in development, and while the interpreter certainly can be used in interesting use cases like running Python in WASM and embedding into a Rust project, do note that RustPython is not totally production-ready.

Contribution is more than welcome! See our contribution section for more information on this.

Conference videos

Checkout those talks on conferences:

Use cases

Although RustPython is a fairly young project, a few people have used it to make cool projects:

  • GreptimeDB: an open-source, cloud-native, distributed time-series database. Using RustPython for embedded scripting.
  • pyckitup: a game engine written in rust.
  • Robot Rumble: an arena-based AI competition platform
  • Ruff: an extremely fast Python linter, written in Rust

Goals

  • Full Python-3 environment entirely in Rust (not CPython bindings)
  • A clean implementation without compatibility hacks

Documentation

Currently along with other areas of the project, documentation is still in an early phase.

You can read the online documentation for the latest release, or the user guide.

You can also generate documentation locally by running:

cargo doc # Including documentation for all dependencies
cargo doc --no-deps --all # Excluding all dependencies

Documentation HTML files can then be found in the target/doc directory or you can append --open to the previous commands to have the documentation open automatically on your default browser.

For a high level overview of the components, see the architecture document.

Contributing

Contributions are welcome and highly appreciated. To get started, check out the contributing guidelines.

You can also join us on Discord.

Compiling to WebAssembly

See this doc

Community

Discord Banner

Chat with us on Discord.

Code of conduct

Our code of conduct can be found here.

Credit

The initial work was based on windelbouwman/rspython and shinglyu/RustPython

These are some useful links to related projects:

License

This project is licensed under the MIT license. Please see the LICENSE file for more details.

The project logo is licensed under the CC-BY-4.0 license. Please see the LICENSE-logo file for more details.

Languages
Rust 87%
Python 12.5%
JavaScript 0.2%
NSIS 0.2%