mirrors/RustPython
0
0 Fork 0
mirror of https://github.com/RustPython/RustPython.git synced 2026-10-28 10:44:36 +08:00
Code Activity

ucd: Rewrite parser to be faster & more flexible (#8611)

Browse Source 
Related: #8599

My original parser was monolithic and inflexible. It worked as intended
for the derived data files, but anything more complicated required hacky
code. For example, the original parser always expected to build a vector
of values, yet sometimes we required other data structures such as
BTreeMaps.

I split up the parser into helper functions that are both cleaner and
more flexible. The actual parser should still yield the same results -
this doesn't "fix" anything yet. However, this is preliminary work for
fixing more of UCD since we will need a more flexible parser for fixes
like the linked PR.
This commit is contained in:
Joshua Megnauth authored and GitHub committed 2026-09-01 08:48:52 +09:00
1 parent 6aac4653e6
commit 7ec2682153
2 files changed
+582 -482
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

crates/unicode/build.rs
+573 -477
View File
File diff suppressed because it is too large. Load diff
crates/unicode/src/data.rs
+9 -5
View File
@@ -19,14 +19,18 @@ use icu_properties::props::{
};
use rustpython_wtf8::CodePoint;
include!(concat!(env!("OUT_DIR"), "/generated/membership_3_2.rs"));
include!(concat!(env!("OUT_DIR"), "/generated/unicode_3_2.rs"));
include!(concat!(env!("OUT_DIR"), "/generated/unicode_latest.rs"));
include!(concat!(env!("OUT_DIR"), "/generated/unicode_num_type.rs"));
include!(concat!(env!("OUT_DIR"), "/generated/bidi_class_3_2.rs"));
include!(concat!(env!("OUT_DIR"), "/generated/binary_props_3_2.rs"));
include!(concat!(
env!("OUT_DIR"),
"/generated/unicode_numeric_value.rs"
"/generated/combining_class_3_2.rs"
));
include!(concat!(env!("OUT_DIR"), "/generated/decomp.rs"));
include!(concat!(env!("OUT_DIR"), "/generated/eaw_3_2.rs"));
include!(concat!(env!("OUT_DIR"), "/generated/gen_cat_3_2.rs"));
include!(concat!(env!("OUT_DIR"), "/generated/membership_3_2.rs"));
include!(concat!(env!("OUT_DIR"), "/generated/numeric_value_3_2.rs"));
include!(concat!(env!("OUT_DIR"), "/generated/num_type_3_2.rs"));
#[derive(Clone, Copy)]
enum DecompositionType {