* Implement native Unicode 17 grapheme iteration
Add the CPython extended grapheme state machine, full-codepoint properties, and traced Segment/iterator types with positional index handling. Refresh canonical unicodedata tests through update_lib while retaining pending markers. Include a losslessly range-compressed exact-final property oracle and differential test.
Source-reviewed implementation; candidate runtime validation remains pending. Reference property generation completed against the pinned CPython Unicode extension.
Assisted-by: OpenAI Codex:model version unavailable
* Bind grapheme Segment methods to the Python object
Use the required Py<Segment> receiver for native descriptors; preserve payload ownership and representation behavior.
Assisted-by: OpenAI Codex:model version unavailable
* Use explicit object receivers for Segment descriptors
Keep the inherent impl on the local payload and receive &Py<Self>, as required for stdlib classes outside the VM crate.
Assisted-by: OpenAI Codex:model version unavailable
* Mark grapheme property lookups as must-use
Satisfy strict Clippy for the three pure property functions without changing their behavior.
Assisted-by: OpenAI Codex:model version unavailable
* Enable validated Unicode grapheme canonical tests
Remove seven stale expected-failure markers through update_lib metadata after native property parity, canonical methods, full Unicode 17 grapheme corpus, and reference-cycle GC validation.
Assisted-by: OpenAI Codex:model version unavailable
* Accept platform line endings in grapheme oracle header
Compare the complete provenance line without its line ending so the same exact Unicode 17 reference is accepted after Windows checkout conversion. Keep the complete code-point property sweep unchanged.
Assisted-by: OpenAI Codex:model version unavailable
* Keep grapheme property regression in test module
Move the existing complete Unicode 17 property regression into the existing cfg(test) module to satisfy strict Clippy. Preserve the fixture, assertions, code-point coverage, and platform-independent provenance line check unchanged.
Assisted-by: OpenAI Codex:model version unavailable
* Qualify grapheme iterator return type for generated bindings
Assisted-by: OpenAI Codex:model version unavailable
* Fix deprecated PyO3 code input import in benchmarks
Use the canonical code module path introduced in PyO3 0.30 without suppressing deprecation warnings.
Assisted-by: OpenAI Codex:model version unavailable
* ucd: Parse Unihan data for 3.2.0; unmask test
Python's 3.2.0 path uses the Unihan data for supplementary numeric
values. The Unihan data isn't bundled with the "full" data which means
we have to parse it ourselves. The full data is large, so I compressed
it with zlib.
AI disclosure: I asked an LLM to elaborate on why the full Unicode 3.2.0
data precludes \u5793. It explained that the code point is part of
Unihan. I wrote the parser myself. I linted the zlib snippet with AI.
Assisted-by: GPT-5.6: Luna
* Use spare_capacity_mut + set_len over transmute
This commit fixes most of UCD's names. The two main fixes are:
* Handling chars missing in 3.2.0 by checking for membership #8548
* Handling names not present in unicodenames
AI disclosure: I used AI for the initial research of this patch to
determine why the Tangut characters are missing. AI also assisted with a
bug fix where I neglected to parse the Tangut u32 as hex. I wrote all of
the code.
Assisted-by: GPT 5.6 Luna
We can save space by calculating the diffs between modern Unicode and
3.2.0 and storing membership information where valid. This avoids
storing entire tables for 3.2.0, and is also more correct in the long
run since it handles absence from 3.2.0 correctly. I switched over Bidi
to this new method which partially fixed the test. Unfortunately, our
Unicode data is more up to date than Python 3.14 so the test fails for
modern Unicode. I updated the test to 3.15 as per review, and now our
bidi passes the test.
* Update `test_unicodedata.py` to 3.14.4
* Add test asset
* Mark failing tests
* Add `expectedSuccess` docorator
* Add support for nee decorator in patch_spec
* Reapply patches
The uppercase/titlecase branch of PyStr::title() pushed characters
unchanged when starting a new word, which left Latin Extended-B
digraphs (U+01F1 'DZ', U+01C4 'DŽ', etc.) in their uppercase form
instead of mapping them to their distinct titlecase counterparts
(U+01F2 'Dz', U+01C5 'Dž'). For ASCII letters and characters where
to_titlecase is identity this had no effect, hiding the bug for the
common case.
Mirror the lowercase branch — which already calls to_titlecase()
when starting a new word — so both branches symmetrically apply
the titlecase mapping. char::to_titlecase is identity for already-
titlecase and ASCII-uppercase characters, so existing cases stay
correct.
Also unmasks test_unicodedata.UnicodeMiscTest.test_bug_4971, which
asserts exactly this behavior (`'DŽ'.title() == 'Dž'` etc.)
and was marked expectedFailure with reason `+ Dž`.
Closes#7527 (the only example from that issue still failing on
3.14.4; the other four examples already pass on current main).
youknowdot