* Accept Unicode decimal digits in int(), Decimal() and complex()
CPython runs a string argument through
_PyUnicode_TransformDecimalAndSpaceToASCII before parsing it, so decimal
digits from any script are accepted:
int('١٢٣') # 123
int('0x١f', 16) # 31
Decimal('١٢٣') # Decimal('123')
complex('１+２j') # (1+2j)
RustPython only did this for float(), which had the transform inlined.
int() handed the raw UTF-8 bytes to bytes_to_int(), whose digit check is
is_ascii_alphanumeric(), so every non-ASCII digit was rejected — even
though float() accepted the same string.
Lift the inlined transform out of float_from_string() into
common::str::transform_decimal_and_space_to_ascii() and apply it to the
str paths of int() and complex() too. The result is always ASCII: as in
CPython, a character that is neither ASCII, whitespace nor a decimal
digit becomes '?' and truncates the string, which no parser accepts at
any base, leaving the caller to raise the error from the original string.
Bytes-like input keeps going straight to the parser, matching CPython's
split between PyLong_FromUnicodeObject and PyLong_FromString.
This unmarks two expectedFailure tests: test_int.test_unicode and
test_decimal.test_unicode_digits.
* Share one PyStr-to-numeric-literal step across int, float and complex
All three constructors need the same thing from a str argument: trim it,
fold Unicode decimal digits and whitespace to ASCII, and give up on a
string holding surrogates. Each expressed that last part differently —
float matched PyKindStr and returned b"", complex leaned on to_str()
returning None, int returned an empty Cow — so the rule lived in three
places at once.
Move it into protocol::numeric_literal_from_str() and have all three call
it. CPython repeats this per type because its wrapper is three lines over
a single PyUnicode representation; ours has to match over Ascii/Utf8/Wtf8,
which is worth writing once.
Only the shared step moves. int keeps its base handling, int and float
keep accepting bytes-like input, complex keeps rejecting it, and each
keeps raising its own error, because none of that is shared.
No behavior change: the CPython differential suite is byte-identical
before and after.
* Drop the now-empty test_unicode_digits override in test_decimal
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
This directory contains all of the Python files that make up the standard
library for RustPython.
Most of these files are copied over from the CPython repository (the 3.7
branch), with slight modifications to allow them to work under RustPython. The
current goal is to complete the standard library with as few modifications as
possible. Current modifications are just temporary workarounds for bugs/missing
feature within the RustPython implementation.
The first big module we are targeting is unittest, so we can leverage the
CPython test suite.
