Commit Graph
15 Commits
Author SHA1 Message Date
youknowdotandyouknowdot c70e9b8443 Sync selected Python 3.15 core library tests and repr type errors (#9321)
* Sync Python 3.15 core library tests and repr type errors

Synchronize ten selected library paths from CPython 3.15.0 at
d220409a13b77de7a6c30abe16ea90db01528edc using the canonical updater,
retaining existing RustPython adapters and failure markers.

Match generic __repr__ invalid-result errors to the qualified type-name
format required by the canonical ExceptionGroup regression, preserving
string results and exceptions raised by the implementation.

Implementation, source review, and validation were AI-assisted. The model
version identifier is unavailable; automated checks are not human
verification. Bounded validation covers 43 passes and one canonical skip,
not the full repository suites.

Assisted-by: OpenAI Codex:model version unavailable

* Preserve original escape decoder consumed length after callbacks

Keep the original input length as consumed after successful unicode-escape
and raw-unicode-escape decoding, including error-handler input replacement.
Only report the current cursor when a non-final decode stops on incomplete
input, matching CPython's codec wrappers and incomplete exits.

Fix the canonical Unicode escape mutation regression that failed CI on
Linux, macOS and Windows. Preserve all Lib tests and existing assertions.
Bounded Linux validation: 55 passes and one canonical skip, CLI build and
strict common/VM Clippy passed. Full repository suites were not run.

Implementation, source review, and validation were AI-assisted. The model
version identifier is unavailable; automated checks are not human
verification.

Assisted-by: OpenAI Codex:model version unavailable

---------

Co-authored-by: youknowdot <336763669+youknowdot@users.noreply.github.com>
2026-10-23 18:01:14 +09:00
Jeong, YunWon 66a3d9daa7 common: implement remaining builtin text codecs (#8687)
* common: implement remaining builtin text codecs

Move utf-16/32/7, unicode-escape, raw-unicode-escape and the
bytes escape transform into rustpython-common and call them from
_codecs instead of _pycodecs.

Assisted-by: Claude:grok-4.6

* host_env: share locale grouping, uuid and pipe state

Add localeconv_numeric so grouping keeps a CHAR_MAX terminator,
UuidCreateSequential for _uuid, and GetNamedPipeHandleStateW.

Assisted-by: Claude:grok-4.6

* Fix utf-7 clippy and the utf-16-be snippet length

Assisted-by: Claude:grok-4.6

* codecs: treat escape decode as final and accept str

unicode_escape_decode and raw_unicode_escape_decode default final to
true and accept a str as UTF-8. Fix clippy on wide_order, escape
encode, and encode closures. Remove expectedFailure markers that now
pass.

Assisted-by: Grok:grok-4.6
2026-09-12 15:51:28 +09:00
Jeong, YunWon f9b177719b Fix formatting, numeric, unicode, codec, complex and marshal mismatches found by differential testing (#8600)
* Fix %-formatting of bytes precision, width and %c errors

`format_bytes` subtracted the byte length from the field width without
saturating, so `b'%1s' % (b'abcde',)` aborted with a capacity overflow.
A precision written as a bare dot now truncates to nothing, and `%r`/`%a`
on bytes now honour the precision and width like the other conversions.

`%c` with a bytes, bytearray or str argument that is not one unit long
now names the argument by its length instead of by its type.

Assisted-by: Claude Code:claude-opus-5

* Fix format spec handling of grouping, bool, 'c' and complex

A grouping option next to an unrecognised presentation type now reports
"Cannot specify ',' with 'a'." instead of falling through to the type
check. `format(True, spec)` now validates the spec the way an int does.

The 'c' type now pads and aligns its character, and an integer outside
the C long range raises OverflowError rather than reporting a code point
out of range.

A complex formatted with the 'n'/'g' types or with a precision and no
type now goes through the general format without forcing a fraction.

Assisted-by: Claude Code:claude-opus-5

* Round floats at a negative ndigits from the exact decimal digits

Dividing by a power of ten rounded twice: once when the quotient was
stored and again at the tie, so e.g. `round(2.5e2, -2)` came out as 300.0.
The digits now come from the exact decimal expansion of the value.

Assisted-by: Claude Code:claude-opus-5

* Report the requested base in int literal errors and reject a bare sign

An invalid literal parsed with base 0 named the base that the prefix
turned out to mean rather than 0. A string of nothing but a sign was
taken for the old octal form and returned 0 instead of raising.

Assisted-by: Claude Code:claude-opus-5

* Split, strip and expand tabs over the Py_UNICODE_ISSPACE set

`str.split`, `rsplit`, `strip`, `lstrip` and `rstrip` with no separator
used Rust's whitespace predicate, which differs from the 29 code points
`Py_UNICODE_ISSPACE` names: U+001C-U+001F and U+0085 were skipped and
U+200B was treated as a separator.

`expandtabs` went through `str`, so it dropped the lone surrogates of a
WTF-8 string; it now works on `Wtf8` throughout.

Indexing a str out of range now says "string index out of range".

Assisted-by: Claude Code:claude-opus-5

* Read Unicode numeric values as exact rationals

`DerivedNumericValues.txt` field 1 carries a rounded decimal, so
`unicodedata.numeric('⅐')` and its neighbours came back short of the
value CPython reports. The generator now reads the rational in field 3,
and takes the 3.2 values from field 8 of `UnicodeData-3.2.0.txt`.

Assisted-by: Claude Code:claude-opus-5

* Return a mappingproxy from Pattern.groupindex

`pattern_groupindex` wraps the group index in a dict proxy, and hands
back a plain dict only when the pattern has no named group.

Assisted-by: Claude Code:claude-opus-5

* Fix the utf-16/32/7 codecs' error handling

The utf-32 decoder dropped a code point above U+10FFFF instead of calling
the error handler, and both the utf-16 and utf-32 handlers were told the
encoding was "utf-16"/"utf-32" whichever byte order the data was in.

The encoders now take a bytes replacement from the handler as it stands,
rejecting one that does not fill whole code units, rather than iterating
over it as text; a str replacement from the utf-16 encoder now splits an
astral code point into a surrogate pair.

The utf-7 decoder names its encoding "utf7", as its codec does.

Assisted-by: Claude Code:claude-opus-5

* Rework complex multiplication, division and power

Division now divides by whichever part of the divisor is larger, the way
`_Py_c_quot` does, and both it and multiplication recover the infinities
that the plain formulas turn into nan, following C11 Annex G. An operand
that is a real number is now multiplied and divided component-wise, which
keeps the sign of a zero and the other part of an infinity; a real
numerator has its own quotient so that its zero keeps a sign too.

An integer exponent no greater than 100 in magnitude is now reached by
repeated multiplication rather than through the polar form, so e.g.
`(1+2j) ** 2` is exact. A zero exponent gives 1, and an exponent with no
imaginary part no longer reaches for a logarithm that turns an infinite
magnitude into a nan. A result that came out infinite raises
OverflowError.

`complex() arg is a malformed string` is replaced by `could not convert
string to complex: ...` when the string carries a misplaced underscore,
`complex division by zero` by `division by zero`, and the message for
zero raised to a negative or complex power no longer names the base.

Assisted-by: Claude Code:claude-opus-5

* Intern the string constants that read as names

A code object's string constants were interned by length, so a short
string that is not a name was interned and a long name was not. They are
now picked out the way `all_name_chars` does. The one-character latin-1
strings are interned as well, so that every route to one of them lands on
the same object.

Assisted-by: Claude Code:claude-opus-5

* Honour the marshal version and separate its EOF errors

Versions below 2 now write a float or complex as its digits under
TYPE_FLOAT/TYPE_COMPLEX, versions from 4 on write a tuple of fewer than
256 items under TYPE_SMALL_TUPLE and a long ascii string under
TYPE_ASCII, and a string is marked interned only when it is.

`marshal.loads` now tells apart the three ways its data can run out:
"EOF read where object expected" where an object was to start, "EOF read
where not expected" where a single byte was, and "marshal data too short"
where a run of bytes was cut short.

Assisted-by: Claude Code:claude-opus-5
2026-08-29 22:43:11 +09:00
Jeong, YunWon 819b4b86fd Fix malformed Unicode error constructors (#8497)
* Fix malformed Unicode error constructors

Replace message-only Unicode decode and encode errors with fully initialized exceptions, preserve source objects and failure ranges, and remove the obsolete constructors.

Assisted-by: Codex:gpt-5

* Fix Windows Unicode CI regressions

Assisted-by: Codex:gpt-5
2026-08-17 00:33:56 +09:00
Jeong, YunWon 0dac28ff50 Implement UTF-32 encode/decode and fix UTF-16 empty encode
- Add UTF-32, UTF-32-LE, UTF-32-BE encode/decode in _pycodecs.py
- Register utf_32 codec functions in codecs.rs via delegate_pycodecs
- Fix PyUnicode_EncodeUTF16 returning "" instead of [] for empty input
- Remove resolved expectedFailure decorators in test_codecs.py
- Add failure reasons to remaining expectedFailure comments
2026-02-02 12:50:34 +09:00
Jiseok CHOI 9e7d291b63 Add callable validation to codecs.register_error (#6229)
Validate that the handler argument passed to codecs.register_error
is callable, raising TypeError with message 'handler must be callable'
if it is not. This matches CPython behavior.

This fix enables test_badregistercall in test_codeccallbacks.py to pass.
2025-10-31 00:08:04 +09:00
Noa c6cab4c43a Parse surrogates in string literals properly 2025-03-26 22:44:42 -05:00
Noa f323d14ed3 Refactor codecs 2025-03-26 02:24:01 -05:00
Jeong YunWon 7db813dbc6 mark failing tests again 2022-07-24 19:47:31 +09:00
CPython developers ce0ac2b11b Update encodings/test_codecs 2022-07-24 16:40:51 +09:00
CPython developers 559442c2f3 crlf -> lf test_codecs 2022-07-24 16:40:51 +09:00
Padraic Fanning 1e48510f5e Clean up skips in test_codeccallbacks 2021-10-10 19:11:24 -04:00
Jeong YunWon 7d859f54b9 unmark resolved test 2021-10-01 22:32:44 +09:00
snowapril 4014b91780 add decorators on failed & error tests
Signed-off-by: snowapril <sinjihng@gmail.com>
2021-10-01 14:10:58 +09:00
snowapril c2f11e593d add test_codeccallbacks.py from cpython 3.8
Signed-off-by: snowapril <sinjihng@gmail.com>
2021-10-01 14:10:58 +09:00