mirror of
https://github.com/RustPython/RustPython.git
synced 2026-10-28 18:59:04 +08:00
Jeong, YunWon
322f294a21
* common: add PyPy CJK codec engines
Assisted-by: Codex:GPT-5
* common: mark CJK mapping tables generated
Assisted-by: Codex:GPT-5
* common: add reusable byte and text engines
Assisted-by: Codex:GPT-5
* common: record opt-in engine dependencies
Assisted-by: Codex:GPT-5
* stdlib: add `_multibytecodec` and the six `_codecs_*` modules
Wire the CJK codec engines in `rustpython-common` up to the VM. `_multibytecodec`
holds `MultibyteCodec`, `MultibyteIncrementalEncoder`, `MultibyteIncrementalDecoder`,
`MultibyteStreamReader` and `MultibyteStreamWriter`, the encode/decode drivers and
the error-handler protocol; each `_codecs_*` module exposes `getcodec` for the
encodings of its region.
`rustpython-stdlib` now enables the `cjk-codecs` feature of `rustpython-common`.
Add `cjk::decode_reset`, which the decoder `reset()` methods need. It differs from
building a fresh decoder state: ISO 2022 keeps its G1..G3 designations and the
escape-throughout flag, and resets only G0 and the shifted flag.
Drop the markers from the CJK codec tests and from the `test_codecs`, `test_io`,
`test_email`, `test_zipfile` and `test_traceback` cases that needed a CJK codec.
`test_codecs.BasicUnicodeTest.{test_basics,test_decoder_state}` stay expected
failures for an unrelated reason: `codecs.charmap_encode` rejects a `None` mapping.
Assisted-by: Claude Code:claude-opus-5
* stdlib: use the byte and text engines from `rustpython-common`
`binascii`, `_json` and `_socket`'s address converters now call the engines in
`rustpython-common` instead of carrying their own copies, and `rustpython-stdlib`
enables the `binascii`, `inet` and `json` features for them. That leaves
`base64` and `crc32fast` with no caller in `rustpython-stdlib`, and
`crates/stdlib/src/json/machinery.rs`, the last of the json_in_type-derived code,
goes away with its last caller.
The address converters change behaviour. `inet_aton` reads the lenient forms
(`127.1`, `16909060`, `0x7f.1`, `0177.0.0.1`), `inet_pton` rejects an octet with a
redundant leading zero and a group of five hex digits, `inet_ntop` writes the
IPv4-compatible form as `::c000:201`, and an address family with no converter
raises `EAFNOSUPPORT` instead of a bare `OSError`.
Give the JSON engine the closing-quote scan and the allocation-free `\uXXXX`
escape it needs to carry `_json`'s workload. Against the code it replaces,
`json.dumps` runs 2.3x faster with `ensure_ascii=True` and 1.7x without it, and
`json.loads` 1.2x.
`binascii.a2b_base64` now reports the messages `binascii.c` does: no
`error decoding base64: ` prefix, and the data-character count in parentheses.
Assisted-by: Claude Code:claude-opus-5
* test: enable the Windows cp932 codec case
The shared CJK engine now satisfies the Windows code-page adapter, so keeping this test marked as an expected failure turns its success into a CI failure.
Assisted-by: Codex:GPT-5
143 lines
3.0 KiB
JSON
143 lines
3.0 KiB
JSON
// See: https://github.com/streetsidesoftware/cspell/tree/master/packages/cspell
|
|
{
|
|
"version": "0.2",
|
|
"import": [
|
|
"@cspell/dict-en_us/cspell-ext.json",
|
|
// "@cspell/dict-cpp/cspell-ext.json",
|
|
"@cspell/dict-python/cspell-ext.json",
|
|
"@cspell/dict-rust/cspell-ext.json",
|
|
"@cspell/dict-win32/cspell-ext.json",
|
|
"@cspell/dict-shell/cspell-ext.json",
|
|
],
|
|
"allowCompoundWords": true,
|
|
// language - current active spelling language
|
|
"language": "en",
|
|
// dictionaries - list of the names of the dictionaries to use
|
|
"dictionaries": [
|
|
"cpython", // Sometimes keeping same terms with cpython is easy
|
|
"python-more", // Python API terms not listed in python
|
|
"rust-more", // Rust API terms not listed in rust
|
|
"rustpython", // RustPython derive macros and internal terms
|
|
"en_US",
|
|
"softwareTerms",
|
|
"c",
|
|
"cpp",
|
|
"python",
|
|
"rust",
|
|
"shell",
|
|
"win32"
|
|
],
|
|
// dictionaryDefinitions - this list defines any custom dictionaries to use
|
|
"dictionaryDefinitions": [
|
|
{
|
|
"name": "cpython",
|
|
"path": "./.cspell.dict/cpython.txt"
|
|
},
|
|
{
|
|
"name": "python-more",
|
|
"path": "./.cspell.dict/python-more.txt"
|
|
},
|
|
{
|
|
"name": "rust-more",
|
|
"path": "./.cspell.dict/rust-more.txt"
|
|
},
|
|
{
|
|
"name": "rustpython",
|
|
"path": "./.cspell.dict/rustpython.txt"
|
|
}
|
|
],
|
|
"ignorePaths": [
|
|
"**/__pycache__/**",
|
|
"target/**",
|
|
"Lib/**",
|
|
"crates/host_env/**"
|
|
],
|
|
"overrides": [
|
|
{
|
|
"filename": "crates/common/src/encodings/cjk/**",
|
|
"words": [
|
|
"CHOSEONG",
|
|
"EUCKR",
|
|
"famille",
|
|
"GBCOMMON",
|
|
"jong",
|
|
"JONGSEONG",
|
|
"JUNGSEONG",
|
|
"PHINT",
|
|
"Xabc"
|
|
]
|
|
}
|
|
],
|
|
// words - list of words to be always considered correct
|
|
// (compound words like pyarg, baseclass, microbenchmark are handled by allowCompoundWords)
|
|
"words": [
|
|
"aiterable",
|
|
"alnum",
|
|
"csock",
|
|
"coro",
|
|
"contig",
|
|
"Crnl",
|
|
"dedentations",
|
|
"dedents",
|
|
"deduped",
|
|
"deoptimized",
|
|
"deoptimize",
|
|
"emscripten",
|
|
"excs",
|
|
"fdigits",
|
|
"feff",
|
|
"flufl",
|
|
"fnfe",
|
|
"fsdefault",
|
|
"ifexp",
|
|
"implicits",
|
|
"inity",
|
|
"interps",
|
|
"jitted",
|
|
"jitting",
|
|
"kwonly",
|
|
"lossily",
|
|
"mcache",
|
|
"oparg",
|
|
"opargs",
|
|
"pointee",
|
|
"pyc",
|
|
"qmark",
|
|
"reborrow",
|
|
"reborrows",
|
|
"reparenting",
|
|
"reraises",
|
|
"reraising",
|
|
"significand",
|
|
"summands",
|
|
"TESTFN",
|
|
"TZPATH",
|
|
"unraisable",
|
|
"wasi",
|
|
"weaked",
|
|
// unix
|
|
"posixshmem",
|
|
"shm",
|
|
"CLOEXEC",
|
|
"endgrent",
|
|
"gethrvtime",
|
|
"getrusage",
|
|
"sigaction",
|
|
"WRLCK",
|
|
// win32
|
|
"IFEXEC"
|
|
],
|
|
// flagWords - list of words to be always considered incorrect
|
|
"flagWords": [
|
|
],
|
|
"ignoreRegExpList": [
|
|
],
|
|
// languageSettings - allow for per programming language configuration settings.
|
|
"languageSettings": [
|
|
{
|
|
"languageId": "python",
|
|
"locale": "en"
|
|
}
|
|
]
|
|
}
|