0 Fork 0

mirror of https://github.com/RustPython/RustPython.git synced 2026-10-28 18:59:04 +08:00

Files

T

Jeong, YunWon 55737ede63 Rewrite _tokenize with 2-phase model

Replace per-line reparsing with single-pass tokenization:
- Read all lines via readline, parse once, yield tokens
- Fix token type values (COMMENT=65, NL=66, OP=55)
- Fix NEWLINE/NL end positions and implicit newline handling
- Fix DEDENT positions via look-ahead to next non-DEDENT token
- Handle FSTRING_MIDDLE brace unescaping ({{ → {, }} → })
- Emit implicit NL before ENDMARKER when source lacks trailing newline
- Raise IndentationError from lexer errors
- Remove 13 expectedFailure marks for now-passing tests

2026-03-10 11:28:16 +09:00

__phello__

Update __hello__ and __phello__ from cpython

2023-02-25 22:59:57 +09:00

_pyrepl

Add _pyrepl (#5540 )

2025-04-11 09:30:58 +09:00

asyncio

Update asyncio from v3.14.2

2026-02-02 00:43:07 +09:00

collections

Restore defaultdict fallback and update test markers

2026-03-04 18:44:27 +09:00

compression

Replace _compression with compression and update related too (#6788 )

2026-01-19 00:38:51 +09:00

concurrent

Update the concurrent library + Added tests (#6673 )

2026-01-09 15:28:12 +09:00

ctypes

Update ctypes from v3.14.3

2026-02-18 03:09:55 +09:00

dbm

Update dbm from 3.13.11, implement sqlite3 autocommit (#6616 )

2026-01-01 14:44:53 +09:00

Update email library v3.13.11 (#6642 )

2026-01-17 18:15:35 +09:00

encodings

Update encodings from v3.14.3

2026-02-06 12:05:17 +09:00

ensurepip

Update ensurepip from 3.14.2

2026-01-17 19:22:00 +09:00

html

Update html module from Python 3.14.2 (#6855 )

2026-01-24 19:12:23 +09:00

http

Update http from 3.14.3 (#7137 )

2026-02-27 05:44:37 +00:00

importlib

Update importlib from v3.14.2

2026-02-05 00:21:34 +09:00

json

Bump json to 3.14.2 (#6774 )

2026-01-18 19:16:48 +09:00

logging

Update logging from 3.13.5 (#6003 )

2025-07-20 10:33:02 +09:00

multiprocessing

Update subprocess from v3.14.2

2026-02-04 11:16:15 +09:00

pathlib

Update pathlib, glob, fnmatch from 3.14.3

2026-02-13 22:15:19 +09:00

pydoc_data

Update pydoc.py from 3.14.3 (#7104 )

2026-02-13 18:07:25 +09:00

Bump re to 3.14.2

2026-01-20 15:29:05 +09:00

sqlite3

Update sqlite3 and the tests to CPython 3.14.2 (#6787 )

2026-01-19 02:45:47 +09:00

string

Upgrade string from CPython 3.14.2

2026-01-17 19:21:11 +09:00

sysconfig

Update sysconfig from v3.14.3

2026-03-02 01:24:15 +09:00

test

Rewrite _tokenize with 2-phase model

2026-03-10 11:28:16 +09:00

tkinter

_tkinter pt. 2 (#5640 )

2025-04-06 17:21:28 +09:00

tomllib

Update tomllib from 3.14.3 (#7048 )

2026-02-08 12:51:38 +00:00

unittest

Upgrade unittest from 3.14.2

2026-01-18 19:56:54 +09:00

urllib

Update urllib from 3.14.3

2026-02-13 13:44:28 +09:00

venv

upgrade venvlauncher

2026-01-17 19:21:10 +09:00

wsgiref

Updated the wsgiref + xmlrpc libraries + associated tests - v3.13.11 (#6634 )

2026-01-03 16:29:10 +09:00

xml

Update xml to 3.14.3

2026-02-27 09:44:41 +01:00

xmlrpc

Update xmlrpc

2026-02-14 18:18:53 +02:00

zipfile

Update zipfile from v3.14.3

2026-02-24 14:23:19 +09:00

zoneinfo

Update zoneinfo from v3.14.3

2026-03-01 20:51:10 +09:00

__future__.py

Update __future__ from CPython 3.12.2

2024-04-23 11:57:46 +09:00

__hello__.py

Update __hello__ and __phello__ from cpython

2023-02-25 22:59:57 +09:00

_aix_support.py

add _aix_support

2025-03-20 14:28:26 +09:00

_android_support.py

Update changed files from 3.13.7 -> 3.13.8

2025-10-22 10:57:47 +03:00

_apple_support.py

add os support modules

2025-03-10 11:43:53 +09:00

_ast_unparse.py

Update ast from v3.14.2

2026-02-02 01:20:23 +09:00

_collections_abc.py

Update _collections_abc from v3.14.2 (#6819 )

2026-01-21 01:01:09 +09:00

_colorize.py

Update _colorize from CPython 3.14.2

2026-01-17 19:22:00 +09:00

_compat_pickle.py

Update pickle.py from 3.14.2 (#6982 )

2026-02-05 19:27:10 +09:00

_dummy_os.py

Upgrade wasm deps + fix demo

2025-01-11 18:48:27 +09:00

_dummy_thread.py

Upgrade threading to 3.13.11; sys.setprofile & impl more threading (#6691 )

2026-01-12 15:33:28 +09:00

_ios_support.py

add os support modules

2025-03-10 11:43:53 +09:00

_markupbase.py

Update _markupbase.py from 3.14.3 (#7129 )

2026-02-15 00:37:14 +09:00

_opcode_metadata.py

align psuedo ops to 3.14.2

2026-01-24 16:09:40 +09:00

_osx_support.py

_osx_support update to 3.13

2025-03-10 23:13:32 -07:00

_py_abc.py

Upgrade abc.py _py_abc.py to 3.13.5 (#5865 )

2025-06-29 12:49:14 +09:00

_py_warnings.py

Update Lib/warnings to CPython 3.14 (#6762 )

2026-01-18 13:59:49 +09:00

_pycodecs.py

Impl more codecs and _codecs._unregister_error (#7025 )

2026-02-06 20:37:18 +09:00

_pydatetime.py

Update datetime to 3.14.3 (#7084 )

2026-02-24 01:07:30 +00:00

_pydecimal.py

Update decimal from 3.14.2

2026-01-30 16:38:36 +02:00

_pyio.py

Update io from v3.14.3

2026-02-13 14:13:30 +09:00

_pylong.py

Update _pylong.py from 3.14.3

2026-02-13 13:55:05 +09:00

_sitebuiltins.py

Update site from CPython v3.12.3 (#5313 )

2024-05-09 23:04:31 +09:00

_strptime.py

Update datetime to 3.14.3 (#7084 )

2026-02-24 01:07:30 +00:00

_threading_local.py

Allow heap getset creation (#5854 )

2025-06-29 03:03:19 +09:00

_weakrefset.py

Update weakref from v3.14.2

2026-01-30 19:21:10 +09:00

abc.py

Upgrade abc.py _py_abc.py to 3.13.5 (#5865 )

2025-06-29 12:49:14 +09:00

annotationlib.py

Update annotationlib from v3.14.3

2026-02-24 16:22:44 +09:00

antigravity.py

feat(antigravity): update to 3.11

2023-02-28 17:34:18 +00:00

argparse.py

Update some libs&tests to 3.14.3 (#7006 )

2026-02-07 19:09:42 +09:00

ast.py

Update ast from v3.14.2

2026-02-02 01:20:23 +09:00

base64.py

Bump base64 to 3.14.2 (#6776 )

2026-01-19 00:04:06 +09:00

bdb.py

Update bdb from v3.14.3

2026-02-28 12:09:59 +09:00

bisect.py

Update bisect.py from CPython v3.12.0

2023-12-24 03:54:18 +08:00

bz2.py

Replace _compression with compression and update related too (#6788 )

2026-01-19 00:38:51 +09:00

calendar.py

Update calendar from CPython 3.14.2

2026-01-17 19:22:00 +09:00

cmd.py

Upgrade cmd module from CPython v3.14.2 (#6853 )

2026-01-24 11:37:32 +09:00

code.py

Update code from v3.14.2-288-g06f9c8ca1c

2026-02-01 18:55:19 -05:00

codecs.py

Update codecs from v3.14.2

2026-01-31 10:11:58 +09:00

codeop.py

Fix traceback, syntax errors, and exception handling (#7015 )

2026-02-07 13:41:45 +09:00

colorsys.py

update colorsys to 3.13.3

2025-04-21 20:45:06 -07:00

compileall.py

Update compileall.py to 3.14.3 (#7225 )

2026-02-25 21:15:55 +09:00

configparser.py

Update some libs&tests to 3.14.3 (#7006 )

2026-02-07 19:09:42 +09:00

contextlib.py

Update contextlib from 3.13.5, (#6056 )

2025-08-01 22:39:00 +09:00

contextvars.py

thread_inherit_context, upgrade threading & contextvar (#6727 )

2026-01-18 08:53:41 +09:00

copy.py

Update copy module and test_copy from CPython v3.14.3 (#7062 )

2026-02-10 00:05:15 +09:00

copyreg.py

Update copyreg from v3.14.2

2026-01-24 00:24:12 +09:00

csv.py

Update csv.py from 3.14.3 (#7181 )

2026-02-18 00:43:49 +09:00

dataclasses.py

Update dataclasses.py from 3.14.3

2026-02-08 19:38:26 +09:00

datetime.py

Update datetime to 3.14.3 (#7084 )

2026-02-24 01:07:30 +00:00

decimal.py

Update decimal from 3.14.2

2026-01-30 16:38:36 +02:00

difflib.py

Update difflib.py from 3.14.3

2026-02-08 20:36:35 +09:00

dis.py

Update dis from v3.14.2

2026-01-26 10:40:22 +09:00

doctest.py

Update doctest from v3.14.2

2026-02-11 16:47:03 +09:00

dummy_threading.py

Upgrade threading to 3.13.11; sys.setprofile & impl more threading (#6691 )

2026-01-12 15:33:28 +09:00

enum.py

Update enum from v3.14.3

2026-02-15 00:38:01 +09:00

filecmp.py

Updated enum + file_cmp library + tests - v3.13.10 (#6606 )

2026-01-03 08:27:18 +09:00

fileinput.py

update fileinput to 3.13.3

2025-04-30 14:33:52 +09:00

fnmatch.py

Update pathlib, glob, fnmatch from 3.14.3

2026-02-13 22:15:19 +09:00

fractions.py

Update fractions from 3.14.2

2026-01-30 07:48:29 +09:00

ftplib.py

Update ftplib from v3.14.2-288-g06f9c8ca1c

2026-02-02 09:09:19 +09:00

functools.py

Update functools from v3.14.2

2026-01-20 17:32:52 +09:00

genericpath.py

Update genericpath.py from 3.13.5 (#6065 )

2025-08-04 20:10:02 +09:00

getopt.py

Update get{opt,pass}.py from 3.14.3

2026-02-08 16:35:56 +09:00

getpass.py

Update get{opt,pass}.py from 3.14.3

2026-02-08 16:35:56 +09:00

gettext.py

Update optparse,gettext,textwrap from 3.14.2 (#6896 )

2026-01-30 20:48:57 +09:00

glob.py

Update pathlib, glob, fnmatch from 3.14.3

2026-02-13 22:15:19 +09:00

graphlib.py

Update graphlib, heapq, ipaddress to 3.14.2 (#6758 )

2026-01-18 01:23:34 +09:00

gzip.py

Replace _compression with compression and update related too (#6788 )

2026-01-19 00:38:51 +09:00

hashlib.py

Update hashlib from v3.14.3

2026-02-14 21:40:04 +09:00

heapq.py

Update graphlib, heapq, ipaddress to 3.14.2 (#6758 )

2026-01-18 01:23:34 +09:00

hmac.py

Update hmac from v3.14.3

2026-02-15 21:04:32 +09:00

imaplib.py

Update imaplib from v3.14.2-288-g06f9c8ca1c

2026-02-01 19:40:43 -05:00

inspect.py

Update some libs&tests to 3.14.3 (#7006 )

2026-02-07 19:09:42 +09:00

io.py

Update io from v3.14.3

2026-02-13 14:13:30 +09:00

ipaddress.py

Update graphlib, heapq, ipaddress to 3.14.2 (#6758 )

2026-01-18 01:23:34 +09:00

keyword.py

Update keyword.py and test_keyword.py from CPython v3.12.0

2024-06-21 22:37:15 +09:00

linecache.py

Update linecache from v3.14.3

2026-02-09 12:56:53 +09:00

locale.py

Update locale from v3.14.3

2026-03-06 06:00:08 +09:00

lzma.py

Replace _compression with compression and update related too (#6788 )

2026-01-19 00:38:51 +09:00

mailbox.py

Update mailbox.py

2026-02-14 18:19:57 +02:00

mimetypes.py

Bump mimetypes to 3.14.2 (#6808 )

2026-01-21 20:14:30 +09:00

netrc.py

Update netrc.py from 3.13.6 and make pwd accesible on Android (#6083 )

2025-08-20 14:08:05 +09:00

ntpath.py

Upgrade nt_path from v3.14.2

2026-01-25 00:18:55 +09:00

nturl2path.py

Update nturl2path from v3.14.2

2026-01-25 00:18:55 +09:00

numbers.py

Update{runpy,numbers}.py from 3.13.7 (#6141 )

2025-09-11 22:43:11 +09:00

opcode.py

instruction CACHE (#7251 )

2026-03-01 14:51:54 +09:00

operator.py

Update operator from v3.14.2

2026-01-23 23:57:31 +09:00

optparse.py

Update optparse,gettext,textwrap from 3.14.2 (#6896 )

2026-01-30 20:48:57 +09:00

os.py

Update os from v3.14.2

2026-02-03 22:10:31 +09:00

pdb.py

apply tests

2026-02-11 16:50:23 +09:00

pickle.py

Update pickle.py from 3.14.2 (#6982 )

2026-02-05 19:27:10 +09:00

pickletools.py

Impl more codecs and _codecs._unregister_error (#7025 )

2026-02-06 20:37:18 +09:00

pkgutil.py

Update pkgutil.py from 3.14.3

2026-03-03 20:00:50 +09:00

platform.py

Updated the platform + plistlib libraries + associated tests v.3.13.10 (#6611 )

2026-01-04 10:02:24 +09:00

plistlib.py

Updated the platform + plistlib libraries + associated tests v.3.13.10 (#6611 )

2026-01-04 10:02:24 +09:00

posixpath.py

Update posixpath.py from 3.14.3

2026-02-13 20:24:57 +09:00

pprint.py

Update pprint.py from 3.14.3

2026-02-06 01:14:55 +09:00

PSF-LICENSE

…

pty.py

Update pty + tty + associated libraries - v3.13.10 (#6630 )

2026-01-05 14:38:04 +09:00

py_compile.py

Update py_compile from 3.14.2 (#6897 )

2026-01-30 19:24:33 +09:00

pyclbr.py

Added pyclbr from 3.13.5 (#6028 )

2025-07-25 10:50:17 +09:00

pydoc.py

Update pydoc.py from 3.14.3 (#7104 )

2026-02-13 18:07:25 +09:00

queue.py

Update queue from 3.14.2

2026-02-04 18:32:45 +09:00

quopri.py

Upgrade quopri from Python 3.14.0 (#6852 )

2026-01-24 10:57:22 +09:00

random.py

Update random from v3.14.3

2026-02-28 22:50:51 +09:00

README.md

…

reprlib.py

Update reprlib from v3.14.2

2026-01-21 15:10:40 +09:00

rlcompleter.py

Update rlcompleter from 3.13.5 (#5990 )

2025-07-17 13:44:18 +09:00

runpy.py

Update{runpy,numbers}.py from 3.13.7 (#6141 )

2025-09-11 22:43:11 +09:00

sched.py

update sched to 3.13.3

2025-04-21 21:11:42 -07:00

secrets.py

Update secrets from 3.13.5 (#5991 )

2025-07-17 00:04:12 +09:00

selectors.py

Update selector.py from 3.13.5 (#6002 )

2025-07-20 10:31:58 +09:00

shelve.py

Update shelve.py from 3.13.11 (#6483 )

2026-01-01 17:15:00 +09:00

shlex.py

Update shlex from v3.14.2-288-g06f9c8ca1c

2026-02-02 08:10:15 +09:00

shutil.py

Update shutil from v3.14.3

2026-02-15 09:02:24 +09:00

signal.py

Update signal from CPython 3.12.3

2024-04-26 01:39:02 +09:00

site.py

upgrade site to 3.14.2

2026-01-17 19:21:10 +09:00

smtplib.py

Update smtplib.py from 3.14.3

2026-03-04 17:47:40 +09:00

socket.py

Update socket from v3.14.2-288-g06f9c8ca1c

2026-02-01 20:23:50 -05:00

socketserver.py

Update the signal + socket libraries + associated tests - v3.13.11 (#6631 )

2026-01-03 16:27:25 +09:00

sre_compile.py

Replace Lib/sre_* from CPython

2024-03-22 14:54:02 +09:00

sre_constants.py

Updated the re library + test (#6648 )

2026-01-05 13:55:07 +09:00

sre_parse.py

Replace Lib/sre_* from CPython

2024-03-22 14:54:02 +09:00

ssl.py

Update ssl from v3.14.3

2026-03-01 11:36:46 +09:00

stat.py

Update stat.py from 3.14.3

2026-02-14 09:18:09 +09:00

statistics.py

Update statistics.py from 3.14.3

2026-02-11 18:01:41 +09:00

stringprep.py

…

struct.py

Update struct from v3.14.2 (#6824 )

2026-01-21 03:05:05 +09:00

subprocess.py

Update subprocess.py to 3.14.3 (#7244 )

2026-02-28 10:36:09 +09:00

symtable.py

Update symtable from 3.14.2

2026-01-30 19:13:59 +02:00

tabnanny.py

Update tabnanny from v3.14.2-288-g06f9c8ca1c

2026-02-02 11:51:53 +09:00

tarfile.py

Update tarfile from v3.14.3

2026-02-27 16:53:55 +09:00

tempfile.py

Update tempfile from v3.14.3

2026-03-01 20:50:21 +09:00

textwrap.py

Update optparse,gettext,textwrap from 3.14.2 (#6896 )

2026-01-30 20:48:57 +09:00

this.py

…

threading.py

thread_inherit_context, upgrade threading & contextvar (#6727 )

2026-01-18 08:53:41 +09:00

timeit.py

Remove RustPython-specific workaround in timeit module (#6923 )

2026-02-01 14:52:36 +09:00

token.py

Update token from v3.14.2

2026-01-22 08:02:43 +09:00

tokenize.py

Update tokenize from v3.14.3

2026-03-10 11:28:16 +09:00

trace.py

Update trace.py from 3.14.3 (#7327 )

2026-03-04 05:38:58 +09:00

traceback.py

Update traceback from v3.14.3

2026-02-06 07:45:17 +09:00

tty.py

Update pty + tty + associated libraries - v3.13.10 (#6630 )

2026-01-05 14:38:04 +09:00

types.py

Update types from v3.14.3

2026-02-18 17:19:49 +09:00

typing.py

Upgrade typing from 3.14.2 and more impl (#7057 )

2026-02-11 09:39:04 +09:00

uuid.py

Update uuid from v3.14.2-288-g06f9c8ca1c

2026-02-02 14:01:59 +09:00

warnings.py

Update Lib/warnings to CPython 3.14 (#6762 )

2026-01-18 13:59:49 +09:00

wave.py

Update wave.py from 3.14.3

2026-02-13 13:42:22 +09:00

weakref.py

Update weakref from v3.14.2

2026-01-30 19:21:10 +09:00

webbrowser.py

update webbrowser and test_webbrowser to 3.13.3

2025-04-21 16:54:00 +09:00

zipapp.py

Update zipapp from v3.14.3

2026-03-05 17:25:59 +09:00

zipimport.py

Upgrade zipimport to Python 3.14.2 (#6857 )

2026-01-24 16:11:36 +09:00

README.md

Standard Library for RustPython

This directory contains all of the Python files that make up the standard library for RustPython.

Most of these files are copied over from the CPython repository (the 3.7 branch), with slight modifications to allow them to work under RustPython. The current goal is to complete the standard library with as few modifications as possible. Current modifications are just temporary workarounds for bugs/missing feature within the RustPython implementation.

The first big module we are targeting is unittest, so we can leverage the CPython test suite.