Commit Graph
22 Commits
Author SHA1 Message Date
Jeong, YunWon a1ba6a662c Align compiler and AST behavior with CPython (#8138)
* Align compiler and AST behavior with CPython

Update the compiler pipeline, AST module support, and related VM compile/eval paths toward CPython 3.14 behavior.

Use upstream Ruff parser crates directly and keep RustPython-specific syntax preflight handling in the compile path.

Refresh opcode metadata, snapshots, and targeted tests for the aligned bytecode and AST behavior.

* fix

* Fix PR 8138 CI failures

* Fix match mapping wildcard scan complexity

* fixes

* Use Ruff runtime AST fields

* Minimize residual patch diff

* refactor error handling

* Remove _ast conversion context wrappers

* Clean up _ast conversion glue and compiler warning escalation

Rename the Node conversion parameter `ctx` to `vm` throughout the _ast
module.

Remove the unused compile_program, compile_program_single,
compile_block_expression, and compile_expression forwarders; compile_top
stays as the entry point.

Stash an escalated compiler SyntaxWarning so a non-SyntaxWarning category
propagates unchanged instead of being rewritten to SyntaxError, matching
PyErr_ExceptionMatches(SyntaxWarning) in compiler_warn. Drop the now-unused
CompileWarningError::into_codegen_error.

Replace the PositionalArguments two-variant enum with a struct holding a
shared range and a PositionalArgumentsKind enum.

Assisted-by: Claude

* Represent compiler flags as a CompilerFlags bitflags

Replace the loose PY_CF_*/CO_* i32 constants in compile_mode with a
CompilerFlags bitflags covering the PyCF_* flags and __future__
CO_FUTURE_* bits. The Python-visible ast.PyCF_* attribute values are
derived from it via bits(); parser input modes stay plain consts.

builtins::compile and the compile_string flag handling test flags
through CompilerFlags::contains instead of raw bit math.

Assisted-by: Claude

* Fix PR 8138 test regressions

* Fix PR 8138 CI failures

* Avoid shell interpolation in Linux deps action

* Fix review findings in compiler/AST changes

- frame.rs: compute the MATCH_CLASS class name lazily so a successful
  match no longer takes the type name lock or allocates; the
  __match_args__-not-a-tuple branch reuses it instead of recomputing.
- _ast/constant.rs: add the missing leading space to the with_recursion
  label ("maximum recursion depth exceeded during compilation").
- sys.set_int_max_str_digits: restore the "maxdigits must be 0 or larger
  than {n}" error text.
- codegen: set CO_MAXBLOCKS to 20 in both ir.rs and compile.rs.
- _ast/exception.rs: extract the duplicated excepthandler node guard
  into ensure_excepthandler_node.
- codegen: correct the validate_keywords doc comment.
- Drop redundant "CPython" qualifiers from comments added in this branch.

Assisted-by: Claude
2026-06-26 18:22:08 +09:00
Shahar Naveh fbdf7c1052 Align more error messages with CPython 3.14.5 (#7988)
* Cannot assign to True/False/None

* cannot use assignment expressions with ...

* cannot assign to function call/expression
2026-05-28 11:19:03 +09:00
Shahar Naveh 9326430926 Align more error messages with CPython 3.14.5 (#7933)
* Align more error messages with CPython 3.14.5

* Align patches for `test_eof.py`

* Unmark passing test

* Fix more
2026-05-20 22:20:13 +09:00
Shahar Naveh e82753086d Align more error messages with CPython 3.14.5 (#7928) 2026-05-20 08:59:59 +09:00
Shahar Naveh 1bb42adc73 Update test_grammar.py to 3.14.5 (#7913)
* Update `test_grammar.py` to 3.14.5

* Update `test_syntax.py` to 3.14.5
2026-05-19 20:59:00 +09:00
Shahar Naveh 6addb91bf3 Update ruff to 0.15.4 (#7246)
* Update ruff to 0.15.4

* Unmark passing test
2026-02-27 22:59:41 +09:00
Jeong, YunWonandCPython Developers 3df178298a Fix traceback, syntax errors, and exception handling (#7015)
* Update codeop from v3.14.3

* Fix traceback, syntax errors, and exception handling

- Improve unclosed bracket detection with "'(' was never closed" message
- Fix IndentationError location to point to end of line
- Implement frame.clear() with proper checks for executing/suspended frames
- Fix exception context chaining for propagated exceptions
- Add traceback.__dir__() and prevent tb_next deletion
- Fix subscript operation source range restoration in compiler
- Change "duplicate parameter" to "duplicate argument" error message
- Refactor duplicate code in asyncgenerator.rs and frame.rs

---------

Co-authored-by: CPython Developers <>

Assisted-by: Codex:gpt-6
2026-02-07 13:41:45 +09:00
Shahar Naveh 6ccf3b5104 Update ruff to 0.14.10 (#6271)
* Update `ruff` to `0.14.10`

* Mark failing test
2025-12-31 21:13:29 +09:00
Shahar NavehandNoa fcf196935e Update ruff 0.14.1 (#6195)
* Update ruff to 0.14.1

* Fix test regression in `test_compile`

* Unmark passing test

* Update `test_syntax` from 3.13.9

---------

Co-authored-by: Noa <coolreader18@gmail.com>
2025-10-20 22:46:46 +09:00
a6b4ef7f5d Replace Python parser with ruff parser (#5494)
* stage1

* compiler pass build

* introduce rustpython-compiler-source

* stage2

* fixup

* pass compile

* Fix hello world compiler test

* Fix code generation for if-elif-else statement

* Fix code generation for lambda expression

* Fix code generation for integers

* Fix code generation for fstrings

* Fix code generation for if statement

* Fix code generation for if statement

* Fix code generation for if statement

* Fix code generation for fstring

* Fix code generation for class definition

* Replace feature flags

* Initialize frozen core modules

* Allow __future__ import after module doc comment

* Disable ast module

* Commit remaining fixes for compile errors in examples

* Fix some warnings

* Update ast stdlib module

* Update ast stdlib module

* Update ast stdlib module

* Update ast stdlib module

* Update ast stdlib module

* Split ast stdlib module into files

* Fix codegen for positional arguments with defaults

* Update ast stdlib module

* Update ast stdlib module

* Extract string and constant handling from expression.rs

* Always add required fields to AST nodes

* Compile doc strings correctly again

* Enable "ast" Cargo feature by default

* Refactor compilation of big integer literal

* Update ast stdlib module

* Update ast stdlib module

* Update ast stdlib module

* Reset barebones example

* Fix some left-over warnings

* Undo accidential change

* Adapt shell to ruff parser

* Pin parser to v0.4.10

* fix clippy

* Add TODO about interactive mode

* Fix compilation of complex number expression

* Remove moved code

* Update test case to ruff v0.4.10

* Apply suggestion

Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>

* Apply suggestion

Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>

* Fix compilation of fstring expression

* Fix compilation of fstring expression

* Fix wasm compile errors

* Attach correct source locations to ast objects

* Fix some more wasm compile errors

* Consider compile mode and enable AST stdlib module again

* Fix incorrect AST source location end column

* Fix compile error if "compiler" feature is not enabled

* Fix regrtests

* Fix some test_ast tests

* Add source range to type ignore

* Fix incompatibility with Rust 2024 edition

* Fix todos by implementing missing ast conversions and deleting unused code

* Appease clippy

* Fix remaining ast tests

* Fix remaining ast tests

* Mark/fix remaining tests

* Fix more

* Hacky windows fix

---------

Co-authored-by: Kangzhi Shi <shikangzhi@gmail.com>
Co-authored-by: Jeong YunWon <jeong@youknowone.org>
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
Co-authored-by: Noa <coolreader18@gmail.com>
2025-03-19 21:06:03 -05:00
Jeong YunWon dc569e9921 Refactoring 2023-05-10 19:35:27 +09:00
DimitrisJim 2da4b70e3a Mark failing tests. 2023-03-01 13:00:17 +02:00
DimitrisJim f58014e2b3 Add test_syntax from CPython 3.11.0 2023-03-01 12:58:51 +02:00
Anders Kaseorg 52ce437951 Prohibit starred arguments after double-starred arguments
CPython prohibits ‘f(**kwargs, *args)’; we should too.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-12-27 12:49:50 -08:00
dvermd f2bfb78d3c Fix ast types' _fields and use 0-based column 2022-10-27 03:09:38 +09:00
Myeongseon Choi c66d22da84 Except operator cases from delete error rule 2022-07-20 18:10:42 +09:00
Myeongseon Choi b490f5255b Update Lib/test/test_syntax.py to be runable 2022-07-13 16:47:31 +09:00
Myeongseon Choi 82efdeb391 Update Lib/test/test_syntax.py from CPython v3.10.5 2022-07-13 16:47:31 +09:00
snowapril 13bbefdd4e compiler: impl syntax error detect for global kind
This commit fixes `test_global_parap_err_first` test in
[test_syntax.py](https://github.com/RustPython/RustPython/blob/master/Lib/test/test_syntax.py#L678) and implement the other syntax errors detection as cpython 3.8 provided for global symbol.

Below syntax error conditions are added.
* name {} is parameter and global
```python
>>>>> def test(a):
.....   global a
.....
SyntaxError: name 'a' is parameter and global at line 2 column 2
        global a
```
* annotated name {} can't be global
```python
>>>>> def test():
.....   a: int
.....   global a
.....
SyntaxError: annotated name 'a' can't be global at line 3 column 2
        global a
```
* name {} is assigned to before global description
```python
>>>>> a = 100
>>>>> def test():
.....   a = 10
.....   global a
.....
SyntaxError: name 'a' is assigned to before global declaration at line 3 column 2
        global a
```
* name {} is used prior to global declaration
```python
>>>>> a = 10
>>>>> def test():
.....   print(a)
.....   global a
.....
SyntaxError: name 'a' is used prior to global declaration at line 3 column 2
        global a
```

Signed-off-by: snowapril <sinjihng@gmail.com>
2021-08-17 21:14:13 +09:00
Padraic Fanning 55d81fe40f Bypass doctests (causing CI to time out) 2021-05-16 13:48:27 -04:00
Padraic Fanning 2187e93032 Mark failing tests 2021-05-16 13:23:03 -04:00
Padraic Fanning 3298d88688 Add test_syntax from CPython 3.8 2021-05-16 13:23:02 -04:00