Validate decimal integer literals from the parser token stream before
lowering the parsed AST. Reuse the validator for normal compilation and
the `_ast` parsing path so `PyCF_ONLY_AST`, `ast.parse()`, and
`ast.literal_eval()` honor `int_max_str_digits` consistently.
Remove the now-passing `test_literal_eval_str_int_limit` expected-failure
marker. The token-based validator excludes non-decimal integers, floats,
complex literals, strings, and comments without duplicating lexer logic.
Assisted-by: Codex:gpt-5.6-sol
Validate parameter annotations while converting ast.arguments, before
merging positional and keyword-only defaults. This gives invalid annotation
contexts precedence over default-list shape errors, as in CPython.
Merge positional defaults before keyword-only defaults so an excess
positional-default error also takes precedence over a keyword-only default
length mismatch.
Remove the expected-failure markers from the FunctionDef and Lambda AST
validator tests that now pass.
Assisted-by: Codex:gpt-5.6-sol
* Align compiler and AST behavior with CPython
Update the compiler pipeline, AST module support, and related VM compile/eval paths toward CPython 3.14 behavior.
Use upstream Ruff parser crates directly and keep RustPython-specific syntax preflight handling in the compile path.
Refresh opcode metadata, snapshots, and targeted tests for the aligned bytecode and AST behavior.
* fix
* Fix PR 8138 CI failures
* Fix match mapping wildcard scan complexity
* fixes
* Use Ruff runtime AST fields
* Minimize residual patch diff
* refactor error handling
* Remove _ast conversion context wrappers
* Clean up _ast conversion glue and compiler warning escalation
Rename the Node conversion parameter `ctx` to `vm` throughout the _ast
module.
Remove the unused compile_program, compile_program_single,
compile_block_expression, and compile_expression forwarders; compile_top
stays as the entry point.
Stash an escalated compiler SyntaxWarning so a non-SyntaxWarning category
propagates unchanged instead of being rewritten to SyntaxError, matching
PyErr_ExceptionMatches(SyntaxWarning) in compiler_warn. Drop the now-unused
CompileWarningError::into_codegen_error.
Replace the PositionalArguments two-variant enum with a struct holding a
shared range and a PositionalArgumentsKind enum.
Assisted-by: Claude
* Represent compiler flags as a CompilerFlags bitflags
Replace the loose PY_CF_*/CO_* i32 constants in compile_mode with a
CompilerFlags bitflags covering the PyCF_* flags and __future__
CO_FUTURE_* bits. The Python-visible ast.PyCF_* attribute values are
derived from it via bits(); parser input modes stay plain consts.
builtins::compile and the compile_string flag handling test flags
through CompilerFlags::contains instead of raw bit math.
Assisted-by: Claude
* Fix PR 8138 test regressions
* Fix PR 8138 CI failures
* Avoid shell interpolation in Linux deps action
* Fix review findings in compiler/AST changes
- frame.rs: compute the MATCH_CLASS class name lazily so a successful
match no longer takes the type name lock or allocates; the
__match_args__-not-a-tuple branch reuses it instead of recomputing.
- _ast/constant.rs: add the missing leading space to the with_recursion
label ("maximum recursion depth exceeded during compilation").
- sys.set_int_max_str_digits: restore the "maxdigits must be 0 or larger
than {n}" error text.
- codegen: set CO_MAXBLOCKS to 20 in both ir.rs and compile.rs.
- _ast/exception.rs: extract the duplicated excepthandler node guard
into ensure_excepthandler_node.
- codegen: correct the validate_keywords doc comment.
- Drop redundant "CPython" qualifiers from comments added in this branch.
Assisted-by: Claude
* Enforce int_max_str_digits on int-to-str conversions
The str-to-int direction already enforced sys.get_int_max_str_digits()
via bytes_to_int; the int-to-str direction did not. CPython 3.14 enforces
both per PEP 644.
Adds check_int_to_str_digits helper in builtins::int (bit-count fast path
+ digit upper-bound from log10(2)), wired into the four Python-level
entry points: repr, the str fast path in protocol::object, int.__format__
(decimal/n/empty spec only — binary bases x/o/b are exempt per CPython),
and the DecimalD/I/U branches of vm::cformat for both str % and bytes %.
Unmasks 8 expectedFailure tests across test_int (max_str_digits, DoS
prevention, int_from_other_bases — each mirrored in IntSubclass),
test_ast (test_repr_large_input_crash) and test_reprlib (test_numbers).
Boundary cases (4299/4300/4301 digits at limit=4300) match CPython 3.14.4.
* Skip int-to-str DoS test on platforms without time.get_clock_info
The test_denial_of_service_prevented_int_to_str regression test uses
support.Stopwatch, which calls time.get_clock_info('monotonic'). In
RustPython that function is gated to unix/windows targets only, so on
wasm32-wasip1 it surfaces as AttributeError and breaks the wasm-wasi CI.
Guard the test with skipUnless(hasattr(time, 'get_clock_info'), ...) so
it runs everywhere it can and is skipped on wasm.
Also narrow is_decimal_int_format to Number(Case::Lower): 'N' is rejected
by format_int as UnknownFormatCode, so excluding it preserves that error
path instead of intercepting it with the digit-limit check.
* Add TODO: RUSTPYTHON marker to skipUnless reason
scripts/update_lib uses TODO: RUSTPYTHON markers inside unittest
decorator reason strings to identify and migrate custom RustPython
patches across CPython library updates.
* Use expectedFailureIf for wasm get_clock_info gap
skipUnless silently hides the test forever; expectedFailureIf surfaces
unexpected success once RustPython implements time.get_clock_info on
wasm, prompting marker removal.
Match CPython's "field 'X' is required for Y" error format when
required scalar AST fields receive None during ast_from_object
conversion. Previously these produced TypeError or a generic
"None disallowed in expression list" ValueError.
Fields covered (Alias.name, Arg.arg, Comprehension.target/iter,
Keyword.value, MatchCase.pattern, WithItem.context_expr,
YieldFrom.value): all now raise the CPython-compatible message
verbatim.
Add a get_node_field_required helper alongside the existing
get_node_field / get_node_field_opt and switch the eight call
sites that read these required scalar fields. The helper rejects
None with the proper ValueError before the value flows into the
type converter where Rust's strong typing would otherwise force a
generic catch-all error. Optional fields (read via
get_node_field_opt) and fields where None is valid (e.g.
Constant.value) are unaffected.
This complements the existing AST validator pass at
crates/vm/src/stdlib/_ast/validate.rs (hooked at _ast.rs:763):
the validator handles post-conversion semantic invariants
(ExprContext, empty body, cross-field rules) but cannot handle
required-scalar-field None because Rust's non-Option types reject
None at conversion time, before validation runs.
Verified byte-identical with CPython 3.14.4 for all seven probe
cases. test.test_ast.test_ast: 227 tests, expected failures
38 -> 36 (test_empty_yield_from and test_none_checks unmasked,
no regressions). test.test_compile: 0 regressions.
Sibling fix to #7656 (which handled Vec<Expr>). The same catch-all
TypeError ("expected some sort of stmt, but got {}") in
Stmt::ast_from_object silently swallowed None, so compile() on an
AST with None in a statement-list field (ClassDef.body, Try.body,
For.body, etc.) raised TypeError where CPython raises
ValueError("None disallowed in statement list").
Add an is_none check before the catch-all, matching the Expr-side
shape introduced in #7656. Option<Stmt> positions are unaffected —
they short-circuit None earlier in node.rs.
Unmasks test_classdef (body=[None] sub-case). Full
test.test_ast.test_ast run: 227 tests, 0 unexpected successes, 38
expected failures (was 39).
* Raise ValueError for None in Expr lists to match CPython validator
CPython raises ValueError("None disallowed in expression list") when
compile() encounters None as an element in a Vec<Expr> position
(Set/List/Tuple.elts, BoolOp.values, Call.args, comprehension generators,
etc.). RustPython reached the catch-all TypeError("expected some sort of
expr, but got {}") branch instead, failing a dozen test_ast validator
tests.
Add an explicit is_none() check in Expr::ast_from_object before the
catch-all so None short-circuits with the CPython-compatible ValueError.
Option<Expr> positions (e.g. Dict.keys for **unpack) are unaffected
since node.rs Option<T>::ast_from_object handles None earlier.
Unmasks 12 skip / expectedFailure markers in
Lib/test/test_ast/test_ast.py:
- test_boolop, test_set, test_list, test_tuple
- test_call, test_delete, test_assign
- test_dict, test_dictcomp, test_generatorexp, test_listcomp, test_setcomp
Addresses part of the skip-test inventory in #7611.
The Stmt-level analog (body=[None] -> "None disallowed in statement list",
used by test_classdef) needs a sibling change in statement.rs; out of
scope here.
* Drop redundant .to_owned() on new_value_error argument
Matches the other new_value_error call sites in the file.
Co-authored-by: Shahar Naveh <50263213+ShaharNaveh@users.noreply.github.com>
---------
Co-authored-by: Shahar Naveh <50263213+ShaharNaveh@users.noreply.github.com>
- Add FieldType enum and FIELD_TYPES static table mapping all
AST node classes to their ASDL field types
- Resolve markers to real Python type objects (GenericAlias,
Union, plain types) at module init in populate_field_types()
- Set class-level None defaults for optional fields
- Replace hardcoded LIST_FIELDS and class-name checks in
slot_init with _field_types-based lookup
- Add expr_context default to Load(), fix ImportFrom.level
default (now None instead of 0)
- Fix __class_getitem__ None check, compile() formatting
- Remove 14 @expectedFailure decorators from test_ast
* Downgraded skips in tests
* Fixed failing tests
* Fixed test_ftplib + test_socket + test_ssl + test_threaded_import failures
* Removed comments about which tests are run in which environment
* Addressed PR comments
* Annotated skips on failing tests
* Removed unneeded tests
* Removed unneeded sys import from test_ftplib
* Added annotation to test_ftplib
* Readded skipIf to test_cleanup_with_symlink_modes with a more general ENV_POLLUTING_TESTS_WINDOWS
* Addressed PR comments
* Made changes to minimize diff in PR
* Apply suggestion from @youknowone
---------
Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com>
* Use `ast.unparse` for decorator generation and every ut_method
* Ensure ut_method type for external patches
* use textwrap
* Apply patches to `test_os.py`
* Apoly on `test_xml_etree.py`
* Run on some test files
* Update `test_str.py`
* Update `test_logging.py` from 3.13.7
이광효