mirrors/RustPython
0
0 Fork 0
mirror of https://github.com/RustPython/RustPython.git synced 2026-10-28 18:59:04 +08:00
Code Activity
Files
main
RustPython/extra_tests/snippets/syntax_try.py
T
Jeong, YunWon f87cfe7e7a Match CPython on code objects, ast, symtable and syntax error positions (#8580) 
* Guard all/any/tuple(genexpr) calls when the generator is a coroutine

`maybe_optimize_function_call()` reserves a `skip_optimization` label for
every `name(genexpr)` call and, for `all`, `any` and `tuple`, emits an
identity guard against the builtin plus an inlined loop. The port skipped
both whenever the generator expression's symbol table was a coroutine, so
`all(await x for x in xs)` and `all(x async for x in xs)` compiled to a
plain call.

An `await` or an `async for` in the generator does not disqualify the
shape: the inlined `FOR_ITER` raises the same `TypeError` that calling the
builtin on an async generator raises, which is what
test_builtin test_builtin_call_async_genexpr_no_crash asserts.

This was the last code-level difference against CPython across `Lib/`:
comparing every file's code tree by structure, opcode and constant value
(qualnames aside, which moved in 3.14.3+, and set constants compared by
value rather than by hash order) now matches on 1720 of 1720 comparable
files, up from 1719.

Assisted-by: Claude Code:claude-opus-5

* Name the ellipsis type `ellipsis`

`type(...).__name__`, its `__qualname__`, and `types.EllipsisType.__name__`
read `EllipsisType`.

Assisted-by: Claude Code:claude-opus-5

* Align compiler front end and instruction positions with CPython

Reject a bare generator expression in a class header and a format spec
nested more than two deep. Both parse here; the reference grammar has no
rule for either, since a class header only takes `arguments` and the
tokenizer runs out of nesting levels at the third format spec.

Scan the interpolations a format spec carries inside another format spec,
so `f"{x:{y:{z}}}"` reaches `z` when building the symbol table.

Give every generic function's type params scope a `.defaults` slot,
whether or not the signature has any default, so `co_varnames` and
`co_nlocals` match. Classes and type aliases keep theirs empty.

Decide an unparenthesized sole generator argument by the parser's
`parenthesized` flag instead of scanning the source for the shape of its
element. `_sum((d := x - c) * d for x in data)` opens with a parenthesized
group the rest of the expression continues, which the scan read as the
generator's own parentheses and left the call's out of its range.

Drop empty f-string fragments unconditionally. A concatenation whose
fragments are all empty now loads its empty string at the whole
concatenation instead of at whichever fragments happened to be plain
string literals.

Keep a byte order mark in source handed over as text, and report it as an
invalid non-printable character. Reading a file to freeze strips it, as
the encoded-bytes path already did.

Carry the filename into the errors `compile(..., PyCF_ONLY_AST)` raises;
they reported `<unknown>` or an empty name.

Across `Lib/`, the 1729 files that compile on both sides now agree on
opcodes, constants, flags, variable names, names, and exception tables,
and on instruction positions in all but 7 files, where a compound
statement whose body ends in a trailing `;` ends one column short.

Assisted-by: Claude Code:claude-opus-5

* Align ast module output with CPython

- Keep lone surrogates in string, f-string and t-string literal values by
  reading such a literal from its source; the literal-value helpers move
  out of `Compiler` into `rustpython_codegen` and `ConstantLiteral::Str`
  holds `Wtf8Buf`.
- Give `{expr=}` interpolations their leading text as a Constant and
  default their conversion to `repr`, in both f-strings and t-strings.
- Span folded consecutive literals from the first of them to the last.
- Take the `:` into a format spec's range only when the parser left it
  out, instead of always widening by one.
- Report a node end that lands on a line start as that line, column 0.
- Give a generator expression written straight into a call's parentheses
  the range of those parentheses, in the preprocess pass, replacing
  codegen's source scan and `_ast`'s one-column adjustment.
- Drop the expectedFailure on test_ast_line_numbers_with_parentheses.

Assisted-by: Claude Code:claude-opus-5

* Align symbol table output with CPython

- Register an import as `DEF_IMPORT` alone, and an annotated parameter as
  a plain parameter, so `import x; global x` is accepted and annotated
  parameters are not reported as annotated names.
- Record a `global` declaration in the module block as well, and resolve a
  name a `global` in an enclosing scope covers as an implicit global, so a
  class body reading such a name emits `LOAD_NAME`.
- Resolve a name bound nowhere as an implicit global instead of leaving
  its scope unknown.
- Require a binding only for a name an explicit `nonlocal` declared;
  a free variable that reached the scope any other way already resolved.
- Name a generator expression's block `genexpr`; the code object keeps
  `<genexpr>`.
- Locate a `def`/`class` block at its keyword rather than at the first
  decorator. `decorated_definition_range()` moves from `Compiler` to
  `rustpython_codegen` for that.
- Carry a comprehension symbol's flags over when inlining it into a parent
  entry that only holds a free variable propagated from a child.
- Leave `__conditional_annotations__` out of an annotation block's
  symbols and have the compiler cook up the free variable, which places
  it after the ones the symbol table supplies.
- Check `from __future__` placement while building the symbol table, and
  give `symtable.symtable()` the same docstring stripping and future
  feature validation the compiler does.
- Floor `co_stacksize` at 1.

Assisted-by: Claude Code:claude-opus-5

* Locate the eval return and the interactive print like CPython

Leave `eval` mode's `RETURN_VALUE` without a location, so the exit block
is duplicated per reaching path and each copy carries the location of the
branch that jumps to it. `a and b` now ends with two returns as it does
in CPython, and `a if b else c` gives the first one the `a` it returns.

Emit an interactive `CALL_INTRINSIC_1(Print)` at the expression statement
rather than wherever the expression left the location, which differ for
`(a := 10)` and `f'{0:fz}'`.

Assisted-by: Claude Code:claude-opus-5

* Give interactive mode the conditional annotations cell

`compile_program_single()` emitted the `__conditional_annotations__` set
and its name accesses but never added the cell that holds it, so a module
compiled in `single` mode was missing the `MAKE_CELL` and the cellvar that
the same source gets in `exec` mode.

Assisted-by: Claude Code:claude-opus-5

* Give compiler-raised SyntaxErrors an end position

`CodegenError` and `SymbolTableError` now carry an `end_location`
alongside `location`, and `CompileError::python_end_location()` returns
it, so errors such as "'await' outside function" set `end_lineno` and
`end_offset` instead of leaving them None.

`SymbolTableBuilder::error_ranged()` builds both positions from a range,
replacing the inline `source_location()` calls at the error sites.

Drop the `expectedFailure` on
`test_exceptions.SyntaxErrorTests.test_file_source`, which now passes.

Assisted-by: Claude Code:claude-opus-5

* Report parser diagnostics with CPython's wording

`InvalidAnnotatedAssignmentTarget` becomes "illegal target for
annotation", `UnexpectedExpressionToken` becomes plain "invalid syntax"
rather than prefixing the parser's text, and the list-recovery
diagnostics ("Expected an expression or a '}'" and its siblings) join the
two that already mapped to "invalid syntax".

Assisted-by: Claude Code:claude-opus-5

* Punctuate the code object repr like CPython

`repr(code)` was missing the comma after the address and escaped the
filename through Rust's Debug formatting.

Drop the `expectedFailure` on the six `test_dis` tests that compare
disassembly text, which now pass.

Assisted-by: Claude Code:claude-opus-5

* Match CPython's parser error positions

A syntax error the parser reports counts its columns in characters, so
`source_location()` converts with UTF-32 rather than UTF-8; an offset
inside a character walks back to where that character starts.

Spans the parser reported differently:

- an error between two tokens covered nothing, and now covers one
  character, since the narrowest token still covers one
- a "was never closed" bracket ends at column 0
- an unterminated string literal ends where it starts
- "leading zeros in decimal integer literals" spans the run of zeros
- an exponent with no digits and no sign is blamed on the digit before
  the exponent letter, which the tokenizer puts back

Assisted-by: Claude Code:claude-opus-5

* codegen: seek to the finally body's own scopes before copying it

An early exit from a try block emits an extra copy of the finally body.
Nested scopes are handed out by position, and the cursors still sit inside
the try block's own run of scopes there, so the copy took those instead of
the ones the finally body opens: a generator expression got a sibling's
symbol table and raised "the symbol 'k' must be present in the symbol
table", or, where the sibling was a named scope, built a code object with
no `.0` argument that raised UnboundLocalError when it ran.

Seek the cursors to the first scope beginning on or after the finally
body's first line before compiling the copy. The existing restore then
leaves the try statement's own copies starting from the same place.

Assisted-by: Claude
2026-08-23 14:32:56 +09:00

471 lines
8.8 KiB
Python
Raw Permalink Blame History

from testutils import assert_raises
try:
raise BaseException()
except BaseException as ex:
print(ex)
print(type(ex))
# print(ex.__traceback__)
# print(type(ex.__traceback__))
try:
raise ZeroDivisionError
except ZeroDivisionError as ex:
pass
class E(Exception):
def __init__(self):
asdf # noqa: F821
try:
raise E
except NameError as ex:
pass
l = []
try:
l.append(1)
assert 0
l.append(2)
except:
l.append(3)
print('boom')
finally:
l.append(4)
print('kablam')
assert l == [1, 3, 4]
l = []
try:
l.append(1)
assert 0
l.append(2)
except AssertionError as ex:
l.append(3)
print('boom', type(ex))
finally:
l.append(4)
print('kablam')
assert l == [1, 3, 4]
l = []
try:
l.append(1)
assert 1
l.append(2)
except AssertionError as ex:
l.append(3)
print('boom', type(ex))
finally:
l.append(4)
print('kablam')
assert l == [1, 2, 4]
l = []
try:
try:
l.append(1)
assert 0
l.append(2)
finally:
l.append(3)
print('kablam')
except AssertionError as ex:
l.append(4)
print('boom', type(ex))
assert l == [1, 3, 4]
l = []
try:
l.append(1)
fubar
l.append(2)
except NameError as ex:
l.append(3)
print('boom', type(ex))
assert l == [1, 3]
l = []
try:
l.append(1)
raise 1
except TypeError as ex:
l.append(3)
print('boom', type(ex))
assert l == [1, 3]
cause = None
try:
try:
raise ZeroDivisionError
except ZeroDivisionError as ex:
assert ex.__cause__ == None
cause = ex
raise NameError from ex
except NameError as ex2:
assert ex2.__cause__ == cause
assert ex2.__context__ == cause
try:
raise ZeroDivisionError from None
except ZeroDivisionError as ex:
assert ex.__cause__ == None
try:
raise ZeroDivisionError
except ZeroDivisionError as ex:
assert ex.__cause__ == None
with assert_raises(TypeError):
raise ZeroDivisionError from 5
try:
raise ZeroDivisionError from NameError
except ZeroDivisionError as ex:
assert type(ex.__cause__) == NameError
with assert_raises(NameError):
try:
raise NameError
except:
raise
with assert_raises(RuntimeError):
raise
context = None
try:
try:
raise ZeroDivisionError
except ZeroDivisionError as ex:
assert ex.__context__ == None
context = ex
raise NameError
except NameError as ex2:
assert ex2.__context__ == context
assert type(ex2.__context__) == ZeroDivisionError
try:
raise ZeroDivisionError
except ZeroDivisionError as ex:
assert ex.__context__ == None
try:
raise ZeroDivisionError from NameError
except ZeroDivisionError as ex:
assert type(ex.__cause__) == NameError
assert ex.__context__ == None
try:
try:
raise ZeroDivisionError
except ZeroDivisionError as ex:
pass
finally:
raise NameError
except NameError as ex2:
assert ex2.__context__ == None
def f():
raise
with assert_raises(ZeroDivisionError):
try:
1/0
except:
f()
with assert_raises(ZeroDivisionError):
try:
1/0
except ZeroDivisionError:
try:
raise
except NameError:
pass
raise
# try-return-finally behavior:
l = []
def foo():
try:
return 33
finally:
l.append(1337)
r = foo()
assert r == 33
assert l == [1337]
# Regression https://github.com/RustPython/RustPython/issues/867
for _ in [1, 2]:
try:
raise ArithmeticError()
except ArithmeticError as e:
continue
def g():
try:
1/0
except ArithmeticError:
return 5
try:
g()
raise NameError
except NameError as ex:
assert ex.__context__ == None
def y():
try:
1/0
except ArithmeticError:
yield 5
try:
y()
raise NameError
except NameError as ex:
assert ex.__context__ == None
try:
{}[1]
except KeyError:
try:
raise RuntimeError()
except RuntimeError:
pass
try:
try:
raise ZeroDivisionError
except ZeroDivisionError as ex:
raise NameError from ex
except NameError as ex2:
assert isinstance(ex2.__cause__, ZeroDivisionError)
else:
assert False, "no raise"
try:
try:
try:
raise ZeroDivisionError
except ZeroDivisionError as ex:
raise NameError from ex
except NameError:
raise
except NameError as ex2:
assert isinstance(ex2.__cause__, ZeroDivisionError)
else:
assert False, "no raise"
# the else clause requires at least one except clause:
with assert_raises(SyntaxError):
exec("""
try:
pass
else:
pass
""")
# Try requires at least except or finally (or both)
with assert_raises(SyntaxError):
exec("""
try:
pass
""")
# leaving the try block early emits an extra copy of the finally body, which
# must not consume the symbol tables of the nested scopes it contains
def return_from_try():
log = []
try:
return "returned"
finally:
log.append((lambda x: x * 2)(3))
log.append({t for t in [1, 2]})
log.append([t for t in [3]])
log.append({k: k for k in [4]})
def nested():
return 5
class Nested:
value = 6
assert log == [6, {1, 2}, [3], {4: 4}], log
assert nested() == 5
assert Nested.value == 6
assert return_from_try() == "returned"
def break_and_continue_from_try():
seen = []
for i in range(4):
try:
if i == 1:
continue
if i == 3:
break
seen.append(i)
finally:
seen.append({t for t in [i]})
return seen
assert break_and_continue_from_try() == [0, {0}, {1}, 2, {2}, {3}]
def return_from_try_runs_finally_once():
log = []
def inner():
try:
return "value"
finally:
log.append(sorted({t for t in "ab"}))
assert inner() == "value"
return log
assert return_from_try_runs_finally_once() == [["a", "b"]]
def generator_return_from_try():
log = []
def gen():
try:
return (yield "yielded")
finally:
log.append([t for t in "z"])
g = gen()
assert g.send(None) == "yielded"
try:
g.send("sent")
except StopIteration as stop:
assert stop.value == "sent", stop.value
else:
assert False, "generator did not stop"
return log
assert generator_return_from_try() == [["z"]]
# the copy of the finally body is emitted where the try block is left, so its
# nested scopes must be looked up past the ones the rest of the try block opens
def scopes_after_the_early_exit():
log = []
def run(data, leave_early):
try:
if leave_early:
return "early"
return list(s * 2 for s in data)
finally:
log.append(sorted(k for k in data))
assert run([1, 2], True) == "early"
assert run([3, 1], False) == [6, 2]
return log
assert scopes_after_the_early_exit() == [[1, 2], [1, 3]]
# a nested function in the try block is a scope too: taking its symbol table
# for the generator expression below built one without the `.0` argument
def named_scope_after_the_early_exit():
log = []
def run(data, leave_early):
try:
if leave_early:
return "early"
def inner():
return [x + 1 for x in data]
return inner()
finally:
log.append(sorted(k for k in data))
assert run([2, 1], True) == "early"
assert run([2, 1], False) == [3, 2]
return log
assert named_scope_after_the_early_exit() == [[1, 2], [1, 2]]
# scopes that share a name resolve by position, so `inner` must not be found
# where the try block declares it
def same_name_scope_after_the_early_exit():
log = []
def run(value, leave_early):
try:
if leave_early:
return "early"
def inner():
return value
return inner()
finally:
def inner():
return log
assert inner() is log
log.append((lambda x: x + value)(1))
assert run(10, True) == "early"
assert run(20, False) == 20
return log
assert same_name_scope_after_the_early_exit() == [11, 21]
# breaking and continuing out of a loop copy the finally body the same way
def loop_exit_with_scopes_after_it():
seen = []
for i in range(4):
try:
if i == 1:
continue
if i == 3:
break
seen.append(sorted(t for t in [i]))
finally:
seen.append(sorted(k for k in [i, i + 1]))
return seen
assert loop_exit_with_scopes_after_it() == [
[0],
[0, 1],
[1, 2],
[2],
[2, 3],
[3, 4],
], loop_exit_with_scopes_after_it()
View Git Blame Copy Permalink