mirror of
https://github.com/RustPython/RustPython.git
synced 2026-10-29 03:04:38 +08:00
Jeong, YunWon
a7981dccc7
* Align compiler futures and annotations with CPython Handle Barry parsing consistently across AST, type-comment, REPL, and WASM paths. Preserve deferred annotation source and scope metadata, and make _symtable conversion cached and linear-time. Assisted-by: Codex:gpt-5.4 * Fix f-string debug ranges and update Barry/annotate test expectations Compute the debug f-string source range from the raw leading and trailing text instead of the comment-stripped text, so a comment inside a multi-line replacement field no longer shifts the emitted LOAD_CONST location. Restrict the obsolete-operator location shift to `<>` by also requiring a `>` at the reported offset; without it any `ExpectedExpression` that followed a `<` (`2 <;`) was reported one column to the left. Add `CodeFlags::FUTURE_MASK` and use it in builtins, the shell, and the WASM VM instead of repeating the eight-flag union, and collapse the duplicated Barry diagnostic sequences into `BarrySource::diagnostic`. Update the `__annotate__` disassembly snapshot to LOAD_FAST_BORROW and replace the RustPython-only `barry_as_FLUFL` co_flags assertion in extra_tests with the CPython-matching one. Assisted-by: Claude Code:claude-opus-5 * Restore the annotated function in __annotate__ qualnames Signature annotation scopes are compiled in the scope enclosing the annotated function, so the function name has to be folded back into the qualname: `f.__annotate__`, `C.m.__annotate__`, `outer.<locals>.inner.__annotate__`. test_type_annotations test_annotate_qualname covers this. test_pyrepl test_future_barry_as_flufl now passes, so drop its expectedFailure marker. Assisted-by: Claude Code:claude-opus-5 * Pass safepoints in the attached-thread interpreter tests `main_and_subinterpreter_run_sections_overlap` parked its workers on a condvar while attached, and `busy_main_interpreter_does_not_block_subinterpreter` looped over protocol calls that never reach a safepoint. An attached thread that never reaches a safepoint cannot be suspended, so a concurrent process-wide stop-the-world (from the gc test running in parallel) never completes and the already-stopped sibling interpreter stays stopped; the second worker then waits in `wait_while_suspended` until the 30s deadline. Both loops now call `vm.check_signals()` each iteration, and the condvar wait uses `wait_timeout` so the lock is released between safepoints. Reproduced on Linux (4 CPUs): the two tests with `--test-threads=2` failed 2/40 before and 0/40 after; the full `rustpython-vm` lib binary failed 1/30 before and 0/30 after. Assisted-by: Claude Code:claude-opus-5 * Report the obsolete `<>` operator from one place `<>` was located twice: `BarrySource` recorded the token ranges it rewrote to `!=`, and `cpython_parse_diagnostic_override` separately peeked at the bytes around an `ExpectedExpression` location. Both produced "invalid syntax" spanning the whole operator. `prepare_barry_as_flufl_source` now records the operator ranges in the non-Barry case too -- by plain text search, since nothing is rewritten there -- so `BarrySource::invalid_legacy_operator` covers both, and the byte-peeking helper is gone. It also defers to an unclosed bracket earlier in the source, which the diagnostic chain it used to sit in did for it. A `<>` that starts a statement now spans both characters: the parser stops at the `<` rather than one character in, which the byte peek did not handle. Assisted-by: Claude Code:claude-opus-5 * Drop a py object's payload before its class `PyInner` declares `typ` before `payload`, so `drop_in_place` dropped the class first and `PyAtomicRef::drop` left it null. A `PyWeak` payload is still linked into the weakref list of the object it points at until its own `Drop` unlinks it under the stripe lock, which happens after that. `WeakRefList::add` walks the same list looking for a proxy to reuse and reads `node.class()` off every node it passes, so it could dereference the null class of a node on its way out. `PyInner::dealloc` now runs the two destructors in the other order at all three of its exit paths, with a compile-time check that no other field has one. Reproduced by the new test in about 0.1s, as the panic CI hit in the c-api `weakrefobject::tests::new_proxy` run: panicked at crates/vm/src/object/ext.rs:312: unsafe precondition(s) violated: hint::unreachable_unchecked must never be reached Assisted-by: Claude Code:claude-opus-5
303 lines
9.8 KiB
Python
Vendored
303 lines
9.8 KiB
Python
Vendored
import contextlib
|
|
import io
|
|
import warnings
|
|
import unittest
|
|
from unittest.mock import patch
|
|
from textwrap import dedent
|
|
|
|
from test.support import force_not_colorized
|
|
|
|
from _pyrepl.console import InteractiveColoredConsole
|
|
from _pyrepl.simple_interact import _more_lines
|
|
|
|
class TestSimpleInteract(unittest.TestCase):
|
|
def test_multiple_statements(self):
|
|
namespace = {}
|
|
code = dedent("""\
|
|
class A:
|
|
def foo(self):
|
|
|
|
|
|
pass
|
|
|
|
class B:
|
|
def bar(self):
|
|
pass
|
|
|
|
a = 1
|
|
a
|
|
""")
|
|
console = InteractiveColoredConsole(namespace, filename="<stdin>")
|
|
f = io.StringIO()
|
|
with (
|
|
patch.object(InteractiveColoredConsole, "showsyntaxerror") as showsyntaxerror,
|
|
patch.object(InteractiveColoredConsole, "runsource", wraps=console.runsource) as runsource,
|
|
contextlib.redirect_stdout(f),
|
|
):
|
|
more = console.push(code, filename="<stdin>", _symbol="single") # type: ignore[call-arg]
|
|
self.assertFalse(more)
|
|
showsyntaxerror.assert_not_called()
|
|
|
|
|
|
def test_multiple_statements_output(self):
|
|
namespace = {}
|
|
code = dedent("""\
|
|
b = 1
|
|
b
|
|
a = 1
|
|
a
|
|
""")
|
|
console = InteractiveColoredConsole(namespace, filename="<stdin>")
|
|
f = io.StringIO()
|
|
with contextlib.redirect_stdout(f):
|
|
more = console.push(code, filename="<stdin>", _symbol="single") # type: ignore[call-arg]
|
|
self.assertFalse(more)
|
|
self.assertEqual(f.getvalue(), "1\n")
|
|
|
|
@force_not_colorized
|
|
def test_multiple_statements_fail_early(self):
|
|
console = InteractiveColoredConsole()
|
|
code = dedent("""\
|
|
raise Exception('foobar')
|
|
print('spam', 'eggs', sep='&')
|
|
""")
|
|
f = io.StringIO()
|
|
with contextlib.redirect_stderr(f):
|
|
console.runsource(code)
|
|
self.assertIn('Exception: foobar', f.getvalue())
|
|
self.assertNotIn('spam&eggs', f.getvalue())
|
|
|
|
def test_empty(self):
|
|
namespace = {}
|
|
code = ""
|
|
console = InteractiveColoredConsole(namespace, filename="<stdin>")
|
|
f = io.StringIO()
|
|
with contextlib.redirect_stdout(f):
|
|
more = console.push(code, filename="<stdin>", _symbol="single") # type: ignore[call-arg]
|
|
self.assertFalse(more)
|
|
self.assertEqual(f.getvalue(), "")
|
|
|
|
def test_runsource_compiles_and_runs_code(self):
|
|
console = InteractiveColoredConsole()
|
|
source = "print('Hello, world!')"
|
|
with patch.object(console, "runcode") as mock_runcode:
|
|
console.runsource(source)
|
|
mock_runcode.assert_called_once()
|
|
|
|
def test_runsource_returns_false_for_successful_compilation(self):
|
|
console = InteractiveColoredConsole()
|
|
source = "print('Hello, world!')"
|
|
f = io.StringIO()
|
|
with contextlib.redirect_stdout(f):
|
|
result = console.runsource(source)
|
|
self.assertFalse(result)
|
|
|
|
@force_not_colorized
|
|
def test_runsource_returns_false_for_failed_compilation(self):
|
|
console = InteractiveColoredConsole()
|
|
source = "print('Hello, world!'"
|
|
f = io.StringIO()
|
|
with contextlib.redirect_stderr(f):
|
|
result = console.runsource(source)
|
|
self.assertFalse(result)
|
|
self.assertIn('SyntaxError', f.getvalue())
|
|
|
|
@unittest.expectedFailure # TODO: RUSTPYTHON
|
|
@force_not_colorized
|
|
def test_runsource_show_syntax_error_location(self):
|
|
console = InteractiveColoredConsole()
|
|
source = "def f(x, x): ..."
|
|
f = io.StringIO()
|
|
with contextlib.redirect_stderr(f):
|
|
result = console.runsource(source)
|
|
self.assertFalse(result)
|
|
r = """
|
|
def f(x, x): ...
|
|
^
|
|
SyntaxError: duplicate argument 'x' in function definition"""
|
|
self.assertIn(r, f.getvalue())
|
|
|
|
def test_runsource_shows_syntax_error_for_failed_compilation(self):
|
|
console = InteractiveColoredConsole()
|
|
source = "print('Hello, world!'"
|
|
with patch.object(console, "showsyntaxerror") as mock_showsyntaxerror:
|
|
console.runsource(source)
|
|
mock_showsyntaxerror.assert_called_once()
|
|
source = dedent("""\
|
|
match 1:
|
|
case {0: _, 0j: _}:
|
|
pass
|
|
""")
|
|
with patch.object(console, "showsyntaxerror") as mock_showsyntaxerror:
|
|
console.runsource(source)
|
|
mock_showsyntaxerror.assert_called_once()
|
|
|
|
def test_runsource_survives_null_bytes(self):
|
|
console = InteractiveColoredConsole()
|
|
source = "\x00\n"
|
|
f = io.StringIO()
|
|
with contextlib.redirect_stdout(f), contextlib.redirect_stderr(f):
|
|
result = console.runsource(source)
|
|
self.assertFalse(result)
|
|
self.assertIn("source code string cannot contain null bytes", f.getvalue())
|
|
|
|
def test_no_active_future(self):
|
|
console = InteractiveColoredConsole()
|
|
source = dedent("""\
|
|
x: int = 1
|
|
print(__annotate__(1))
|
|
""")
|
|
f = io.StringIO()
|
|
with contextlib.redirect_stdout(f):
|
|
result = console.runsource(source)
|
|
self.assertFalse(result)
|
|
self.assertEqual(f.getvalue(), "{'x': <class 'int'>}\n")
|
|
|
|
def test_future_annotations(self):
|
|
console = InteractiveColoredConsole()
|
|
source = dedent("""\
|
|
from __future__ import annotations
|
|
def g(x: int): ...
|
|
print(g.__annotations__)
|
|
""")
|
|
f = io.StringIO()
|
|
with contextlib.redirect_stdout(f):
|
|
result = console.runsource(source)
|
|
self.assertFalse(result)
|
|
self.assertEqual(f.getvalue(), "{'x': 'int'}\n")
|
|
|
|
def test_future_barry_as_flufl(self):
|
|
console = InteractiveColoredConsole()
|
|
f = io.StringIO()
|
|
with contextlib.redirect_stdout(f):
|
|
result = console.runsource("from __future__ import barry_as_FLUFL\n")
|
|
result = console.runsource("""print("black" <> 'blue')\n""")
|
|
self.assertFalse(result)
|
|
self.assertEqual(f.getvalue(), "True\n")
|
|
|
|
|
|
class TestMoreLines(unittest.TestCase):
|
|
def test_invalid_syntax_single_line(self):
|
|
namespace = {}
|
|
code = "if foo"
|
|
console = InteractiveColoredConsole(namespace, filename="<stdin>")
|
|
self.assertFalse(_more_lines(console, code))
|
|
|
|
def test_empty_line(self):
|
|
namespace = {}
|
|
code = ""
|
|
console = InteractiveColoredConsole(namespace, filename="<stdin>")
|
|
self.assertFalse(_more_lines(console, code))
|
|
|
|
def test_valid_single_statement(self):
|
|
namespace = {}
|
|
code = "foo = 1"
|
|
console = InteractiveColoredConsole(namespace, filename="<stdin>")
|
|
self.assertFalse(_more_lines(console, code))
|
|
|
|
def test_multiline_single_assignment(self):
|
|
namespace = {}
|
|
code = dedent("""\
|
|
foo = [
|
|
1,
|
|
2,
|
|
3,
|
|
]""")
|
|
console = InteractiveColoredConsole(namespace, filename="<stdin>")
|
|
self.assertFalse(_more_lines(console, code))
|
|
|
|
def test_multiline_single_block(self):
|
|
namespace = {}
|
|
code = dedent("""\
|
|
def foo():
|
|
'''docs'''
|
|
|
|
return 1""")
|
|
console = InteractiveColoredConsole(namespace, filename="<stdin>")
|
|
self.assertTrue(_more_lines(console, code))
|
|
|
|
def test_multiple_statements_single_line(self):
|
|
namespace = {}
|
|
code = "foo = 1;bar = 2"
|
|
console = InteractiveColoredConsole(namespace, filename="<stdin>")
|
|
self.assertFalse(_more_lines(console, code))
|
|
|
|
def test_multiple_statements(self):
|
|
namespace = {}
|
|
code = dedent("""\
|
|
import time
|
|
|
|
foo = 1""")
|
|
console = InteractiveColoredConsole(namespace, filename="<stdin>")
|
|
self.assertTrue(_more_lines(console, code))
|
|
|
|
def test_multiple_blocks(self):
|
|
namespace = {}
|
|
code = dedent("""\
|
|
from dataclasses import dataclass
|
|
|
|
@dataclass
|
|
class Point:
|
|
x: float
|
|
y: float""")
|
|
console = InteractiveColoredConsole(namespace, filename="<stdin>")
|
|
self.assertTrue(_more_lines(console, code))
|
|
|
|
def test_multiple_blocks_empty_newline(self):
|
|
namespace = {}
|
|
code = dedent("""\
|
|
from dataclasses import dataclass
|
|
|
|
@dataclass
|
|
class Point:
|
|
x: float
|
|
y: float
|
|
""")
|
|
console = InteractiveColoredConsole(namespace, filename="<stdin>")
|
|
self.assertFalse(_more_lines(console, code))
|
|
|
|
def test_multiple_blocks_indented_newline(self):
|
|
namespace = {}
|
|
code = (
|
|
"from dataclasses import dataclass\n"
|
|
"\n"
|
|
"@dataclass\n"
|
|
"class Point:\n"
|
|
" x: float\n"
|
|
" y: float\n"
|
|
" "
|
|
)
|
|
console = InteractiveColoredConsole(namespace, filename="<stdin>")
|
|
self.assertFalse(_more_lines(console, code))
|
|
|
|
def test_incomplete_statement(self):
|
|
namespace = {}
|
|
code = "if foo:"
|
|
console = InteractiveColoredConsole(namespace, filename="<stdin>")
|
|
self.assertTrue(_more_lines(console, code))
|
|
|
|
|
|
class TestWarnings(unittest.TestCase):
|
|
def test_pep_765_warning(self):
|
|
"""
|
|
Test that a SyntaxWarning emitted from the
|
|
AST optimizer is only shown once in the REPL.
|
|
"""
|
|
# gh-131927
|
|
console = InteractiveColoredConsole()
|
|
code = dedent("""\
|
|
def f():
|
|
try:
|
|
return 1
|
|
finally:
|
|
return 2
|
|
""")
|
|
|
|
with warnings.catch_warnings(record=True) as caught:
|
|
warnings.simplefilter("always")
|
|
console.runsource(code)
|
|
|
|
count = sum("'return' in a 'finally' block" in str(w.message)
|
|
for w in caught)
|
|
self.assertEqual(count, 1)
|