mirror of
https://github.com/RustPython/RustPython.git
synced 2026-10-28 18:59:04 +08:00
* Add _interpreters and _interpchannels modules
Apply the patch baseline for the low-level subinterpreter modules:
- crates/vm/src/stdlib/_interpreters.rs: create/destroy/list_all/get_main/
get_current/is_running/exec/call/run_string/run_func/set___main___attrs/
is_shareable/whence and the Interpreter*Error exceptions
- crates/vm/src/stdlib/_interpchannels.rs: channel create/destroy/send/recv/
list_all/list_interpreters/release/close and the Channel*Error exceptions
- crates/vm/src/vm/crossinterp.rs: cross-interpreter data protocol
- crates/vm/src/vm/{interpreter,mod,runtime,vm_new}.rs: interpreter registry
and lifecycle plumbing the modules need
- Lib/concurrent/interpreters/: high-level PEP 734 package
Assisted-by: Claude:claude-opus-5
* Correct _interpreters and _interpchannels against CPython 3.14
Argument matching:
- Add `function::ArgSpec`, a `PyArg_ParseTupleAndKeywords` equivalent that
applies the `|`, `$` and `:` markers of a format string to a `kwlist`, and
route every module function of both modules through it. This enforces
keyword-only parameters, rejects unexpected keywords, accepts keyword forms
that were previously positional-only (`whence(id=)`, `get_config(id=)`,
`set___main___attrs(id=, updates=)`, `capture_exception(exc=)`,
`_register_end_types(send=, recv=)`), and reproduces the arity messages.
- `O!` slots (`shared`, `updates`, `call`'s `args`/`kwargs`) now reject None,
and `_PyArg_BadArgument` renders None as "None" rather than "NoneType".
- `new_config` takes at most one positional `str`; `list_all` of
`_interpchannels` is argument-less.
- Convert arguments in `kwlist` order, so an argument's own error is raised
ahead of the checks that follow it.
_interpreters:
- Add the `CrossInterpreterBufferView` type and build the received memoryview
on it, keeping the sending interpreter's exporter out of the destination.
- Build `excinfo.errdisplay` from `traceback.TracebackException`.
Feature flags:
- `os.fork` checks finalization before `allow_fork`, matching `os_fork_impl`.
- `_thread.start_new_thread` and `start_joinable_thread` reject an interpreter
without `allow_threads`.
Lib/concurrent/interpreters/_crossinterp.py: restore the stripped docstrings
and comments.
Assisted-by: Claude:claude-opus-5
* Correct the stateless checks and error shapes of _interpreters against CPython 3.14
verify_stateless_function follows _PyFunction_VerifyStateless: it rejects
non-dict builtins, a non-empty __defaults__, __kwdefaults__ or __closure__,
and code whose LOAD_GLOBAL names are held by the function's globals or
absent from its builtins.
code_returns_only_none follows _PyCode_ReturnsOnlyNone: generator,
coroutine and async-generator code is rejected up front, the instruction
walk skips inline caches and maps specialized and instrumented opcodes
back, and the LOAD_CONST preceding each RETURN_VALUE is compared against
the index of None in co_consts.
ExcInfo::capture keeps an empty exception message instead of dropping it,
and builds `formatted` as `module.qualname: msg`, leaving out the builtins
and __main__ modules.
_interpreters.call packs the callable through _PyFunction_GetXIData before
pickle and re-raises the stateless-check failure when both fail. A func,
args or kwargs that cannot be rebuilt in the target, and a result without
cross-interpreter data, now raise NotShareableError with the snapshot as
the cause rather than being returned as excinfo.
_interpreters.capture_exception() without an argument returns None.
pickle_loads and _PyFunction_GetXIData attach the underlying failure as
__cause__.
channel_send converts the object to cross-interpreter data after the
channel's `closing` check.
ChannelID rich comparison returns the result of comparing its id with the
other number instead of coercing that result to bool.
PyMemoryView_FromObjectAndFlags raises "memoryview: a bytes-like object is
required, not 'X'" for an object that is not a buffer.
Assisted-by: Claude:claude-opus-5
* Correct cross-interpreter data conversion and channel argument parsing
Bound shareable ints by isize, raising OverflowError("try sending as
bytes") outside that range and keeping it as the NotShareableError cause.
Report an unshareable object by its repr, and attach a failing conversion
as __cause__ of the NotShareableError raised for the object it was called
for, so each level of a nested tuple appears in the chain.
Guard each tuple item conversion with with_recursion("while sharing a
tuple").
Gate the memoryview getdata function on `_interpreters` having been
imported, and route channel_send_buffer through a memoryview so it uses
that function and the resolved fallback.
Create the cross-interpreter exception types from the module_exec of
either module that raises them, instead of only `_interpreters`.
parse_cid raises OverflowError("int too big to convert"); int_arg
reproduces the `i` converter's three overflow messages and is also used
while parsing channel_send arguments.
Add BASETYPE to _queue.Empty, and gate nt.execv/nt.execve on allow_exec.
Assisted-by: Claude:claude-opus-5
* Correct interpreter ID parsing and the channel_send defaults lookup
_PyInterpreterState_ObjectToID accepts any object with __index__, raises
OverflowError("int too big to convert") outside the int64 range, and
reports a negative ID with the repr of the original object.
channelsmod_send reads the channel's default unboundop and fallback only
when one of the two arguments is negative, so an explicit pair is
validated before the channel is looked up.
Assisted-by: Claude:claude-opus-5
* Add the _interpqueues module and concurrent.interpreters._queues
_interpqueues holds a process-wide queue table and exposes create,
destroy, list_all, put, get, bind, release, get_maxsize,
get_queue_defaults, is_full, get_count and _register_heap_types, raising
QueueError and QueueNotFoundError. QueueEmpty and QueueFull, which
subclass queue.Empty and queue.Full, are registered per interpreter by
concurrent.interpreters._queues.
Cross-interpreter data carries a queue as a refcounted queue ID that
binds when the data is captured and releases when it is dropped, and
is_shareable reports a registered Queue as shareable.
Queue and channel items owned by an interpreter are now cleared from
Interpreter::finalize, after module finalization, rather than from
destroy_owned_interpreter, so values sent by an atexit callback also
become unbound.
is_shareable gates memoryview on the same registration flag as the
getdata lookup.
Assisted-by: Claude:claude-opus-5
* Add the high-level concurrent.interpreters API
Interpreter, ExecutionFailed and the queue aliases, copied verbatim from
CPython 3.14.
Assisted-by: Claude:claude-opus-5
* Destroy leftover subinterpreters when the main interpreter finalizes
Interpreter::finalize now runs finalize_subinterpreters between the atexit
handlers and the finalizing flag: when the main interpreter still owns
subinterpreters it emits the RuntimeWarning "remaining subinterpreters;
close them with Interpreter.close()" and then finalizes each of them.
runtime::owned_interpreter_ids lists the runtime-owned interpreters.
Assisted-by: Claude:claude-opus-5
* Absolutize the script path in run_file
_Py_abspath and _PyPathConfig_ComputeSysPath0 arrive as abs_path and
script_sys_path0. run_file hands the absolutized path to get_importer
and to the runner, so __file__ and co_filename no longer stay relative,
and inserts the symlink-resolved script directory as sys.path[0]
instead of the argument's parent.
test_cmd_line_script.test_script_abspath no longer expects a failure.
Assisted-by: Claude:claude-opus-5
* Name types in error messages the way tp_name does
The object, number, sequence and mapping protocols, the
unsupported-operand helpers, and the type-specific messages fed from
them now read PyType::slot_name() where they read PyType::name(),
so a type declared with a module keeps it.
Message texts corrected along the way:
- PySequence_Size, PyMapping_Size, PySequence_GetItem and
PySequence_SetItem/DelItem split their merged wording in two, and
PyObject_SetItem/DelItem say "object".
- PyObject_GenericSetAttr reports a read-only attribute and the
missing __dict__.
- check_class drops the type name it appended.
- __int__ and __index__ drop the class prefix; __float__ keeps it.
- float(), complex(), round(), next(), str subscripting and list
subscripting take their own wording.
- str.join and bytes.join number the item they turn down and name what
they wanted, and report a non-iterable as "can only join an
iterable".
test_class.testObjectAttributeAccessErrorMessages,
string_tests.test_subscript and the test_descrtut doctest no longer
expect a failure.
Assisted-by: Claude:claude-opus-5
* Gate script_code's source compilation on the compiler feature
crossinterp::script_code called vm.compile unconditionally, which only
exists with the rustpython-compiler feature. Builds without it, such as
example_projects/barebone, now raise a TypeError instead of failing to
compile.
Assisted-by: Claude:claude-opus-5
* Skip test_pyinit_function_raises_exception
The test imports _testsinglephase in its body without a decorator
guarding it, and now that _interpreters exists requires_subinterpreters
no longer skips it.
Assisted-by: Claude:claude-opus-5
484 lines
11 KiB
Python
Vendored
484 lines
11 KiB
Python
Vendored
# This contains most of the executable examples from Guido's descr
|
|
# tutorial, once at
|
|
#
|
|
# https://www.python.org/download/releases/2.2.3/descrintro/
|
|
#
|
|
# A few examples left implicit in the writeup were fleshed out, a few were
|
|
# skipped due to lack of interest (e.g., faking super() by hand isn't
|
|
# of much interest anymore), and a few were fiddled to make the output
|
|
# deterministic.
|
|
|
|
from test.support import sortdict # noqa: F401
|
|
import doctest
|
|
import unittest
|
|
|
|
|
|
class defaultdict(dict):
|
|
def __init__(self, default=None):
|
|
dict.__init__(self)
|
|
self.default = default
|
|
|
|
def __getitem__(self, key):
|
|
try:
|
|
return dict.__getitem__(self, key)
|
|
except KeyError:
|
|
return self.default
|
|
|
|
def get(self, key, *args):
|
|
if not args:
|
|
args = (self.default,)
|
|
return dict.get(self, key, *args)
|
|
|
|
def merge(self, other):
|
|
for key in other:
|
|
if key not in self:
|
|
self[key] = other[key]
|
|
|
|
test_1 = """
|
|
|
|
Here's the new type at work:
|
|
|
|
>>> print(defaultdict) # show our type
|
|
<class '%(modname)s.defaultdict'>
|
|
>>> print(type(defaultdict)) # its metatype
|
|
<class 'type'>
|
|
>>> a = defaultdict(default=0.0) # create an instance
|
|
>>> print(a) # show the instance
|
|
{}
|
|
>>> print(type(a)) # show its type
|
|
<class '%(modname)s.defaultdict'>
|
|
>>> print(a.__class__) # show its class
|
|
<class '%(modname)s.defaultdict'>
|
|
>>> print(type(a) is a.__class__) # its type is its class
|
|
True
|
|
>>> a[1] = 3.25 # modify the instance
|
|
>>> print(a) # show the new value
|
|
{1: 3.25}
|
|
>>> print(a[1]) # show the new item
|
|
3.25
|
|
>>> print(a[0]) # a non-existent item
|
|
0.0
|
|
>>> a.merge({1:100, 2:200}) # use a dict method
|
|
>>> print(sortdict(a)) # show the result
|
|
{1: 3.25, 2: 200}
|
|
>>>
|
|
|
|
We can also use the new type in contexts where classic only allows "real"
|
|
dictionaries, such as the locals/globals dictionaries for the exec
|
|
statement or the built-in function eval():
|
|
|
|
>>> print(sorted(a.keys()))
|
|
[1, 2]
|
|
>>> a['print'] = print # need the print function here
|
|
>>> exec("x = 3; print(x)", a)
|
|
3
|
|
>>> print(sorted(a.keys(), key=lambda x: (str(type(x)), x)))
|
|
[1, 2, '__builtins__', 'print', 'x']
|
|
>>> print(a['x'])
|
|
3
|
|
>>>
|
|
|
|
Now I'll show that defaultdict instances have dynamic instance variables,
|
|
just like classic classes:
|
|
|
|
>>> a.default = -1
|
|
>>> print(a["noway"])
|
|
-1
|
|
>>> a.default = -1000
|
|
>>> print(a["noway"])
|
|
-1000
|
|
>>> 'default' in dir(a)
|
|
True
|
|
>>> a.x1 = 100
|
|
>>> a.x2 = 200
|
|
>>> print(a.x1)
|
|
100
|
|
>>> d = dir(a)
|
|
>>> 'default' in d and 'x1' in d and 'x2' in d
|
|
True
|
|
>>> print(sortdict(a.__dict__))
|
|
{'default': -1000, 'x1': 100, 'x2': 200}
|
|
>>>
|
|
""" % {'modname': __name__}
|
|
|
|
class defaultdict2(dict):
|
|
__slots__ = ['default']
|
|
|
|
def __init__(self, default=None):
|
|
dict.__init__(self)
|
|
self.default = default
|
|
|
|
def __getitem__(self, key):
|
|
try:
|
|
return dict.__getitem__(self, key)
|
|
except KeyError:
|
|
return self.default
|
|
|
|
def get(self, key, *args):
|
|
if not args:
|
|
args = (self.default,)
|
|
return dict.get(self, key, *args)
|
|
|
|
def merge(self, other):
|
|
for key in other:
|
|
if key not in self:
|
|
self[key] = other[key]
|
|
|
|
test_2 = """
|
|
|
|
The __slots__ declaration takes a list of instance variables, and reserves
|
|
space for exactly these in the instance. When __slots__ is used, other
|
|
instance variables cannot be assigned to:
|
|
|
|
>>> a = defaultdict2(default=0.0)
|
|
>>> a[1]
|
|
0.0
|
|
>>> a.default = -1
|
|
>>> a[1]
|
|
-1
|
|
>>> a.x1 = 1
|
|
Traceback (most recent call last):
|
|
File "<stdin>", line 1, in ?
|
|
AttributeError: 'defaultdict2' object has no attribute 'x1' and no __dict__ for setting new attributes
|
|
>>>
|
|
|
|
"""
|
|
|
|
test_3 = """
|
|
|
|
Introspecting instances of built-in types
|
|
|
|
For instance of built-in types, x.__class__ is now the same as type(x):
|
|
|
|
>>> type([])
|
|
<class 'list'>
|
|
>>> [].__class__
|
|
<class 'list'>
|
|
>>> list
|
|
<class 'list'>
|
|
>>> isinstance([], list)
|
|
True
|
|
>>> isinstance([], dict)
|
|
False
|
|
>>> isinstance([], object)
|
|
True
|
|
>>>
|
|
|
|
You can get the information from the list type:
|
|
|
|
>>> import pprint
|
|
>>> pprint.pprint(dir(list)) # like list.__dict__.keys(), but sorted
|
|
['__add__',
|
|
'__class__',
|
|
'__class_getitem__',
|
|
'__contains__',
|
|
'__delattr__',
|
|
'__delitem__',
|
|
'__dir__',
|
|
'__doc__',
|
|
'__eq__',
|
|
'__format__',
|
|
'__ge__',
|
|
'__getattribute__',
|
|
'__getitem__',
|
|
'__getstate__',
|
|
'__gt__',
|
|
'__hash__',
|
|
'__iadd__',
|
|
'__imul__',
|
|
'__init__',
|
|
'__init_subclass__',
|
|
'__iter__',
|
|
'__le__',
|
|
'__len__',
|
|
'__lt__',
|
|
'__mul__',
|
|
'__ne__',
|
|
'__new__',
|
|
'__reduce__',
|
|
'__reduce_ex__',
|
|
'__repr__',
|
|
'__reversed__',
|
|
'__rmul__',
|
|
'__setattr__',
|
|
'__setitem__',
|
|
'__sizeof__',
|
|
'__str__',
|
|
'__subclasshook__',
|
|
'append',
|
|
'clear',
|
|
'copy',
|
|
'count',
|
|
'extend',
|
|
'index',
|
|
'insert',
|
|
'pop',
|
|
'remove',
|
|
'reverse',
|
|
'sort']
|
|
|
|
The new introspection API gives more information than the old one: in
|
|
addition to the regular methods, it also shows the methods that are
|
|
normally invoked through special notations, e.g. __iadd__ (+=), __len__
|
|
(len), __ne__ (!=). You can invoke any method from this list directly:
|
|
|
|
>>> a = ['tic', 'tac']
|
|
>>> list.__len__(a) # same as len(a)
|
|
2
|
|
>>> a.__len__() # ditto
|
|
2
|
|
>>> list.append(a, 'toe') # same as a.append('toe')
|
|
>>> a
|
|
['tic', 'tac', 'toe']
|
|
>>>
|
|
|
|
This is just like it is for user-defined classes.
|
|
"""
|
|
|
|
test_4 = """
|
|
|
|
Static methods and class methods
|
|
|
|
The new introspection API makes it possible to add static methods and class
|
|
methods. Static methods are easy to describe: they behave pretty much like
|
|
static methods in C++ or Java. Here's an example:
|
|
|
|
>>> class C:
|
|
...
|
|
... @staticmethod
|
|
... def foo(x, y):
|
|
... print("staticmethod", x, y)
|
|
|
|
>>> C.foo(1, 2)
|
|
staticmethod 1 2
|
|
>>> c = C()
|
|
>>> c.foo(1, 2)
|
|
staticmethod 1 2
|
|
|
|
Class methods use a similar pattern to declare methods that receive an
|
|
implicit first argument that is the *class* for which they are invoked.
|
|
|
|
>>> class C:
|
|
... @classmethod
|
|
... def foo(cls, y):
|
|
... print("classmethod", cls, y)
|
|
|
|
>>> C.foo(1)
|
|
classmethod <class '%(modname)s.C'> 1
|
|
>>> c = C()
|
|
>>> c.foo(1)
|
|
classmethod <class '%(modname)s.C'> 1
|
|
|
|
>>> class D(C):
|
|
... pass
|
|
|
|
>>> D.foo(1)
|
|
classmethod <class '%(modname)s.D'> 1
|
|
>>> d = D()
|
|
>>> d.foo(1)
|
|
classmethod <class '%(modname)s.D'> 1
|
|
|
|
This prints "classmethod __main__.D 1" both times; in other words, the
|
|
class passed as the first argument of foo() is the class involved in the
|
|
call, not the class involved in the definition of foo().
|
|
|
|
But notice this:
|
|
|
|
>>> class E(C):
|
|
... @classmethod
|
|
... def foo(cls, y): # override C.foo
|
|
... print("E.foo() called")
|
|
... C.foo(y)
|
|
|
|
>>> E.foo(1)
|
|
E.foo() called
|
|
classmethod <class '%(modname)s.C'> 1
|
|
>>> e = E()
|
|
>>> e.foo(1)
|
|
E.foo() called
|
|
classmethod <class '%(modname)s.C'> 1
|
|
|
|
In this example, the call to C.foo() from E.foo() will see class C as its
|
|
first argument, not class E. This is to be expected, since the call
|
|
specifies the class C. But it stresses the difference between these class
|
|
methods and methods defined in metaclasses (where an upcall to a metamethod
|
|
would pass the target class as an explicit first argument).
|
|
""" % {'modname': __name__}
|
|
|
|
test_5 = """
|
|
|
|
Attributes defined by get/set methods
|
|
|
|
|
|
>>> class property(object):
|
|
...
|
|
... def __init__(self, get, set=None):
|
|
... self.__get = get
|
|
... self.__set = set
|
|
...
|
|
... def __get__(self, inst, type=None):
|
|
... return self.__get(inst)
|
|
...
|
|
... def __set__(self, inst, value):
|
|
... if self.__set is None:
|
|
... raise AttributeError("this attribute is read-only")
|
|
... return self.__set(inst, value)
|
|
|
|
Now let's define a class with an attribute x defined by a pair of methods,
|
|
getx() and setx():
|
|
|
|
>>> class C(object):
|
|
...
|
|
... def __init__(self):
|
|
... self.__x = 0
|
|
...
|
|
... def getx(self):
|
|
... return self.__x
|
|
...
|
|
... def setx(self, x):
|
|
... if x < 0: x = 0
|
|
... self.__x = x
|
|
...
|
|
... x = property(getx, setx)
|
|
|
|
Here's a small demonstration:
|
|
|
|
>>> a = C()
|
|
>>> a.x = 10
|
|
>>> print(a.x)
|
|
10
|
|
>>> a.x = -10
|
|
>>> print(a.x)
|
|
0
|
|
>>>
|
|
|
|
Hmm -- property is builtin now, so let's try it that way too.
|
|
|
|
>>> del property # unmask the builtin
|
|
>>> property
|
|
<class 'property'>
|
|
|
|
>>> class C(object):
|
|
... def __init__(self):
|
|
... self.__x = 0
|
|
... def getx(self):
|
|
... return self.__x
|
|
... def setx(self, x):
|
|
... if x < 0: x = 0
|
|
... self.__x = x
|
|
... x = property(getx, setx)
|
|
|
|
|
|
>>> a = C()
|
|
>>> a.x = 10
|
|
>>> print(a.x)
|
|
10
|
|
>>> a.x = -10
|
|
>>> print(a.x)
|
|
0
|
|
>>>
|
|
"""
|
|
|
|
test_6 = """
|
|
|
|
Method resolution order
|
|
|
|
This example is implicit in the writeup.
|
|
|
|
>>> class A: # implicit new-style class
|
|
... def save(self):
|
|
... print("called A.save()")
|
|
>>> class B(A):
|
|
... pass
|
|
>>> class C(A):
|
|
... def save(self):
|
|
... print("called C.save()")
|
|
>>> class D(B, C):
|
|
... pass
|
|
|
|
>>> D().save()
|
|
called C.save()
|
|
|
|
>>> class A(object): # explicit new-style class
|
|
... def save(self):
|
|
... print("called A.save()")
|
|
>>> class B(A):
|
|
... pass
|
|
>>> class C(A):
|
|
... def save(self):
|
|
... print("called C.save()")
|
|
>>> class D(B, C):
|
|
... pass
|
|
|
|
>>> D().save()
|
|
called C.save()
|
|
"""
|
|
|
|
class A(object):
|
|
def m(self):
|
|
return "A"
|
|
|
|
class B(A):
|
|
def m(self):
|
|
return "B" + super(B, self).m()
|
|
|
|
class C(A):
|
|
def m(self):
|
|
return "C" + super(C, self).m()
|
|
|
|
class D(C, B):
|
|
def m(self):
|
|
return "D" + super(D, self).m()
|
|
|
|
|
|
test_7 = """
|
|
|
|
Cooperative methods and "super"
|
|
|
|
>>> print(D().m()) # "DCBA"
|
|
DCBA
|
|
"""
|
|
|
|
test_8 = """
|
|
|
|
Backwards incompatibilities
|
|
|
|
>>> class A:
|
|
... def foo(self):
|
|
... print("called A.foo()")
|
|
|
|
>>> class B(A):
|
|
... pass
|
|
|
|
>>> class C(A):
|
|
... def foo(self):
|
|
... B.foo(self)
|
|
|
|
>>> C().foo()
|
|
called A.foo()
|
|
|
|
>>> class C(A):
|
|
... def foo(self):
|
|
... A.foo(self)
|
|
>>> C().foo()
|
|
called A.foo()
|
|
"""
|
|
|
|
__test__ = {"tut1": test_1,
|
|
"tut2": test_2,
|
|
"tut3": test_3,
|
|
"tut4": test_4,
|
|
"tut5": test_5,
|
|
"tut6": test_6,
|
|
"tut7": test_7,
|
|
"tut8": test_8}
|
|
|
|
def load_tests(loader, tests, pattern):
|
|
from test.support.rustpython import DocTestChecker # TODO: RUSTPYTHON
|
|
tests.addTest(doctest.DocTestSuite(checker=DocTestChecker())) # TODO: RUSTPYTHON
|
|
return tests
|
|
|
|
|
|
if __name__ == "__main__":
|
|
unittest.main()
|