mirror of
https://github.com/RustPython/RustPython.git
synced 2026-10-28 18:59:04 +08:00
* Correct what the non-unix frame stack is for
The comment said non-unix threading builds have no stop-the-world.
CollectStopTheWorld is gated on `feature = "threading"` alone, and
sys._current_frames stops the world on both paths; the field is the
fallback a reader uses when there is no `top_iframe` to materialize from.
Assisted-by: Claude
* Seek by the whole file position on Windows
`SetFilePointer` answers with the low half of the new position and signals
failure with INVALID_SET_FILE_POINTER, which is also that half of a
position four gigabytes in; telling them apart takes the error code, which
this did not read, so such a seek was reported as an error. Deciding
seekability from it also called such a file unseekable.
`SetFilePointerEx` returns the whole position and a success flag of its
own, which also removes the transmute of the position into halves.
Assisted-by: Claude
* memoryview: tell a value of the wrong kind from one that does not fit
Assigning an item reported every packing failure as a TypeError, so
m[0] = 300 on a 'B' view said the value was the wrong type rather than out
of range. Packing now says which of the two it was, and whether the value's
own code raised, in which case that error is the answer as it is:
m[0] = 300 ValueError: invalid value for format 'B'
m[0] = "x" TypeError: invalid type for format 'B'
m[0] = <__index__ that raises> the raised error
`struct` reports both as `struct.error` and is unchanged; the kind travels
beside the exception for the caller that tells them apart.
Also: None was read as a deletion, so m[0] = None answered "cannot delete
memory" instead of packing it; and deleting through the mapping protocol
never reached the read-only check, which comes first.
Assisted-by: Claude
* Raise MemoryError for allocations sized by Python input
os.read, _RawIOBase.read, int.to_bytes, struct.pack, ctypes array
creation and the _ssl RAND functions sized a Vec from a Python-supplied
length with vec![], which calls handle_alloc_error and, under
panic = "abort", ends the process. They now allocate through
vm.new_zeroed_bytes and raise MemoryError.
itertools.product built its pools without checking that
len(iterables) * repeat is representable; it now raises OverflowError
"repeat argument too large" and reserves the pool and index vectors
fallibly. repeat is read as isize, so a negative one raises ValueError
"repeat argument cannot be negative" instead of the conversion's message.
_ssl.RAND_bytes and RAND_pseudo_bytes read n as i32, matching the int
they are declared with.
Assisted-by: Claude
* Collect with the count in the object header
A collection kept the count it was working with in a table keyed by the
object's address, and the objects it had proved reachable in a second one.
Between them they were hashed once per candidate and twice per edge in the
heap, which is where most of a collection over a live heap went.
The count now lives in `PyInner::gc_refs`, with `GcBits::COLLECTING` saying
it is meaningful, and reachability is `gc_refs == GC_REACHABLE` rather than
membership in a set. Step 5 splits the candidates and clears the bit in one
pass. gcbench, five interleaved pairs, median: a live heap of 423k objects
goes from 0.101s to 0.055s and a dead one from 0.402s to 0.383s.
The bits, generation, owner and count take eight bytes between them. A
64-bit header had those eight as the padding its alignment forces, so it is
unchanged at 48 bytes; a 32-bit header grows from 24 to 28.
Assisted-by: Claude
* Take an explicit thread stack size as a floor in debug builds
A debug build already started Python threads on 8 MiB rather than Rust's
2 MiB default, but an explicit threading.stack_size(N) went through
verbatim. test_threading asks for 256 KiB, and starting a thread on that
walked off the end of the stack: the guard page fault landed in the
prologue of ExecutingFrame::run.
Unoptimized, that prologue reserves 80,848 bytes where the optimized one
reserves 656 -- execute_instruction is #[inline(always)] and LLVM only
colors stack slots from opt-level 1, so the frame is the sum of all 200
instruction arms' temporaries rather than the largest. A Python call costs
88,672 bytes of native stack there, and threading's bootstrap is six
frames deep, so 256 KiB holds less than half of what starting a thread
takes.
The floor reaches thread::Builder only; threading.stack_size() still
answers with what was asked for, and release builds are unchanged.
Assisted-by: Claude
* Check the native stack on every frame entry
The C-stack guard ran on one frame entry in eight. That asks the margin to
cover eight frames rather than one, and it does not: an unoptimized frame
entered through native code takes 88,672 bytes against a debug margin of
262,144. A recursion whose steps re-enter that way -- `__add__` calling
itself, a sort key that sorts -- ran off the end of the stack instead of
raising RecursionError. On a debug build `class Add: __add__ = lambda s, o:
s + o; Add() + 1` segfaulted on the main thread; it now raises, as it does
under CPython and in release builds.
enter_iframe checked and then called enter_iframe_unchecked, which checked
again; it now leaves the check to the one call.
Measured on a call-dominated benchmark, five interleaved pairs: instructions
retired go up 0.17%, about four per call, which is the stack pointer read
and the compare.
Assisted-by: Claude
* Take an iterable's length hint when a list is filled from it
`map_py_iter` read `__length_hint__` only to pass it to `PyIterIter`, and
returned an empty vector when the hint was `isize::MAX` or more. Collecting
through `PyResult` dropped the iterator's lower bound, so nothing reserved
the room the hint asked for.
`list()`, `list.extend()` and `list.__iadd__()` now reserve it and report a
hint they cannot honour as `MemoryError`; a hint that leaves no room for the
elements the list already holds is passed over, as `list_extend()` does.
`tuple()` and the other callers keep filling up without reserving.
`length_hint_opt` errors other than the `TypeError` it already turns into
`None` now reach the caller instead of being dropped.
`__iadd__` and `inplace_concat` went through `extract_cloned`, which reads
`__len__` and not `__length_hint__`; both call `PyList::extend` now, the way
`list_inplace_concat()` calls `list_extend()`.
The tuple, list and dict fast paths of `extract_elements_inner` reserve their
known length, which the `collect()` they used dropped.
Assisted-by: Claude
* Drop map.__length_hint__
`map_methods` has no `__length_hint__`, so `operator.length_hint()` on a map
answers 0, not the length of what it draws from.
The method walked into the length hint of every iterator it holds, and a
chain of maps 10000 long overflowed the native stack answering for the
outermost one. It also took the longest of its iterators, where a map stops
at the shortest.
Assisted-by: Claude
* Ask for a length hint where each caller asks for it
`map_py_iter` asked the iterable it was handed, for every caller, and reported
what asking raised. Only some callers ask it: `list_extend()` and
`_PyBytes_FromIterator()` ask the iterable, `PySequence_Tuple()` asks the
iterator, and the bytearray constructor asks nothing. `tuple()`, `min()`,
`max()`, `collections.deque()` and `f(*x)` raised for an iterable whose
`__len__` or `__length_hint__` does, where they answer.
Which object is asked is now the caller's to say. `sorted()` asks the iterable,
being `PySequence_List()`.
`bytes_from_object()` stood in for `PyBytes_FromObject()`, for
`bytearray_extend()` and for the bytearray constructor, which do not agree on
this: the first two ask, the last does not. It is split, and assigning to a
bytearray slice takes the constructor's side with `PyByteArray_FromObject()`.
`list.extend()` counted what it held before the iterable had been asked, where
`list_extend()` reads `Py_SIZE(self)` after. A `__length_hint__` that adds to
the list made the overflow guard read a count too small and raise `MemoryError`
where nothing is wrong; one that empties it made the guard skip a reservation
that cannot be served.
Assisted-by: Claude
* Settle product's pool count before it reads its arguments
`product_new()` checks `repeat` and works out `npools` before it calls
`PySequence_Tuple()` on any argument, and fills the pools `npools` times.
The pools were filled by repeating the arguments `repeat` times instead, which
walks that many steps even with no arguments to repeat: `product(repeat=2**62)`
counted up to it rather than answering `[()]`. The count was also worked out
after the arguments had been read, so a repeat too large to serve ran their
code first.
Assisted-by: Claude
* Let the bool format answer with the error its value raised
`pack_single()` leaves `'?'` to `PyObject_IsTrue()` and returns what that
raised. Packing classified the error instead, so a `ValueError` from a
`__bool__` came back as "memoryview: invalid value for format '?'".
Assisted-by: Claude
* Release a cell's old value after the lock
`PyCell::set` dropped what it replaced while still holding the mutex guarding
the cell contents. A `__del__` running from that drop and reading the same cell
waited on a lock its own caller held, so `del it` on a closure variable whose
value has such a `__del__` deadlocked.
The replaced value is now released once the guard is gone, as `Py_XSETREF`
stores before it decrefs.
Assisted-by: Claude
* Have a set iterator hold the set it iterates
The iterator kept only a reference to the inner hash table, so in
`it = iter(A(*args))` the `A()` temporary was the last owner and died as the
call returned, before `it` was bound. A `__del__` reading `it` there saw an
unbound name and its error was printed and ignored.
The iterator now holds the set object, as `si_set` does, and releases it once
exhausted. That release happens after the lock is dropped, where
`setiter_iternext()` puts its `Py_DECREF(so)` past `Py_END_CRITICAL_SECTION()`,
so a `__del__` that iterates again does not wait on a lock the call holds.
Assisted-by: Claude
* Release an exhausted iterator's container after the lock
`PositionIterInternal::_next` overwrote its `IterStatus::Active` while the
caller still held the mutex around it. Dropping the container there ran any
`__del__` under that lock, and a `__del__` that iterated the same object again
blocked on it.
`exhaust()` now hands the container back instead of dropping it, and
`locked_step()` releases it after the guard. list, list_reverseiterator, tuple,
str, dict and its views and reverse views, bytes, bytearray, memoryview,
array, deque, and the enumerate and sequence iterators all go through it.
Assisted-by: Claude
* Unskip test_free_after_iterating
Assisted-by: Claude
* Stop listing test_set as an environment polluter
`check_free_after_iterating` no longer leaves an ignored exception behind, and
the job that reruns the listed tests ten times fails once one of them stops
polluting.
Assisted-by: Claude
* Drop winsound imports left unused
`TryFromBorrowedObject`, `exceptions`, and `ToWideString` have no reference in
the module, which fails the Windows clippy line under `-Dwarnings`.
Assisted-by: Claude
* Ask nothing where the caller takes no room
`map_py_iter` asked the iterator for a length hint on behalf of every caller
that does not reserve, and reported what asking raised. Those callers ask
nothing at all: `tuple()`, `f(*x)`, `bytearray(x)`, `min()`, `max()` and
`collections.deque()` answer for an iterator whose `__length_hint__` raises,
where they had been raising it.
The answer was also never spent. It reached `PyIterIter` for a `size_hint()`
the push loop does not read, so the lookup and any call it made were work
thrown away: `tuple()` over a generator drops 22% of its instructions, and 17%
over an iterator with a `__length_hint__` written in Python.
Assisted-by: Claude
* Stop asking an iterator how long it is to walk it
`PyIter::iter` and `PyIter::into_iter` asked for a length hint and reported
what asking raised. Nothing spent the answer: it reached `PyIterIter` for a
`size_hint()` that every caller either loops past or drops, since collecting
into a `Result` reports no lower bound.
23 operations answered for an iterator whose `__length_hint__` raises, where
they had been raising it: `set`, `frozenset` and the nine `set` methods that
take an iterable, `dict.fromkeys` and the dict view operators, `array` and
`array.extend`, `all`, `any`, `sum`, `io.writelines`, `math.fsum`,
`math.prod`, and `csv.writerow` and `writerows`.
Over a generator, `set()` drops 16% of its instructions and `all()` 23%.
`str.join` and `bytes.join` do ask, reaching their elements through
`PySequence_Fast()`, which fills a list from the iterator. They take
`iter_sized()`, which is now the only way to ask. `iter_without_hint` is gone,
its callers being what `iter` already does.
Assisted-by: Claude
* Name the matrix jobs rather than let the matrix name them
A generated job name lists every value in the matrix entry, so `cargo check`
carried the booleans its dependencies and `skip_ssl` keys expand to, and the
snippets job carried its test arguments and timeout. Adding or removing a key
renames the check, which drops it from the required list until that list is
edited to match. Emptying `env_polluting_tests` renamed three checks this way.
`Run rust tests` and `clippy` keep the names they had. `cargo check` drops the
booleans from six of its nine, and the snippets job drops its arguments and
timeout from all three.
Assisted-by: Claude
* Keep a sequence iterator active when an element raises
`PositionIterInternal::_next` exhausted the iterator for any non-`Return`
result, so an error from `__getitem__` ended the walk. `iter_iternext()`
lets go of its sequence for `IndexError` and `StopIteration` alone, and
`PyIterReturn::from_getitem_result` has already turned the first of those
into the second, so only `StopIteration` exhausts now.
Both deque iterators keep exhausting on their own mutation guard, which
`deque_iternext()` does by zeroing the counter before it raises; they
share one step function for it, and the message it raises is lowercased
to match the three other sites in the module.
Assisted-by: Claude
* Name the caller in the bytes conversion TypeError
An object with no iteration protocol reached `PyObject_GetIter`'s "not
iterable" message, and `bytearray.extend()` reported bytes. Each entry
point now checks for the protocol first and names itself:
bytes(object()) cannot convert 'object' object to bytes
bytearray(object()) cannot convert 'object' object to bytearray
bytearray().extend(object()) can't extend bytearray with object
Assisted-by: Claude
* Take a strong count too large to hold as reachable
`start_gc_refs` clipped a count at `GC_REACHABLE - 1`, a number the
per-reference subtraction could still walk down to zero and collect a
live object. It now stores `GC_REACHABLE`, and `subtract_gc_ref` leaves
that value alone.
Assisted-by: Claude
* Raise AssertionError where a snippet asserts False
`assert False` is removed under `-O`, which the snippet suite may run.
Assisted-by: Claude
* Keep raising for a collection that moved under its iterator
A deque, set or dict iterator raised once and then read as spent. The
guard is sticky: `deque_iternext()` looks at the deque's state before the
count it keeps, and `dictiter_iternextkey()` and `setiter_iternext()`
write a size no collection can have, so every later call finds the same
thing and raises again. `dequereviter_next()` is the one exception,
looking at its count first, so it runs out after the first raise.
Both deque iterators now carry `dequeiterobject.counter` rather than
reading a length back from the deque, and the dict and set iterators
compare the size they captured against the collection's own every time
they are asked how much is left, which is what makes that answer nothing
from the moment the collection changes rather than only once the
iterator has raised.
The set's message is capitalized to match `setiter_iternext()`.
Measured against 3.14.6, for each of deque, reversed deque, set, dict
and a reversed dict view: the hint after the change, the error, the hint
after the error, and what a later call answers.
Assisted-by: Claude
* Give apt-get update a deadline to fail on
The step waits on `apt-get update`, which has no deadline of its own, so
a source that takes the connection and then stops answering holds the
job until the workflow's own timeout. The retry that disables the
Microsoft and azure-cli sources runs only when the update exits
non-zero, which a held connection never does; three jobs on this branch
sat on this step for 20 minutes to five and a half hours.
Each attempt is now bounded, and the transports are given a timeout, so
a source that stops answering reaches the retry.
Assisted-by: Claude
1816 lines
57 KiB
Python
Vendored
1816 lines
57 KiB
Python
Vendored
import collections
|
|
import collections.abc
|
|
import gc
|
|
import pickle
|
|
import random
|
|
import re
|
|
import string
|
|
import sys
|
|
import unittest
|
|
import weakref
|
|
from test import support
|
|
from test.support import import_helper
|
|
|
|
|
|
class CustomHash:
|
|
def __init__(self, hash):
|
|
self.hash = hash
|
|
def __hash__(self):
|
|
return self.hash
|
|
def __repr__(self):
|
|
return f'<CustomHash {self.hash} at {id(self):#x}>'
|
|
|
|
|
|
class DictTest(unittest.TestCase):
|
|
|
|
def test_invalid_keyword_arguments(self):
|
|
class Custom(dict):
|
|
pass
|
|
for invalid in {1 : 2}, Custom({1 : 2}):
|
|
with self.assertRaises(TypeError):
|
|
dict(**invalid)
|
|
with self.assertRaises(TypeError):
|
|
{}.update(**invalid)
|
|
|
|
def test_constructor(self):
|
|
# calling built-in types without argument must return empty
|
|
self.assertEqual(dict(), {})
|
|
self.assertIsNot(dict(), {})
|
|
|
|
def test_literal_constructor(self):
|
|
# check literal constructor for different sized dicts
|
|
# (to exercise the BUILD_MAP oparg).
|
|
for n in (0, 1, 6, 256, 400):
|
|
items = [(''.join(random.sample(string.ascii_letters, 8)), i)
|
|
for i in range(n)]
|
|
random.shuffle(items)
|
|
formatted_items = ('{!r}: {:d}'.format(k, v) for k, v in items)
|
|
dictliteral = '{' + ', '.join(formatted_items) + '}'
|
|
self.assertEqual(eval(dictliteral), dict(items))
|
|
|
|
def test_merge_operator(self):
|
|
|
|
a = {0: 0, 1: 1, 2: 1}
|
|
b = {1: 1, 2: 2, 3: 3}
|
|
|
|
c = a.copy()
|
|
c |= b
|
|
|
|
self.assertEqual(a | b, {0: 0, 1: 1, 2: 2, 3: 3})
|
|
self.assertEqual(c, {0: 0, 1: 1, 2: 2, 3: 3})
|
|
|
|
c = b.copy()
|
|
c |= a
|
|
|
|
self.assertEqual(b | a, {1: 1, 2: 1, 3: 3, 0: 0})
|
|
self.assertEqual(c, {1: 1, 2: 1, 3: 3, 0: 0})
|
|
|
|
c = a.copy()
|
|
c |= [(1, 1), (2, 2), (3, 3)]
|
|
|
|
self.assertEqual(c, {0: 0, 1: 1, 2: 2, 3: 3})
|
|
|
|
self.assertIs(a.__or__(None), NotImplemented)
|
|
self.assertIs(a.__or__(()), NotImplemented)
|
|
self.assertIs(a.__or__("BAD"), NotImplemented)
|
|
self.assertIs(a.__or__(""), NotImplemented)
|
|
|
|
self.assertRaises(TypeError, a.__ior__, None)
|
|
self.assertEqual(a.__ior__(()), {0: 0, 1: 1, 2: 1})
|
|
self.assertRaises(ValueError, a.__ior__, "BAD")
|
|
self.assertEqual(a.__ior__(""), {0: 0, 1: 1, 2: 1})
|
|
|
|
def test_bool(self):
|
|
self.assertIs(not {}, True)
|
|
self.assertTrue({1: 2})
|
|
self.assertIs(bool({}), False)
|
|
self.assertIs(bool({1: 2}), True)
|
|
|
|
def test_keys(self):
|
|
d = {}
|
|
self.assertEqual(set(d.keys()), set())
|
|
d = {'a': 1, 'b': 2}
|
|
k = d.keys()
|
|
self.assertEqual(set(k), {'a', 'b'})
|
|
self.assertIn('a', k)
|
|
self.assertIn('b', k)
|
|
self.assertIn('a', d)
|
|
self.assertIn('b', d)
|
|
self.assertRaises(TypeError, d.keys, None)
|
|
self.assertEqual(repr(dict(a=1).keys()), "dict_keys(['a'])")
|
|
|
|
def test_values(self):
|
|
d = {}
|
|
self.assertEqual(set(d.values()), set())
|
|
d = {1:2}
|
|
self.assertEqual(set(d.values()), {2})
|
|
self.assertRaises(TypeError, d.values, None)
|
|
self.assertEqual(repr(dict(a=1).values()), "dict_values([1])")
|
|
|
|
def test_items(self):
|
|
d = {}
|
|
self.assertEqual(set(d.items()), set())
|
|
|
|
d = {1:2}
|
|
self.assertEqual(set(d.items()), {(1, 2)})
|
|
self.assertRaises(TypeError, d.items, None)
|
|
self.assertEqual(repr(dict(a=1).items()), "dict_items([('a', 1)])")
|
|
|
|
def test_views_mapping(self):
|
|
mappingproxy = type(type.__dict__)
|
|
class Dict(dict):
|
|
pass
|
|
for cls in [dict, Dict]:
|
|
d = cls()
|
|
m1 = d.keys().mapping
|
|
m2 = d.values().mapping
|
|
m3 = d.items().mapping
|
|
|
|
for m in [m1, m2, m3]:
|
|
self.assertIsInstance(m, mappingproxy)
|
|
self.assertEqual(m, d)
|
|
|
|
d["foo"] = "bar"
|
|
|
|
for m in [m1, m2, m3]:
|
|
self.assertIsInstance(m, mappingproxy)
|
|
self.assertEqual(m, d)
|
|
|
|
def test_contains(self):
|
|
d = {}
|
|
self.assertNotIn('a', d)
|
|
self.assertFalse('a' in d)
|
|
self.assertTrue('a' not in d)
|
|
d = {'a': 1, 'b': 2}
|
|
self.assertIn('a', d)
|
|
self.assertIn('b', d)
|
|
self.assertNotIn('c', d)
|
|
|
|
self.assertRaises(TypeError, d.__contains__)
|
|
|
|
def test_len(self):
|
|
d = {}
|
|
self.assertEqual(len(d), 0)
|
|
d = {'a': 1, 'b': 2}
|
|
self.assertEqual(len(d), 2)
|
|
|
|
def test_getitem(self):
|
|
d = {'a': 1, 'b': 2}
|
|
self.assertEqual(d['a'], 1)
|
|
self.assertEqual(d['b'], 2)
|
|
d['c'] = 3
|
|
d['a'] = 4
|
|
self.assertEqual(d['c'], 3)
|
|
self.assertEqual(d['a'], 4)
|
|
del d['b']
|
|
self.assertEqual(d, {'a': 4, 'c': 3})
|
|
|
|
self.assertRaises(TypeError, d.__getitem__)
|
|
|
|
class BadEq(object):
|
|
def __eq__(self, other):
|
|
raise Exc()
|
|
def __hash__(self):
|
|
return 24
|
|
|
|
d = {}
|
|
d[BadEq()] = 42
|
|
self.assertRaises(KeyError, d.__getitem__, 23)
|
|
|
|
class Exc(Exception): pass
|
|
|
|
class BadHash(object):
|
|
fail = False
|
|
def __hash__(self):
|
|
if self.fail:
|
|
raise Exc()
|
|
else:
|
|
return 42
|
|
|
|
x = BadHash()
|
|
d[x] = 42
|
|
x.fail = True
|
|
self.assertRaises(Exc, d.__getitem__, x)
|
|
|
|
def test_clear(self):
|
|
d = {1:1, 2:2, 3:3}
|
|
d.clear()
|
|
self.assertEqual(d, {})
|
|
|
|
self.assertRaises(TypeError, d.clear, None)
|
|
|
|
def test_update(self):
|
|
d = {}
|
|
d.update({1:100})
|
|
d.update({2:20})
|
|
d.update({1:1, 2:2, 3:3})
|
|
self.assertEqual(d, {1:1, 2:2, 3:3})
|
|
|
|
d.update()
|
|
self.assertEqual(d, {1:1, 2:2, 3:3})
|
|
|
|
self.assertRaises((TypeError, AttributeError), d.update, None)
|
|
|
|
class SimpleUserDict:
|
|
def __init__(self):
|
|
self.d = {1:1, 2:2, 3:3}
|
|
def keys(self):
|
|
return self.d.keys()
|
|
def __getitem__(self, i):
|
|
return self.d[i]
|
|
d.clear()
|
|
d.update(SimpleUserDict())
|
|
self.assertEqual(d, {1:1, 2:2, 3:3})
|
|
|
|
class Exc(Exception): pass
|
|
|
|
d.clear()
|
|
class FailingUserDict:
|
|
def keys(self):
|
|
raise Exc
|
|
self.assertRaises(Exc, d.update, FailingUserDict())
|
|
|
|
class FailingUserDict:
|
|
def keys(self):
|
|
class BogonIter:
|
|
def __init__(self):
|
|
self.i = 1
|
|
def __iter__(self):
|
|
return self
|
|
def __next__(self):
|
|
if self.i:
|
|
self.i = 0
|
|
return 'a'
|
|
raise Exc
|
|
return BogonIter()
|
|
def __getitem__(self, key):
|
|
return key
|
|
self.assertRaises(Exc, d.update, FailingUserDict())
|
|
|
|
class FailingUserDict:
|
|
def keys(self):
|
|
class BogonIter:
|
|
def __init__(self):
|
|
self.i = ord('a')
|
|
def __iter__(self):
|
|
return self
|
|
def __next__(self):
|
|
if self.i <= ord('z'):
|
|
rtn = chr(self.i)
|
|
self.i += 1
|
|
return rtn
|
|
raise StopIteration
|
|
return BogonIter()
|
|
def __getitem__(self, key):
|
|
raise Exc
|
|
self.assertRaises(Exc, d.update, FailingUserDict())
|
|
|
|
class badseq(object):
|
|
def __iter__(self):
|
|
return self
|
|
def __next__(self):
|
|
raise Exc()
|
|
|
|
self.assertRaises(Exc, {}.update, badseq())
|
|
|
|
self.assertRaises(ValueError, {}.update, [(1, 2, 3)])
|
|
|
|
def test_update_type_error(self):
|
|
with self.assertRaises(TypeError) as cm:
|
|
{}.update([object() for _ in range(3)])
|
|
|
|
self.assertEqual(str(cm.exception), "object is not iterable")
|
|
self.assertEqual(
|
|
cm.exception.__notes__,
|
|
['Cannot convert dictionary update sequence element #0 to a sequence'],
|
|
)
|
|
|
|
def badgen():
|
|
yield "key"
|
|
raise TypeError("oops")
|
|
yield "value"
|
|
|
|
with self.assertRaises(TypeError) as cm:
|
|
dict([badgen() for _ in range(3)])
|
|
|
|
self.assertEqual(str(cm.exception), "oops")
|
|
self.assertEqual(
|
|
cm.exception.__notes__,
|
|
['Cannot convert dictionary update sequence element #0 to a sequence'],
|
|
)
|
|
|
|
def test_update_shared_keys(self):
|
|
class MyClass: pass
|
|
|
|
# Subclass str to enable us to create an object during the
|
|
# dict.update() call.
|
|
class MyStr(str):
|
|
def __hash__(self):
|
|
return super().__hash__()
|
|
|
|
def __eq__(self, other):
|
|
# Create an object that shares the same PyDictKeysObject as
|
|
# obj.__dict__.
|
|
obj2 = MyClass()
|
|
obj2.a = "a"
|
|
obj2.b = "b"
|
|
obj2.c = "c"
|
|
return super().__eq__(other)
|
|
|
|
obj = MyClass()
|
|
obj.a = "a"
|
|
obj.b = "b"
|
|
|
|
x = {}
|
|
x[MyStr("a")] = MyStr("a")
|
|
|
|
# gh-132617: this previously raised "dict mutated during update" error
|
|
x.update(obj.__dict__)
|
|
|
|
self.assertEqual(x, {
|
|
MyStr("a"): "a",
|
|
"b": "b",
|
|
})
|
|
|
|
def test_fromkeys(self):
|
|
self.assertEqual(dict.fromkeys('abc'), {'a':None, 'b':None, 'c':None})
|
|
d = {}
|
|
self.assertIsNot(d.fromkeys('abc'), d)
|
|
self.assertEqual(d.fromkeys('abc'), {'a':None, 'b':None, 'c':None})
|
|
self.assertEqual(d.fromkeys((4,5),0), {4:0, 5:0})
|
|
self.assertEqual(d.fromkeys([]), {})
|
|
def g():
|
|
yield 1
|
|
self.assertEqual(d.fromkeys(g()), {1:None})
|
|
self.assertRaises(TypeError, {}.fromkeys, 3)
|
|
class dictlike(dict): pass
|
|
self.assertEqual(dictlike.fromkeys('a'), {'a':None})
|
|
self.assertEqual(dictlike().fromkeys('a'), {'a':None})
|
|
self.assertIsInstance(dictlike.fromkeys('a'), dictlike)
|
|
self.assertIsInstance(dictlike().fromkeys('a'), dictlike)
|
|
class mydict(dict):
|
|
def __new__(cls):
|
|
return collections.UserDict()
|
|
ud = mydict.fromkeys('ab')
|
|
self.assertEqual(ud, {'a':None, 'b':None})
|
|
self.assertIsInstance(ud, collections.UserDict)
|
|
self.assertRaises(TypeError, dict.fromkeys)
|
|
|
|
class Exc(Exception): pass
|
|
|
|
class baddict1(dict):
|
|
def __init__(self):
|
|
raise Exc()
|
|
|
|
self.assertRaises(Exc, baddict1.fromkeys, [1])
|
|
|
|
class BadSeq(object):
|
|
def __iter__(self):
|
|
return self
|
|
def __next__(self):
|
|
raise Exc()
|
|
|
|
self.assertRaises(Exc, dict.fromkeys, BadSeq())
|
|
|
|
class baddict2(dict):
|
|
def __setitem__(self, key, value):
|
|
raise Exc()
|
|
|
|
self.assertRaises(Exc, baddict2.fromkeys, [1])
|
|
|
|
# test fast path for dictionary inputs
|
|
res = dict(zip(range(6), [0]*6))
|
|
d = dict(zip(range(6), range(6)))
|
|
self.assertEqual(dict.fromkeys(d, 0), res)
|
|
# test fast path for set inputs
|
|
d = set(range(6))
|
|
self.assertEqual(dict.fromkeys(d, 0), res)
|
|
# test slow path for other iterable inputs
|
|
d = list(range(6))
|
|
self.assertEqual(dict.fromkeys(d, 0), res)
|
|
|
|
# test fast path when object's constructor returns large non-empty dict
|
|
class baddict3(dict):
|
|
def __new__(cls):
|
|
return d
|
|
d = {i : i for i in range(1000)}
|
|
res = d.copy()
|
|
res.update(a=None, b=None, c=None)
|
|
self.assertEqual(baddict3.fromkeys({"a", "b", "c"}), res)
|
|
|
|
# test slow path when object is a proper subclass of dict
|
|
class baddict4(dict):
|
|
def __init__(self):
|
|
dict.__init__(self, d)
|
|
d = {i : i for i in range(1000)}
|
|
res = d.copy()
|
|
res.update(a=None, b=None, c=None)
|
|
self.assertEqual(baddict4.fromkeys({"a", "b", "c"}), res)
|
|
|
|
def test_copy(self):
|
|
d = {1: 1, 2: 2, 3: 3}
|
|
self.assertIsNot(d.copy(), d)
|
|
self.assertEqual(d.copy(), d)
|
|
self.assertEqual(d.copy(), {1: 1, 2: 2, 3: 3})
|
|
|
|
copy = d.copy()
|
|
d[4] = 4
|
|
self.assertNotEqual(copy, d)
|
|
|
|
self.assertEqual({}.copy(), {})
|
|
self.assertRaises(TypeError, d.copy, None)
|
|
|
|
def test_copy_fuzz(self):
|
|
for dict_size in [10, 100, 1000, 10000, 100000]:
|
|
dict_size = random.randrange(
|
|
dict_size // 2, dict_size + dict_size // 2)
|
|
with self.subTest(dict_size=dict_size):
|
|
d = {}
|
|
for i in range(dict_size):
|
|
d[i] = i
|
|
|
|
d2 = d.copy()
|
|
self.assertIsNot(d2, d)
|
|
self.assertEqual(d, d2)
|
|
d2['key'] = 'value'
|
|
self.assertNotEqual(d, d2)
|
|
self.assertEqual(len(d2), len(d) + 1)
|
|
|
|
def test_copy_maintains_tracking(self):
|
|
class A:
|
|
pass
|
|
|
|
key = A()
|
|
|
|
for d in ({}, {'a': 1}, {key: 'val'}):
|
|
d2 = d.copy()
|
|
self.assertEqual(gc.is_tracked(d), gc.is_tracked(d2))
|
|
|
|
def test_copy_noncompact(self):
|
|
# Dicts don't compact themselves on del/pop operations.
|
|
# Copy will use a slow merging strategy that produces
|
|
# a compacted copy when roughly 33% of dict is a non-used
|
|
# keys-space (to optimize memory footprint).
|
|
# In this test we want to hit the slow/compacting
|
|
# branch of dict.copy() and make sure it works OK.
|
|
d = {k: k for k in range(1000)}
|
|
for k in range(950):
|
|
del d[k]
|
|
d2 = d.copy()
|
|
self.assertEqual(d2, d)
|
|
|
|
def test_get(self):
|
|
d = {}
|
|
self.assertIs(d.get('c'), None)
|
|
self.assertEqual(d.get('c', 3), 3)
|
|
d = {'a': 1, 'b': 2}
|
|
self.assertIs(d.get('c'), None)
|
|
self.assertEqual(d.get('c', 3), 3)
|
|
self.assertEqual(d.get('a'), 1)
|
|
self.assertEqual(d.get('a', 3), 1)
|
|
self.assertRaises(TypeError, d.get)
|
|
self.assertRaises(TypeError, d.get, None, None, None)
|
|
|
|
def test_setdefault(self):
|
|
# dict.setdefault()
|
|
d = {}
|
|
self.assertIs(d.setdefault('key0'), None)
|
|
d.setdefault('key0', [])
|
|
self.assertIs(d.setdefault('key0'), None)
|
|
d.setdefault('key', []).append(3)
|
|
self.assertEqual(d['key'][0], 3)
|
|
d.setdefault('key', []).append(4)
|
|
self.assertEqual(len(d['key']), 2)
|
|
self.assertRaises(TypeError, d.setdefault)
|
|
|
|
class Exc(Exception): pass
|
|
|
|
class BadHash(object):
|
|
fail = False
|
|
def __hash__(self):
|
|
if self.fail:
|
|
raise Exc()
|
|
else:
|
|
return 42
|
|
|
|
x = BadHash()
|
|
d[x] = 42
|
|
x.fail = True
|
|
self.assertRaises(Exc, d.setdefault, x, [])
|
|
|
|
def test_setdefault_atomic(self):
|
|
# Issue #13521: setdefault() calls __hash__ and __eq__ only once.
|
|
class Hashed(object):
|
|
def __init__(self):
|
|
self.hash_count = 0
|
|
self.eq_count = 0
|
|
def __hash__(self):
|
|
self.hash_count += 1
|
|
return 42
|
|
def __eq__(self, other):
|
|
self.eq_count += 1
|
|
return id(self) == id(other)
|
|
hashed1 = Hashed()
|
|
y = {hashed1: 5}
|
|
hashed2 = Hashed()
|
|
y.setdefault(hashed2, [])
|
|
self.assertEqual(hashed1.hash_count, 1)
|
|
self.assertEqual(hashed2.hash_count, 1)
|
|
self.assertEqual(hashed1.eq_count + hashed2.eq_count, 1)
|
|
|
|
def test_setitem_atomic_at_resize(self):
|
|
class Hashed(object):
|
|
def __init__(self):
|
|
self.hash_count = 0
|
|
self.eq_count = 0
|
|
def __hash__(self):
|
|
self.hash_count += 1
|
|
return 42
|
|
def __eq__(self, other):
|
|
self.eq_count += 1
|
|
return id(self) == id(other)
|
|
hashed1 = Hashed()
|
|
# 5 items
|
|
y = {hashed1: 5, 0: 0, 1: 1, 2: 2, 3: 3}
|
|
hashed2 = Hashed()
|
|
# 6th item forces a resize
|
|
y[hashed2] = []
|
|
self.assertEqual(hashed1.hash_count, 1)
|
|
self.assertEqual(hashed2.hash_count, 1)
|
|
self.assertEqual(hashed1.eq_count + hashed2.eq_count, 1)
|
|
|
|
def test_popitem(self):
|
|
# dict.popitem()
|
|
for copymode in -1, +1:
|
|
# -1: b has same structure as a
|
|
# +1: b is a.copy()
|
|
for log2size in range(12):
|
|
size = 2**log2size
|
|
a = {}
|
|
b = {}
|
|
for i in range(size):
|
|
a[repr(i)] = i
|
|
if copymode < 0:
|
|
b[repr(i)] = i
|
|
if copymode > 0:
|
|
b = a.copy()
|
|
for i in range(size):
|
|
ka, va = ta = a.popitem()
|
|
self.assertEqual(va, int(ka))
|
|
kb, vb = tb = b.popitem()
|
|
self.assertEqual(vb, int(kb))
|
|
self.assertFalse(copymode < 0 and ta != tb)
|
|
self.assertFalse(a)
|
|
self.assertFalse(b)
|
|
|
|
d = {}
|
|
self.assertRaises(KeyError, d.popitem)
|
|
|
|
def test_pop(self):
|
|
# Tests for pop with specified key
|
|
d = {}
|
|
k, v = 'abc', 'def'
|
|
d[k] = v
|
|
self.assertRaises(KeyError, d.pop, 'ghi')
|
|
|
|
self.assertEqual(d.pop(k), v)
|
|
self.assertEqual(len(d), 0)
|
|
|
|
self.assertRaises(KeyError, d.pop, k)
|
|
|
|
self.assertEqual(d.pop(k, v), v)
|
|
d[k] = v
|
|
self.assertEqual(d.pop(k, 1), v)
|
|
|
|
self.assertRaises(TypeError, d.pop)
|
|
|
|
class Exc(Exception): pass
|
|
|
|
class BadHash(object):
|
|
fail = False
|
|
def __hash__(self):
|
|
if self.fail:
|
|
raise Exc()
|
|
else:
|
|
return 42
|
|
|
|
x = BadHash()
|
|
d[x] = 42
|
|
x.fail = True
|
|
self.assertRaises(Exc, d.pop, x)
|
|
|
|
def test_mutating_iteration(self):
|
|
# changing dict size during iteration
|
|
d = {}
|
|
d[1] = 1
|
|
with self.assertRaises(RuntimeError):
|
|
for i in d:
|
|
d[i+1] = 1
|
|
|
|
def test_mutating_iteration_delete(self):
|
|
# change dict content during iteration
|
|
d = {}
|
|
d[0] = 0
|
|
with self.assertRaises(RuntimeError):
|
|
for i in d:
|
|
del d[0]
|
|
d[0] = 0
|
|
|
|
def test_mutating_iteration_delete_over_values(self):
|
|
# change dict content during iteration
|
|
d = {}
|
|
d[0] = 0
|
|
with self.assertRaises(RuntimeError):
|
|
for i in d.values():
|
|
del d[0]
|
|
d[0] = 0
|
|
|
|
def test_mutating_iteration_delete_over_items(self):
|
|
# change dict content during iteration
|
|
d = {}
|
|
d[0] = 0
|
|
with self.assertRaises(RuntimeError):
|
|
for i in d.items():
|
|
del d[0]
|
|
d[0] = 0
|
|
|
|
def test_mutating_lookup(self):
|
|
# changing dict during a lookup (issue #14417)
|
|
class NastyKey:
|
|
mutate_dict = None
|
|
|
|
def __init__(self, value):
|
|
self.value = value
|
|
|
|
def __hash__(self):
|
|
# hash collision!
|
|
return 1
|
|
|
|
def __eq__(self, other):
|
|
if NastyKey.mutate_dict:
|
|
mydict, key = NastyKey.mutate_dict
|
|
NastyKey.mutate_dict = None
|
|
del mydict[key]
|
|
return self.value == other.value
|
|
|
|
key1 = NastyKey(1)
|
|
key2 = NastyKey(2)
|
|
d = {key1: 1}
|
|
NastyKey.mutate_dict = (d, key1)
|
|
d[key2] = 2
|
|
self.assertEqual(d, {key2: 2})
|
|
|
|
def test_repr(self):
|
|
d = {}
|
|
self.assertEqual(repr(d), '{}')
|
|
d[1] = 2
|
|
self.assertEqual(repr(d), '{1: 2}')
|
|
d = {}
|
|
d[1] = d
|
|
self.assertEqual(repr(d), '{1: {...}}')
|
|
|
|
class Exc(Exception): pass
|
|
|
|
class BadRepr(object):
|
|
def __repr__(self):
|
|
raise Exc()
|
|
|
|
d = {1: BadRepr()}
|
|
self.assertRaises(Exc, repr, d)
|
|
|
|
@unittest.skip("TODO: RUSTPYTHON; segfault")
|
|
@support.skip_wasi_stack_overflow()
|
|
@support.skip_emscripten_stack_overflow()
|
|
def test_repr_deep(self):
|
|
d = {}
|
|
for i in range(support.exceeds_recursion_limit()):
|
|
d = {1: d}
|
|
self.assertRaises(RecursionError, repr, d)
|
|
|
|
def test_eq(self):
|
|
self.assertEqual({}, {})
|
|
self.assertEqual({1: 2}, {1: 2})
|
|
|
|
class Exc(Exception): pass
|
|
|
|
class BadCmp(object):
|
|
def __eq__(self, other):
|
|
raise Exc()
|
|
def __hash__(self):
|
|
return 1
|
|
|
|
d1 = {BadCmp(): 1}
|
|
d2 = {1: 1}
|
|
|
|
with self.assertRaises(Exc):
|
|
d1 == d2
|
|
|
|
def test_keys_contained(self):
|
|
self.helper_keys_contained(lambda x: x.keys())
|
|
self.helper_keys_contained(lambda x: x.items())
|
|
|
|
def helper_keys_contained(self, fn):
|
|
# Test rich comparisons against dict key views, which should behave the
|
|
# same as sets.
|
|
empty = fn(dict())
|
|
empty2 = fn(dict())
|
|
smaller = fn({1:1, 2:2})
|
|
larger = fn({1:1, 2:2, 3:3})
|
|
larger2 = fn({1:1, 2:2, 3:3})
|
|
larger3 = fn({4:1, 2:2, 3:3})
|
|
|
|
self.assertTrue(smaller < larger)
|
|
self.assertTrue(smaller <= larger)
|
|
self.assertTrue(larger > smaller)
|
|
self.assertTrue(larger >= smaller)
|
|
|
|
self.assertFalse(smaller >= larger)
|
|
self.assertFalse(smaller > larger)
|
|
self.assertFalse(larger <= smaller)
|
|
self.assertFalse(larger < smaller)
|
|
|
|
self.assertFalse(smaller < larger3)
|
|
self.assertFalse(smaller <= larger3)
|
|
self.assertFalse(larger3 > smaller)
|
|
self.assertFalse(larger3 >= smaller)
|
|
|
|
# Inequality strictness
|
|
self.assertTrue(larger2 >= larger)
|
|
self.assertTrue(larger2 <= larger)
|
|
self.assertFalse(larger2 > larger)
|
|
self.assertFalse(larger2 < larger)
|
|
|
|
self.assertTrue(larger == larger2)
|
|
self.assertTrue(smaller != larger)
|
|
|
|
# There is an optimization on the zero-element case.
|
|
self.assertTrue(empty == empty2)
|
|
self.assertFalse(empty != empty2)
|
|
self.assertFalse(empty == smaller)
|
|
self.assertTrue(empty != smaller)
|
|
|
|
# With the same size, an elementwise compare happens
|
|
self.assertTrue(larger != larger3)
|
|
self.assertFalse(larger == larger3)
|
|
|
|
def test_errors_in_view_containment_check(self):
|
|
class C:
|
|
def __eq__(self, other):
|
|
raise RuntimeError
|
|
|
|
d1 = {1: C()}
|
|
d2 = {1: C()}
|
|
with self.assertRaises(RuntimeError):
|
|
d1.items() == d2.items()
|
|
with self.assertRaises(RuntimeError):
|
|
d1.items() != d2.items()
|
|
with self.assertRaises(RuntimeError):
|
|
d1.items() <= d2.items()
|
|
with self.assertRaises(RuntimeError):
|
|
d1.items() >= d2.items()
|
|
|
|
d3 = {1: C(), 2: C()}
|
|
with self.assertRaises(RuntimeError):
|
|
d2.items() < d3.items()
|
|
with self.assertRaises(RuntimeError):
|
|
d3.items() > d2.items()
|
|
|
|
def test_dictview_set_operations_on_keys(self):
|
|
k1 = {1:1, 2:2}.keys()
|
|
k2 = {1:1, 2:2, 3:3}.keys()
|
|
k3 = {4:4}.keys()
|
|
|
|
self.assertEqual(k1 - k2, set())
|
|
self.assertEqual(k1 - k3, {1,2})
|
|
self.assertEqual(k2 - k1, {3})
|
|
self.assertEqual(k3 - k1, {4})
|
|
self.assertEqual(k1 & k2, {1,2})
|
|
self.assertEqual(k1 & k3, set())
|
|
self.assertEqual(k1 | k2, {1,2,3})
|
|
self.assertEqual(k1 ^ k2, {3})
|
|
self.assertEqual(k1 ^ k3, {1,2,4})
|
|
|
|
def test_dictview_set_operations_on_items(self):
|
|
k1 = {1:1, 2:2}.items()
|
|
k2 = {1:1, 2:2, 3:3}.items()
|
|
k3 = {4:4}.items()
|
|
|
|
self.assertEqual(k1 - k2, set())
|
|
self.assertEqual(k1 - k3, {(1,1), (2,2)})
|
|
self.assertEqual(k2 - k1, {(3,3)})
|
|
self.assertEqual(k3 - k1, {(4,4)})
|
|
self.assertEqual(k1 & k2, {(1,1), (2,2)})
|
|
self.assertEqual(k1 & k3, set())
|
|
self.assertEqual(k1 | k2, {(1,1), (2,2), (3,3)})
|
|
self.assertEqual(k1 ^ k2, {(3,3)})
|
|
self.assertEqual(k1 ^ k3, {(1,1), (2,2), (4,4)})
|
|
|
|
def test_items_symmetric_difference(self):
|
|
rr = random.randrange
|
|
for _ in range(100):
|
|
left = {x:rr(3) for x in range(20) if rr(2)}
|
|
right = {x:rr(3) for x in range(20) if rr(2)}
|
|
with self.subTest(left=left, right=right):
|
|
expected = set(left.items()) ^ set(right.items())
|
|
actual = left.items() ^ right.items()
|
|
self.assertEqual(actual, expected)
|
|
|
|
def test_dictview_mixed_set_operations(self):
|
|
# Just a few for .keys()
|
|
self.assertTrue({1:1}.keys() == {1})
|
|
self.assertTrue({1} == {1:1}.keys())
|
|
self.assertEqual({1:1}.keys() | {2}, {1, 2})
|
|
self.assertEqual({2} | {1:1}.keys(), {1, 2})
|
|
# And a few for .items()
|
|
self.assertTrue({1:1}.items() == {(1,1)})
|
|
self.assertTrue({(1,1)} == {1:1}.items())
|
|
self.assertEqual({1:1}.items() | {2}, {(1,1), 2})
|
|
self.assertEqual({2} | {1:1}.items(), {(1,1), 2})
|
|
|
|
def test_missing(self):
|
|
# Make sure dict doesn't have a __missing__ method
|
|
self.assertNotHasAttr(dict, "__missing__")
|
|
self.assertNotHasAttr({}, "__missing__")
|
|
# Test several cases:
|
|
# (D) subclass defines __missing__ method returning a value
|
|
# (E) subclass defines __missing__ method raising RuntimeError
|
|
# (F) subclass sets __missing__ instance variable (no effect)
|
|
# (G) subclass doesn't define __missing__ at all
|
|
class D(dict):
|
|
def __missing__(self, key):
|
|
return 42
|
|
d = D({1: 2, 3: 4})
|
|
self.assertEqual(d[1], 2)
|
|
self.assertEqual(d[3], 4)
|
|
self.assertNotIn(2, d)
|
|
self.assertNotIn(2, d.keys())
|
|
self.assertEqual(d[2], 42)
|
|
|
|
class E(dict):
|
|
def __missing__(self, key):
|
|
raise RuntimeError(key)
|
|
e = E()
|
|
with self.assertRaises(RuntimeError) as c:
|
|
e[42]
|
|
self.assertEqual(c.exception.args, (42,))
|
|
|
|
class F(dict):
|
|
def __init__(self):
|
|
# An instance variable __missing__ should have no effect
|
|
self.__missing__ = lambda key: None
|
|
f = F()
|
|
with self.assertRaises(KeyError) as c:
|
|
f[42]
|
|
self.assertEqual(c.exception.args, (42,))
|
|
|
|
class G(dict):
|
|
pass
|
|
g = G()
|
|
with self.assertRaises(KeyError) as c:
|
|
g[42]
|
|
self.assertEqual(c.exception.args, (42,))
|
|
|
|
def test_tuple_keyerror(self):
|
|
# SF #1576657
|
|
d = {}
|
|
with self.assertRaises(KeyError) as c:
|
|
d[(1,)]
|
|
self.assertEqual(c.exception.args, ((1,),))
|
|
|
|
def test_bad_key(self):
|
|
# Dictionary lookups should fail if __eq__() raises an exception.
|
|
class CustomException(Exception):
|
|
pass
|
|
|
|
class BadDictKey:
|
|
def __hash__(self):
|
|
return hash(self.__class__)
|
|
|
|
def __eq__(self, other):
|
|
if isinstance(other, self.__class__):
|
|
raise CustomException
|
|
return other
|
|
|
|
d = {}
|
|
x1 = BadDictKey()
|
|
x2 = BadDictKey()
|
|
d[x1] = 1
|
|
for stmt in ['d[x2] = 2',
|
|
'z = d[x2]',
|
|
'x2 in d',
|
|
'd.get(x2)',
|
|
'd.setdefault(x2, 42)',
|
|
'd.pop(x2)',
|
|
'd.update({x2: 2})']:
|
|
with self.assertRaises(CustomException):
|
|
exec(stmt, locals())
|
|
|
|
def test_resize1(self):
|
|
# Dict resizing bug, found by Jack Jansen in 2.2 CVS development.
|
|
# This version got an assert failure in debug build, infinite loop in
|
|
# release build. Unfortunately, provoking this kind of stuff requires
|
|
# a mix of inserts and deletes hitting exactly the right hash codes in
|
|
# exactly the right order, and I can't think of a randomized approach
|
|
# that would be *likely* to hit a failing case in reasonable time.
|
|
|
|
d = {}
|
|
for i in range(5):
|
|
d[i] = i
|
|
for i in range(5):
|
|
del d[i]
|
|
for i in range(5, 9): # i==8 was the problem
|
|
d[i] = i
|
|
|
|
def test_resize2(self):
|
|
# Another dict resizing bug (SF bug #1456209).
|
|
# This caused Segmentation faults or Illegal instructions.
|
|
|
|
class X(object):
|
|
def __hash__(self):
|
|
return 5
|
|
def __eq__(self, other):
|
|
if resizing:
|
|
d.clear()
|
|
return False
|
|
d = {}
|
|
resizing = False
|
|
d[X()] = 1
|
|
d[X()] = 2
|
|
d[X()] = 3
|
|
d[X()] = 4
|
|
d[X()] = 5
|
|
# now trigger a resize
|
|
resizing = True
|
|
d[9] = 6
|
|
|
|
def test_empty_presized_dict_in_freelist(self):
|
|
# Bug #3537: if an empty but presized dict with a size larger
|
|
# than 7 was in the freelist, it triggered an assertion failure
|
|
with self.assertRaises(ZeroDivisionError):
|
|
d = {'a': 1 // 0, 'b': None, 'c': None, 'd': None, 'e': None,
|
|
'f': None, 'g': None, 'h': None}
|
|
d = {}
|
|
|
|
@unittest.expectedFailure # TODO: RUSTPYTHON
|
|
def test_container_iterator(self):
|
|
# Bug #3680: tp_traverse was not implemented for dictiter and
|
|
# dictview objects.
|
|
class C(object):
|
|
pass
|
|
views = (dict.items, dict.values, dict.keys)
|
|
for v in views:
|
|
obj = C()
|
|
ref = weakref.ref(obj)
|
|
container = {obj: 1}
|
|
obj.v = v(container)
|
|
obj.x = iter(obj.v)
|
|
del obj, container
|
|
gc.collect()
|
|
self.assertIs(ref(), None, "Cycle was not collected")
|
|
|
|
def make_shared_key_dict(self, n):
|
|
class C:
|
|
pass
|
|
|
|
dicts = []
|
|
for i in range(n):
|
|
a = C()
|
|
a.x, a.y, a.z = 1, 2, 3
|
|
dicts.append(a.__dict__)
|
|
|
|
return dicts
|
|
|
|
@support.cpython_only
|
|
def test_splittable_setdefault(self):
|
|
"""split table must keep correct insertion
|
|
order when attributes are adding using setdefault()"""
|
|
a, b = self.make_shared_key_dict(2)
|
|
|
|
a['a'] = 1
|
|
size_a = sys.getsizeof(a)
|
|
a['b'] = 2
|
|
b.setdefault('b', 2)
|
|
size_b = sys.getsizeof(b)
|
|
b['a'] = 1
|
|
|
|
self.assertEqual(list(a), ['x', 'y', 'z', 'a', 'b'])
|
|
self.assertEqual(list(b), ['x', 'y', 'z', 'b', 'a'])
|
|
|
|
@support.cpython_only
|
|
def test_splittable_del(self):
|
|
"""split table must be combined when del d[k]"""
|
|
a, b = self.make_shared_key_dict(2)
|
|
|
|
orig_size = sys.getsizeof(a)
|
|
|
|
del a['y'] # split table is combined
|
|
with self.assertRaises(KeyError):
|
|
del a['y']
|
|
|
|
self.assertEqual(list(a), ['x', 'z'])
|
|
self.assertEqual(list(b), ['x', 'y', 'z'])
|
|
|
|
# Two dicts have different insertion order.
|
|
a['y'] = 42
|
|
self.assertEqual(list(a), ['x', 'z', 'y'])
|
|
self.assertEqual(list(b), ['x', 'y', 'z'])
|
|
|
|
@support.cpython_only
|
|
def test_splittable_pop(self):
|
|
a, b = self.make_shared_key_dict(2)
|
|
|
|
a.pop('y')
|
|
with self.assertRaises(KeyError):
|
|
a.pop('y')
|
|
|
|
self.assertEqual(list(a), ['x', 'z'])
|
|
self.assertEqual(list(b), ['x', 'y', 'z'])
|
|
|
|
# Two dicts have different insertion order.
|
|
a['y'] = 42
|
|
self.assertEqual(list(a), ['x', 'z', 'y'])
|
|
self.assertEqual(list(b), ['x', 'y', 'z'])
|
|
|
|
@support.cpython_only
|
|
def test_splittable_pop_pending(self):
|
|
"""pop a pending key in a split table should not crash"""
|
|
a, b = self.make_shared_key_dict(2)
|
|
|
|
a['a'] = 4
|
|
with self.assertRaises(KeyError):
|
|
b.pop('a')
|
|
|
|
@support.cpython_only
|
|
def test_splittable_popitem(self):
|
|
"""split table must be combined when d.popitem()"""
|
|
a, b = self.make_shared_key_dict(2)
|
|
|
|
orig_size = sys.getsizeof(a)
|
|
|
|
item = a.popitem() # split table is combined
|
|
self.assertEqual(item, ('z', 3))
|
|
with self.assertRaises(KeyError):
|
|
del a['z']
|
|
|
|
self.assertGreater(sys.getsizeof(a), orig_size)
|
|
self.assertEqual(list(a), ['x', 'y'])
|
|
self.assertEqual(list(b), ['x', 'y', 'z'])
|
|
|
|
@support.cpython_only
|
|
def test_splittable_update(self):
|
|
"""dict.update(other) must preserve order in other."""
|
|
class C:
|
|
def __init__(self, order):
|
|
if order:
|
|
self.a, self.b, self.c = 1, 2, 3
|
|
else:
|
|
self.c, self.b, self.a = 1, 2, 3
|
|
o = C(True)
|
|
o = C(False) # o.__dict__ has reversed order.
|
|
self.assertEqual(list(o.__dict__), ["c", "b", "a"])
|
|
|
|
d = {}
|
|
d.update(o.__dict__)
|
|
self.assertEqual(list(d), ["c", "b", "a"])
|
|
|
|
@support.cpython_only
|
|
def test_splittable_to_generic_combinedtable(self):
|
|
"""split table must be correctly resized and converted to generic combined table"""
|
|
class C:
|
|
pass
|
|
|
|
a = C()
|
|
a.x = 1
|
|
d = a.__dict__
|
|
d[2] = 2 # split table is resized to a generic combined table
|
|
|
|
self.assertEqual(list(d), ['x', 2])
|
|
|
|
def test_iterator_pickling(self):
|
|
for proto in range(pickle.HIGHEST_PROTOCOL + 1):
|
|
data = {1:"a", 2:"b", 3:"c"}
|
|
it = iter(data)
|
|
d = pickle.dumps(it, proto)
|
|
it = pickle.loads(d)
|
|
self.assertEqual(list(it), list(data))
|
|
|
|
it = pickle.loads(d)
|
|
try:
|
|
drop = next(it)
|
|
except StopIteration:
|
|
continue
|
|
d = pickle.dumps(it, proto)
|
|
it = pickle.loads(d)
|
|
del data[drop]
|
|
self.assertEqual(list(it), list(data))
|
|
|
|
def test_itemiterator_pickling(self):
|
|
for proto in range(pickle.HIGHEST_PROTOCOL + 1):
|
|
data = {1:"a", 2:"b", 3:"c"}
|
|
# dictviews aren't picklable, only their iterators
|
|
itorg = iter(data.items())
|
|
d = pickle.dumps(itorg, proto)
|
|
it = pickle.loads(d)
|
|
# note that the type of the unpickled iterator
|
|
# is not necessarily the same as the original. It is
|
|
# merely an object supporting the iterator protocol, yielding
|
|
# the same objects as the original one.
|
|
# self.assertEqual(type(itorg), type(it))
|
|
self.assertIsInstance(it, collections.abc.Iterator)
|
|
self.assertEqual(dict(it), data)
|
|
|
|
it = pickle.loads(d)
|
|
drop = next(it)
|
|
d = pickle.dumps(it, proto)
|
|
it = pickle.loads(d)
|
|
del data[drop[0]]
|
|
self.assertEqual(dict(it), data)
|
|
|
|
def test_valuesiterator_pickling(self):
|
|
for proto in range(pickle.HIGHEST_PROTOCOL + 1):
|
|
data = {1:"a", 2:"b", 3:"c"}
|
|
# data.values() isn't picklable, only its iterator
|
|
it = iter(data.values())
|
|
d = pickle.dumps(it, proto)
|
|
it = pickle.loads(d)
|
|
self.assertEqual(list(it), list(data.values()))
|
|
|
|
it = pickle.loads(d)
|
|
drop = next(it)
|
|
d = pickle.dumps(it, proto)
|
|
it = pickle.loads(d)
|
|
values = list(it) + [drop]
|
|
self.assertEqual(sorted(values), sorted(list(data.values())))
|
|
|
|
def test_reverseiterator_pickling(self):
|
|
for proto in range(pickle.HIGHEST_PROTOCOL + 1):
|
|
data = {1:"a", 2:"b", 3:"c"}
|
|
it = reversed(data)
|
|
d = pickle.dumps(it, proto)
|
|
it = pickle.loads(d)
|
|
self.assertEqual(list(it), list(reversed(data)))
|
|
|
|
it = pickle.loads(d)
|
|
try:
|
|
drop = next(it)
|
|
except StopIteration:
|
|
continue
|
|
d = pickle.dumps(it, proto)
|
|
it = pickle.loads(d)
|
|
del data[drop]
|
|
self.assertEqual(list(it), list(reversed(data)))
|
|
|
|
def test_reverseitemiterator_pickling(self):
|
|
for proto in range(pickle.HIGHEST_PROTOCOL + 1):
|
|
data = {1:"a", 2:"b", 3:"c"}
|
|
# dictviews aren't picklable, only their iterators
|
|
itorg = reversed(data.items())
|
|
d = pickle.dumps(itorg, proto)
|
|
it = pickle.loads(d)
|
|
# note that the type of the unpickled iterator
|
|
# is not necessarily the same as the original. It is
|
|
# merely an object supporting the iterator protocol, yielding
|
|
# the same objects as the original one.
|
|
# self.assertEqual(type(itorg), type(it))
|
|
self.assertIsInstance(it, collections.abc.Iterator)
|
|
self.assertEqual(dict(it), data)
|
|
|
|
it = pickle.loads(d)
|
|
drop = next(it)
|
|
d = pickle.dumps(it, proto)
|
|
it = pickle.loads(d)
|
|
del data[drop[0]]
|
|
self.assertEqual(dict(it), data)
|
|
|
|
def test_reversevaluesiterator_pickling(self):
|
|
for proto in range(pickle.HIGHEST_PROTOCOL + 1):
|
|
data = {1:"a", 2:"b", 3:"c"}
|
|
# data.values() isn't picklable, only its iterator
|
|
it = reversed(data.values())
|
|
d = pickle.dumps(it, proto)
|
|
it = pickle.loads(d)
|
|
self.assertEqual(list(it), list(reversed(data.values())))
|
|
|
|
it = pickle.loads(d)
|
|
drop = next(it)
|
|
d = pickle.dumps(it, proto)
|
|
it = pickle.loads(d)
|
|
values = list(it) + [drop]
|
|
self.assertEqual(sorted(values), sorted(data.values()))
|
|
|
|
def test_instance_dict_getattr_str_subclass(self):
|
|
class Foo:
|
|
def __init__(self, msg):
|
|
self.msg = msg
|
|
f = Foo('123')
|
|
class _str(str):
|
|
pass
|
|
self.assertEqual(f.msg, getattr(f, _str('msg')))
|
|
self.assertEqual(f.msg, f.__dict__[_str('msg')])
|
|
|
|
def test_object_set_item_single_instance_non_str_key(self):
|
|
class Foo: pass
|
|
f = Foo()
|
|
f.__dict__[1] = 1
|
|
f.a = 'a'
|
|
self.assertEqual(f.__dict__, {1:1, 'a':'a'})
|
|
|
|
def check_reentrant_insertion(self, mutate):
|
|
# This object will trigger mutation of the dict when replaced
|
|
# by another value. Note this relies on refcounting: the test
|
|
# won't achieve its purpose on fully-GCed Python implementations.
|
|
class Mutating:
|
|
def __del__(self):
|
|
mutate(d)
|
|
|
|
d = {k: Mutating() for k in 'abcdefghijklmnopqr'}
|
|
for k in list(d):
|
|
d[k] = k
|
|
|
|
def test_reentrant_insertion(self):
|
|
# Reentrant insertion shouldn't crash (see issue #22653)
|
|
def mutate(d):
|
|
d['b'] = 5
|
|
self.check_reentrant_insertion(mutate)
|
|
|
|
def mutate(d):
|
|
d.update(self.__dict__)
|
|
d.clear()
|
|
self.check_reentrant_insertion(mutate)
|
|
|
|
def mutate(d):
|
|
while d:
|
|
d.popitem()
|
|
self.check_reentrant_insertion(mutate)
|
|
|
|
def test_merge_and_mutate(self):
|
|
class X:
|
|
def __hash__(self):
|
|
return 0
|
|
|
|
def __eq__(self, o):
|
|
other.clear()
|
|
return False
|
|
|
|
l = [(i,0) for i in range(1, 1337)]
|
|
other = dict(l)
|
|
other[X()] = 0
|
|
d = {X(): 0, 1: 1}
|
|
self.assertRaises(RuntimeError, d.update, other)
|
|
|
|
def test_free_after_iterating(self):
|
|
support.check_free_after_iterating(self, iter, dict)
|
|
support.check_free_after_iterating(self, lambda d: iter(d.keys()), dict)
|
|
support.check_free_after_iterating(self, lambda d: iter(d.values()), dict)
|
|
support.check_free_after_iterating(self, lambda d: iter(d.items()), dict)
|
|
|
|
@unittest.expectedFailure # TODO: RUSTPYTHON
|
|
def test_equal_operator_modifying_operand(self):
|
|
# test fix for seg fault reported in bpo-27945 part 3.
|
|
class X():
|
|
def __del__(self):
|
|
dict_b.clear()
|
|
|
|
def __eq__(self, other):
|
|
dict_a.clear()
|
|
return True
|
|
|
|
def __hash__(self):
|
|
return 13
|
|
|
|
dict_a = {X(): 0}
|
|
dict_b = {X(): X()}
|
|
self.assertTrue(dict_a == dict_b)
|
|
|
|
# test fix for seg fault reported in bpo-38588 part 1.
|
|
class Y:
|
|
def __eq__(self, other):
|
|
dict_d.clear()
|
|
return True
|
|
|
|
dict_c = {0: Y()}
|
|
dict_d = {0: set()}
|
|
self.assertTrue(dict_c == dict_d)
|
|
|
|
def test_fromkeys_operator_modifying_dict_operand(self):
|
|
# test fix for seg fault reported in issue 27945 part 4a.
|
|
class X(int):
|
|
def __hash__(self):
|
|
return 13
|
|
|
|
def __eq__(self, other):
|
|
if len(d) > 1:
|
|
d.clear()
|
|
return False
|
|
|
|
d = {} # this is required to exist so that d can be constructed!
|
|
d = {X(1): 1, X(2): 2}
|
|
try:
|
|
dict.fromkeys(d) # shouldn't crash
|
|
except RuntimeError: # implementation defined
|
|
pass
|
|
|
|
def test_fromkeys_operator_modifying_set_operand(self):
|
|
# test fix for seg fault reported in issue 27945 part 4b.
|
|
class X(int):
|
|
def __hash__(self):
|
|
return 13
|
|
|
|
def __eq__(self, other):
|
|
if len(d) > 1:
|
|
d.clear()
|
|
return False
|
|
|
|
d = {} # this is required to exist so that d can be constructed!
|
|
d = {X(1), X(2)}
|
|
try:
|
|
dict.fromkeys(d) # shouldn't crash
|
|
except RuntimeError: # implementation defined
|
|
pass
|
|
|
|
def test_dictitems_contains_use_after_free(self):
|
|
class X:
|
|
def __eq__(self, other):
|
|
d.clear()
|
|
return NotImplemented
|
|
|
|
d = {0: set()}
|
|
(0, X()) in d.items()
|
|
|
|
def test_dict_contain_use_after_free(self):
|
|
# bpo-40489
|
|
class S(str):
|
|
def __eq__(self, other):
|
|
d.clear()
|
|
return NotImplemented
|
|
|
|
def __hash__(self):
|
|
return hash('test')
|
|
|
|
d = {S(): 'value'}
|
|
self.assertFalse('test' in d)
|
|
|
|
def test_init_use_after_free(self):
|
|
class X:
|
|
def __hash__(self):
|
|
pair[:] = []
|
|
return 13
|
|
|
|
pair = [X(), 123]
|
|
dict([pair])
|
|
|
|
def test_oob_indexing_dictiter_iternextitem(self):
|
|
class X(int):
|
|
def __del__(self):
|
|
d.clear()
|
|
|
|
d = {i: X(i) for i in range(8)}
|
|
|
|
def iter_and_mutate():
|
|
for result in d.items():
|
|
if result[0] == 2:
|
|
d[2] = None # free d[2] --> X(2).__del__ was called
|
|
|
|
self.assertRaises(RuntimeError, iter_and_mutate)
|
|
|
|
def test_reversed(self):
|
|
d = {"a": 1, "b": 2, "foo": 0, "c": 3, "d": 4}
|
|
del d["foo"]
|
|
r = reversed(d)
|
|
self.assertEqual(list(r), list('dcba'))
|
|
self.assertRaises(StopIteration, next, r)
|
|
|
|
def test_reverse_iterator_for_empty_dict(self):
|
|
# bpo-38525: reversed iterator should work properly
|
|
|
|
# empty dict is directly used for reference count test
|
|
self.assertEqual(list(reversed({})), [])
|
|
self.assertEqual(list(reversed({}.items())), [])
|
|
self.assertEqual(list(reversed({}.values())), [])
|
|
self.assertEqual(list(reversed({}.keys())), [])
|
|
|
|
# dict() and {} don't trigger the same code path
|
|
self.assertEqual(list(reversed(dict())), [])
|
|
self.assertEqual(list(reversed(dict().items())), [])
|
|
self.assertEqual(list(reversed(dict().values())), [])
|
|
self.assertEqual(list(reversed(dict().keys())), [])
|
|
|
|
def test_reverse_iterator_for_shared_shared_dicts(self):
|
|
class A:
|
|
def __init__(self, x, y):
|
|
if x: self.x = x
|
|
if y: self.y = y
|
|
|
|
self.assertEqual(list(reversed(A(1, 2).__dict__)), ['y', 'x'])
|
|
self.assertEqual(list(reversed(A(1, 0).__dict__)), ['x'])
|
|
self.assertEqual(list(reversed(A(0, 1).__dict__)), ['y'])
|
|
|
|
def test_dict_copy_order(self):
|
|
# bpo-34320
|
|
od = collections.OrderedDict([('a', 1), ('b', 2)])
|
|
od.move_to_end('a')
|
|
expected = list(od.items())
|
|
|
|
copy = dict(od)
|
|
self.assertEqual(list(copy.items()), expected)
|
|
|
|
# dict subclass doesn't override __iter__
|
|
class CustomDict(dict):
|
|
pass
|
|
|
|
pairs = [('a', 1), ('b', 2), ('c', 3)]
|
|
|
|
d = CustomDict(pairs)
|
|
self.assertEqual(pairs, list(dict(d).items()))
|
|
|
|
class CustomReversedDict(dict):
|
|
def keys(self):
|
|
return reversed(list(dict.keys(self)))
|
|
|
|
__iter__ = keys
|
|
|
|
def items(self):
|
|
return reversed(dict.items(self))
|
|
|
|
d = CustomReversedDict(pairs)
|
|
self.assertEqual(pairs[::-1], list(dict(d).items()))
|
|
|
|
@support.cpython_only
|
|
def test_dict_items_result_gc(self):
|
|
# bpo-42536: dict.items's tuple-reuse speed trick breaks the GC's
|
|
# assumptions about what can be untracked. Make sure we re-track result
|
|
# tuples whenever we reuse them.
|
|
it = iter({None: []}.items())
|
|
gc.collect()
|
|
# That GC collection probably untracked the recycled internal result
|
|
# tuple, which is initialized to (None, None). Make sure it's re-tracked
|
|
# when it's mutated and returned from __next__:
|
|
self.assertTrue(gc.is_tracked(next(it)))
|
|
|
|
@support.cpython_only
|
|
def test_dict_items_result_gc_reversed(self):
|
|
# Same as test_dict_items_result_gc above, but reversed.
|
|
it = reversed({None: []}.items())
|
|
gc.collect()
|
|
self.assertTrue(gc.is_tracked(next(it)))
|
|
|
|
def test_store_evilattr(self):
|
|
class EvilAttr:
|
|
def __init__(self, d):
|
|
self.d = d
|
|
|
|
def __del__(self):
|
|
if 'attr' in self.d:
|
|
del self.d['attr']
|
|
gc.collect()
|
|
|
|
class Obj:
|
|
pass
|
|
|
|
obj = Obj()
|
|
obj.__dict__ = {}
|
|
for _ in range(10):
|
|
obj.attr = EvilAttr(obj.__dict__)
|
|
|
|
def test_str_nonstr(self):
|
|
# cpython uses a different lookup function if the dict only contains
|
|
# `str` keys. Make sure the unoptimized path is used when a non-`str`
|
|
# key appears.
|
|
|
|
class StrSub(str):
|
|
pass
|
|
|
|
eq_count = 0
|
|
# This class compares equal to the string 'key3'
|
|
class Key3:
|
|
def __hash__(self):
|
|
return hash('key3')
|
|
|
|
def __eq__(self, other):
|
|
nonlocal eq_count
|
|
if isinstance(other, Key3) or isinstance(other, str) and other == 'key3':
|
|
eq_count += 1
|
|
return True
|
|
return False
|
|
|
|
key3_1 = StrSub('key3')
|
|
key3_2 = Key3()
|
|
key3_3 = Key3()
|
|
|
|
dicts = []
|
|
|
|
# Create dicts of the form `{'key1': 42, 'key2': 43, key3: 44}` in a
|
|
# bunch of different ways. In all cases, `key3` is not of type `str`.
|
|
# `key3_1` is a `str` subclass and `key3_2` is a completely unrelated
|
|
# type.
|
|
for key3 in (key3_1, key3_2):
|
|
# A literal
|
|
dicts.append({'key1': 42, 'key2': 43, key3: 44})
|
|
|
|
# key3 inserted via `dict.__setitem__`
|
|
d = {'key1': 42, 'key2': 43}
|
|
d[key3] = 44
|
|
dicts.append(d)
|
|
|
|
# key3 inserted via `dict.setdefault`
|
|
d = {'key1': 42, 'key2': 43}
|
|
self.assertEqual(d.setdefault(key3, 44), 44)
|
|
dicts.append(d)
|
|
|
|
# key3 inserted via `dict.update`
|
|
d = {'key1': 42, 'key2': 43}
|
|
d.update({key3: 44})
|
|
dicts.append(d)
|
|
|
|
# key3 inserted via `dict.__ior__`
|
|
d = {'key1': 42, 'key2': 43}
|
|
d |= {key3: 44}
|
|
dicts.append(d)
|
|
|
|
# `dict(iterable)`
|
|
def make_pairs():
|
|
yield ('key1', 42)
|
|
yield ('key2', 43)
|
|
yield (key3, 44)
|
|
d = dict(make_pairs())
|
|
dicts.append(d)
|
|
|
|
# `dict.copy`
|
|
d = d.copy()
|
|
dicts.append(d)
|
|
|
|
# dict comprehension
|
|
d = {key: 42 + i for i,key in enumerate(['key1', 'key2', key3])}
|
|
dicts.append(d)
|
|
|
|
for d in dicts:
|
|
with self.subTest(d=d):
|
|
self.assertEqual(d.get('key1'), 42)
|
|
|
|
# Try to make an object that is of type `str` and is equal to
|
|
# `'key1'`, but (at least on cpython) is a different object.
|
|
noninterned_key1 = 'ke'
|
|
noninterned_key1 += 'y1'
|
|
if support.check_impl_detail(cpython=True):
|
|
# suppress a SyntaxWarning
|
|
interned_key1 = 'key1'
|
|
self.assertFalse(noninterned_key1 is interned_key1)
|
|
self.assertEqual(d.get(noninterned_key1), 42)
|
|
|
|
self.assertEqual(d.get('key3'), 44)
|
|
self.assertEqual(d.get(key3_1), 44)
|
|
self.assertEqual(d.get(key3_2), 44)
|
|
|
|
# `key3_3` itself is definitely not a dict key, so make sure
|
|
# that `__eq__` gets called.
|
|
#
|
|
# Note that this might not hold for `key3_1` and `key3_2`
|
|
# because they might be the same object as one of the dict keys,
|
|
# in which case implementations are allowed to skip the call to
|
|
# `__eq__`.
|
|
eq_count = 0
|
|
self.assertEqual(d.get(key3_3), 44)
|
|
self.assertGreaterEqual(eq_count, 1)
|
|
|
|
def test_overwrite_managed_dict(self):
|
|
# GH-130327: Overwriting an object's managed dictionary with another object's
|
|
# skipped traversal in favor of inline values, causing the GC to believe that
|
|
# the __dict__ wasn't reachable.
|
|
import gc
|
|
|
|
class Shenanigans:
|
|
pass
|
|
|
|
to_be_deleted = Shenanigans()
|
|
to_be_deleted.attr = "whatever"
|
|
holds_reference = Shenanigans()
|
|
holds_reference.__dict__ = to_be_deleted.__dict__
|
|
holds_reference.ref = {"circular": to_be_deleted, "data": 42}
|
|
|
|
del to_be_deleted
|
|
gc.collect()
|
|
self.assertEqual(holds_reference.ref['data'], 42)
|
|
self.assertEqual(holds_reference.attr, "whatever")
|
|
|
|
@unittest.expectedFailure # TODO: RUSTPYTHON
|
|
def test_unhashable_key(self):
|
|
d = {'a': 1}
|
|
key = [1, 2, 3]
|
|
|
|
def check_unhashable_key():
|
|
msg = "cannot use 'list' as a dict key (unhashable type: 'list')"
|
|
return self.assertRaisesRegex(TypeError, re.escape(msg))
|
|
|
|
with check_unhashable_key():
|
|
key in d
|
|
with check_unhashable_key():
|
|
d[key]
|
|
with check_unhashable_key():
|
|
d[key] = 2
|
|
with check_unhashable_key():
|
|
d.setdefault(key, 2)
|
|
with check_unhashable_key():
|
|
d.pop(key)
|
|
with check_unhashable_key():
|
|
d.get(key)
|
|
|
|
# Only TypeError exception is overriden,
|
|
# other exceptions are left unchanged.
|
|
class HashError:
|
|
def __hash__(self):
|
|
raise KeyError('error')
|
|
|
|
key2 = HashError()
|
|
with self.assertRaises(KeyError):
|
|
key2 in d
|
|
with self.assertRaises(KeyError):
|
|
d[key2]
|
|
with self.assertRaises(KeyError):
|
|
d[key2] = 2
|
|
with self.assertRaises(KeyError):
|
|
d.setdefault(key2, 2)
|
|
with self.assertRaises(KeyError):
|
|
d.pop(key2)
|
|
with self.assertRaises(KeyError):
|
|
d.get(key2)
|
|
|
|
def test_clear_at_lookup(self):
|
|
# gh-140551 dict crash if clear is called at lookup stage
|
|
class X:
|
|
def __hash__(self):
|
|
return 1
|
|
def __eq__(self, other):
|
|
nonlocal d
|
|
d.clear()
|
|
|
|
d = {}
|
|
for _ in range(10):
|
|
d[X()] = None
|
|
|
|
self.assertEqual(len(d), 1)
|
|
|
|
d = {}
|
|
for _ in range(10):
|
|
d.setdefault(X(), None)
|
|
|
|
self.assertEqual(len(d), 1)
|
|
|
|
def test_split_table_update_with_str_subclass(self):
|
|
# gh-142218: inserting into a split table dictionary with a non str
|
|
# key that matches an existing key.
|
|
class MyStr(str): pass
|
|
class MyClass: pass
|
|
obj = MyClass()
|
|
obj.attr = 1
|
|
obj.__dict__[MyStr('attr')] = 2
|
|
self.assertEqual(obj.attr, 2)
|
|
|
|
def test_split_table_insert_with_str_subclass(self):
|
|
# gh-143189: inserting into split table dictionary with a non str
|
|
# key that matches an existing key in the shared table but not in
|
|
# the dict yet.
|
|
|
|
class MyStr(str): pass
|
|
class MyClass: pass
|
|
|
|
obj = MyClass()
|
|
obj.attr1 = 1
|
|
|
|
obj2 = MyClass()
|
|
d = obj2.__dict__
|
|
d[MyStr("attr1")] = 2
|
|
self.assertIsInstance(list(d)[0], MyStr)
|
|
|
|
def test_hash_collision_remove_add(self):
|
|
self.maxDiff = None
|
|
# There should be enough space, so all elements with unique hash
|
|
# will be placed in corresponding cells without collision.
|
|
n = 64
|
|
items = [(CustomHash(h), h) for h in range(n)]
|
|
# Keys with hash collision.
|
|
a = CustomHash(n)
|
|
b = CustomHash(n)
|
|
items += [(a, 'a'), (b, 'b')]
|
|
d = dict(items)
|
|
self.assertEqual(len(d), len(items), d)
|
|
del d[a]
|
|
# "a" has been replaced with a dummy.
|
|
del items[n]
|
|
self.assertEqual(len(d), len(items), d)
|
|
self.assertEqual(d, dict(items))
|
|
d[b] = 'c'
|
|
# "b" should not replace the dummy.
|
|
items[n] = (b, 'c')
|
|
self.assertEqual(len(d), len(items), d)
|
|
self.assertEqual(d, dict(items))
|
|
|
|
def test_clear_reentrant_embedded(self):
|
|
# gh-130555: dict.clear() must be safe when values are embedded
|
|
# in an object and a destructor mutates the dict.
|
|
class MyObj: pass
|
|
class ClearOnDelete:
|
|
def __del__(self):
|
|
nonlocal x
|
|
del x
|
|
|
|
x = MyObj()
|
|
x.a = ClearOnDelete()
|
|
|
|
d = x.__dict__
|
|
d.clear()
|
|
|
|
def test_clear_reentrant_cycle(self):
|
|
# gh-130555: dict.clear() must be safe for embedded dicts when the
|
|
# object is part of a reference cycle and the last reference to the
|
|
# dict is via the cycle.
|
|
class MyObj: pass
|
|
obj = MyObj()
|
|
obj.f = obj
|
|
obj.attr = "attr"
|
|
|
|
d = obj.__dict__
|
|
del obj
|
|
|
|
d.clear()
|
|
|
|
def test_clear_reentrant_force_combined(self):
|
|
# gh-130555: dict.clear() must be safe when a destructor forces the
|
|
# dict from embedded/split to combined (setting ma_values to NULL).
|
|
class MyObj: pass
|
|
class ForceConvert:
|
|
def __del__(self):
|
|
d[1] = "trigger"
|
|
|
|
x = MyObj()
|
|
x.a = ForceConvert()
|
|
x.b = "other"
|
|
|
|
d = x.__dict__
|
|
d.clear()
|
|
|
|
def test_clear_reentrant_delete(self):
|
|
# gh-130555: dict.clear() must be safe when a destructor deletes
|
|
# a key from the same embedded dict.
|
|
class MyObj: pass
|
|
class DelKey:
|
|
def __del__(self):
|
|
try:
|
|
del d['b']
|
|
except KeyError:
|
|
pass
|
|
|
|
x = MyObj()
|
|
x.a = DelKey()
|
|
x.b = "value_b"
|
|
x.c = "value_c"
|
|
|
|
d = x.__dict__
|
|
d.clear()
|
|
|
|
|
|
class CAPITest(unittest.TestCase):
|
|
|
|
# Test _PyDict_GetItem_KnownHash()
|
|
@support.cpython_only
|
|
def test_getitem_knownhash(self):
|
|
_testinternalcapi = import_helper.import_module('_testinternalcapi')
|
|
dict_getitem_knownhash = _testinternalcapi.dict_getitem_knownhash
|
|
|
|
d = {'x': 1, 'y': 2, 'z': 3}
|
|
self.assertEqual(dict_getitem_knownhash(d, 'x', hash('x')), 1)
|
|
self.assertEqual(dict_getitem_knownhash(d, 'y', hash('y')), 2)
|
|
self.assertEqual(dict_getitem_knownhash(d, 'z', hash('z')), 3)
|
|
|
|
# not a dict
|
|
self.assertRaises(SystemError, dict_getitem_knownhash, [], 1, hash(1))
|
|
# key does not exist
|
|
self.assertRaises(KeyError, dict_getitem_knownhash, {}, 1, hash(1))
|
|
|
|
class Exc(Exception): pass
|
|
class BadEq:
|
|
def __eq__(self, other):
|
|
raise Exc
|
|
def __hash__(self):
|
|
return 7
|
|
|
|
k1, k2 = BadEq(), BadEq()
|
|
d = {k1: 1}
|
|
self.assertEqual(dict_getitem_knownhash(d, k1, hash(k1)), 1)
|
|
self.assertRaises(Exc, dict_getitem_knownhash, d, k2, hash(k2))
|
|
|
|
|
|
from test import mapping_tests
|
|
|
|
class GeneralMappingTests(mapping_tests.BasicTestMappingProtocol):
|
|
type2test = dict
|
|
|
|
class Dict(dict):
|
|
pass
|
|
|
|
class SubclassMappingTests(mapping_tests.BasicTestMappingProtocol):
|
|
type2test = Dict
|
|
|
|
|
|
if __name__ == "__main__":
|
|
unittest.main()
|