It turns out that there are many other tests that can impact
test_many_opens by leaving unclosed file handles. Rather than fix them
all, it is easier to simply increase the threshold for the problematic
test.
There seems to have been a bug in the libregrtest code which unloaded
modules between tests. The previous state was calculated using
`sys.modules.keys()`, which is actually a mutable object that is updated
as the underlying `sys.modules` is updated. The result was that modules
were not unloaded between tests, which is the root cause for
`test_unittest` failing when run after `test_import` and
`test_importlib`.
This code is copied from 3.12. Ideally all of `libregrtest` should
probably be updated as it seems wildly out of date, but that's a lot
more work.
This test was marked as an expected failure. Because the garbage
collector is missing, that meant that the `os.scandir` object went
unclosed. This object was squatting on the file descriptors of all the
files contained in the test directory, which was breaking test_zipfile.
After you suggestion in https://github.com/python/cpython/issues/116504#issuecomment-1999239012 I went to take a look at `test_cmd_line` in RustPython (it was so long ago I contributed to this amazing project, so may thing had changed!), and I've noticed this.
This is a problem, here' the simplest demo:
```python
import unittest
class TestMe(unittest.TestCase):
@unittest.expectedFailure
def test_me(self):
def run():
raise ValueError
with self.subTest(run=run):
run()
if __name__ == '__main__':
unittest.main()
```
This works as expected:
```
» ./python.exe ex.py
x
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK (expected failures=1)
```
This does not:
```python
import unittest
class TestMe(unittest.TestCase):
def test_me(self):
@unittest.expectedFailure
def run():
raise ValueError
with self.subTest(run=run):
run()
if __name__ == '__main__':
unittest.main()
```
Produces:
```
» ./python.exe ex.py
E
======================================================================
ERROR: test_me (__main__.TestMe.test_me) (run=<function TestMe.test_me.<locals>.run at 0x1057a2150>)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/sobolev/Desktop/cpython2/ex.py", line 10, in test_me
run()
~~~^^
File "/Users/sobolev/Desktop/cpython2/ex.py", line 7, in run
raise ValueError
ValueError
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (errors=1)
```
So, I propose to remove these decorators, let's only keep `TODO` comments to indicate separate failures.
I had previously `test_locale` as expected to fail, as it did indeed
fail on my system due to unimplemented functionality. As it happens, it
passes in CI because the locale settings used there (`C`, I believe)
just happen to format integers the same with "%d" as "%n".
I mistakenly un-marked it because I thought I misunderstood the problem.
Technically speaking, my system was misconfigured, leading me to disable
the test in the first place.
`test_locale` calls `locale.setlocale(locale.LC_ALL, '')`, which reads
the value of the `LANG` environment variable and uses that to look up
and reset all the locale settings. My system has `LANG=en_US.UTF-8`,
which is apparently not what this test was expecting. If `LANG` is unset
or set to `C`, the test passes, as it does in CI.
There are a substantial number of socket tests that are disabled due to
`bind(): bad family` errors. It seems like RustPython only supports a
small subset of the required connection families, so the failing tests
are broken for the same reasons.
The failing test was unsetting `PYTHONPATH`, but neglecting to unset
`RUSTPYTHONPATH`, which obviously was not significant for the original
CPython test. Including `RUSTPYTHONPATH` in the test fixes it.
Jeong, YunWon