New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ValueError warning in test_urllib due to io.IOBase destructor #81099
Comments
Issue for https://bugs.python.org/issue18748#msg340059. comment : Is there someone interested to debug remaining "Exception ignored:" logs in test_urllib? test_invalid_redirect (test.test_urllib.urlopen_HttpTests) ...
Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f24be0>
Traceback (most recent call last):
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
super().close() # set "closed" flag
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
self.fp.flush()
ValueError: I/O operation on closed file.
ok
test_read_bogus (test.test_urllib.urlopen_HttpTests) ...
Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f24cd0>
Traceback (most recent call last):
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
super().close() # set "closed" flag
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
self.fp.flush()
ValueError: I/O operation on closed file.
ok
test_redirect_limit_independent (test.test_urllib.urlopen_HttpTests) ...
Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f24cd0>
Traceback (most recent call last):
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
super().close() # set "closed" flag
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
self.fp.flush()
ValueError: I/O operation on closed file.
Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f24c30>
Traceback (most recent call last):
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
super().close() # set "closed" flag
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
self.fp.flush()
ValueError: I/O operation on closed file.
Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f24460>
Traceback (most recent call last):
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
super().close() # set "closed" flag
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
self.fp.flush()
ValueError: I/O operation on closed file.
Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f24a50>
Traceback (most recent call last):
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
super().close() # set "closed" flag
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
self.fp.flush()
ValueError: I/O operation on closed file.
Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f247d0>
Traceback (most recent call last):
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
super().close() # set "closed" flag
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
self.fp.flush()
ValueError: I/O operation on closed file.
Exception ignored in: <http.client.HTTPResponse object at 0x7fca52ea95f0>
Traceback (most recent call last):
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
super().close() # set "closed" flag
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
self.fp.flush()
ValueError: I/O operation on closed file.
Exception ignored in: <http.client.HTTPResponse object at 0x7fca52ea99b0>
Traceback (most recent call last):
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
super().close() # set "closed" flag
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
self.fp.flush()
ValueError: I/O operation on closed file.
Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f2f820>
Traceback (most recent call last):
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
super().close() # set "closed" flag
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
self.fp.flush()
ValueError: I/O operation on closed file.
Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f2fbe0>
Traceback (most recent call last):
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
super().close() # set "closed" flag
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
self.fp.flush()
ValueError: I/O operation on closed file.
Exception ignored in: <http.client.HTTPResponse object at 0x7fca52f2fd70>
Traceback (most recent call last):
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 402, in close
super().close() # set "closed" flag
File "/home/vstinner/prog/python/master/Lib/http/client.py", line 415, in flush
self.fp.flush()
ValueError: I/O operation on closed file.
ok It's unclear to be if it's a bug in http.client, a bug in the test... or something else. |
I think this is a bug in BasicIO. close() should be idempotent. Calling it on the closed file should have no effect. |
No, BasicIO.close() is correct. This is a bug in the garbage collector: the underlying file is closed before closing HTTPResponse. The HTTPResponse instance has a reference to the file object, the file object does not have a reference to the HTTPResponse instance, therefore the HTTPResponse instance should be destroyed first. |
My analysis was that close was called on fakesocket which is internally closed when it's counter resets to zero and the destructor was trying to flush on a closed object during destructor call. Comment from https://bugs.python.org/issue18748#msg341106 The ValueError warnings in test_urllib noted in msg340059 feels like an issue with the test where FakeSocket calls close by itself once it's internal io_refs counter is 0 and during destructor again close is called on an already closed object causing the ValueError.
Maybe a check could be added during flush to make sure the object is not closed by testing for self.closed but I am not sure if closed attribute is guaranteed to be present. Removing the manual call to close in fakesocket could help since the destructor should be taking care of it and I could see no test failures or warnings removing the close as io_refs gets to 0. [1] cpython/Lib/test/test_urllib.py Line 61 in be6dbfb
[2] cpython/Lib/test/test_urllib.py Line 67 in be6dbfb
[3] Line 919 in be6dbfb
[4] Line 2145 in be6dbfb
|
Well, maybe Python GC can be enhanced. In the meanwhile, I would like to make these warnings quiet since they are very annoying when I have to analyze buildbot warnings. Moreover, this issues blocks my bpo-37069. Maybe open a separated issue to propose to enhance the GC, if you have an idea how to change it. |
I proposed PR 13955: a fix in 2 lines. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: