New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AttributeError: 'NoneType' in http/client.py when using select when file descriptor is closed. #63353
Comments
In Lib/http/client.py +682 (Formerly httplib) def fileno(self):
return self.fp.fileno() This function should be modified to be able to handle the case where the http request is already completed and so "fp" is closed. Ex.: I encountered the issue in the following context: Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/nappstore/server_comm.py", line 211, in download_file
ready = select.select([req], [], [], timeout)[0]
File "/usr/lib/python2.7/socket.py", line 313, in fileno
return self._sock.fileno()
File "/usr/lib/python2.7/httplib.py", line 655, in fileno
return self.fp.fileno()
AttributeError: 'NoneType' object has no attribute 'fileno' For the returned value, I'm not sure because there is currently 2 different cases for other objects returning a fileno. |
Changing the API of HTTPResponse.fileno is a feature change, not a bug fix. I do not think it should be changed as it would break existing code to no purpose. Instead, calls should continue to wrapped in try: except ValueError when failure is possible. And this issue closed as rejected. -- I do not think the call in _fileobject.fileno should be so wrapped because I think the bug is elsewhere. _fileinput()._sock is supposed to be a socket, not an HTTPResponse. This says to me that a) self._sock.fileno() is supposed to call socket.fileno, not HTTPResonse.fileno; and b) nappstore/server_comm.py is calling socket(_sock=<HTTPResonse object>) (or directly modifying the .sock attribute). If b) is true, that strikes me as a bug because _sock is an undocumented private parameter, subject to change. Indeed it disappeared in 3.x when socket became a _socket.socket subclass rather than a wrapper thereof. (Also, the pseudo _fileobject was replaced with an appropriate, real, io class.) |
Hi Terry, Please really understand that here I speak about the "def fileno()" function that is inside Lib/http/client.py" and not about the "def fileno()" that is in socket.py. |
We are both talking about 2.7 httplib.HTTPResponse.fileno. I should have said that users should try: ...fileno(); except AttributeError rather than ValueError. Any caller can understand AttributeError as well as ValueError or any other exception. It is not an error for a python function to raise an exception. In fact, it is the normal thing to do when it cannot do as requested. API changes are not allowed in 2.7 and this one would be dubious even in a future 3.x. I do think the doc should be more informative and that Return(s) the fileno of the underlying socket. should be extended to indicate the specific exception raised when that is not possible. Return the fileno of the underlying socket if there is one or raise AttributeError. This should also be added as a docstring (currently missing). The above is also true for 3.x as self.fp continues to be replaced with None on closing. The HTTPResponse docs are known to be deficient. bpo-3430 -- |
Thank you for your reply. This is how is defined req.: then: I found this issue with python 2.7, but I don't care a lot for it to be fixed in 2.7. As I saw that the code looks unchanged in python 3.x, I just reported the issue for it to be fixed/better handled in 3.x :-) |
It seems to me that there is indeed an issue of some sort here, but its locus is (to me) unclear. I haven't commented before this because I wanted to read the docs...but I haven't had time yet :) One question is, is it even expected that passing a Request to select will work? If it *is* expected, then what is the API that req should be conforming to? This API may be an implicit one that is not documented, or perhaps it is documented in select (I haven't checked). If req is conforming to the explicit or implicit API, then the bug would be in select. Otherwise it is in httplib. Or, if this isn't something we've been supporting in the past, then as Terry says it is a new feature. |
s/httplib/urllib/ |
R. David, what you say is correct, supporting "select" that would be nice but i'm also not sure that is supposed to, and in that case, maybe select as to be fixed for that. But:
And for the 2 firsts, one is finally using the fileno function of the other. |
OK, I've looked at the docs and code, and as far as I can see this bug does not exist in Python3. Or at least in 3.4, which is the only place I'd feel safe about making a change to the exception type. To summarize: in 3.4 socket logic is based on RawIOBase, as is HTTPResponse. And RawIOBase checks to see if the file is closed and raises a ValueError if it is, when fileno is called on the 'fp' attribute of the HTTPResponse. At least, that's what I think based on reading the code. So, unless you can reproduce the error in 3.3 and/or 3.4, I think we should close this issue as out of date, since as Terry said there is a non-trivial danger of backward incompatibility if we were to change the exception type in 2.7. |
Florent, for future reference, marking an issue for 2.7 says to us "I want this fixed for 2.7". I agree that having 3 different error indicators for 3 similar functions is nasty. But this is partly due to the difference between object that *has* a fd (socket) versus a object that *wraps such an object. The problem is that changing any of them could break code that just uses one of them. The select doc says that a 'waitable' object must have .fileno() that return a valid fd int. It says nothing about an allowed error return. This could be interpreted as meaning that not returning an fd int makes the object (such as a closed HTTPResponse) unwaitable. Or this could be interpreted as a deficiency in the select doc or code. In the module/selectmodule.c code, the various poll methods interpret -1 as the error return for a closed fd. I do not know how it deals with socket.fileno raising ValueError (or returning None?). In the client doc, none of several examples of using HTTPResponses use select. They just use the various methods to retrieve data. I could interpret this to mean that they are not intended to work with select. The fact that they do not, reinforces that. |
I think ValueError should generally be raised when calling fileno() on a closed file. However, this is not something we're gonna change in a bugfix release, so it would go in 3.4 at the earliest. |
@terryjreedy, @pitrou, @bitdancer, @fviard @ambv This issue is 10 years old and appears to only affect 2.7, which is no longer supported. This issue should be closed in order to clean up the issue list. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: