Issue41345
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2020-07-20 11:43 by Michal Arbet, last changed 2022-04-11 14:59 by admin. This issue is now closed.
Files | ||||
---|---|---|---|---|
File name | Uploaded | Description | Edit | |
mycap.pcap | Michal Arbet, 2020-07-20 11:43 |
Messages (8) | |||
---|---|---|---|
msg374000 - (view) | Author: Michal Arbet (Michal Arbet) | Date: 2020-07-20 11:43 | |
Hi, I'm not sure if this is really python bug, but I hope that you can check the issue. Issue is that from time to time i'm getting exception from python when sending request to server which has http keepalive option turned on. Requests send a request but in few miliseconds apache2 server is closing persistent connection by sending FIN packet which generate traceback. I can reproduce it by following simple script. #!/usr/bin/python3 import requests from time import sleep import logging logging.basicConfig(level=logging.DEBUG) s = requests.Session() s.verify = False # self-signed cert counter = 0 txt = "test" while True: counter = counter + 1 s.post('http://localhost', data={counter:txt}) sleep(5) Everything is working fine, but from time to time I get following traceback. When FIN is received right after request was sent. michalarbet@pixla:~/work$ ./request_test.py DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:80 DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 0 Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 421, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 416, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.8/http/client.py", line 1332, in getresponse response.begin() File "/usr/lib/python3.8/http/client.py", line 303, in begin version, status, reason = self._read_status() File "/usr/lib/python3.8/http/client.py", line 272, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/michalarbet/.local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 719, in urlopen retries = retries.increment( File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 400, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/lib/python3/dist-packages/six.py", line 702, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 421, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 416, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.8/http/client.py", line 1332, in getresponse response.begin() File "/usr/lib/python3.8/http/client.py", line 303, in begin version, status, reason = self._read_status() File "/usr/lib/python3.8/http/client.py", line 272, in _read_status raise RemoteDisconnected("Remote end closed connection without" urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "./request_test.py", line 26, in <module> s.post('http://localhost', data={counter:txt}) File "/home/michalarbet/.local/lib/python3.8/site-packages/requests/sessions.py", line 578, in post return self.request('POST', url, data=data, json=json, **kwargs) File "/home/michalarbet/.local/lib/python3.8/site-packages/requests/sessions.py", line 530, in request resp = self.send(prep, **send_kwargs) File "/home/michalarbet/.local/lib/python3.8/site-packages/requests/sessions.py", line 643, in send r = adapter.send(request, **kwargs) File "/home/michalarbet/.local/lib/python3.8/site-packages/requests/adapters.py", line 498, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) Apache using keepalive. /etc/apache2/apache2.conf:KeepAliveTimeout 5 /etc/apache2/apache2.conf:MaxKeepAliveRequests 100 /etc/apache2/apache2.conf:KeepAlive On Sending also pcap file from different reproduce test, but behaviour is same. |
|||
msg374004 - (view) | Author: Ilia Androshchuk (an7e) * | Date: 2020-07-20 12:54 | |
Hi Michal, Which version of operating system and apache are you running? I'll try to reproduce |
|||
msg374008 - (view) | Author: Michal Arbet (Michal Arbet) | Date: 2020-07-20 13:48 | |
I'm using Ubuntu 20.04 with dist apache2 and default settings : michalarbet@pixla:~/work$ dpkg -l | grep apache ii apache2 2.4.41-4ubuntu3 amd64 Apache HTTP Server ii apache2-bin 2.4.41-4ubuntu3 amd64 Apache HTTP Server (modules and other binary files) ii apache2-data 2.4.41-4ubuntu3 all Apache HTTP Server (common files) ii apache2-utils 2.4.41-4ubuntu3 amd64 Apache HTTP Server (utility programs for web servers) ii libapache2-mod-php 2:7.4+75 all server-side, HTML-embedded scripting language (Apache 2 module) (default) ii libapache2-mod-php7.4 7.4.3-4ubuntu2.2 amd64 server-side, HTML-embedded scripting language (Apache 2 module) michalarbet@pixla:~/work$ uname -a Linux pixla 5.4.0-40-generic #44-Ubuntu SMP Tue Jun 23 00:01:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux When reproducing issue, be patient , you have to wait several iterations before it will fail. Some bug reports and discussion related to : https://bugs.python.org/issue41345 https://github.com/psf/requests/issues/4664 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=899406 po 20. 7. 2020 v 14:54 odesílatel Ilia Androshchuk <report@bugs.python.org> napsal: > > Ilia Androshchuk <an7e@outlook.com> added the comment: > > Hi Michal, > Which version of operating system and apache are you running? > I'll try to reproduce > > ---------- > nosy: +an7e > > _______________________________________ > Python tracker <report@bugs.python.org> > <https://bugs.python.org/issue41345> > _______________________________________ > |
|||
msg374034 - (view) | Author: Ilia Androshchuk (an7e) * | Date: 2020-07-20 23:17 | |
Hi, Michal It took a lot of time, but I achieved exactly the same behavior. My Traceback : DEBUG:urllib3.connectionpool:Resetting dropped connection: localhost DEBUG:urllib3.connectionpool:http://localhost:80 "POST / HTTP/1.1" 200 3138 Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 421, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 416, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.8/http/client.py", line 1332, in getresponse response.begin() File "/usr/lib/python3.8/http/client.py", line 303, in begin version, status, reason = self._read_status() File "/usr/lib/python3.8/http/client.py", line 272, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 719, in urlopen retries = retries.increment( File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 400, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/lib/python3/dist-packages/six.py", line 702, in reraise raise value.with_traceback(tb) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 421, in _make_request six.raise_from(e, None) File "<string>", line 3, in raise_from File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 416, in _make_request httplib_response = conn.getresponse() File "/usr/lib/python3.8/http/client.py", line 1332, in getresponse response.begin() File "/usr/lib/python3.8/http/client.py", line 303, in begin version, status, reason = self._read_status() File "/usr/lib/python3.8/http/client.py", line 272, in _read_status raise RemoteDisconnected("Remote end closed connection without" urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "test1.py", line 14, in <module> s.post('http://localhost', data={counter:txt}) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 581, in post return self.request('POST', url, data=data, json=json, **kwargs) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request resp = self.send(prep, **send_kwargs) File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3/dist-packages/requests/adapters.py", line 498, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) I'm using same dist and settings: lerfi@UbuntuVM:~/work/psf/41345$ uname -a Linux UbuntuVM 5.4.0-40-generic #44-Ubuntu SMP Tue Jun 23 00:01:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux erfi@UbuntuVM:~/work/psf/41345$ dpkg -l | grep apache ii apache2 2.4.41-4ubuntu3 amd64 Apache HTTP Server ii apache2-bin 2.4.41-4ubuntu3 amd64 Apache HTTP Server (modules and other binary files) ii apache2-data 2.4.41-4ubuntu3 all Apache HTTP Server (common files) ii apache2-utils 2.4.41-4ubuntu3 amd64 Apache HTTP Server (utility programs for web servers) pip3 requests==2.22.0 it looks like there's a problem with this urllib3 package |
|||
msg374066 - (view) | Author: Ilia Androshchuk (an7e) * | Date: 2020-07-21 13:18 | |
Hi Michal, I found some information in the package documentation Please check this page https://requests.readthedocs.io/en/master/user/advanced/#keep-alive |
|||
msg374200 - (view) | Author: Guido van Rossum (gvanrossum) * ![]() |
Date: 2020-07-24 18:28 | |
This surely is some application corner case. Closing. |
|||
msg374365 - (view) | Author: Michal Arbet (Michal Arbet) | Date: 2020-07-27 08:04 | |
Hello, Thank you very much for help, I created pull request for urllib3 -> https://github.com/urllib3/urllib3/pull/1911 Can u just confirm that this could be fixed in urllib3 as it is in pull request ? Have I fixed it by good way ? Thanks. Michal Arbet ( kevko ) pá 24. 7. 2020 v 20:29 odesílatel Guido van Rossum <report@bugs.python.org> napsal: > > Guido van Rossum <guido@python.org> added the comment: > > This surely is some application corner case. Closing. > > ---------- > nosy: +gvanrossum > resolution: -> not a bug > stage: -> resolved > status: open -> closed > > _______________________________________ > Python tracker <report@bugs.python.org> > <https://bugs.python.org/issue41345> > _______________________________________ > |
|||
msg374466 - (view) | Author: Martin Panter (martin.panter) * ![]() |
Date: 2020-07-28 05:47 | |
Previous report about Requests to the Python bug tracker: Issue 33620. I suspect this is an unavoidable race condition with trying a POST (or other non-idempotent) request on an idle HTTP connection. I think it has to be up to the higher-level application or user to decide if it is safe to retry a POST request. Otherwise you risk e.g. accidentally ordering two pizzas because the server received two POST requests but something interfered with the response of the first response. On the other hand, I noticed some browsers seem to automatically retry a POST once if it is interrupted, which makes me uneasy. A concrete example of the problem is a firmware upload that triggers a reboot. If the reboot is too quick and prevents the POST response being sent, I found that a web browser will repeat the firmware upload once more after my firmware boots up again. If it is not safe to retry the POST request, other options would be to avoid the server thinking the connection is stale: * always do the POST request on a fresh HTTP connection * "ping" the connection with a dummy request (e.g. OPTIONS, HEAD, or GET) immediately before the POST request Another option that comes to mind is to try using the 100 Continue mechanism, but this is not a general solution and depends on the application. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:33 | admin | set | github: 85517 |
2020-07-28 05:47:14 | martin.panter | set | nosy:
+ martin.panter messages: + msg374466 |
2020-07-27 08:04:37 | Michal Arbet | set | messages: + msg374365 |
2020-07-24 18:28:51 | gvanrossum | set | status: open -> closed nosy: + gvanrossum messages: + msg374200 resolution: not a bug stage: resolved |
2020-07-21 13:18:28 | an7e | set | messages: + msg374066 |
2020-07-20 23:17:21 | an7e | set | messages: + msg374034 |
2020-07-20 13:48:46 | Michal Arbet | set | messages: + msg374008 |
2020-07-20 12:54:48 | an7e | set | nosy:
+ an7e messages: + msg374004 |
2020-07-20 11:43:46 | Michal Arbet | create |