Author juliadolgova
Recipients juliadolgova, paul.moore, steve.dower, tim.golden, zach.ware
Date 2017-02-11.08:43:25
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1486802607.23.0.387912582288.issue29533@psf.upfronthosting.co.za>
In-reply-to
Content
I've found that urllib works sometimes slowly on windows with proxy.

To reproduce the issue:
on Windows:
1. Turn on the option "use proxy" in "browser settings" in "control panel".
No real proxy needed. The problem will come out before addressing to proxy. Just don't pay attention to exception.
2. Make sure that the list of addresses for proxy bypass is not empty
3. Execute checklib.py with socket.py (attached here) in the same directory

The result output could be:
A (not a problem):
Before call to _socket.gethostbyaddr("docs.python.org")
After call to _socket.gethostbyaddr("docs.python.org")

B (little problem):
Before call to _socket.gethostbyaddr("docs.python.org")
Exception in call to _socket.gethostbyaddr("docs.python.org")

C (worse problem):
Before call to _socket.gethostbyaddr("docs.python.org")
(Delay)
Exception in call to _socket.gethostbyaddr("docs.python.org")

The result A,B or C depends on what DNS server you use, what url you pass into urllib2.urlopen(), and could differ at different time because dns is not a stable thing. 
However, no matter what result you have, this test shows that a hostname is passed into gethostbyaddr instead of IP as expected and described in MSDN. It should be changed to gethostbyname_ex here.

test.py compare performance of gethostbyaddr and gethostbyname_ex. 
It sets different dns servers on the system and calls these functions with different hostnames passed into. Run on my computer shows that gethostbyname_ex is 3 times more productive and doesn't raise exceptions.

-----------------------------
Attached files:
checklib.py - just make a call to urllib2.urlopen("https://docs.python.org")
socket.py - not a patched lib. Has debug lines near 141 line. Use it with checklib.py.
test.py - compare performance of gethostbyaddr with gethostbyname_ex
log.txt - result of test.py on my computer (Windows 8, 64 bit)
socket.patch - socket.py patch
History
Date User Action Args
2017-02-11 08:43:27juliadolgovasetrecipients: + juliadolgova, paul.moore, tim.golden, zach.ware, steve.dower
2017-02-11 08:43:27juliadolgovasetmessageid: <1486802607.23.0.387912582288.issue29533@psf.upfronthosting.co.za>
2017-02-11 08:43:27juliadolgovalinkissue29533 messages
2017-02-11 08:43:25juliadolgovacreate