classification
Title: urllib2: urlopen() gets exception(kwargs bug?)
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Douman, alex, christian.heimes, demian.brecht, r.david.murray, spaceone
Priority: normal Keywords:

Created on 2015-01-15 06:01 by Douman, last changed 2016-09-26 14:54 by spaceone. This issue is now closed.

Files
File name Uploaded Description Edit
callstack_urllib2 Douman, 2015-01-15 06:01 Callstack
callstack_urllib2_full Douman, 2015-01-15 17:05 full callstack
Messages (13)
msg234055 - (view) Author: Douman (Douman) Date: 2015-01-15 06:01
I get strange callstack from urllib2

It seems that python thinks that HTTPSConnection doesn't have context argument.
Which is entirely incorrect. 
I have suspicions that this is related to the way how context argument is passed(kwargs)
This happens starting from python 2.7.9
msg234073 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-01-15 13:18
More likely http_class isn't what you think it is (ie: it might be an HTTPSConnection subclass that hasn't been updated to deal with 2.7.9.  Can you check that?
msg234082 - (view) Author: Douman (Douman) Date: 2015-01-15 16:12
Yes, i checked what is http_class. It is passed as httplib.HTTPSConnection
Before submitting this issue i checked httplib.py in my installation of py2 and there https://hg.python.org/cpython/file/2.7/Lib/httplib.py

HTTPSConnection has argument "context".

Btw, it would be nice to update comments in urllib2 so that it would be more accurate
msg234083 - (view) Author: Douman (Douman) Date: 2015-01-15 16:16
Btw, i also tried to replace **kwargs with usual argument and it didn't throw exception
msg234084 - (view) Author: Douman (Douman) Date: 2015-01-15 16:36
Also according to documentation this class was specifically updated with context in 2.7.9
I guess then there should commit related to adding of "context" to HTTPSConnection
msg234085 - (view) Author: Douman (Douman) Date: 2015-01-15 16:42
It seems to be this one
https://hg.python.org/cpython/rev/1882157b298a
msg234088 - (view) Author: Douman (Douman) Date: 2015-01-15 17:05
I made additional experiments and now i'm confused
When i tried to execute urlopen in python interpeter it worked fine.
But it fails for me when i attempt to do so via some helper function in search engine of qBittorent it throw exception

https://github.com/qbittorrent/qBittorrent/blob/master/src/searchengine/nova/helpers.py

What is curious... all exceptions should be passed without notices, but for some reason python does throw exception even though all are catched and passed
msg236433 - (view) Author: Douman (Douman) Date: 2015-02-23 10:26
Just to up issue.
It seems that there is some changes in 2.7.9 that breaks usage of urllib2.urlopen()
msg236508 - (view) Author: Douman (Douman) Date: 2015-02-24 15:50
No problem in python implementation of HTTPSConnection
Sorry
msg277421 - (view) Author: SpaceOne (spaceone) * Date: 2016-09-26 14:28
Hello,
The resolution of this bug is "not a bug". If that is the case can you please add information how to fix/workaround this.
I have got the following valid-seeming code:
"""
import cookielib
import urllib
import urllib2
cookie = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookie))
opener.open('https://www.google.com', timeout=2)
"""

Which results in:
Traceback (most recent call last):
  File "httplib_context_bug.py", line 6, in <module>
    opener.open('https://www.google.com', timeout=2)
  File "/usr/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1240, in https_open
    context=self._context)
  File "/usr/lib/python2.7/urllib2.py", line 1166, in do_open
    h = http_class(host, timeout=req.timeout, **http_conn_args)
TypeError: __init__() got an unexpected keyword argument 'context'

$ python 
Python 2.7.9 (default, Mar  1 2015, 12:57:24)
$ cat /etc/issue
Debian GNU/Linux 8 \n \l
msg277422 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2016-09-26 14:31
You get advice on the Python user mailing list, https://mail.python.org/mailman/listinfo/python-list . The bug tracker is not a support/help forum.
msg277425 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-09-26 14:51
Christian: it looks like, unlike the original report, this one involves only stdlib code.  So maybe there really is a bug here.
msg277426 - (view) Author: SpaceOne (spaceone) * Date: 2016-09-26 14:54
Nice that you investigate again into this issue. Could you please test if this still happens on python 2.7.10 as I unfortunately have no environment with that version. This would be very kind of you!
History
Date User Action Args
2016-09-26 14:54:06spaceonesetmessages: + msg277426
2016-09-26 14:51:33r.david.murraysetmessages: + msg277425
2016-09-26 14:31:22christian.heimessetnosy: + christian.heimes

messages: + msg277422
stage: resolved
2016-09-26 14:28:38spaceonesetnosy: + spaceone
messages: + msg277421
2015-02-24 15:50:41Doumansetstatus: open -> closed
resolution: not a bug
messages: + msg236508
2015-02-23 20:55:52demian.brechtsetnosy: + demian.brecht
2015-02-23 10:26:30Doumansetmessages: + msg236433
2015-01-15 17:05:44Doumansetfiles: + callstack_urllib2_full

messages: + msg234088
2015-01-15 16:42:59Doumansetmessages: + msg234085
2015-01-15 16:36:58Doumansetmessages: + msg234084
2015-01-15 16:16:49Doumansetmessages: + msg234083
2015-01-15 16:12:25Doumansetmessages: + msg234082
2015-01-15 13:18:31r.david.murraysetnosy: + r.david.murray, alex
messages: + msg234073

components: + Library (Lib), - Extension Modules
type: compile error -> behavior
2015-01-15 06:01:34Doumancreate