This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: AttributeError thrown by urllib.open_http
Type: Stage:
Components: Library (Lib) Versions: Python 2.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: akuchling, barry, georg.brandl, georg.brandl, jjlee, rhettinger, robzed, varmaa, zenzen
Priority: high Keywords:

Created on 2003-07-07 12:52 by zenzen, last changed 2022-04-10 16:09 by admin. This issue is now closed.

Messages (13)
msg16924 - (view) Author: Stuart Bishop (zenzen) Date: 2003-07-07 12:52
In 2.3b2, looks like an error condition isn't being picked up 
on line 300 or 301 of urllib.py.

The code that triggered this traceback was simply:
        url = urllib.urlopen(action, data)


Traceback (most recent call last):
  File "autospamrep.py", line 170, in ?
    current_page = handle_spamcop_page(current_page)
  File "autospamrep.py", line 140, in handle_spamcop_page
    url = urllib.urlopen(action, data)
  File "/Library/Frameworks/Python.framework/Versions/2.3/
lib/python2.3/urllib.py", line 78, in urlopen
    return opener.open(url, data)
  File "/Library/Frameworks/Python.framework/Versions/2.3/
lib/python2.3/urllib.py", line 183, in open
    return getattr(self, name)(url, data)
  File "/Library/Frameworks/Python.framework/Versions/2.3/
lib/python2.3/urllib.py", line 308, in open_http
    return self.http_error(url, fp, errcode, errmsg, headers, 
data)
  File "/Library/Frameworks/Python.framework/Versions/2.3/
lib/python2.3/urllib.py", line 323, in http_error
    return self.http_error_default(url, fp, errcode, errmsg, 
headers)
  File "/Library/Frameworks/Python.framework/Versions/2.3/
lib/python2.3/urllib.py", line 551, in http_error_default
    return addinfourl(fp, headers, "http:" + url)
  File "/Library/Frameworks/Python.framework/Versions/2.3/
lib/python2.3/urllib.py", line 837, in __init__
    addbase.__init__(self, fp)
  File "/Library/Frameworks/Python.framework/Versions/2.3/
lib/python2.3/urllib.py", line 787, in __init__
    self.read = self.fp.read
AttributeError: 'NoneType' object has no attribute 'read'
msg16925 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2003-07-09 05:50
Logged In: YES 
user_id=80475

What were the values of 'action' and 'data' when the call was 
made?
msg16926 - (view) Author: John J Lee (jjlee) Date: 2003-07-12 13:14
Logged In: YES 
user_id=261020

HTTPResponse.read returns '' if its .fp is None, but the 
backwards-compat HTTP class' .getfile() just returns self.file, 
which it previously grabbed from HTTPResponse in .getreply(). 
 
Wild guess: maybe HTTP.getreply should just do 
 
self.file = response 
 
rather than 
 
self.file = response.fp 
 
The object returned by HTTP.getfile() was documented as 
returning an object supporting .readline() and .readlines(), 
while HTTPResponse only supports .read(), so that's obviously 
not the whole solution. 
 
msg16927 - (view) Author: Stuart Bishop (zenzen) Date: 2003-07-17 04:34
Logged In: YES 
user_id=46639

I've finally managed to get another traceback with some more 
information, using an assert I inserted into urllib.py a while ago to 
catch .fp == None:

Traceback (most recent call last):
  File "/Users/zen/bin/autospamrep.py", line 168, in ?
    current_page = urllib.urlopen(start_url).read()
  File "/Library/Frameworks/Python.framework/Versions/2.3/lib/
python2.3/urllib.py", line 76, in urlopen
    return opener.open(url)
  File "/Library/Frameworks/Python.framework/Versions/2.3/lib/
python2.3/urllib.py", line 181, in open
    return getattr(self, name)(url)
  File "/Library/Frameworks/Python.framework/Versions/2.3/lib/
python2.3/urllib.py", line 305, in open_http
    assert fp is not None, 'errcode %r, errmsg %r, headers %r' % 
(errcode, errmsg, headers)
AssertionError: errcode -1, errmsg '', headers None
msg16928 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2003-07-29 03:23
Logged In: YES 
user_id=12800

Please provide a self-contained, complete example that we
can use to reproduce this problem.  Without enough
information, I can't see us fixing this for Python 2.3, and
time for that is rapidly running out.

Lowering to priority 6.
msg16929 - (view) Author: Rob Probin (robzed) Date: 2004-03-17 22:24
Logged In: YES 
user_id=1000470

""" 
This comment is program to reproduce the problem. Sorry it's not an 
attachment - as a relative Sourceforge newbie I have no idea how to 
attach to an existing bug. More notes available via email if required - 
including all local variables for each function from post mortem.

As said before, seems to be fp = None. Although the exception is caused 
by the 'self.read = self.fp.read', it looks like 'fp = h.getfile()' inside 
open_http()

This is repeatable, but you may have to run this more than once. 
(Apologies to noaa.gov).

*** PLEASE: Run only where absolutely necessary for reproduction of 
bug!!! ***

"""

""" Attribute Error test case  - Python 2.3 """

import urllib

url = "http://adds.aviationweather.noaa.gov/metars/index.php"

params = urllib.urlencode({ "station_ids" : "KJFK", 
				"hoursStr" : "most recent only", 
				"std_trans" : "standard", 
				"chk_metars" : "on",
				"submit":"Submit"})

print "test"

for i in range(1, 1000):
	x = urllib.urlopen(url, params)
	string = x.read()
	print i

"""
Local variables for middle level routine...

	classURLopener
	open_http(self, url, data=None)
		args	('User-agent', 'Python-urllib/1.15')
		auth	None
		data	
'hoursStr=most+recent+only&station_ids=KJFK&std_trans=standard&sub
mit=Submit&chk_metars=on'
		errcode	-1
		errmsg	''
		fp	None
		h	<httplib.HTTP instance at 0x507df30>
		headers	None
		host	'adds.aviationweather.noaa.gov'
		httplib	<module 'httplib' from '/System/Library/Frameworks/
Python.framework/Versions/2.3/lib/python2.3/httplib.pyc'>
		realhost	'adds.aviationweather.noaa.gov'
		selector	'/metars/index.php'
		self	<urllib.FancyURLopener instance at 0x465f3c8>
		url	'//adds.aviationweather.noaa.gov/metars/index.php'
		user_passwd	None
"""
msg16930 - (view) Author: Rob Probin (robzed) Date: 2004-03-18 23:43
Logged In: YES 
user_id=1000470

The file pointer (fp) is None (inside urllib) from httplib. This appears to 
be caused by a BadStatusLine exception in getreply() (line1016 httplib). 

This sets self.file to self._conn.sock.makefile('rb', 0) then does a 
self.close() which sets self.file to None. 

Being new to this peice of code, I'm not sure whether it's urllib assuming 
the file isn't going to be closed, or the BadStatusLine exception clearing 
the file. Certainly it looks like the error -1 is not being trapped by 
open_http() in urllib upon calling h.getreply() and assuming that the file 
still exists even in an error condition?

It maybe a coincidence but it appears to occur more when a web browser 
on the same machine is refreshing. 

Regards
Rob
msg16931 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2005-01-07 01:37
Logged In: YES 
user_id=80475

Andrew, are you still working on this one?
msg16932 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2005-01-07 12:39
Logged In: YES 
user_id=11375

No, not at this point in time.  Unassigning (or, if this bug
is on the radar for 2.3.5/2.4.1, I can find time to work on it).
-
msg16933 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2005-01-07 12:39
Logged In: YES 
user_id=11375

No, not at this point in time.  Unassigning (or, if this bug
is on the radar for 2.3.5/2.4.1, I can find time to work on it).
-
msg16934 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2005-12-15 22:11
Logged In: YES 
user_id=1188172

Further information can be found in #1163401 which has been
closed as a duplicate.
msg16935 - (view) Author: Atul Varma (varmaa) Date: 2007-02-25 00:58
I have attempted to fix this bug in patch 1668132:

http://sourceforge.net/tracker/index.php?func=detail&aid=1668132&group_id=5470&atid=305470
msg16936 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2007-03-14 08:28
Fixed in rev. 54376, 54377 (2.5). Raises IOError now.
History
Date User Action Args
2022-04-10 16:09:47adminsetgithub: 38809
2003-07-07 12:52:22zenzencreate