classification
Title: urllib.request: opener not resetting content-length
Type: behavior Stage: resolved
Components: Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: asvetlov Nosy List: asvetlov, haypo, kachayev, orsenthil, python-dev, terry.reedy
Priority: normal Keywords: patch

Created on 2012-11-13 01:57 by terry.reedy, last changed 2014-03-19 16:34 by haypo. This issue is now closed.

Files
File name Uploaded Description Edit
issue16464.diff kachayev, 2012-11-17 22:29 review
issue16464_fixed.diff kachayev, 2012-11-22 17:02 review
Messages (10)
msg175485 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2012-11-13 01:57
Code based on python-list post by a do-not-wish-to-register urllib user.

import urllib.request
opener = urllib.request.build_opener()
request = urllib.request.Request("http://example.com/", headers =
        {"Content-Type": "application/x-www-form-urlencoded"})
print(request.data, '\n', request.header_items())

opener.open(request, "1".encode("us-ascii"))
print(request.data, '\n', request.header_items())

opener.open(request, "123456789".encode("us-ascii"))
print(request.data, '\n', request.header_items())
>>> 
None 
 [('Content-type', 'application/x-www-form-urlencoded')]
b'1' 
 [('Content-length', '1'), ('Host', 'example.com'), ('User-agent', 'Python-urllib/3.3'), ('Content-type', 'application/x-www-form-urlencoded')]
b'123456789' 
 [('Content-length', '1'), ('Host', 'example.com'), ('User-agent', 'Python-urllib/3.3'), ('Content-type', 'application/x-www-form-urlencoded')]

The first opener.open adds data and several headers to request, including content-length. The second changes the data but not the content-length. The docs do not say anything about this either way.
msg175814 - (view) Author: Alexey Kachayev (kachayev) * Date: 2012-11-17 22:29
This is special case for more "general" problem. When request is executed with HTTP client and data is not None, it calculates content length and adds special header to request. Then one can change request.data attribute value, but header "Content-length" is not changed in this case.

Patch is attached.

I implemented request.data as property and added method "remove_header" to deal problem. Test cases are also provided.
msg176108 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2012-11-22 14:52
I'm agree with solution, see my comments in review for the patch.
msg176109 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2012-11-22 14:52
Perhaps it's a bit new behavior and should be applied to 3.4 only.
msg176114 - (view) Author: Alexey Kachayev (kachayev) * Date: 2012-11-22 17:02
Fixed patch is attached.
Documentation is updated.
msg176492 - (view) Author: Roundup Robot (python-dev) Date: 2012-11-27 21:06
New changeset 618ea5612e83 by Andrew Svetlov in branch 'default':
Issue #16464: reset Request's Content-Length header on .data change.
http://hg.python.org/cpython/rev/618ea5612e83
msg176493 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2012-11-27 21:07
Pushed.
Thanks, Alexey.
msg184732 - (view) Author: Roundup Robot (python-dev) Date: 2013-03-20 04:15
New changeset b1579eb4e1bc by R David Murray in branch 'default':
#17485: Delete the Content-Length header if the data attribute is deleted.
http://hg.python.org/cpython/rev/b1579eb4e1bc
msg213097 - (view) Author: Roundup Robot (python-dev) Date: 2014-03-10 22:11
New changeset e6d862886e5c by R David Murray in branch 'default':
whatsnew: urllib Request objects are now reusable.
http://hg.python.org/cpython/rev/e6d862886e5c
msg214109 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-03-19 16:34
changeset:   89857:ad0c75b7bd7d
tag:         tip
parent:      89855:9120196b3114
parent:      89856:68335b8afb1f
user:        Victor Stinner <victor.stinner@gmail.com>
date:        Wed Mar 19 17:34:12 2014 +0100
description:
(Merge 3.4) Skip test_urllib2.test_issue16464() is the ssl module is missing


changeset:   89856:68335b8afb1f
branch:      3.4
parent:      89852:c44258b4b7a4
user:        Victor Stinner <victor.stinner@gmail.com>
date:        Wed Mar 19 17:31:20 2014 +0100
files:       Lib/test/test_urllib2.py
description:
Skip test_urllib2.test_issue16464() is the ssl module is missing
History
Date User Action Args
2014-03-19 16:34:43hayposetnosy: + haypo
messages: + msg214109
2014-03-10 22:11:21python-devsetmessages: + msg213097
2013-03-20 04:15:59python-devsetmessages: + msg184732
2012-11-27 21:19:34asvetlovsetstatus: open -> closed
assignee: asvetlov
resolution: fixed
stage: needs patch -> resolved
2012-11-27 21:07:52asvetlovsetmessages: + msg176493
versions: + Python 3.4, - Python 3.3
2012-11-27 21:06:31python-devsetnosy: + python-dev
messages: + msg176492
2012-11-22 17:03:00kachayevsetfiles: + issue16464_fixed.diff

messages: + msg176114
2012-11-22 14:52:55asvetlovsetmessages: + msg176109
2012-11-22 14:52:08asvetlovsetnosy: + asvetlov
messages: + msg176108
2012-11-17 22:29:13kachayevsetfiles: + issue16464.diff

nosy: + kachayev
messages: + msg175814

keywords: + patch
2012-11-13 01:57:33terry.reedycreate