classification
Title: xmlrpclib confuses unicode and string
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: ezio.melotti, python-dev, uis, vstinner, wosc
Priority: normal Keywords: patch

Created on 2011-09-07 14:30 by wosc, last changed 2012-05-22 14:26 by uis. This issue is now closed.

Files
File name Uploaded Description Edit
xmlrpclib_unicode_host-2.patch vstinner, 2011-09-07 14:39
Messages (7)
msg143680 - (view) Author: Wolfgang Schnerring (wosc) Date: 2011-09-07 14:30
This is a similar issue to http://bugs.python.org/issue7093, but more insiduous:

This works:

xmlrpclib.ServerProxy(u'http://localhost:8080').foo(dict(baz=u'bär'))

While this fails with a UnicodeDecodeError (note the trailing slash in the URI):

xmlrpclib.ServerProxy(u'http://localhost:8080/').foo(dict(baz=u'bär'))

  File "/usr/local/python2.7/lib/python2.7/httplib.py", line 937, in endheaders
    self._send_output(message_body)
  File "/usr/local/python2.7/lib/python2.7/httplib.py", line 795, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 139: ordinal not in range(128)


So, somewhere in xmlrpclib, confusion happens, since even though the URI is passed in as unicode both times, it is stored as string in the first case (thus compatible with the serialized, utf-8 encoded string of the message body), but in the second case it remains unicode (thus failing, as #7093 tells, which I personally wouldn't have closed wontfix).
msg143682 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-09-07 14:36
> (thus failing, as #7093 tells, which I personally wouldn't have
> closed wontfix).

I don't know the right encoding to encode a HTTP header. In Python 3, http.client.HTTPConnection.putheader() encodes header name to ASCII and header values to ISO-8859-1.
msg143683 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-09-07 14:39
New patch using ISO-8859-1 instead of the default encoding (ASCII).
msg143697 - (view) Author: Wolfgang Schnerring (wosc) Date: 2011-09-07 17:49
I guess it should use the configured encoding[1] (which is utf-8 by default) to do that, shouldn't it? Since that's the encoding that is used for the message body, too.


[1] http://docs.python.org/library/xmlrpclib.html#xmlrpclib.ServerProxy
msg143698 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-09-07 17:52
"I guess it should use the configured encoding[1] (which is utf-8 by default) to do that, shouldn't it? Since that's the encoding that is used for the message body, too."

The URI is only used in HTTP headers, not in the body.
msg144431 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2011-09-22 23:31
New changeset c02e790c4535 by Victor Stinner in branch '2.7':
Issue #12931: xmlrpclib now encodes Unicode URI to ISO-8859-1, instead of
http://hg.python.org/cpython/rev/c02e790c4535

New changeset 5ceab07bcd02 by Victor Stinner in branch '3.2':
Issue #12931: Add a test with Unicode URI to test_xmlrpc
http://hg.python.org/cpython/rev/5ceab07bcd02

New changeset 3b46f2e2d280 by Victor Stinner in branch 'default':
Merge 3.2: Issue #12931: Add a test with Unicode URI to test_xmlrpc
http://hg.python.org/cpython/rev/3b46f2e2d280
msg161355 - (view) Author: Ulrich Seidl (uis) Date: 2012-05-22 14:26
The change set committed for 2.7 introduces another problem. At the beginning of xmlrpclib.py, there is an explicit test for the availability of unicode:

try:
    unicode
except NameError:
    unicode = None # unicode support not available

In case unicode was set to None, a TypeError:
isinstance() arg 2 must be a class, type, or tuple of classes and types

will be raised by the code introduced to ServerProxy:

if isinstance(uri, unicode):
    uri = uri.encode('ISO-8859-1')
History
Date User Action Args
2012-05-22 14:26:01uissetnosy: + uis
messages: + msg161355
2011-09-22 23:48:37ezio.melottisetstage: resolved
2011-09-22 23:32:31vstinnersetstatus: open -> closed
resolution: fixed
2011-09-22 23:31:10python-devsetnosy: + python-dev
messages: + msg144431
2011-09-17 15:52:38ezio.melottisetnosy: + ezio.melotti

versions: - Python 2.6
2011-09-07 17:52:22vstinnersetmessages: + msg143698
2011-09-07 17:49:26woscsetmessages: + msg143697
2011-09-07 14:39:09vstinnersetfiles: + xmlrpclib_unicode_host-2.patch
keywords: + patch
messages: + msg143683
2011-09-07 14:36:49vstinnersetnosy: + vstinner
messages: + msg143682
2011-09-07 14:30:06wosccreate