classification
Title: Imap lib implicit conversion from bytes to string
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.1, Python 3.2
process
Status: closed Resolution: fixed
Dependencies: Superseder: IMAP4 missing support for starttls
View: 4471
Assigned To: Nosy List: marcin.bachry, pitrou, r.david.murray, surprising42, vstinner
Priority: normal Keywords: patch

Created on 2009-08-19 13:17 by surprising42, last changed 2011-06-20 03:03 by r.david.murray. This issue is now closed.

Files
File name Uploaded Description Edit
imaplib.py surprising42, 2009-08-19 13:17 updated imap.py
imaplib-login.diff marcin.bachry, 2009-08-21 10:57
Messages (9)
msg91729 - (view) Author: Eric (surprising42) Date: 2009-08-19 13:17
using imaplib IMAP4_SSL, would fail at login due to TypeError, implicit
conversion from bytes object to string around line 1068 involving
function "_quote".

My fix:  
def _quote(self, arg):
        #Could not implicitly convert to bytes object to string
        arg = arg.encode('utf-8') #added this line to solve problem
        arg = arg.replace(b'\\', b'\\\\')
        arg = arg.replace(b'"', b'\\"')

        return b'"' + arg + b'"'
msg91733 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-08-19 15:10
See issue 1210 for background on the imaplib bytes/string issues.

A quick glance at the imaplib code leaves me confused.  _checkquote,
for example, appears to be mixing string comparisons and byte
comparisons.  Issue 1210 says imaplib _command should only quote
strings, but it does not appear at a quick glance to do any
quoting (the call to _checkquote is commented out).

It looks like the fix that would be consonant with the rest of
the code would be to convert the password to bytes using
the ASCII codec in the login method before calling _quote.

I'm adding Victor as nosy since he did the 1210 patch.
msg91748 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2009-08-19 21:29
IMAP4 protocol uses bytes, not characters. There is no "standard
charset", so you have to encode (manually) your login and password as
bytes. login method prototype:
  IMAP4.login(login: bytes, password: bytes)

You server may use UTF-8, but another server may use ISO-8859-1 or any
other charset.

The documentation should explain why the Python library uses bytes and
not "simply" characters.
msg91765 - (view) Author: Eric (surprising42) Date: 2009-08-20 09:34
I checked the latest documentation for 3.1.1
(http://docs.python.org/3.1/library/imaplib.html), but I can't find any
reference to needing to encode information myself for the login
procedure. Is there some other documentation you are referring to?

In any case, the error wasn't returned by the server, but by imaplib. If
the arg needs to be encoded for the whole process, then the arg should
be checked for the appropriate type (I think the doc even says "plain
text password"), or an optional encoding argument for the relevant
functions (or a catchall used when connecting to the server?) with a
default encoding attempted.
msg91767 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2009-08-20 10:28
> I can't find any reference to needing to encode information 
> myself for the login procedure. Is there some other 
> documentation you are referring to?

Exactly, the Python imaplib documentation should be fixed (the doc, not
the code).
msg91768 - (view) Author: Marcin Bachry (marcin.bachry) Date: 2009-08-20 10:53
It seems most IMAP4 methods accept str as arguments right now (I
checked: list, lsub, myrights, select, status, search, fetch) and
login() is a sole exception. I know the protocol is mostly ascii only,
but still having possibility of using str in the API feels convenient
and reasonable to me. Can't we convert str parameters to login() to
ascii too? It already done in other methods (the _command() method
iterates over args and converts them).
msg91775 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2009-08-20 14:24
> It seems most IMAP4 methods accept str as arguments right now (I
> checked: list, lsub, myrights, select, status, search, fetch) and
> login() is a sole exception. I know the protocol is mostly ascii only,
> but still having possibility of using str in the API feels convenient
> and reasonable to me.

Ok, that sounds like a good compromise. Can you write a patch?
msg91815 - (view) Author: Marcin Bachry (marcin.bachry) Date: 2009-08-21 10:57
Ok, it think it's good to leave internal _quote() function operating on
bytes and convert password argument in login(). The method now works
with both str and bytes as arguments.
msg138685 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-06-20 03:03
This was fixed in issue 4471 by Antoine when he added some tests that call login.  He fixed it by changing _quote to work with strings.  Per the discussion here I'm not sure this is the best fix, but until someone reports a bug with it it we may as well let it stand.
History
Date User Action Args
2011-06-20 03:03:26r.david.murraysetstatus: open -> closed

superseder: IMAP4 missing support for starttls

nosy: + pitrou
messages: + msg138685
resolution: fixed
stage: test needed -> resolved
2010-01-15 18:32:33r.david.murraylinkissue6897 superseder
2009-08-21 10:57:21marcin.bachrysetfiles: + imaplib-login.diff
keywords: + patch
messages: + msg91815
2009-08-20 14:24:46vstinnersetmessages: + msg91775
2009-08-20 10:53:43marcin.bachrysetnosy: + marcin.bachry
messages: + msg91768
2009-08-20 10:28:36vstinnersetmessages: + msg91767
2009-08-20 09:34:17surprising42setmessages: + msg91765
2009-08-19 21:29:43vstinnersetmessages: + msg91748
2009-08-19 15:10:35r.david.murraysetpriority: normal
versions: + Python 3.2
nosy: + vstinner, r.david.murray

messages: + msg91733

stage: test needed
2009-08-19 13:17:59surprising42create