classification
Title: email.parser: impossible to read messages encoded in a different encoding
Type: behavior Stage: test needed
Components: Library (Lib) Versions: Python 3.2, Python 3.1
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: barry Nosy List: Alex Quinn, barry, dato, r.david.murray, tercero12 (5)
Priority: high Keywords

Created on 2008-12-14 16:55 by dato, last changed 2009-11-15 14:19 by tercero12.

Messages (5)
msg77807 - (view) Author: Adeodato Simó (dato) Date: 2008-12-14 16:55
Currently, email.parser/feedparser can only parse messages that come 
as a string, or from a file opened in text mode.

Email messages, however, can contain 8bit characters in any encoding 
other than the local one (yet still be valid e-mails, of course), so I 
think a method is needed to have the parser be able to receive bytes. 
At the moment, and as far as I can see, it's not possible to parse 
some perfectly valid messages with python 3.0.

I don't think it's appropriate to ask that files be opened with the 
proper encoding, and then for the parser to read them. First, it is 
not possible to know what encoding that would be without parsing the 
message. And second, a message could contain parts in different 
encoding, and many mailboxes with random messages most certainly do.

Also, message objects will need a way to return a bytes repreentation, 
for the reasons explained above, and particularly if one wants to 
write back the message without modifying it.
msg89508 - (view) Author: R. David Murray (r.david.murray) Date: 2009-06-18 16:44
Is there any use case for feedparser accepting strings as input that
isn't a design error waiting to bite the programmer?
msg89509 - (view) Author: Barry A. Warsaw (barry) Date: 2009-06-18 17:48
dato: We've started some branches that try to address this, by exposing
both a read-a-buncha-bytes interface and a read-a-string interface.

rdm: As it turns out, yes.  There are use cases for reading a string
containing only ascii bytes.

In general, this is part of the big tricky problem in fixing the email
package for Python 3.x.  We had great debates about this at Pycon with
no resolution, and I think everyone has just been too busy to engage on
this since then. :(

email-sig is the best place to try to rally some effort.
msg91737 - (view) Author: Alex Quinn (Alex Quinn) Date: 2009-08-19 19:56
This bug also prevents the cgi module from handling POST data with 
multipart/form-data.  Consequently, 3.x cannot be readily used to write 
web apps for uploading files.  See #4953:
   http://bugs.python.org/issue4953
msg95293 - (view) Author: Timothy Farrell (tercero12) Date: 2009-11-15 14:19
Just an update for people interested:

The email team has a goal of fixing the email module in time for the 3.2
release.  There is the possibility of having to change some interfaces.
 See this document: http://wiki.python.org/moin/Email%20SIG/DesignThoughts
History
Date User Action Args
2009-11-15 14:19:41tercero12setmessages: + msg95293
2009-08-19 19:56:30Alex Quinnsetnosy: + Alex Quinn
messages: + msg91737
2009-08-18 18:59:09tercero12setnosy: + tercero12
2009-06-18 17:48:30barrysetmessages: + msg89509
2009-06-18 16:44:50r.david.murrayset
nosy: + r.david.murray
versions: + Python 3.1, Python 3.2, - Python 3.0
messages: + msg89508
priority: high
type: behavior
stage: test needed
2009-03-30 22:56:23ajaksu2linkissue1685453 dependencies
2008-12-14 17:24:43benjamin.petersonsetassignee: barry
nosy: + barry
2008-12-14 16:55:45datocreate