Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

email package should work better with unicode #44753

Closed
warsaw opened this issue Mar 21, 2007 · 7 comments
Closed

email package should work better with unicode #44753

warsaw opened this issue Mar 21, 2007 · 7 comments
Assignees
@bitdancer
Labels
stdlib Python modules in the Lib dir topic-unicode type-bug An unexpected behavior, bug, or error

Comments

@warsaw
Copy link
Member

warsaw commented Mar 21, 2007

BPO 1685453
Nosy @warsaw, @devdanzin, @merwok, @bitdancer

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = 'https://github.com/bitdancer'
closed_at = <Date 2010-12-27.17:04:58.053>
created_at = <Date 2007-03-21.18:39:03.000>
labels = ['type-bug', 'library', 'expert-unicode']
title = 'email package should work better with unicode'
updated_at = <Date 2011-12-05.18:14:12.309>
user = 'https://github.com/warsaw'

bugs.python.org fields:

activity = <Date 2011-12-05.18:14:12.309>
actor = 'r.david.murray'
assignee = 'r.david.murray'
closed = True
closed_date = <Date 2010-12-27.17:04:58.053>
closer = 'r.david.murray'
components = ['Library (Lib)', 'Unicode']
creation = <Date 2007-03-21.18:39:03.000>
creator = 'barry'
dependencies = []
files = []
hgrepos = []
issue_num = 1685453
keywords = []
message_count = 7.0
messages = ['31612', '84700', '84753', '100550', '124715', '148880', '148882']
nosy_count = 9.0
nosy_names = ['barry', 'ocean-city', 'ajaksu2', 'eric.araujo', 'r.david.murray', 'bgamari', 'l0nwlf', 'pebbe', 'sivang']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue1685453'
versions = ['Python 2.6', 'Python 3.1', 'Python 2.7', 'Python 3.2']
@warsaw
Copy link
Member Author

warsaw commented Mar 21, 2007

This is a catch-all issue for improving the email package's handling of unicode. For now, please add issues/problems you find with email & unicode to this tracker item.

For example:

MIMEText()'s first argument should accept a unicode if _charset is also given. It should not be necessary to manually encode the first argument into an 8-bit string.

@warsaw warsaw self-assigned this Mar 21, 2007
@warsaw warsaw added the stdlib Python modules in the Lib dir label Mar 21, 2007
@warsaw warsaw self-assigned this Mar 21, 2007
@warsaw warsaw added the stdlib Python modules in the Lib dir label Mar 21, 2007
@devdanzin
Copy link
Mannequin

devdanzin mannequin commented Mar 30, 2009

Link to bpo-1681333, bpo-4487, bpo-1443875, bpo-1555842, bpo-4661, bpo-1078919, bpo-963906,
bpo-1379416 and bpo-1368247.

@devdanzin devdanzin mannequin added topic-unicode type-bug An unexpected behavior, bug, or error labels Mar 30, 2009
@ocean-city
Copy link
Mannequin

ocean-city mannequin commented Mar 31, 2009

Probably these are related too. bpo-5259 bpo-5304

@pebbe
Copy link
Mannequin

pebbe mannequin commented Mar 6, 2010

In Python 3.1.1, email.mime.text.MIMEText accepts an 8-bit charset, but not utf-8.

I think you should not have to specify a charset. All strings are unicode now, so I think the package should choose an appropriate charset based on the characters in the text, us-ascii, some iso-8859 charset, or utf-8, whatever fits.

Python 3.1.1 (r311:74480, Oct  2 2009, 11:50:52)                                                                                              
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2                                                                                 
Type "help", "copyright", "credits" or "license" for more information.                                                                        
>>> from email.mime.text import MIMEText                                                                                                      
>>> text = 'H\u00e9'                                                                                                                          
>>> msg = MIMEText(text, 'plain', 'iso-8859-1')                                                                                               
>>> print(msg.as_string())                                                                                                                    
Content-Type: text/plain; charset="iso-8859-1"                                                                                                
MIME-Version: 1.0                                                                                                                             
Content-Transfer-Encoding: quoted-printable                                                                                                   
                                                                                                                                              
H=E9                                                                                                                                          
>>> msg = MIMEText(text, 'plain', 'utf-8')                                                                                                    
Traceback (most recent call last):                                                                                                            
  File "/my/opt/Python-3/lib/python3.1/email/message.py", line 269, in set_charset                                                            
    cte(self)                                                                                                                                 
TypeError: 'str' object is not callable                                                                                                       
                                                                                                                                              
During handling of the above exception, another exception occurred:                                                                           
                                                                                                                                              
Traceback (most recent call last):                                                                                                            
  File "<stdin>", line 1, in <module>                                                                                                         
  File "/my/opt/Python-3/lib/python3.1/email/mime/text.py", line 30, in __init__                                                              
    self.set_payload(_text, _charset)                                                                                                         
  File "/my/opt/Python-3/lib/python3.1/email/message.py", line 234, in set_payload                                                            
    self.set_charset(charset)                                                                                                                 
  File "/my/opt/Python-3/lib/python3.1/email/message.py", line 271, in set_charset                                                            
    self._payload = charset.body_encode(self._payload)                                                                                        
  File "/my/opt/Python-3/lib/python3.1/email/charset.py", line 380, in body_encode                                                            
    return email.base64mime.body_encode(string)                                                                                               
  File "/my/opt/Python-3/lib/python3.1/email/base64mime.py", line 94, in body_encode                                                          
    enc = b2a_base64(s[i:i + max_unencoded]).decode("ascii")                                                                                  
TypeError: must be bytes or buffer, not str                                                                                                   
>>>

@warsaw warsaw assigned bitdancer and unassigned warsaw May 5, 2010
@bitdancer
Copy link
Member

Now that we are primarily focused on Python3 development, collecting "unicode" issues is not really all that useful (at least not to me, and I'm currently doing the email maintenance), so I'm closing this. All the relevant issues are assigned to me anyway, so I'll be dealing with them by and by.

@sivang
Copy link
Mannequin

sivang mannequin commented Dec 5, 2011

I am having hard time parsing all the text/html and text/plain parts of a message, concatenating them into a string. I am thinking of writing some custom code to do manual handling of this...

If this could be fixed that would be great. The issues are converting from and to ascii/unicode or whatever encoding/charset the part uses.

@bitdancer
Copy link
Member

That particular problem will get fixed in the next version of the email package (hopefully in Python3.3), but that isn't ready yet.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bitdancer bitdancer

Labels
stdlib Python modules in the Lib dir topic-unicode type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@warsaw @bitdancer