classification
Title: email.message.Message flatten dies of list index out of range
Type: behavior Stage: resolved
Components: email, Library (Lib) Versions: Python 3.4, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: HJarausch, barry, janmalte, python-dev, r.david.murray
Priority: normal Keywords:

Created on 2012-12-29 10:23 by HJarausch, last changed 2015-11-20 15:04 by r.david.murray. This issue is now closed.

Messages (7)
msg178482 - (view) Author: Helmut Jarausch (HJarausch) Date: 2012-12-29 10:23
The following code triggers the bug:

#!/usr/bin/python3.3
#-*- coding: latin1 -*-
from email.message import Message
from email import policy
from email.parser import FeedParser

Parser= FeedParser(policy=policy.SMTP)
Parser.feed('From jarausch@igpm.rwth-aachen.de  Tue Apr 24 15:09:24 2012\n')
Parser.feed('X-Status:    \n')  # this triggers the bug
Parser.feed('From: Helmut Jarausch <jarausch@igpm.rwth-aachen.de>\n')

Msg= Parser.close()
Msg_as_str= Msg.as_string(unixfrom=True)

This triggers
  File "Email_Parse_Bug.py", line 13, in <module>
    Msg_as_str= Msg.as_string(unixfrom=True)
  File "/usr/lib64/python3.3/email/message.py", line 151, in as_string
    g.flatten(self, unixfrom=unixfrom)
  File "/usr/lib64/python3.3/email/generator.py", line 112, in flatten
    self._write(msg)
  File "/usr/lib64/python3.3/email/generator.py", line 171, in _write
    self._write_headers(msg)
  File "/usr/lib64/python3.3/email/generator.py", line 198, in _write_headers
    self.write(self.policy.fold(h, v))
  File "/usr/lib64/python3.3/email/policy.py", line 153, in fold
    return self._fold(name, value, refold_binary=True)
  File "/usr/lib64/python3.3/email/policy.py", line 176, in _fold
    (len(lines[0])+len(name)+2 > maxlen or
IndexError: list index out of range


If I strip the '\n' from the lines feeded to the Parser,
the bug does not occur.

Thanks for looking into it,
Helmut.

This is with  Python 3.3.0+ 3.3:ccc372b37fbb+
msg178502 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-12-29 15:25
Thanks for the report.

We thought we fixed this once already (in issue 11401).  It looks like that only fixed the "old" folding algorithm, and that the new one has the same bug.  I thought the test ran against both algorithms, but I must have made a mistake.
msg181385 - (view) Author: Roundup Robot (python-dev) Date: 2013-02-04 20:25
New changeset e64b74227198 by R David Murray in branch '3.3':
#16811: Fix folding of headers with no value in provisional policies.
http://hg.python.org/cpython/rev/e64b74227198

New changeset fe7f3e2e49ce by R David Murray in branch 'default':
Merge #16811: Fix folding of headers with no value in provisional policies.
http://hg.python.org/cpython/rev/fe7f3e2e49ce
msg181386 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-02-04 20:28
Fixed, thanks.  There are some other issues with folding values consisting of only blanks, but I'll deal with that in the context of other issues.  With this fix the new folding algorithm works at least as well as the old folding algorithm on blank values.
msg181443 - (view) Author: Roundup Robot (python-dev) Date: 2013-02-05 15:18
New changeset 4553dfcafac7 by R David Murray in branch '3.3':
News item for issue #16811 fix.
http://hg.python.org/cpython/rev/4553dfcafac7

New changeset 68be406e76e1 by R David Murray in branch 'default':
Merge: News item for issue #16811 fix.
http://hg.python.org/cpython/rev/68be406e76e1
msg254974 - (view) Author: Jan Malte (janmalte) Date: 2015-11-20 13:41
seems like this happens still, if the CC header starts with a new line character.

Python 3.4.3
msg254986 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-11-20 15:04
Please open a new issue and provide a reproducer.  Thanks.
History
Date User Action Args
2015-11-20 15:04:47r.david.murraysetmessages: + msg254986
2015-11-20 13:41:29janmaltesetnosy: + janmalte
messages: + msg254974
2013-02-05 15:18:57python-devsetmessages: + msg181443
2013-02-04 20:28:05r.david.murraysetstatus: open -> closed
resolution: fixed
messages: + msg181386

stage: needs patch -> resolved
2013-02-04 20:25:22python-devsetnosy: + python-dev
messages: + msg181385
2012-12-29 15:25:10r.david.murraysetmessages: + msg178502
2012-12-29 15:13:58r.david.murraysetversions: + Python 3.4
nosy: + barry, r.david.murray

components: + email
type: crash -> behavior
stage: needs patch
2012-12-29 10:23:14HJarauschcreate