Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IndexError thrown on email.message.Message.get #73598

Closed
uckelman mannequin opened this issue Feb 1, 2017 · 13 comments
Closed

IndexError thrown on email.message.Message.get #73598

uckelman mannequin opened this issue Feb 1, 2017 · 13 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes topic-email type-bug An unexpected behavior, bug, or error

Comments

@uckelman
Copy link
Mannequin

uckelman mannequin commented Feb 1, 2017

BPO 29412
Nosy @warsaw, @bitdancer, @maxking, @zhangyangyu, @uckelman
PRs
  • bpo-29412: patched string index out of range error in get_word function of _head… #6907
  • bpo-29412: Fix indexError when parsing a header value ending unexpectedly #14387
  • [3.8] bpo-29412: Fix indexError when parsing a header value ending unexpectedly (GH-14387) #14411
  • [3.7] bpo-29412: Fix indexError when parsing a header value ending unexpectedly (GH-14387) #14412
    		•
    Files
  • 29412.patch: patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2019-08-17.03:01:56.392>
created_at = <Date 2017-02-01.13:53:21.553>
labels = ['3.8', 'type-bug', '3.7', 'expert-email', '3.9']
title = 'IndexError thrown on email.message.Message.get'
updated_at = <Date 2019-08-17.03:01:56.384>
user = 'https://github.com/uckelman'

    bugs.python.org fields:

    activity = <Date 2019-08-17.03:01:56.384>
actor = 'maxking'
assignee = 'none'
closed = True
closed_date = <Date 2019-08-17.03:01:56.392>
closer = 'maxking'
components = ['email']
creation = <Date 2017-02-01.13:53:21.553>
creator = 'uckelman'
dependencies = []
files = ['46562']
hgrepos = []
issue_num = 29412
keywords = ['patch']
message_count = 13.0
messages = ['286631', '286633', '286635', '286845', '286867', '287232', '287242', '345631', '345632', '345633', '346676', '346689', '346690']
nosy_count = 5.0
nosy_names = ['barry', 'r.david.murray', 'maxking', 'xiang.zhang', 'uckelman']
pr_nums = ['6907', '14387', '14411', '14412']
priority = 'normal'
resolution = None
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue29412'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']
    @uckelman
    Copy link
    Mannequin Author

    uckelman mannequin commented Feb 1, 2017

    Test case:

      import email
  import email.policy
 
  txt = '''From: juckelman@strozfriedberg.co.uk
  To: (Recipient list suppressed)
  Date: Thu, 22 Aug 2013 04:13:02 +0000
  Subject: ADSF-1082
 
  Hey!
  '''
 
  msg = email.message_from_string(txt)
  msg.get('to')
  msg = email.message_from_string(txt, policy=email.policy.default)
  msg.get('to')

    The second msg.get() throws an IndexError:

      Traceback (most recent call last):
    File "test.py", line 16, in <module>
      print(msg.get('to'))    # throws IndexError
    File "/usr/lib64/python3.5/email/message.py", line 472, in get
      return self.policy.header_fetch_parse(k, v)
    File "/usr/lib64/python3.5/email/policy.py", line 153, in header_fetch_parse
      return self.header_factory(name, ''.join(value.splitlines()))
    File "/usr/lib64/python3.5/email/headerregistry.py", line 586, in __call__
      return self[name](name, value)
    File "/usr/lib64/python3.5/email/headerregistry.py", line 197, in __new__
      cls.parse(value, kwds)
    File "/usr/lib64/python3.5/email/headerregistry.py", line 337, in parse
      kwds['parse_tree'] = address_list = cls.value_parser(value)
    File "/usr/lib64/python3.5/email/headerregistry.py", line 328, in value_parser
      address_list, value = parser.get_address_list(value)
    File "/usr/lib64/python3.5/email/_header_value_parser.py", line 2336, in get_address_list
      token, value = get_address(value)
    File "/usr/lib64/python3.5/email/_header_value_parser.py", line 2313, in get_address
      token, value = get_group(value)
    File "/usr/lib64/python3.5/email/_header_value_parser.py", line 2269, in get_group
      token, value = get_display_name(value)
    File "/usr/lib64/python3.5/email/_header_value_parser.py", line 2095, in get_display_name
      token, value = get_phrase(value)
    File "/usr/lib64/python3.5/email/_header_value_parser.py", line 1770, in get_phrase
      token, value = get_word(value)
    File "/usr/lib64/python3.5/email/_header_value_parser.py", line 1745, in get_word
      if value[0]=='"':
  IndexError: string index out of range

    The docs say that email.policy.default has raise_on_defect set to False, hence parse errors ought to be reported via EmailMessage.defects, not by throwing an exception.

    @uckelman uckelman mannequin added topic-email type-bug An unexpected behavior, bug, or error labels Feb 1, 2017
    @uckelman uckelman mannequin changed the title IndexError thrown on email.message.M IndexError thrown on email.message.EmailMessage.get Feb 1, 2017
    @bitdancer
    Copy link
    Member

    Does the patch from bpo-27931 fix your problem as well? I haven't looked closely enough to see if I think it should, I'm just hoping :)

    @uckelman
    Copy link
    Mannequin Author

    uckelman mannequin commented Feb 1, 2017

    No dice. I get the same exception with issue27931_v2.patch. I briefly looked at the other two, and don't expect those will help, either.

    @uckelman uckelman mannequin changed the title IndexError thrown on email.message.EmailMessage.get IndexError thrown on email.message.Message.get Feb 1, 2017
    @zhangyangyu
    Copy link
    Member

    This seems not related to bpo-27931.

    The problem is that the receiver's content is only CFWS. It's just like it's empty and for the default policy, it checks value[0] == '"' in get_word.

    @bitdancer
    Copy link
    Member

    I'm really short on time to even review patches these days, but I'll see if I can pry any loose if someone wants to propose a patch.

    @bitdancer bitdancer added the 3.7 (EOL) end of life label Feb 3, 2017
    @uckelman
    Copy link
    Mannequin Author

    uckelman mannequin commented Feb 7, 2017

    I'm working on a patch now.

    @uckelman
    Copy link
    Mannequin Author

    uckelman mannequin commented Feb 7, 2017

    Here's a patch, complete with tests.

    If the value is all CFWS, then get_cfws(value)[1], which is what's left after the CFWS is extracted, is the empty string---which is why value[0] throws in this case.

    @maxking
    Copy link
    Contributor

    maxking commented Jun 14, 2019

    I can't reproduce this problem with the latest master branch, it was perhaps fixed with some other PR.

    This is also a dupe of bpo-31445.

    @barry, @david: I think this issue can be closed.

    @maxking
    Copy link
    Contributor

    maxking commented Jun 14, 2019

    For the record, this is how I tested using the master branch:

    >>> msg = email.message_from_string('  To: (Recipient list suppressed)')
>>> msg['To']
>>> import email.policy
>>> msg = email.message_from_string('  To: (Recipient list suppressed)', policy=email.policy.default)
>>> msg
<email.message.EmailMessage object at 0x7f377512b370>
>>> msg['To']
>>> msg.get('to')

    @maxking
    Copy link
    Contributor

    maxking commented Jun 14, 2019

    Nevermind, I was wrong, I was able to reproduce it:

    >>> msg = email.message_from_string('To: (Recipient list suppressed)', policy=email.policy.default))
  File "<stdin>", line 1
SyntaxError: unmatched ')'
>>> msg = email.message_from_string('To: (Recipient list suppressed)', policy=email.policy.default)
>>> msg.get('to')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/maxking/Documents/cpython/Lib/email/message.py", line 471, in get
    return self.policy.header_fetch_parse(k, v)
  File "/home/maxking/Documents/cpython/Lib/email/policy.py", line 163, in header_fetch_parse
    return self.header_factory(name, value)
  File "/home/maxking/Documents/cpython/Lib/email/headerregistry.py", line 589, in __call__
    return self[name](name, value)
  File "/home/maxking/Documents/cpython/Lib/email/headerregistry.py", line 197, in __new__
    cls.parse(value, kwds)
  File "/home/maxking/Documents/cpython/Lib/email/headerregistry.py", line 340, in parse
    kwds['parse_tree'] = address_list = cls.value_parser(value)
  File "/home/maxking/Documents/cpython/Lib/email/headerregistry.py", line 331, in value_parser
    address_list, value = parser.get_address_list(value)
  File "/home/maxking/Documents/cpython/Lib/email/_header_value_parser.py", line 1951, in get_address_list
    token, value = get_address(value)
  File "/home/maxking/Documents/cpython/Lib/email/_header_value_parser.py", line 1928, in get_address
    token, value = get_group(value)
  File "/home/maxking/Documents/cpython/Lib/email/_header_value_parser.py", line 1884, in get_group
    token, value = get_display_name(value)
  File "/home/maxking/Documents/cpython/Lib/email/_header_value_parser.py", line 1710, in get_display_name
    token, value = get_phrase(value)
  File "/home/maxking/Documents/cpython/Lib/email/_header_value_parser.py", line 1385, in get_phrase
    token, value = get_word(value)
  File "/home/maxking/Documents/cpython/Lib/email/_header_value_parser.py", line 1360, in get_word
    if value[0]=='"':
IndexError: string index out of range

    @warsaw
    Copy link
    Member

    warsaw commented Jun 26, 2019

    New changeset 7213df7 by Barry Warsaw (Abhilash Raj) in branch 'master':
    bpo-29412: Fix indexError when parsing a header value ending unexpectedly (GH-14387)
    7213df7

    @warsaw
    Copy link
    Member

    warsaw commented Jun 26, 2019

    New changeset b950cdb by Barry Warsaw (Miss Islington (bot)) in branch '3.7':
    bpo-29412: Fix indexError when parsing a header value ending unexpectedly (GH-14387) (GH-14412)
    b950cdb

    @warsaw
    Copy link
    Member

    warsaw commented Jun 26, 2019

    New changeset 82654a0 by Barry Warsaw (Miss Islington (bot)) in branch '3.8':
    bpo-29412: Fix indexError when parsing a header value ending unexpectedly (GH-14387) (GH-14411)
    82654a0

    @maxking maxking added 3.8 only security fixes 3.9 only security fixes labels Aug 17, 2019
    @maxking maxking closed this as completed Aug 17, 2019
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes topic-email type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @warsaw @bitdancer @maxking @zhangyangyu