Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AttributeError when parsing multipart email with invalid non-decodable Content-Transfer-Encoding #75018

Open
ajdlinux mannequin opened this issue Jul 3, 2017 · 5 comments
Open

AttributeError when parsing multipart email with invalid non-decodable Content-Transfer-Encoding #75018

ajdlinux mannequin opened this issue Jul 3, 2017 · 5 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes stdlib Python modules in the Lib dir topic-email type-bug An unexpected behavior, bug, or error

Comments

@ajdlinux
Copy link
Mannequin

ajdlinux mannequin commented Jul 3, 2017

BPO 30835
Nosy @warsaw, @bitdancer, @ajdlinux, @miss-islington, @websurfer5
PRs
  • bpo-30835: email: Fix AttributeError when parsing invalid Content-Tra… #2544
  • bpo-30835: email: Fix AttributeError when parsing invalid CTE #13598
  • [3.7] bpo-30835: email: Fix AttributeError when parsing invalid CTE (GH-13598) #13820
  • [3.6] bpo-30835: email: Fix AttributeError when parsing invalid CTE (GH-13598) #13821
    		•
    Files
  • testprog.py: Example program that demonstrates the error
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2017-07-03.13:37:03.005>
labels = ['3.7', '3.8', 'type-bug', 'library', 'expert-email']
title = 'AttributeError when parsing multipart email with invalid non-decodable Content-Transfer-Encoding'
updated_at = <Date 2019-06-05.10:23:41.770>
user = 'https://github.com/ajdlinux'

    bugs.python.org fields:

    activity = <Date 2019-06-05.10:23:41.770>
actor = 'miss-islington'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'email']
creation = <Date 2017-07-03.13:37:03.005>
creator = 'Andrew Donnellan'
dependencies = []
files = ['46989']
hgrepos = []
issue_num = 30835
keywords = ['patch']
message_count = 4.0
messages = ['297584', '297771', '344620', '344698']
nosy_count = 5.0
nosy_names = ['barry', 'r.david.murray', 'Andrew Donnellan', 'miss-islington', 'Jeffrey.Kintscher']
pr_nums = ['2544', '13598', '13820', '13821']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30835'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']
    @ajdlinux
    Copy link
    Mannequin Author

    ajdlinux mannequin commented Jul 3, 2017

    Parsing an email containing a multipart Content-Type, along with a Content-Transfer-Encoding containing an invalid (non-ASCII-decodable) byte will fail.

    email.feedparser.FeedParser._parsegen() calls "self._cur.get('content-transfer-encoding', '8bit')" to get the header.

    It then tries to check whether the C-T-E is in the allowable set of ('7bit', '8bit', 'binary'), and to do so case-insensitively, it tries to convert the header to lowercase. However, because there's an invalid character in there, it's dealing with a Header object rather than a str. Hence it throws an AttributeError.

    Correct behaviour would be to convert the Header to a str, see that it's not valid, and continue on to handle the defect as usual.

    Thanks to Daniel Axtens for finding this bug as he was running the AFL fuzzer on the email parsing code in Patchwork (https://github.com/getpatchwork/patchwork).

    Pull request incoming.

    @ajdlinux ajdlinux mannequin added 3.7 (EOL) end of life stdlib Python modules in the Lib dir topic-email type-bug An unexpected behavior, bug, or error labels Jul 3, 2017
    @bitdancer
    Copy link
    Member

    There's a deeper problem here involving how Header is used in compat32 that I've been aware of for a while but haven't had time to try to think through a fix for (there may not be one, given the history of the compat32 code). In the meantime, the proposed fix is reasonable. (It isn't needed for the new policies, but it doesn't hurt.)

    @serhiy-storchaka serhiy-storchaka added the 3.8 only security fixes label Dec 6, 2018
    @warsaw
    Copy link
    Member

    warsaw commented Jun 4, 2019

    New changeset aa79707 by Barry Warsaw (Abhilash Raj) in branch 'master':
    bpo-30835: email: Fix AttributeError when parsing invalid CTE (GH-13598)
    aa79707

    @miss-islington
    Copy link
    Contributor

    New changeset f62a372 by Miss Islington (bot) in branch '3.7':
    bpo-30835: email: Fix AttributeError when parsing invalid CTE (GH-13598)
    f62a372

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @furkanonder
    Copy link
    Contributor

    @warsaw The issue seems to be solved. Can we close it?

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life 3.8 only security fixes stdlib Python modules in the Lib dir topic-email type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    5 participants
    @warsaw @bitdancer @serhiy-storchaka @furkanonder @miss-islington