Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

email.policy.SMTP.fold() issue for long filenames with spaces #74971

Closed
jhillacre mannequin opened this issue Jun 27, 2017 · 5 comments
Closed

email.policy.SMTP.fold() issue for long filenames with spaces #74971

jhillacre mannequin opened this issue Jun 27, 2017 · 5 comments
Labels
3.7 (EOL) end of life topic-email

Comments

@jhillacre
Copy link
Mannequin

jhillacre mannequin commented Jun 27, 2017

BPO 30788
Nosy @warsaw, @bitdancer, @jhillacre
PRs
  • bpo-30788: email.policy.SMTP.fold() issue for long filenames with spaces #2622
  • bpo-27240 Rewrite the email header folding algorithm. #3488
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2017-12-04.00:50:36.654>
created_at = <Date 2017-06-27.19:58:41.120>
labels = ['3.7', 'expert-email']
title = 'email.policy.SMTP.fold() issue for long filenames with spaces'
updated_at = <Date 2017-12-04.00:50:36.652>
user = 'https://github.com/jhillacre'

    bugs.python.org fields:

    activity = <Date 2017-12-04.00:50:36.652>
actor = 'r.david.murray'
assignee = 'none'
closed = True
closed_date = <Date 2017-12-04.00:50:36.654>
closer = 'r.david.murray'
components = ['email']
creation = <Date 2017-06-27.19:58:41.120>
creator = 'jhillacre'
dependencies = []
files = []
hgrepos = []
issue_num = 30788
keywords = ['patch']
message_count = 5.0
messages = ['297072', '299437', '301797', '302664', '307531']
nosy_count = 3.0
nosy_names = ['barry', 'r.david.murray', 'jhillacre']
pr_nums = ['2622', '3488']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue30788'
versions = ['Python 3.6', 'Python 3.7']
    @jhillacre
    Copy link
    Mannequin Author

    jhillacre mannequin commented Jun 27, 2017

    Found an issue in email.policy.SMTP.fold on "Content-Disposition" headers for long filenames with spaces. Managed to simply the issue to a test case as follows:

    >> from email.policy import SMTP; SMTP.fold('Content-Disposition', 'attachment; filename="{}"'.format('1234 67890' + '1234567890' * 6))

    Below are the tracebacks various python versions produces for this code. I think that the difference between python 3.5 - 3.6 and 3.7 are related to the fix in bpo-30532.

    Python 3.6.1 (default, Apr  7 2017, 09:32:32) & Python 3.5.3 (default, Jan 17 2017, 14:34:36)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python3.6/email/policy.py", line 183, in fold
    return self._fold(name, value, refold_binary=True)
  File "/usr/lib64/python3.6/email/policy.py", line 213, in _fold
    return self.header_factory(name, ''.join(lines)).fold(policy=self)
  File "/usr/lib64/python3.6/email/headerregistry.py", line 255, in fold
    return header.fold(policy=policy)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 300, in fold
    self._fold(folded)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 1228, in _fold
    rest._fold(folded)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 347, in _fold
    if folded.append_if_fits(part):
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 149, in append_if_fits
    token._fold(self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 338, in _fold
    if folded.append_if_fits(part, tstr):
... last 4 lines repeated 482 more times ...
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 149, in append_if_fits
    token._fold(self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 325, in _fold
    tstr = str(part)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 196, in __str__
    return ''.join(str(x) for x in self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 196, in <genexpr>
    return ''.join(str(x) for x in self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 196, in __str__
    return ''.join(str(x) for x in self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 196, in <genexpr>
    return ''.join(str(x) for x in self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 196, in __str__
    return ''.join(str(x) for x in self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 196, in <genexpr>
    return ''.join(str(x) for x in self)
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 633, in __str__
    return quote_string(''.join(str(x) for x in self))
  File "/usr/lib64/python3.6/email/_header_value_parser.py", line 633, in <genexpr>
    return quote_string(''.join(str(x) for x in self))
RecursionError: maximum recursion depth exceeded while getting the str of an object

Python 3.7.0a0 (heads/master:e613e6add5, Jun 27 2017, 12:17:18) 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/joel/PycharmProjects/cpython/Lib/email/policy.py", line 183, in fold
    return self._fold(name, value, refold_binary=True)
  File "/home/joel/PycharmProjects/cpython/Lib/email/policy.py", line 213, in _fold
    return self.header_factory(name, ''.join(lines)).fold(policy=self)
  File "/home/joel/PycharmProjects/cpython/Lib/email/headerregistry.py", line 255, in fold
    return header.fold(policy=policy)
  File "/home/joel/PycharmProjects/cpython/Lib/email/_header_value_parser.py", line 300, in fold
    self._fold(folded)
  File "/home/joel/PycharmProjects/cpython/Lib/email/_header_value_parser.py", line 1226, in _fold
    rest._fold(folded)
  File "/home/joel/PycharmProjects/cpython/Lib/email/_header_value_parser.py", line 345, in _fold
    if folded.append_if_fits(part):
  File "/home/joel/PycharmProjects/cpython/Lib/email/_header_value_parser.py", line 145, in append_if_fits
    ws = token.pop_leading_fws()
  File "/home/joel/PycharmProjects/cpython/Lib/email/_header_value_parser.py", line 272, in pop_leading_fws
    return self[0].pop_leading_fws()
  File "/home/joel/PycharmProjects/cpython/Lib/email/_header_value_parser.py", line 272, in pop_leading_fws
    return self[0].pop_leading_fws()
  File "/home/joel/PycharmProjects/cpython/Lib/email/_header_value_parser.py", line 270, in pop_leading_fws
    if self[0].token_type == 'fws':
IndexError: list index out of range

    Note that the following example without a space does not cause a traceback:

    Python 3.7.0a0 (heads/master:e613e6add5, Jun 27 2017, 12:17:18) 
>>> from email.policy import SMTP; SMTP.fold('Content-Disposition', 'attachment; filename="{}"'.format('1234567890' + '1234567890' * 6))
'Content-Disposition: attachment;\r\n filename="1234567890123456789012345678901234567890123456789012345678901234567890"\r\n'

    @jhillacre jhillacre mannequin added 3.7 (EOL) end of life topic-email labels Jun 27, 2017
    @jhillacre jhillacre mannequin changed the title email.policy.SMTP.fold() issue with long filenames with spaces email.policy.SMTP.fold() issue for long filenames with spaces Jun 27, 2017
    @jhillacre
    Copy link
    Mannequin Author

    jhillacre mannequin commented Jul 28, 2017

    Just wanted to ping this and see if there was someone available to review my PR associated with this issue.

    @bitdancer
    Copy link
    Member

    Thanks for the PR, but I've managd to finish my rewrite of the folder so that it not only doesn't traceback on this, but correctly folds it. I'll probably post the PR tomorrow.

    @jhillacre
    Copy link
    Mannequin Author

    jhillacre mannequin commented Sep 20, 2017

    I just took a quick peek at your PR to see if a similar test exists from this bug and it is looking good. Thanks for your work.

    @bitdancer
    Copy link
    Member

    The PR has been committed.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life topic-email
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @bitdancer