This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: While parsing email message id: UnboundLocalError
Type: crash Stage: resolved
Components: email Versions: Python 3.9, Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Nikita Hoffmann, barry, maxking, miss-islington, r.david.murray, xtreak
Priority: normal Keywords: patch

Created on 2019-11-05 13:03 by Nikita Hoffmann, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
samplescript.py Nikita Hoffmann, 2019-11-05 14:28 Sample script
Pull Requests
URL Status Linked Edit
PR 17277 merged Claudiu.Popa, 2019-11-20 08:33
PR 17476 merged miss-islington, 2019-12-05 17:22
PR 17503 merged maxking, 2019-12-08 05:56
PR 17514 merged miss-islington, 2019-12-09 01:35
Messages (10)
msg356030 - (view) Author: Nikita Hoffmann (Nikita Hoffmann) Date: 2019-11-05 13:03
Parsing an invalid email message id can throw a header parser error. A bug in parse_message_ still tries to append the unset token to a variable.

File "/opt/python-3.8.0/lib/python3.8/email/_header_value_parser.py", line 2116, in parse_message_
id
    message_id.append(token)
UnboundLocalError: local variable 'token' referenced before assignment

Version 3.7 is not affected.
msg356032 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-11-05 13:07
Thanks for the report. Can you please attach a sample script to reproduce this error?
msg356108 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-11-06 09:45
This was also reported in issue38708 with the original code added in issue35805. Commenting out the except clause also doesn't raise any error in test suite so I assume the code path was not tested. Maybe the script could be added as part of the test suite.

$ git diff
diff --git a/Lib/email/_header_value_parser.py b/Lib/email/_header_value_parser.py
index 1668b4a14e..9b6ca3a268 100644
--- a/Lib/email/_header_value_parser.py
+++ b/Lib/email/_header_value_parser.py
@@ -2110,10 +2110,10 @@ def parse_message_id(value):
     message_id = MessageID()
     try:
         token, value = get_msg_id(value)
+        message_id.append(token)
     except errors.HeaderParseError:
         message_id.defects.append(errors.InvalidHeaderDefect(
             "Expected msg-id but found {!r}".format(value)))
-    message_id.append(token)
     return message_id

 #
msg357405 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2019-11-24 18:00
More tests are always good :)

The "correct" solution here (as far as I remember, its has been a while since I've had time to even looked at the _header_value_parser code) would be to add a new 'invalid-msg-id' token, and do this:

    message_id = MessageID()
    try:
        token, value = get_msg_id(value)
        message_id.append(token)
    except HeaderParseError as ex:
        message_id = InvalidMessageID(value)
        message_id.defects.append(InvalidHeaderDefect(
            f"Invalid msg_id: {ex}"))
    return message_id
msg357407 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2019-11-24 18:47
Actually, the success path there should also check that value is empty, and if it is not register a defect for that as well.
msg357836 - (view) Author: miss-islington (miss-islington) Date: 2019-12-05 03:14
New changeset bb815499af855b1759c02535f8d7a9d0358e74e8 by Miss Islington (bot) (Claudiu Popa) in branch 'master':
bpo-38698: Prevent UnboundLocalError to pop up in parse_message_id (GH-17277)
https://github.com/python/cpython/commit/bb815499af855b1759c02535f8d7a9d0358e74e8
msg357874 - (view) Author: miss-islington (miss-islington) Date: 2019-12-05 17:42
New changeset e21aa61e96f8343200e765d119ebe778873a6bf1 by Miss Islington (bot) in branch '3.8':
bpo-38698: Prevent UnboundLocalError to pop up in parse_message_id (GH-17277)
https://github.com/python/cpython/commit/e21aa61e96f8343200e765d119ebe778873a6bf1
msg358044 - (view) Author: Abhilash Raj (maxking) * (Python committer) Date: 2019-12-09 01:35
New changeset 68157da8b42b26408af5d157d2dba4fcf29c6320 by Abhilash Raj in branch 'master':
bpo-38698: Add a new InvalidMessageID token to email header parser. (GH-17503)
https://github.com/python/cpython/commit/68157da8b42b26408af5d157d2dba4fcf29c6320
msg358046 - (view) Author: miss-islington (miss-islington) Date: 2019-12-09 02:11
New changeset f66f4a09d0b6817fe6a86a567fd506aa223f1563 by Miss Islington (bot) in branch '3.8':
bpo-38698: Add a new InvalidMessageID token to email header parser. (GH-17503)
https://github.com/python/cpython/commit/f66f4a09d0b6817fe6a86a567fd506aa223f1563
msg358049 - (view) Author: Abhilash Raj (maxking) * (Python committer) Date: 2019-12-09 02:36
Closing this as fixed.
History
Date User Action Args
2022-04-11 14:59:22adminsetgithub: 82879
2019-12-25 12:52:17xtreaklinkissue39119 superseder
2019-12-09 02:36:06maxkingsetstatus: open -> closed
resolution: fixed
messages: + msg358049

stage: patch review -> resolved
2019-12-09 02:11:37miss-islingtonsetmessages: + msg358046
2019-12-09 01:35:51miss-islingtonsetpull_requests: + pull_request16991
2019-12-09 01:35:44maxkingsetmessages: + msg358044
2019-12-08 05:56:27maxkingsetpull_requests: + pull_request16980
2019-12-05 17:42:08miss-islingtonsetmessages: + msg357874
2019-12-05 17:22:46miss-islingtonsetpull_requests: + pull_request16957
2019-12-05 03:14:33miss-islingtonsetnosy: + miss-islington
messages: + msg357836
2019-11-24 18:47:58r.david.murraysetmessages: + msg357407
2019-11-24 18:00:51r.david.murraysetmessages: + msg357405
2019-11-20 08:33:51Claudiu.Popasetkeywords: + patch
stage: patch review
pull_requests: + pull_request16770
2019-11-06 09:45:31xtreaksetmessages: + msg356108
versions: + Python 3.9
2019-11-05 14:28:42Nikita Hoffmannsetfiles: + samplescript.py
2019-11-05 13:07:36xtreaksetnosy: + xtreak, maxking
messages: + msg356032
2019-11-05 13:03:28Nikita Hoffmanncreate