classification
Title: Not all defects pass through email policy
Type: behavior Stage: patch review
Components: email Versions: Python 3.6, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: barry, martin.panter, r.david.murray
Priority: normal Keywords: easy, patch

Created on 2015-06-02 16:39 by r.david.murray, last changed 2016-06-25 21:03 by r.david.murray.

Files
File name Uploaded Description Edit
defect-fixes.patch martin.panter, 2016-06-20 11:49
defect-fixes.v2.patch martin.panter, 2016-06-25 10:27 review
Messages (6)
msg244679 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-06-02 16:39
This is a note to myself.  While looking at another issue I noticed that I never completed the work to make sure that all message defects discovered by feedparser are passed through the policy defect handler.  The fix is simple, the time consuming part is writing the tests.
msg268642 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-06-16 04:38
In particular, I noticed InvalidHeaderDefect is not raised if a header line begins with a colon (:), and also MisplacedEnvelopeHeaderDefect, if a header line begins "From ", including a space, and is neither the first or last line of the header section.
msg268895 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-06-20 11:49
Here is a patch to fix the two defects in Lib/email/feedparser.py. I also made some extra changes to the tests:

* Remove some tests from Lib/test/test_email/test_email.py, because I think they were redundant with methods of the same name in test_defect_handling.py.

* Change a couple of the tests in test_defect_handling.py to verify there are no defects recorded at all even in raise_on_defect=True mode.
msg268987 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-06-21 12:45
test_email/test_email tests the compat32 policy.  test_defect_handling tests the default policy.  Thus they are not redundant, except of course that they are...it would be better to run the same test with both policies in turn, but I never got as far as setting up that infrastructure.

That said, for defects it may not matter, and for the other "redudant" tests sometimes the result is different, which is the point of the new policy.  So, things are in a bit of a limbo currently and could use some sorting out.  You'll note that in some cases in test_email I've noted where the corresponding test for the new policy is in the new test files.  There are a bunch more tests in test_email that need equivalents for the new policies (and I'm sure we'll find some bugs when that happens, unfortunately).
msg269224 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-06-25 10:27
I see that “compat32” and “default” are actually different things (Compat32 vs EmailPolicy classes). IMO “default” was a bad choice of name, but I guess that ship has already sailed.

This patch adds a test_defect_handling.TestCompat32 subclass, so the test methods are now run with compat32 as well as the “default” policy.
msg269257 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2016-06-25 21:03
The name default is future proofing.  It will be the default...eventually :)  And it is the default now if you use EmailMessage to construct your messages.
History
Date User Action Args
2016-06-25 21:03:42r.david.murraysetmessages: + msg269257
2016-06-25 10:27:58martin.pantersetfiles: + defect-fixes.v2.patch

messages: + msg269224
2016-06-21 12:45:53r.david.murraysetmessages: + msg268987
2016-06-20 11:49:07martin.pantersetfiles: + defect-fixes.patch
versions: - Python 3.4
messages: + msg268895

keywords: + patch
stage: needs patch -> patch review
2016-06-20 08:49:57martin.panterlinkissue27328 dependencies
2016-06-16 04:38:17martin.pantersetnosy: + martin.panter
messages: + msg268642
2015-06-02 16:39:13r.david.murraycreate