classification
Title: None.splitlines raises AttributeError in email.feedparser
Type: enhancement Stage: resolved
Components: email Versions: Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: barry, r.david.murray, sshnaidm, terry.reedy, xtreak
Priority: normal Keywords: patch

Created on 2018-07-12 12:13 by sshnaidm, last changed 2018-07-13 21:44 by terry.reedy. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8266 closed sshnaidm, 2018-07-12 12:25
Messages (7)
msg321544 - (view) Author: Sagi (Sergey) Shnaidman (sshnaidm) * Date: 2018-07-12 12:13
sudo pip install ansible-lint
Requirement already satisfied: ansible-lint in /usr/lib/python2.7/site-packages (3.4.21)
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pip/_internal/basecommand.py", line 228, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 291, in run
    resolver.resolve(requirement_set)
  File "/usr/lib/python2.7/site-packages/pip/_internal/resolve.py", line 103, in resolve
    self._resolve_one(requirement_set, req)
  File "/usr/lib/python2.7/site-packages/pip/_internal/resolve.py", line 262, in _resolve_one
    check_dist_requires_python(dist)
  File "/usr/lib/python2.7/site-packages/pip/_internal/utils/packaging.py", line 46, in check_dist_requires_python
    feed_parser.feed(metadata)
  File "/usr/lib64/python2.7/email/feedparser.py", line 178, in feed
    self._input.push(data)
  File "/usr/lib64/python2.7/email/feedparser.py", line 99, in push
    print(parts)
UnboundLocalError: local variable 'parts' referenced before assignment

pip2 --version
pip 10.0.1 from /usr/lib/python2.7/site-packages/pip (python 2.7)
msg321547 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-07-12 13:23
Hi,

I couldn't see the print statement in Lib/email/feedparser.py with latest 2.7 branch and also `git log -p Lib/email/feedparser.py | grep print` doesn't return me anything. `pip install ansible-lint` also works fine with Python 2.7. I think this statement was added by mistake for debugging by the developer before `parts = data.splitlines(True)`. Can you please point me to the commit where the print statement was added or a test case to reproduce this?

Thanks
msg321553 - (view) Author: Sagi (Sergey) Shnaidman (sshnaidm) * Date: 2018-07-12 13:55
@xtreak , sorry, you're right, I pasted a wrong traceback.
This is original traceback:

Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/site-packages/pip/commands/install.py", line 346, in run
    requirement_set.prepare_files(finder)
  File "/usr/lib/python2.7/site-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python2.7/site-packages/pip/req/req_set.py", line 666, in _prepare_file
    check_dist_requires_python(dist)
  File "/usr/lib/python2.7/site-packages/pip/utils/packaging.py", line 48, in check_dist_requires_python
    feed_parser.feed(metadata)
  File "/usr/lib64/python2.7/email/feedparser.py", line 177, in feed
    self._input.push(data)
  File "/usr/lib64/python2.7/email/feedparser.py", line 99, in push
    parts = data.splitlines(True)
AttributeError: 'NoneType' object has no attribute 'splitlines'


And I pasted by mistake the one I used to debug.
msg321556 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-07-12 14:05
Thanks @sshnaidm. The patch makes sense. Can you add a test case for this that fails without the patch? I have tried `pip install ansible-lint` and it works fine for me in Python 2.7.

I think a NEWS entry will help too : https://devguide.python.org/committing/#what-s-new-and-news-entries 

Thanks
msg321562 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2018-07-12 15:05
This appears to be a bug in pip or the ansible-lint package, similar, and possibly identical, to issue 31361.  I believe None is being passed in as the argument to feed_parser.feed, which is an invalid use of that API and so correctly results in an error.
msg321635 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-07-13 21:34
An exception is not a crash.  An intended exception is not even a bug.  A bug is a discrepancy between behavior and doc.  In this case, email.parser.Feedparser is imported from email.feedparser. Its doc
https://docs.python.org/2.7/library/email.parser.html#feedparser-api
says
"feed(data)
    Feed the FeedParser some more data. data should be a string containing one or more lines."
In addition, html.parser.HTMLParser.feed(data) say "data must be str."

For either, behavior upon passing anything else is undefined.  Defining the behavior for None, and defining it to be 'return something' rather than 'raise something', would be an enhancement.  Such enhancements are only added, if deemed desirable, in future versions.

(Note: I recommend against resubmitting this for 3.8.  Masking bugs in 3rd party code is contrary to policy.  There would have to be a positive use-case of benefit large enough to multiple users to overcome the masking deficit.)
msg321636 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-07-13 21:44
What you can do is report to pip and ansible-lint that there is a bug somewhere in one of the two packages, or try to figure out which and report it to one of them.
History
Date User Action Args
2018-07-13 21:44:17terry.reedysetmessages: + msg321636
2018-07-13 21:36:24terry.reedysettitle: local variable 'parts' referenced before assignment in feedparser in email module -> None.splitlines raises AttributeError in email.feedparser
2018-07-13 21:34:39terry.reedysetstatus: open -> closed

type: crash -> enhancement

nosy: + terry.reedy
messages: + msg321635
resolution: not a bug
stage: patch review -> resolved
2018-07-12 15:05:54r.david.murraysetmessages: + msg321562
2018-07-12 14:05:53xtreaksetmessages: + msg321556
2018-07-12 13:55:34sshnaidmsetmessages: + msg321553
2018-07-12 13:23:58xtreaksetnosy: + xtreak
messages: + msg321547
2018-07-12 12:25:06sshnaidmsetkeywords: + patch
stage: patch review
pull_requests: + pull_request7800
2018-07-12 12:13:49sshnaidmcreate