classification
Title: fileinput.FileInput.readline() always returns str object at the end even if in 'rb' mode
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.6, Python 3.4, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: Ryosuke Ito, python-dev, r.david.murray, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2015-10-29 14:12 by Ryosuke Ito, last changed 2015-11-01 21:02 by r.david.murray. This issue is now closed.

Files
File name Uploaded Description Edit
fileinput.py.diff Ryosuke Ito, 2015-10-29 14:12 A patch for fileinput.py to fix this issue
test_fileinput.py.diff Ryosuke Ito, 2015-11-01 14:04 A patch for test_fileinput.py (parent: https://hg.python.org/cpython/rev/f7db966c9fee)
Messages (8)
msg253675 - (view) Author: Ryosuke Ito (Ryosuke Ito) * Date: 2015-10-29 14:12
In Python3, fileinput.FileInput.readline() always returns str object at the end,
even if in 'rb' mode.
Here's a test code.

import fileinput

fi = fileinput.input('test_fileinput.py', mode='rb')

while True:
    line = fi.readline()
    assert isinstance(line, bytes)
    if not len(line):
        break

It fails in Python3.2 to 3.5.

I wrote a patch for this.
With it, the test above passes.
msg253679 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-10-29 15:33
Would be nice to add a test in Lib/test/test_fileinput.py.
msg253680 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-10-29 15:51
We need to think carefully about backward compatibility here.
msg253844 - (view) Author: Ryosuke Ito (Ryosuke Ito) * Date: 2015-11-01 14:04
Add a test for fileinput.FileInput.readline() in 'rb' mode.

I'm sure this test fails on Python 3.5 and fixed in the patch(fileinput.py.diff).
msg253848 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-11-01 14:37
LGTM.

I don't think we should care about breaking backward compatibility here. Most code tests end-of-files with "not line" or "len(line) == 0", and this work with fixed fileinput as well as with broken. That is why we have received this bug report now, after many years since releasing 3.0.
msg253849 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-11-01 14:46
New changeset e27c589e1c6a by Serhiy Storchaka in branch '3.4':
Issue #25510: fileinput.FileInput.readline() now returns b'' instead of ''
https://hg.python.org/cpython/rev/e27c589e1c6a

New changeset 321b34824020 by Serhiy Storchaka in branch '3.5':
Issue #25510: fileinput.FileInput.readline() now returns b'' instead of ''
https://hg.python.org/cpython/rev/321b34824020

New changeset 4f0e293e6eb0 by Serhiy Storchaka in branch 'default':
Issue #25510: fileinput.FileInput.readline() now returns b'' instead of ''
https://hg.python.org/cpython/rev/4f0e293e6eb0
msg253850 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-11-01 14:50
Thank you Ryosuke for your contribution.
msg253875 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-11-01 21:02
Ah, I misunderstood the bug report.  Yes, this should be OK (I hope :).
History
Date User Action Args
2015-11-01 21:02:14r.david.murraysetmessages: + msg253875
2015-11-01 14:50:32serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg253850

stage: commit review -> resolved
2015-11-01 14:46:49python-devsetnosy: + python-dev
messages: + msg253849
2015-11-01 14:37:34serhiy.storchakasetassignee: serhiy.storchaka
stage: test needed -> commit review
messages: + msg253848
versions: + Python 3.4, Python 3.6
2015-11-01 14:04:52Ryosuke Itosetfiles: + test_fileinput.py.diff

messages: + msg253844
2015-10-29 15:51:56r.david.murraysetnosy: + r.david.murray
messages: + msg253680
2015-10-29 15:33:13serhiy.storchakasetnosy: + serhiy.storchaka

messages: + msg253679
stage: test needed
2015-10-29 14:13:43Ryosuke Itosettype: behavior
2015-10-29 14:13:21Ryosuke Itosetcomponents: + Library (Lib)
versions: + Python 3.5
2015-10-29 14:12:19Ryosuke Itocreate