Title: Add fileinput.islastline()
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Digitalxero, John Tapsell, ajaksu2, berker.peksag, damngamerz, dharriso, eric.araujo, isandler, pitrou, relm, rhettinger, serhiy.storchaka, sonderblade
Priority: low Keywords: easy, patch

Created on 2004-06-07 09:45 by relm, last changed 2017-04-29 20:12 by John Tapsell.

File name Uploaded Description Edit
fileinput.diff relm, 2004-06-07 09:45 Patch to add fileinput.islastline() relm, 2004-06-15 18:42 A "word count" program with and without fileinput.islastline()
cpython_rev68060.patch Digitalxero, 2011-02-27 03:22
Messages (15)
msg46132 - (view) Author: Relm Arrowny (relm) Date: 2004-06-07 09:45
You can test for the first line of a file with 
fileinput.isfirstline(), but there is no corresponding 
fileinput.islastline() to test for the last line of a file.

Note that there is already an unapplied patch for this at 
request ID 776100, but it is faulty in that it only tests 
for end of buffer and not end of file.

I was not sure how to append this patch to the existing 
request, so have started a new one.  Hope this is okay.

This patch also includes documentation and test 
msg46133 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2004-06-15 12:05
Logged In: YES 

I'm -0 on complicating this venerable interface.   Also, I
find it unnatural for the inside of a loop to be able to
ascertain whether it is in its final iteration.  The closest
approximation is the for-else clause which is vary rarely
used in normal python programming.  While the OP was able to
sketch a use case, it involved only saving a couple of lines
over what can be done with the existing API.
msg46134 - (view) Author: Relm Arrowny (relm) Date: 2004-06-15 18:42
Logged In: YES 

The OP's use case was too simplified to convey
the usefulness of fileinput.islastline().  It does not
just save a couple of lines.  I am attaching a "wc" or
"word count" program to demonstrate.  Note the following
problems with the current work-around:

1. Instead of doing end-of-file processing at the end of
   a file, we must do it at the beginning of the *next*

2. Useful functions such as fileinput.filelineno() and
   fileinput.filename() now refer to the *wrong* file,
   so now we need to manually keep track of these values

3. Perhaps a minor point, but this work-around only works
   at the *beginning* of the loop instead of the end.  It
   feels somewhat awkward.

4. End-of-file processing must occur both in the loop and
   once outside.  A function avoids code duplication, but
   to refer to variables in the loop, they must be passed
   into and out of the function (6 in, 2 out for the "wc"
   example... maybe keyword arguments would have helped).

5. Most important is readablility.  It's not obvious that
   `fileinput.isfirstline() and fileinput.lineno() != 1`
   means we are the end of the (previous!) file.  Using
   `fileinput.islastline()` would be much clearer.

And not that it is a reason, but Perl and Ruby solve
this with an `eof` function and `eof?` method,
msg46135 - (view) Author: Dave Harrison (dharriso) Date: 2004-07-04 11:24
Logged In: YES 

I would say that I'm -0.5 on this feature.  After looking
over the current code for in the cvs tree, it
looks like the current code implements an iterator model. 
Since uses the next() call to move along its
input, and raises StopIteration if there are no more lines
to be returned, this function would have to be polled to
find out if the final line had been reached - where a
try/except block that caught StopIteration would seem more
intuitive to me (Inkeeping with the current code).

However it might be considered that the nextfile() function
be changed to raise StopIteration after the last file had
been completed to maintain consistency between the lines and
files aspects of the class.
msg46136 - (view) Author: Ilya Sandler (isandler) Date: 2004-08-05 18:17
Logged In: YES 

An alternative to islastline() would be to add a single-line
look-ahead capability:

nextline()  which would return a structure containing
  next line, and its file name
(or have 2 functions
and nextlinefile()

This would cover most of the OP's needs but would also
provide extra useful functionality....E.g. when processing
files which have a 

header line
[data line]*

its very convenient to know in advance when section ends
while processing the last line of section not when
enountering the header line of the next section...

Another use case would be processing of files with line groups:


(grouping could be for example on  the value of a certian
field in the line)

again it would be very useful to know when a group ends
while still processing this group
msg46137 - (view) Author: Björn Lindqvist (sonderblade) Date: 2007-03-14 01:25
I have applied this patch and it works as specified. Although it does not apply cleanly and it contains an error; self._lastline = None should probably be in the __init__() method. Please reconsider this patch and see Relms attached wc program. It is definitely sensible and useful for fileinput to have a islastline method. In fact, every program that uses fileinput and wants to make a summary for each read file. 
msg81711 - (view) Author: Daniel Diniz (ajaksu2) * (Python triager) Date: 2009-02-12 02:58
Issue issue 776100 closed as duplicated.
msg125560 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-01-06 15:49
Anyone wants to produce an up-to-date patch for py3k?
msg129609 - (view) Author: Dj Gilcrease (Digitalxero) Date: 2011-02-27 03:22
Updated the patch and tests for py3.3
msg137209 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-05-29 17:34
Would storing len(self._buffer) in the instance be premature optimization?
msg187432 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2013-04-20 14:57
The latest patch still applies cleanly, can we have it reviewed please.
msg286385 - (view) Author: Saurav Singh (damngamerz) Date: 2017-01-27 21:26
last patch works fine i have applied and tested the patch.Its documented well.Can we please have it reviewed?
msg287069 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2017-02-06 02:47
Thanks for the updated patch, but I'm not sure FileInput class needs a islastline() method. I couldn't find any similar or duplicate feature requests on the tracker after this opened in 2004 (which is a sign that the need for this method is low.)

Also, the patch needs some work. Some comments:

* There is no documentation changes to Doc/library/fileinput.rst
* 'mark' is unused in Lib/
* Not sure why we need 't1 = t2 = t3 = None' in tests
* We could use self.addCleanup() instead of 'try...finally'
* fileinput.islastline() is not tested. We could reuse Test_fileinput_isfirstline.

According to commit history of Lib/, Serhiy has done a lot of work in the past few years. I just added him to nosy list to get his feedback.
msg287080 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-02-06 06:15
The patch is outdated. It uses private _buffer attribute which no longer exist (see issue15068). But even when _buffer was existing the patch was not correct, because _buffer contained only a part of the file.

It is possible to implement islastline(), but at the cost of additional buffering. This effectively negates the result of issue15068. The user would need to enter two lines first than fileinput could return the fist one from stdin. And additional complication slows down reading for all users even if they don't need islastline().

If you need islastline() you can use a wrapper:

def yield_line_and_islastline(f):
        prevline = next(f)
    except StopIteration:
    for line in f:
        yield (prevline, f.isfirstline())
        prevline = line
    yield prevline, True
msg292601 - (view) Author: John Tapsell (John Tapsell) Date: 2017-04-29 20:12
> If you need islastline() you can use a wrapper:

This doesn't work because fileinput.filename()  and fileinput.filelineno()  are going to be wrong and pointing to the next file.
Date User Action Args
2017-04-29 20:12:23John Tapsellsetnosy: + John Tapsell
messages: + msg292601
2017-02-06 06:15:55serhiy.storchakasetmessages: + msg287080
2017-02-06 02:47:18berker.peksagsetversions: + Python 3.7, - Python 3.4
nosy: + berker.peksag, serhiy.storchaka

messages: + msg287069

stage: needs patch -> patch review
2017-01-27 21:26:58damngamerzsetnosy: + damngamerz
messages: + msg286385
2014-02-03 18:31:45BreamoreBoysetnosy: - BreamoreBoy
2013-04-20 14:57:43BreamoreBoysetnosy: + BreamoreBoy

messages: + msg187432
versions: + Python 3.4, - Python 3.3
2011-05-29 17:34:50eric.araujosetnosy: + eric.araujo
messages: + msg137209
2011-02-27 03:22:08Digitalxerosetfiles: + cpython_rev68060.patch

nosy: + Digitalxero
messages: + msg129609

keywords: + patch
2011-01-06 15:49:15pitrousettype: enhancement
versions: + Python 3.3, - Python 2.6
keywords: + easy, - patch
nosy: + pitrou

messages: + msg125560
stage: needs patch
2009-02-12 02:58:05ajaksu2setpriority: normal -> low
nosy: + ajaksu2
messages: + msg81711
versions: + Python 2.6, - Python 2.3
2009-02-12 02:56:48ajaksu2linkissue776100 superseder
2004-06-07 09:45:17relmcreate