This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: fileinput causes RecursionErrors when dealing with large numbers of empty files
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: duaneg, josh.r, serhiy.storchaka
Priority: normal Keywords:

Created on 2016-09-08 17:30 by josh.r, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (2)
msg275072 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2016-09-08 17:30
Accidentally discovered this while running code that processes a whole bunch of files, when it turned out the files were empty. The readline method of fileinput.input will make a recursive call to continue processing when it reaches the end of a file. This is fine when files aren't empty, but if you have sufficient (~1000) empty files being processed, this causes recursion errors.

Simple example:

>>> for i in range(10000):
...     with open('test{}'.format(i), 'wb'):
...         pass
...
>>> import fileinput, glob
>>> ''.join(fileinput.input(glob.glob('./test*')))
Traceback ...
... (almost all the levels are showing the self.readline() call at the end of readline) ...
RecursionError: maximum recursion depth exceeded

Admittedly a niche case, but a relatively simple switch from recursion to iteration would solve it. Same problem appears in all versions of Python.
msg276135 - (view) Author: Duane Griffin (duaneg) * Date: 2016-09-12 23:44
This was fixed in 2.7, 3.5 and head when #15068 was fixed.
History
Date User Action Args
2022-04-11 14:58:36adminsetgithub: 72211
2016-09-13 00:12:44berker.peksagsetstatus: open -> closed
resolution: out of date
stage: resolved
2016-09-12 23:44:53duanegsetnosy: + serhiy.storchaka, duaneg
messages: + msg276135
2016-09-08 17:30:58josh.rcreate