This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author JanKanis
Recipients JanKanis, benjamin.peterson, dlesco, docs@python, hynek, lemburg, pitrou, stutzbach, terry.reedy
Date 2014-07-03.09:38:28
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1404380309.75.0.473808735546.issue21910@psf.upfronthosting.co.za>
In-reply-to
Content
The resolution of issue 5445 should be documented somewhere properly, so people can depend on it or not.

IOBase.writelines handles generator arguments without problems, i.e. without first draining the entire generator and then writing the result in one go. That would require large amounts of memory if the generator is large, and fail entirely if the generator is infinite. 

codecs.StreamWriter.writelines uses self.write(''.join(argument)) as implementation, which fails on very large or infinite arguments.

According to issue 5445 it is not part of the file protocol that .writelines must handle (large/infinite) generators, only list-like iterables. However as far as I know this is not documented anywhere, and sometimes people assume that writelines is meant for this case. E.g. jinja (https://github.com/mitsuhiko/jinja2/blob/master/jinja2/environment.py#L1153, the dump method is explicitly documented to stream). The guarantees that .writelines makes or does not make in this regard should be documented somewhere, so that either .writeline implementations that don't handle large generators can be pointed out as bugs, or code that makes assumptions on .writeline handling large generators can be.

I personally think .writelines should handle large generators, since in the python 3 world a lot of apis were iterator-ified and it is wat a lot of people would probably expect. But having a clear and documented decision on this is more important. 

(note: I've copied most of the nosy list from #5445)
History
Date User Action Args
2014-07-03 09:38:29JanKanissetrecipients: + JanKanis, lemburg, terry.reedy, pitrou, benjamin.peterson, stutzbach, dlesco, docs@python, hynek
2014-07-03 09:38:29JanKanissetmessageid: <1404380309.75.0.473808735546.issue21910@psf.upfronthosting.co.za>
2014-07-03 09:38:29JanKanislinkissue21910 messages
2014-07-03 09:38:28JanKaniscreate