This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author lemburg
Recipients dlesco, facundobatista, lemburg
Date 2009-03-10.15:52:53
SpamBayes Score 1.0221268e-12
Marked as misclassified No
Message-id <49B68CD4.6040101@egenix.com>
In-reply-to <1236699401.94.0.646854639239.issue5445@psf.upfronthosting.co.za>
Content
On 2009-03-10 16:36, Daniel Lescohier wrote:
> Daniel Lescohier <daniel.lescohier@cbs.com> added the comment:
> 
> Let me give an example of why it's important that writelines 
> iteratively writes.  For:
> 
> rows = (line[:-1].split('\t') for line in in_file)
> projected = (keep_fields(row, 0, 3, 7) for row in rows)
> filtered = (row for row in projected if row[2]=='1')
> out_file.writelines('\t'.join(row)+'\n' for row in filtered)
> 
> For a large input file, for a regular out_file object, this will work. 
> For a codecs.StreamWriter wrapped out_file object, this won't work, 
> because it's not following the file protocol that writelines should 
> iteratively write.

Of course, it's possible to have a generator producing lots of data,
but that does not warrant making most uses of that method slow.

If you'd like to see such support in .writelines(), please provide
an implementation that follows the approach of the file object
implementation in Python 2.x: it writes the lines in chunks of
1000 lines each, if it finds that the input object is not a sequence
(actually, it's stricter than that for some: it requires a Python
list).

The standard case of passing a list of strings to that method
should not get slower because of this.

BTW: I am not aware of any .writelines() file protocol. If there
were such a protocol, .readlines() would also have to return
an iterator (which it doesn't). The idea behind .writelines()
is to be able to write back a list generated with .readlines().

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
History
Date User Action Args
2009-03-10 15:52:56lemburgsetrecipients: + lemburg, facundobatista, dlesco
2009-03-10 15:52:54lemburglinkissue5445 messages
2009-03-10 15:52:53lemburgcreate