classification
Title: csv.writer.writerow() does not accept generator (must be coerced to list)
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: eric.araujo, jdufresne, martin.panter, python-dev, r.david.murray, serhiy.storchaka
Priority: normal Keywords: easy, patch

Created on 2015-01-05 17:31 by jdufresne, last changed 2015-06-02 19:45 by r.david.murray. This issue is now closed.

Files
File name Uploaded Description Edit
csv-gen.patch jdufresne, 2015-01-06 03:39 patch review
csv_writerow_iterable.patch serhiy.storchaka, 2015-01-06 08:57 review
Messages (9)
msg233470 - (view) Author: Jon Dufresne (jdufresne) * Date: 2015-01-05 17:31
The csv.writer.writerow() does not accept a generator as input. I find this counter-intuitive and against the spirit of similar APIs. If the generator is coerced to a list, everything works as expected. See the following test script which fails on the line "w.writerow(g)". In my opinion, this line should work identically to the line "w.writerow(list(g))".

---
import csv

f = open('foo.csv', 'w')
w = csv.writer(f)
g = (i for i in ['a', 'b', 'c'])
w.writerow(list(g))
g = (i for i in ['a', 'b', 'c'])
w.writerow(g)
---
msg233475 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-01-05 18:30
This seems like a sensible enhancement request to me.

It is possible it could even be considered a bug, the docs aren't exactly clear on what 'row' is expected to be.
msg233501 - (view) Author: Jon Dufresne (jdufresne) * Date: 2015-01-06 03:39
I have created an initial patch such that writerow() now allows generators. I have also added a unit test to demonstrate the fix.

The code now coerces iterators (and generators) to a list, then operates on the result. I would have preferred to simply iterate over the argument, however, there is a special case where the length of the argument is exactly 1. So coercing to a list makes checking the length simpler.

All feedback welcome.
msg233508 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-01-06 08:26
Hmm.  That could be an issue.  If someone passes a generator they will generally expect it to be consumed as a generator, not turned into a list implicitly.  So it may be better to turn this into a doc bug and require the explicit "list" call :(.
msg233509 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-01-06 08:57
The docs mention that "row" should be a sequence, so there is no a bug.

Here is a patch which makes writerow() accept an iterable without converting it to a list. It also adds tests for few corner cases and fixes the docs.
msg239192 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-03-24 22:27
Left a question about handling of the unquoted empty field exception on Rietveld.
msg239570 - (view) Author: Roundup Robot (python-dev) Date: 2015-03-30 06:22
New changeset cf5b62036445 by Serhiy Storchaka in branch 'default':
Issue #23171: csv.Writer.writerow() now supports arbitrary iterables.
https://hg.python.org/cpython/rev/cf5b62036445
msg244689 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-06-02 19:41
Looks like Serhiy forgot to close this, so closing it.
msg244690 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2015-06-02 19:45
No, I just had a stale tab :( :(
History
Date User Action Args
2015-06-02 19:45:10r.david.murraysetmessages: + msg244690
2015-06-02 19:41:10r.david.murraysetassignee: serhiy.storchaka ->
messages: + msg244689
2015-03-30 07:11:02serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2015-03-30 06:22:04python-devsetnosy: + python-dev
messages: + msg239570
2015-03-24 22:27:08martin.pantersetmessages: + msg239192
2015-03-24 20:06:48serhiy.storchakasetassignee: serhiy.storchaka
2015-03-20 02:47:33martin.pantersetnosy: + martin.panter
2015-03-19 21:57:50eric.araujosetnosy: + eric.araujo
2015-01-06 08:57:56serhiy.storchakasetfiles: + csv_writerow_iterable.patch

nosy: + serhiy.storchaka
messages: + msg233509

stage: needs patch -> patch review
2015-01-06 08:26:10r.david.murraysetmessages: + msg233508
2015-01-06 03:39:45jdufresnesetfiles: + csv-gen.patch
keywords: + patch
messages: + msg233501
2015-01-05 18:30:54r.david.murraysettype: behavior -> enhancement
versions: + Python 3.5, - Python 3.3
keywords: + easy
nosy: + r.david.murray

messages: + msg233475
stage: needs patch
2015-01-05 17:31:42jdufresnecreate