This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: csv.DictWriter emits strange errors if fieldnames is an iterator
Type: enhancement Stage: patch review
Components: Library (Lib) Versions: Python 3.11
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: bendotc, iritkatriel, rhettinger, sam_ezeh
Priority: normal Keywords: patch

Created on 2018-01-14 00:41 by bendotc, last changed 2022-04-11 14:58 by admin.

Files
File name Uploaded Description Edit
sam_ezeh.patch sam_ezeh, 2022-04-01 08:35
Pull Requests
URL Status Linked Edit
PR 32225 open sam_ezeh, 2022-04-01 08:41
Messages (4)
msg309904 - (view) Author: Ben Cummings (bendotc) Date: 2018-01-14 00:41
If I pass an iterator to csv.DictWriter as the fieldname field, then DictWriter consumes that iterator pretty quickly, emitting strange errors such as the following when trying to write the headers.

>>> import csv
>>> fields = iter(["a", "b", "c", "d"])
>>> f = open('test.csv', 'w')
>>> writer = csv.DictWriter(f, fieldnames=fields)
>>> writer.writeheader()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/csv.py", line 142, in writeheader
    self.writerow(header)
  File "/usr/lib/python3.4/csv.py", line 153, in writerow
    return self.writer.writerow(self._dict_to_list(rowdict))
  File "/usr/lib/python3.4/csv.py", line 149, in _dict_to_list
    + ", ".join([repr(x) for x in wrong_fields]))
ValueError: dict contains fields not in fieldnames: 'c', 'a'

This is because writeheader and _dict_to_list repeatedly iterate over self.fieldnames. It seems like this could be solved by making a list of fieldnames in the constructor.
msg309907 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-01-14 01:08
This isn't a bug.  The docs specify that *fieldnames* is a sequence, "The fieldnames parameter is a sequence of keys that identify the order in which values in the dictionary passed to the writerow() method are written to the csvfile."

We could make this a feature request. but as far as I can tell there isn't a real use case, nor has one ever arisen in the decade long history of the module.  Adding the list(fieldnames) behavior could be done without a backwards compatibility issue, but it does incur an overhead that would be paid by all existing working code.
msg415573 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-03-19 21:36
Perhaps we could raise an exception with a clearer error message when fieldnames not a sequence?
msg416473 - (view) Author: Sam Ezeh (sam_ezeh) * Date: 2022-04-01 08:32
I've submitted a patch that introduces code that raises TypeError at construction if `fieldnames` is not a sequence
History
Date User Action Args
2022-04-11 14:58:56adminsetgithub: 76728
2022-04-01 08:41:33sam_ezehsetstage: patch review
pull_requests: + pull_request30300
2022-04-01 08:35:10sam_ezehsetfiles: + sam_ezeh.patch
2022-04-01 08:34:54sam_ezehsetfiles: - sam_ezeh.patch
2022-04-01 08:33:33sam_ezehsetfiles: + sam_ezeh.patch
2022-04-01 08:33:05sam_ezehsetfiles: - sam_ezeh.patch
2022-04-01 08:32:18sam_ezehsetfiles: + sam_ezeh.patch

nosy: + sam_ezeh
messages: + msg416473

keywords: + patch
2022-03-19 21:36:23iritkatrielsetnosy: + iritkatriel

messages: + msg415573
versions: + Python 3.11, - Python 3.7
2018-01-14 01:08:18rhettingersetversions: - Python 3.4, Python 3.5, Python 3.6, Python 3.8
nosy: + rhettinger

messages: + msg309907

type: enhancement
2018-01-14 00:41:44bendotccreate