Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

csv.DictWriter emits strange errors if fieldnames is an iterator #76728

Closed
bendotc mannequin opened this issue Jan 14, 2018 · 5 comments
Closed

csv.DictWriter emits strange errors if fieldnames is an iterator #76728

bendotc mannequin opened this issue Jan 14, 2018 · 5 comments
Labels
3.11 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@bendotc
Copy link
Mannequin

bendotc mannequin commented Jan 14, 2018

BPO 32547
Nosy @rhettinger, @bendotc, @iritkatriel, @dignissimus
PRs
  • gh-76728: Coerce DictReader and DictWriter fieldnames argument to a list #32225
  • Files
  • sam_ezeh.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2018-01-14.00:41:44.336>
    labels = ['type-feature', 'library', '3.11']
    title = 'csv.DictWriter emits strange errors if fieldnames is an iterator'
    updated_at = <Date 2022-04-01.08:41:33.520>
    user = 'https://github.com/bendotc'

    bugs.python.org fields:

    activity = <Date 2022-04-01.08:41:33.520>
    actor = 'sam_ezeh'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)']
    creation = <Date 2018-01-14.00:41:44.336>
    creator = 'bendotc'
    dependencies = []
    files = ['50711']
    hgrepos = []
    issue_num = 32547
    keywords = ['patch']
    message_count = 4.0
    messages = ['309904', '309907', '415573', '416473']
    nosy_count = 4.0
    nosy_names = ['rhettinger', 'bendotc', 'iritkatriel', 'sam_ezeh']
    pr_nums = ['32225']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue32547'
    versions = ['Python 3.11']

    @bendotc
    Copy link
    Mannequin Author

    bendotc mannequin commented Jan 14, 2018

    If I pass an iterator to csv.DictWriter as the fieldname field, then DictWriter consumes that iterator pretty quickly, emitting strange errors such as the following when trying to write the headers.

    >>> import csv
    >>> fields = iter(["a", "b", "c", "d"])
    >>> f = open('test.csv', 'w')
    >>> writer = csv.DictWriter(f, fieldnames=fields)
    >>> writer.writeheader()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.4/csv.py", line 142, in writeheader
        self.writerow(header)
      File "/usr/lib/python3.4/csv.py", line 153, in writerow
        return self.writer.writerow(self._dict_to_list(rowdict))
      File "/usr/lib/python3.4/csv.py", line 149, in _dict_to_list
        + ", ".join([repr(x) for x in wrong_fields]))
    ValueError: dict contains fields not in fieldnames: 'c', 'a'

    This is because writeheader and _dict_to_list repeatedly iterate over self.fieldnames. It seems like this could be solved by making a list of fieldnames in the constructor.

    @bendotc bendotc mannequin added 3.8 only security fixes 3.7 (EOL) end of life stdlib Python modules in the Lib dir labels Jan 14, 2018
    @rhettinger
    Copy link
    Contributor

    This isn't a bug. The docs specify that *fieldnames* is a sequence, "The fieldnames parameter is a sequence of keys that identify the order in which values in the dictionary passed to the writerow() method are written to the csvfile."

    We could make this a feature request. but as far as I can tell there isn't a real use case, nor has one ever arisen in the decade long history of the module. Adding the list(fieldnames) behavior could be done without a backwards compatibility issue, but it does incur an overhead that would be paid by all existing working code.

    @rhettinger rhettinger added type-feature A feature request or enhancement and removed 3.8 only security fixes labels Jan 14, 2018
    @iritkatriel
    Copy link
    Member

    Perhaps we could raise an exception with a clearer error message when fieldnames not a sequence?

    @iritkatriel iritkatriel added 3.11 only security fixes and removed 3.7 (EOL) end of life labels Mar 19, 2022
    @dignissimus
    Copy link
    Mannequin

    dignissimus mannequin commented Apr 1, 2022

    I've submitted a patch that introduces code that raises TypeError at construction if fieldnames is not a sequence

    @furkanonder
    Copy link
    Sponsor Contributor

    @iritkatriel The issue seems to have been resolved. Can we close the issue?

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    Status: Done
    Development

    No branches or pull requests

    3 participants