This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: csv DictWriter's internal _dict_to_list raise error unsupported operand
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: Nosy List: afflictor, rhettinger, serhiy.storchaka, xtreak
Priority: normal Keywords:

Created on 2019-11-06 10:04 by afflictor, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (6)
msg356110 - (view) Author: Patrick Schneeweiß (afflictor) Date: 2019-11-06 10:04
I switched over to Python 3.7 and still using my csv DictWriter.

No I'm receiving an error when calling the writerow() method. 
unsupported operand type(s) for -: 'list' and 'list'

Digging deeped in the problem I found out, that the internal _dict_to_list(self, rowdict) method works different compared to Python 2.7
It tries to substract the fieldnames list and the keys of the dict to find keys not included.

But substracting lists is not possible. As a workaround I defined the Dictwriter with the option extrasaction="ignore".

I would be better to use the method like in Python 2.7. I will output my versions of the method:

Python 3.7:
    def _dict_to_list(self, rowdict):
        if self.extrasaction == "raise":
            wrong_fields = rowdict.keys() - self.fieldnames
            if wrong_fields:
                raise ValueError("dict contains fields not in fieldnames: "
                                 + ", ".join([repr(x) for x in wrong_fields]))
        return (rowdict.get(key, self.restval) for key in self.fieldnames)

Python 2.7:
    def _dict_to_list(self, rowdict):
        if self.extrasaction == "raise":
            wrong_fields = [k for k in rowdict if k not in self.fieldnames]
            if wrong_fields:
                raise ValueError("dict contains fields not in fieldnames: "
                                 + ", ".join([repr(x) for x in wrong_fields]))
        return [rowdict.get(key, self.restval) for key in self.fieldnames]
msg356111 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-11-06 10:30
I guess you try to write not a dict, but an object whose keys() method returns a list.
msg356113 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2019-11-06 10:39
Can you please attach a sample script to reproduce the behavior difference? With below script a dictionary is passed then dict.keys() supports subtraction over a list to return the difference. I am not sure how you have received a list with dict.keys() in Python 3.7.

import csv

fields = ["c", "d"]
with open('test.csv', 'w') as f:
    writer = csv.DictWriter(f, fieldnames=fields, extrasaction='raise')
    writer.writeheader()
    writer.writerow({"n": 1})
msg356115 - (view) Author: Patrick Schneeweiß (afflictor) Date: 2019-11-06 10:44
Hello all,

This is my dict:
{'rule': 'WAN-2-Web|unknown (Barracuda Networks CloudGen Firewall)', 'April (2018)': '', 'May (2018)': '', 'June (2018)': '', 'July (2018)': '', 'August (2018)': '', 'April (2019)': '', 'May (2019)': '14', 'June (2019)': '', 'July (2019)': '', 'August (2019)': ''}

And this was my fieldnames list:
['rule', 'April (2018)', 'May (2018)', 'June (2018)', 'July (2018)', 'August (2018)', 'April (2019)', 'May (2019)', 'June (2019)', 'July (2019)', 'August (2019)']


when I call the keys Method on this dict, I receive this:
dict_keys(['rule', 'April (2018)', 'May (2018)', 'June (2018)', 'July (2018)', 'August (2018)', 'April (2019)', 'May (2019)', 'June (2019)', 'July (2019)', 'August (2019)'])
msg356116 - (view) Author: Patrick Schneeweiß (afflictor) Date: 2019-11-06 10:49
Ok in a new sample Script it is working. 

As I used my normal script in a software called Splunk, I saw they are using a custom class OrderedDict.

I will take a look on this first.
msg356118 - (view) Author: Patrick Schneeweiß (afflictor) Date: 2019-11-06 11:05
Sorry guys, it was an issue with the cusotm Splunk Library. They treated the keys Method a little different.

Thanks all for the hints.

Will close this
History
Date User Action Args
2022-04-11 14:59:22adminsetgithub: 82898
2019-11-06 11:05:04afflictorsetstatus: open -> closed
resolution: third party
messages: + msg356118

stage: resolved
2019-11-06 10:49:07afflictorsetmessages: + msg356116
2019-11-06 10:44:36afflictorsetmessages: + msg356115
2019-11-06 10:39:04xtreaksetnosy: + xtreak
messages: + msg356113
2019-11-06 10:30:38serhiy.storchakasetnosy: + rhettinger, serhiy.storchaka
messages: + msg356111
2019-11-06 10:04:48afflictorcreate