classification
Title: csv module: add header row to DictWriter
Type: enhancement Stage: patch review
Components: None Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: djc Nosy List: djc, ed_abraham, florian-rathgeber, flox, ivo, pitrou, skip.montanaro
Priority: normal Keywords: buildbot, patch

Created on 2006-08-09 22:20 by ed_abraham, last changed 2010-08-29 14:14 by florian-rathgeber. This issue is now closed.

Messages (17)
msg61251 - (view) Author: ed_abraham (ed_abraham) Date: 2006-08-09 22:20
I use the DictWriter class from the csv module, and
have to manually write the header row. A mindless chore
which I would like to see eliminated. Can we have a
writeheader method added to the class? Something like
the following:

def writeheader(self, headernames = {}):
    """Write a header row"""
    if not headernames:
        headernames = dict(zip(self.fieldnames,
self.fieldnames))
        self.writerow(headernames)

This would let you either use the fieldnames directly,
or supply your own pretty header names. 

Would be nice to have another keyword argument to
DictWriter, 'header = False'. If header was true, then
the __init__ method could call writeheader(). 


At the moment I have to write things like

fields = ['a','b','c']
w = csv.DictWriter(fid, fields)
w.writerow(dict(zip(fields, fields)))
for row in rows:
    w.writerow(row)

The proposed changes would let me write the simpler

w = csv.DictWriter(fid, ['a','b','c'], header = True)
for row in rows:
    w.writerow(row)


A problem is that including a new keyword argument
would break code which used position to fill the
keyword arguments and to supply arguments through *args
to the writer class.
msg84796 - (view) Author: Skip Montanaro (skip.montanaro) * (Python committer) Date: 2009-03-31 15:03
I don't see a patch.  Is there some reason that if you need this
you can't simply subclass DictWriter?
msg86157 - (view) Author: Matthew Iversen (ivo) Date: 2009-04-19 04:10
Skip, you were arguing in another csv issue on a NamedTupleReader that
the Reader and Writer should work in concert together.

Certainly, making this default functionality for DictWriter would
definitely make it work more in concert with DictReader.

A sample process of reading and writing might be:

1. DictReader reads in header names, reads in data
2. Pass in header names to DictWriter on init
3. DictWriter writes out modified data back to csv file (headers get
written automatically or with a method call)
4. DictReader can now reader in the csv file automatically again with
header names

My feeling is, if DictReader can read in head names, why can't
DictWriter write them back out again? Shouldn't there be a good amount
of symmatry to their function/abilities?

My feeling of how to implement this functionality would be to include a
new init argument, say 'writeheader'. It's default would be True if you
wanted to implement this new feature by default, or False if you wanted
to keep it an option so that older scripts relying on the old
functionality won't break immmediately.
DictWriter would then write the header on the first call to writerow(s),
or perhaps also with an explicit call to writeheader() say.

I certainly am miffed by the fact that DictWriter cannot produce
(normally) a csv file that DictReader can read in automatically.
msg96063 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2009-12-07 18:22
I'd like to commit this, but it would be nice to get a review first:

Index: Lib/csv.py
===================================================================
--- Lib/csv.py	(revision 76697)
+++ Lib/csv.py	(working copy)
@@ -132,6 +132,10 @@
         self.extrasaction = extrasaction
         self.writer = writer(f, dialect, *args, **kwds)
 
+    def writeheader(self):
+        header = dict(zip(self.fieldnames, self.fieldnames))
+        self.writerow(header)
+
     def _dict_to_list(self, rowdict):
         if self.extrasaction == "raise":
             wrong_fields = [k for k in rowdict if k not in self.fieldnames]
Index: Lib/test/test_csv.py
===================================================================
--- Lib/test/test_csv.py	(revision 76697)
+++ Lib/test/test_csv.py	(working copy)
@@ -598,8 +598,10 @@
         fileobj = os.fdopen(fd, "w+b")
         try:
             writer = csv.DictWriter(fileobj, fieldnames = ["f1", "f2",
"f3"])
+            writer.writeheader()
             writer.writerow({"f1": 10, "f3": "abc"})
             fileobj.seek(0)
+            self.assertEqual(fileobj.readline(), "f1,f2,f3\r\n")
             self.assertEqual(fileobj.read(), "10,,abc\r\n")
         finally:
             fileobj.close()

(I think I have commit privileges already.)
msg96069 - (view) Author: Skip Montanaro (skip.montanaro) * (Python committer) Date: 2009-12-07 20:41
I'm sorry, but I don't have time to look at this right now.  On the one
hand, one person asks for more symmetry.  Someone else wants to add a
writeheader method.  If you want symmetry shouldn't the DictWriter
simply write the header without being asked?  I'm confused by the
various options and feel that someone is going to be disappointed by any
solution.  For the time being at least I would prefer that the status
quo remain in place.

S
msg96074 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2009-12-07 21:02
Skip, I agree that it's hard to decide if we should have the class write
the header on __init__(). I figured starting off with a method to make
doing it "manually" is a good start; people can start using that, and if
it's deemed useful we can always add the auto-write later.
msg96078 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-12-07 21:59
We can't change default behaviour because it will break compatibility,
so an additional method looks ok to me.
msg96089 - (view) Author: Skip Montanaro (skip.montanaro) * (Python committer) Date: 2009-12-07 23:42
Antoine> We can't change default behaviour because it will break
    Antoine> compatibility, so an additional method looks ok to me.

Why can't default behavior be changed?

S
msg96091 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-12-07 23:45
Le lundi 07 décembre 2009 à 23:42 +0000, Skip Montanaro a écrit :
> Skip Montanaro <skip@pobox.com> added the comment:
> 
> Antoine> We can't change default behaviour because it will break
>     Antoine> compatibility, so an additional method looks ok to me.
> 
> Why can't default behavior be changed?

Well, because it will break assumptions about the generated documents?
msg96102 - (view) Author: Skip Montanaro (skip.montanaro) * (Python committer) Date: 2009-12-08 01:34
Antoine> We can't change default behaviour because it will break
    Antoine> compatibility, so an additional method looks ok to me.

    >> Why can't default behavior be changed?

    Antoine> Well, because it will break assumptions about the generated documents?

Isn't the alpha period (2.7 and 3.2 in this case) precisely when an API can
change?

Skip
msg96103 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-12-08 01:40
> Isn't the alpha period (2.7 and 3.2 in this case) precisely when an API can
> change?

Well, it can, but only if there are compelling reasons to do so. It
should be the exception rather than the rule.
The reasons here seem far from compelling, and moreover we can't detect
whether the user is expecting the new or the old behaviour.
msg96104 - (view) Author: Skip Montanaro (skip.montanaro) * (Python committer) Date: 2009-12-08 02:00
>> Isn't the alpha period (2.7 and 3.2 in this case) precisely when an
    >> API can change?

    Antoine> Well, it can, but only if there are compelling reasons to do
    Antoine> so. It should be the exception rather than the rule.  The
    Antoine> reasons here seem far from compelling, and moreover we can't
    Antoine> detect whether the user is expecting the new or the old
    Antoine> behaviour.

Fine.  Dirkjan, assuming there are the necessary test cases and
documentation changes, feel free to check in your patch and close the
ticket.

Skip
msg99952 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2010-02-23 21:10
Fixed in SVN, r78384.
msg100404 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-03-04 18:12
according to the buidbots, it hurts some platforms: Windows XP, Windows 7 and sparc Solaris10


 * sparc solaris10

test_writerows (test.test_csv.Test_Csv) ... ok

======================================================================
FAIL: test_write_simple_dict (test.test_csv.TestDictFields)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/test/test_csv.py", line 607, in test_write_simple_dict
    self.assertEqual(fileobj.read(), "10,,abc\r\n")
AssertionError: 'f1,f2,f3\r\n' != '10,,abc\r\n'

----------------------------------------------------------------------
Ran 84 tests in 17.335s

FAILED (failures=1)
test test_csv failed -- Traceback (most recent call last):
  File "/home2/buildbot/slave/trunk.loewis-sun/build/Lib/test/test_csv.py", line 607, in test_write_simple_dict
    self.assertEqual(fileobj.read(), "10,,abc\r\n")
AssertionError: 'f1,f2,f3\r\n' != '10,,abc\r\n'

 * x86 XP and x86 Windows7

test_csv
test test_csv failed -- Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\test\test_csv.py", line 604, in test_write_simple_dict
    writer.writerow({"f1": 10, "f3": "abc"})
  File "D:\cygwin\home\db3l\buildarea\trunk.bolen-windows\build\lib\csv.py", line 148, in writerow
    return self.writer.writerow(self._dict_to_list(rowdict))
IOError: [Errno 0] Error
msg100407 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2010-03-04 18:48
Testing on Windows with this:

Index: Lib/test/test_csv.py
===================================================================
--- Lib/test/test_csv.py	(revision 78430)
+++ Lib/test/test_csv.py	(working copy)
@@ -9,6 +9,7 @@
 import tempfile
 import csv
 import gc
+import io
 from test import test_support
 
 class Test_Csv(unittest.TestCase):
@@ -595,7 +596,7 @@
     ### "short" means there are fewer elements in the row than fieldnames
     def test_write_simple_dict(self):
         fd, name = tempfile.mkstemp()
-        fileobj = os.fdopen(fd, "w+b")
+        fileobj = io.open(fd, 'w+b')
         try:
             writer = csv.DictWriter(fileobj, fieldnames = ["f1", "f2", "f3"])
             writer.writeheader()
msg100409 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2010-03-04 19:23
Committed in r78660 after positive comment from briancurtin re Windows. Hopefully this fixes Solaris, as well.
msg100455 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2010-03-05 09:13
Both the solaris and windows slaves seem to have succeeded this time.
History
Date User Action Args
2010-08-29 14:14:47florian-rathgebersetnosy: + florian-rathgeber
2010-03-05 09:13:34djcsetstatus: open -> closed
resolution: fixed
messages: + msg100455
2010-03-04 19:23:49djcsetmessages: + msg100409
2010-03-04 18:48:23djcsetmessages: + msg100407
2010-03-04 18:24:03floxsetstatus: closed -> open
resolution: fixed -> (no value)
2010-03-04 18:16:07floxsetkeywords: + buildbot
2010-03-04 18:12:33floxsetnosy: + flox
messages: + msg100404
2010-02-23 21:12:39djcsetstatus: open -> closed
assignee: djc
resolution: fixed
2010-02-23 21:10:27djcsetmessages: + msg99952
2009-12-08 02:00:29skip.montanarosetmessages: + msg96104
2009-12-08 01:40:19pitrousetmessages: + msg96103
2009-12-08 01:34:55skip.montanarosetmessages: + msg96102
2009-12-07 23:45:05pitrousetmessages: + msg96091
2009-12-07 23:42:20skip.montanarosetmessages: + msg96089
2009-12-07 21:59:03pitrousetversions: + Python 3.2, - Python 3.1
nosy: + pitrou

messages: + msg96078

stage: test needed -> patch review
2009-12-07 21:02:30djcsetmessages: + msg96074
2009-12-07 20:41:06skip.montanarosetmessages: + msg96069
2009-12-07 18:22:52djcsetmessages: + msg96063
2009-08-06 12:58:42djcsetnosy: + djc
2009-04-19 04:10:26ivosetnosy: + ivo
messages: + msg86157
2009-03-31 15:03:04skip.montanarosetnosy: + skip.montanaro
messages: + msg84796
2009-03-30 04:23:50ajaksu2setkeywords: + patch
stage: test needed
versions: + Python 3.1, Python 2.7
2006-08-09 22:20:01ed_abrahamcreate