classification
Title: configparser.ConfigParser.clean and .update bugs
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.3, Python 3.2
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: lukasz.langa Nosy List: lukasz.langa, python-dev, wolfmanx
Priority: normal Keywords:

Created on 2012-12-30 17:12 by wolfmanx, last changed 2013-01-02 00:03 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
bug_configparser.py wolfmanx, 2012-12-30 17:12 Description and test code
bug_configparser_update_order.py wolfmanx, 2012-12-31 17:42 Exercise preservation/modification of key order for OrderedDict/ConfigParser
Messages (11)
msg178588 - (view) Author: Wolfgang Scherer (wolfmanx) Date: 2012-12-30 17:12
configparser.ConfigParser.clean() always fails:

    >>> cfg = configparser.ConfigParser()
    >>> if not hasattr(configparser.ConfigParser, 'clear'):
    ...     configparser.ConfigParser.clear = configparser_clear_compat
    >>> cfg.clear() #doctest: +ELLIPSIS
    Traceback (most recent call last):
    ...
    ValueError: Cannot remove the default section.

configparser.ConfigParser.update() overwrites all sections except DEFAULT instead of updating them.

See attached test file-
msg178640 - (view) Author: Roundup Robot (python-dev) Date: 2012-12-31 02:44
New changeset dc5adc08f1a8 by Łukasz Langa in branch '3.2':
Fixes `parser.clean()` reported in issue #16820.
http://hg.python.org/cpython/rev/dc5adc08f1a8

New changeset 4fc2fea807e6 by Łukasz Langa in branch '3.3':
Merged `parser.clean()` fix (issue #16820) from 3.2.
http://hg.python.org/cpython/rev/4fc2fea807e6

New changeset c6f9bc5a0cf1 by Łukasz Langa in branch 'default':
Merged `parser.clean()` fix (issue #16820) from 3.2 through 3.3.
http://hg.python.org/cpython/rev/c6f9bc5a0cf1
msg178641 - (view) Author: Łukasz Langa (lukasz.langa) (Python committer) Date: 2012-12-31 03:09
Thanks for your report, Wolfgang. The `clean()` method is now fixed. The
`update()` situation is more complicated, however.

The mapping protocol defines that

    mapping.update({'a': 'b', 'c': 'd'})

and

    mapping.update(a='b', c='d')

are equivalent to

    mapping['a'] = 'b'
    mapping['c'] = 'd'

For `configparser` we decided that setting a section on a parser is
a destructive operation (e.g. it overwrites previous options). If the user
finds this undesirable, she can either use `parser['section'].update()` or
`parser.read_dict()` like you suggested.

The bug here is that __setitem__ for the DEFAULTSECT should also clear options
previously set on this section.
msg178671 - (view) Author: Roundup Robot (python-dev) Date: 2012-12-31 12:57
New changeset 459a23083b66 by Łukasz Langa in branch '3.3':
Fixes `__setitem__` on parser['DEFAULT'] reported in issue #16820.
http://hg.python.org/cpython/rev/459a23083b66

New changeset f6fb5a5748f0 by Łukasz Langa in branch 'default':
Merged `parser['DEFAULT'].__setitem__` fix (issue #16820) from 3.3.
http://hg.python.org/cpython/rev/f6fb5a5748f0
msg178672 - (view) Author: Łukasz Langa (lukasz.langa) (Python committer) Date: 2012-12-31 13:04
For the record, the bug that caused the following to be equivalent:

  parser['DEFAULT'] = {'option': 'value'} 
  parser['DEFAULT'].update({'option': 'value'})

has been fixed for 3.3.1+ only. This way it's going to be easier for users to reason about the fix ("it was broken in 3.2.0 - 3.3.0").

Note that the bug only affected __setitem__ on the default section.
msg178710 - (view) Author: Wolfgang Scherer (wolfmanx) Date: 2012-12-31 17:42
Thanks, works for me.

I only noted the discrepancy and did not give it much thought.
I will just implement a merge method on top of read_dict.
That gives me all options that could be desired :).

However, after implementing the entire compatibility layer, I found one more issue:

Using self.remove_section() changes the section order during an update.
I would prefer that the section be cleared instead of removed in order to preserve the section order. Since OrderedDict does indeed preserve the key order during update, I think that this does not violate the Mapping Protocol.

If this is not desired, just go ahead and close the issue.

See also attached example bug_configparser_update_order.py:

OrderedDict does not change the order of keys upon .update():

    >>> od = OrderedDict((('section1', {}), ('section2', {})))

    >>> list(od.keys())
    ['section1', 'section2']

    >>> od.update((('section1', {}), ('section3', {})))

    >>> list(od.keys())
    ['section1', 'section2', 'section3']

But ConfigParser changes the order of sections upon .update():

    >>> cfg = configparser.ConfigParser()
    >>> cfg.update((('section1', {}), ('section2', {})))

    >>> cfg.sections()
    ['section1', 'section2']

    >>> cfg.update((('section1', {}), ('section3', {})))

    >>> cfg.sections()
    ['section2', 'section1', 'section3']
msg178755 - (view) Author: Łukasz Langa (lukasz.langa) (Python committer) Date: 2013-01-01 20:56
This is a fair point. Stay tuned.
msg178759 - (view) Author: Roundup Robot (python-dev) Date: 2013-01-01 21:37
New changeset f580342b63d8 by Łukasz Langa in branch '3.3':
configparser: preserve section order when using `__setitem__` (issue #16820)
http://hg.python.org/cpython/rev/f580342b63d8

New changeset a758f561a280 by Łukasz Langa in branch 'default':
Merged section order preservation fix when using `__setitem__` (issue #16820)
http://hg.python.org/cpython/rev/a758f561a280
msg178773 - (view) Author: Roundup Robot (python-dev) Date: 2013-01-01 22:51
New changeset 6f0cee62f0c6 by Łukasz Langa in branch '3.2':
configparser: preserve section order when using `__setitem__` (issue #16820)
http://hg.python.org/cpython/rev/6f0cee62f0c6

New changeset 2f5320497017 by Łukasz Langa in branch '3.3':
Null-merged 3.2 section order preservation fix when using `__setitem__` (issue
http://hg.python.org/cpython/rev/2f5320497017

New changeset 27b698395d35 by Łukasz Langa in branch 'default':
Null-merged 3.2 section order preservation fix when using `__setitem__` (issue
http://hg.python.org/cpython/rev/27b698395d35
msg178775 - (view) Author: Łukasz Langa (lukasz.langa) (Python committer) Date: 2013-01-01 22:56
Breaking section order when using `__setitem__` was a bug and as such was fixed in 3.2.4 and 3.3.1.

In the DEFAULTSECT case above someone could potentially use the broken behaviour as a feature so that change was only made for 3.3.1+. In the section order case there is no such controversy.
msg178792 - (view) Author: Roundup Robot (python-dev) Date: 2013-01-02 00:03
New changeset d5c45089df2d by Łukasz Langa in branch '3.3':
Misc/NEWS updated to tell about #14590 and #16820
http://hg.python.org/cpython/rev/d5c45089df2d

New changeset 7938847b2641 by Łukasz Langa in branch '3.2':
Misc/NEWS updated to tell about #16820
http://hg.python.org/cpython/rev/7938847b2641
History
Date User Action Args
2013-01-02 00:03:42python-devsetmessages: + msg178792
2013-01-01 22:56:39lukasz.langasetstatus: open -> closed
resolution: rejected -> fixed
messages: + msg178775
2013-01-01 22:51:23python-devsetmessages: + msg178773
2013-01-01 21:37:58python-devsetmessages: + msg178759
2013-01-01 20:56:35lukasz.langasetmessages: + msg178755
2012-12-31 17:42:49wolfmanxsetstatus: closed -> open
files: + bug_configparser_update_order.py
resolution: fixed -> rejected
messages: + msg178710
2012-12-31 13:04:12lukasz.langasetstatus: open -> closed
resolution: fixed
messages: + msg178672

stage: resolved
2012-12-31 12:57:41python-devsetmessages: + msg178671
2012-12-31 03:10:00lukasz.langasetmessages: + msg178641
versions: + Python 3.3
2012-12-31 02:44:10python-devsetnosy: + python-dev
messages: + msg178640
2012-12-31 00:42:42lukasz.langasetassignee: lukasz.langa
2012-12-30 17:49:51r.david.murraysetnosy: + lukasz.langa
2012-12-30 17:12:53wolfmanxcreate