classification
Title: configparser does not convert defaults to strings
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Anthony Sottile, aragilar, barry, lukasz.langa, miss-islington, r.david.murray, vstinner
Priority: deferred blocker Keywords: 3.7regression, patch

Created on 2015-04-01 04:23 by aragilar, last changed 2018-08-08 15:02 by lukasz.langa. This issue is now closed.

Files
File name Uploaded Description Edit
py27-config-fix.patch aragilar, 2015-04-08 05:26 review
Pull Requests
URL Status Linked Edit
PR 2558 merged aragilar, 2017-07-04 06:10
PR 3176 merged lukasz.langa, 2017-08-21 23:16
PR 3191 merged lukasz.langa, 2017-08-22 19:01
PR 7524 merged lukasz.langa, 2018-06-08 09:19
PR 7529 merged miss-islington, 2018-06-08 11:04
Messages (16)
msg239768 - (view) Author: James Tocknell (aragilar) * Date: 2015-04-01 04:23
ConfigParser(defaults={1:2.4}) and ConfigParser(defaults={"a":5.2}) cause an exception when configparser tries to perform string operations on 1 and 5.2. I didn't see it documented that defaults must only contain strings, and using ConfigParser['DEFAULT'] = {1:2.4} does the necessary string conversion, so it looks like there should be some conversion done to the defaults.
msg240249 - (view) Author: James Tocknell (aragilar) * Date: 2015-04-08 05:26
Here's a patch for 2.7 (based of the head of the 2.7 branch), something similar could be done for 3.4 (I wasn't sure what branch I was supposed to base the patch off, since 3.4 is inactive). The string requirement was already noted in the docstring of the configparser module, but that's not mentioned in the main docs. Also, I wasn't sure where to put a test in because there was not test_configparser.py located in Lib/test.
msg297793 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-07-06 03:41
I'm guessing we can only do something here in 3.7, for backward compatibility reasons, but maybe I'm wrong.  Hopefully Lukasz will notice the activity on the issue and have time to take a look.
msg298205 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2017-07-12 12:16
Responded on the PR.
msg298234 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-07-12 16:50
Thanks.  OK, so you agree a fix is appropriate.  What about the question of backport/backward compatibility?
msg298279 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2017-07-13 10:21
Backwards compatibility will be considered when the patch is ready. I'm not overly worried about fixing a case that currently raises exceptions all over the place. There's few compatibility concerns that we need to consider in this case.
msg300660 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2017-08-21 22:46
New changeset 44e6ad87340d50f48daf53b6a61138377d0d0d10 by Łukasz Langa (James Tocknell) in branch 'master':
bpo-23835: Enforce that configparser defaults are strings (#2558)
https://github.com/python/cpython/commit/44e6ad87340d50f48daf53b6a61138377d0d0d10
msg300661 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2017-08-21 23:23
New changeset ea57923e89aa4a1bde1d4fa1de7d6eacff603683 by Łukasz Langa in branch 'master':
bpo-23835: [docs] configparser converts defaults to strings (#3176)
https://github.com/python/cpython/commit/ea57923e89aa4a1bde1d4fa1de7d6eacff603683
msg300717 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2017-08-22 19:06
I merged the original fix and documented it. I thought about it some more and remembered that RawConfigParser objects do in fact support non-string values by historical coincidence. It's unfortunately a popular idiom with old programs to load some configuration defaults using the `defaults=` keyword and later use the legacy get() and set() API which doesn't check types inside. A config file like this cannot be safely written back to a file, etc.

I would very much like to get rid of RawConfigParser entirely but we're stuck with it due to backwards compatibility. So, to fix the regression, I created PR 3191.
msg300791 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2017-08-24 16:43
New changeset a5fab17fc11433b2418f626dc51e8a3d07b198ca by Łukasz Langa in branch 'master':
bpo-23835: Restore legacy defaults= behavior for RawConfigParser (#3191)
https://github.com/python/cpython/commit/a5fab17fc11433b2418f626dc51e8a3d07b198ca
msg318983 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-06-07 23:21
I think this introduced a regression in 3.7.  See bpo-33802

https://bugs.python.org/issue33802
msg318999 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-06-08 00:38
It seems like this issue introduced a regression according to Barry: bpo-33802.
msg319047 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2018-06-08 11:02
New changeset 214f18e49feb6a9d6c05aa09a4bb304905e81334 by Łukasz Langa in branch 'master':
bpo-33802: Do not interpolate in ConfigParser while reading defaults (GH-7524)
https://github.com/python/cpython/commit/214f18e49feb6a9d6c05aa09a4bb304905e81334
msg319066 - (view) Author: miss-islington (miss-islington) Date: 2018-06-08 14:01
New changeset f44203d782e397941c17d96e6a1f9dc1df08b3e6 by Miss Islington (bot) in branch '3.7':
bpo-33802: Do not interpolate in ConfigParser while reading defaults (GH-7524)
https://github.com/python/cpython/commit/f44203d782e397941c17d96e6a1f9dc1df08b3e6
msg320899 - (view) Author: Anthony Sottile (Anthony Sottile) * Date: 2018-07-02 16:22
Unclear if this regression (from this patch) is intentional or not:

```
$ python3.6 -c 'import configparser; configparser.ConfigParser(defaults={"a": None})'
$ python3.7 -c 'import configparser; configparser.ConfigParser(defaults={"a": None})'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/configparser.py", line 638, in __init__
    self._read_defaults(defaults)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/configparser.py", line 1216, in _read_defaults
    self.read_dict({self.default_section: defaults})
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/configparser.py", line 753, in read_dict
    self.set(section, key, value)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/configparser.py", line 1197, in set
    self._validate_value_types(option=option, value=value)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/configparser.py", line 1182, in _validate_value_types
    raise TypeError("option values must be strings")
TypeError: option values must be strings
```
msg323285 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2018-08-08 15:02
That's intentional. In ConfigParser objects this exception would be raised for any other section's assignment already. You want RawConfigParser if you want to put (invalid) types as option values. See:

>>> cp = ConfigParser()
>>> cp['asd'] = {'a': None}
Traceback (most recent call last):
...
TypeError: option values must be strings
>>> rcp = RawConfigParser()
>>> rcp['asd'] = {'a': None}
>>>
History
Date User Action Args
2018-08-08 15:02:23lukasz.langasetmessages: + msg323285
2018-07-02 16:22:34Anthony Sottilesetnosy: + Anthony Sottile
messages: + msg320899
2018-06-08 17:10:28barrysetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-06-08 14:01:58miss-islingtonsetnosy: + miss-islington
messages: + msg319066
2018-06-08 11:04:00miss-islingtonsetpull_requests: + pull_request7165
2018-06-08 11:02:52lukasz.langasetmessages: + msg319047
2018-06-08 09:19:48lukasz.langasetkeywords: + patch
stage: patch review
pull_requests: + pull_request7153
2018-06-08 00:38:12vstinnersetnosy: + vstinner
messages: + msg318999
2018-06-07 23:21:36barrysetstatus: closed -> open
priority: normal -> deferred blocker


keywords: + 3.7regression, - patch
nosy: + barry
messages: + msg318983
resolution: fixed -> (no value)
stage: resolved -> (no value)
2017-08-27 22:05:43lukasz.langasetstatus: open -> closed
resolution: fixed
stage: resolved
2017-08-24 16:43:58lukasz.langasetmessages: + msg300791
2017-08-22 19:06:09lukasz.langasetmessages: + msg300717
2017-08-22 19:01:59lukasz.langasetpull_requests: + pull_request3230
2017-08-21 23:23:40lukasz.langasetmessages: + msg300661
2017-08-21 23:16:06lukasz.langasetpull_requests: + pull_request3214
2017-08-21 22:46:35lukasz.langasetmessages: + msg300660
2017-08-21 21:18:44lukasz.langasetversions: - Python 2.7, Python 3.5, Python 3.6
2017-07-13 10:21:36lukasz.langasetmessages: + msg298279
2017-07-12 16:50:50r.david.murraysetmessages: + msg298234
2017-07-12 12:16:12lukasz.langasetmessages: + msg298205
2017-07-06 03:41:26r.david.murraysetnosy: + r.david.murray

messages: + msg297793
versions: + Python 3.6, Python 3.7, - Python 3.4
2017-07-04 06:10:32aragilarsetpull_requests: + pull_request2628
2015-04-08 05:26:55aragilarsetfiles: + py27-config-fix.patch
keywords: + patch
messages: + msg240249
2015-04-01 06:59:56serhiy.storchakasetnosy: + lukasz.langa

type: crash -> behavior
versions: + Python 3.5
2015-04-01 04:23:08aragilarcreate