classification
Title: Update ConfigParser doc: accepts iterables, not just lists
Type: behavior Stage: resolved
Components: Documentation, Library (Lib) Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: lukasz.langa Nosy List: Rich.Rauenzahn, ZackerySpytz, brian.curtin, iritkatriel, lukasz.langa, terry.reedy, xiang.zhang
Priority: normal Keywords: patch

Created on 2016-06-19 04:50 by Rich.Rauenzahn, last changed 2020-10-14 01:18 by terry.reedy. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8123 merged ZackerySpytz, 2018-07-11 01:11
PR 9637 merged miss-islington, 2018-09-29 16:16
PR 9638 merged miss-islington, 2018-09-29 16:16
Messages (10)
msg268838 - (view) Author: Rich Rauenzahn (Rich.Rauenzahn) Date: 2016-06-19 04:50
This came up on StackOverflow: http://stackoverflow.com/a/37903779/2077386

I wanted to bring it to your attention in case it hasn't been notice before.

It turns out that if you pass a fileobject (i.e., ConfigParser().read(open("foo"))) ConfigParser.read() will look at the argument, see it isn't a basestring, and then will iterate over it.  fileobjects are iterables.

This results in iterating over the contents of the file 'foo'.  It then attempts to open a file named after every line read from 'foo'.

For example, I made a file foo and filled it with a-g, each letter per line.  

strace shows:

open("foo", O_RDONLY)                     = 3
open("a\n", O_RDONLY)                   = -1 ENOENT (No such file or directory)
open("b\n", O_RDONLY)                   = -1 ENOENT (No such file or directory)
open("c\n", O_RDONLY)                   = -1 ENOENT (No such file or directory)
open("d\n", O_RDONLY)                   = -1 ENOENT (No such file or directory)
open("e\n", O_RDONLY)                   = -1 ENOENT (No such file or directory)
open("f\n", O_RDONLY)                   = -1 ENOENT (No such file or directory)
open("g\n", O_RDONLY)                   = -1 ENOENT (No such file or directory)

...and since the API is designed to ignore files it can't open, it just ignores the open errors.

I wonder if this fileobject case ought to be checked for when checking the arguments passed into ConfigParser.read().  

Thank you.
msg268839 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-06-19 05:05
The doc tells ConfigParser.read accepts a filename or a list of filenames. Why do you pass it fileobject? If you want to use fileobject, why not read_file?
msg268840 - (view) Author: Rich Rauenzahn (Rich.Rauenzahn) Date: 2016-06-19 05:16
Given that write() accepts a fileobject, but read() accepts a list of strings or a string (and readfp() is the one that accepts a fileobject instead), this seems like it could be a common enough error that just iterating over the fileobject could be undesirable and an exception might be thrown instead.

I'm throwing this out here to see if the library maintainers were aware of this odd edge case.
msg268914 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2016-06-20 17:28
Thanks for your report, Rich.  configparser is one of the oldest libraries in the standard library.  Its behavior might sometimes not be intuitive to newcomers but is burdened by years of backwards compatibility.  Changing the behavior now to raise a type error on an open file passed to read would inevitably cause some program in the wild to break.  Given this constraint, there's not much we can do besides documentation.  If you have suggestions how we could improve it, let me know here.
msg268915 - (view) Author: Rich Rauenzahn (Rich.Rauenzahn) Date: 2016-06-20 17:36
Thank you, lukasz.  That's the answer I anticipated -- I can appreciate the backwards compatibility aspect very much.

Regarding the docs, the docs say:

"Attempt to read and parse a list of filenames, returning a list of filenames which were successfully parsed."  

I don't know if the convention in the docs is that list always means *list*, but it could be changed to be iterable since that is the implementation.  

That fileobjects are also iterables could be pointed out here, but I think anyone making this mistake isn't going to make the mistake from misreading the docs, it's from skipping the docs and assuming read() is consistent with write().
msg269215 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-06-24 21:47
Doing what the doc says is not a bug. The doc being obsolete in saying 'list' (which was once true) instead of the now correct 'iterable' is. The 3.6 ConfigParser.__doc___ also says 'list', so both docstrings and docs for all current versions should be fixed.
msg326683 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2018-09-29 16:16
New changeset e45473e3ca31e5b78dc85cab575f5bb60d5b7f8f by Brian Curtin (Zackery Spytz) in branch 'master':
bpo-27351: Fix ConfigParser.read() documentation and docstring (GH-8123)
https://github.com/python/cpython/commit/e45473e3ca31e5b78dc85cab575f5bb60d5b7f8f
msg326685 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2018-09-29 16:33
New changeset b0b8f9bd4e6f78ac7383b4e56cfb6cbacc77da89 by Brian Curtin (Miss Islington (bot)) in branch '3.7':
bpo-27351: Fix ConfigParser.read() documentation and docstring (GH-8123)
https://github.com/python/cpython/commit/b0b8f9bd4e6f78ac7383b4e56cfb6cbacc77da89
msg326686 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2018-09-29 16:39
New changeset 3cd5e8e83c9785d9f505138903c7a50dc964101e by Brian Curtin (Miss Islington (bot)) in branch '3.6':
bpo-27351: Fix ConfigParser.read() documentation and docstring (GH-8123)
https://github.com/python/cpython/commit/3cd5e8e83c9785d9f505138903c7a50dc964101e
msg378278 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-10-08 19:09
This seems complete, can it be closed?
History
Date User Action Args
2020-10-14 01:18:42terry.reedysetstatus: open -> closed
title: Unexpected ConfigParser.read() behavior when passed fileobject -> Update ConfigParser doc: accepts iterables, not just lists
stage: patch review -> resolved
resolution: fixed
versions: - Python 2.7
2020-10-08 19:09:47iritkatrielsetnosy: + iritkatriel
messages: + msg378278
2018-09-29 16:39:42brian.curtinsetmessages: + msg326686
2018-09-29 16:33:09brian.curtinsetmessages: + msg326685
2018-09-29 16:16:16miss-islingtonsetpull_requests: + pull_request9030
2018-09-29 16:16:10miss-islingtonsetpull_requests: + pull_request9029
2018-09-29 16:16:00brian.curtinsetnosy: + brian.curtin
messages: + msg326683
2018-07-11 01:12:56ZackerySpytzsetnosy: + ZackerySpytz

versions: + Python 3.7, Python 3.8, - Python 3.5
2018-07-11 01:11:05ZackerySpytzsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request7771
2016-06-24 21:47:53terry.reedysetversions: + Python 3.5, Python 3.6
nosy: + terry.reedy

messages: + msg269215

components: + Documentation
2016-06-20 17:36:59Rich.Rauenzahnsetmessages: + msg268915
2016-06-20 17:28:09lukasz.langasetassignee: lukasz.langa
messages: + msg268914
stage: needs patch
2016-06-19 05:16:01Rich.Rauenzahnsetmessages: + msg268840
2016-06-19 05:05:57xiang.zhangsetnosy: + xiang.zhang, lukasz.langa
messages: + msg268839
2016-06-19 04:50:27Rich.Rauenzahncreate