classification
Title: ConfigParser .read() - handling of nonexistent files
Type: enhancement Stage: test needed
Components: Library (Lib) Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Adrian Wielgosik, dheiberg, hongweipeng, lukasz.langa, terry.reedy
Priority: normal Keywords:

Created on 2018-12-09 16:35 by Adrian Wielgosik, last changed 2018-12-14 20:05 by terry.reedy.

Messages (2)
msg331439 - (view) Author: Adrian Wielgosik (Adrian Wielgosik) * Date: 2018-12-09 16:35
Documentation of ConfigParser says:

> If a file named in filenames cannot be opened, that file will be ignored. This is designed so that you can specify an iterable of potential configuration file locations (for example, the current directory, the user’s home directory, and some system-wide directory), and all existing configuration files in the iterable will be read.

While this is a useful property, it can also be a footgun. The first read() example in the Quick Start section contains just a single file read:

>>> config.read('example.ini')

I would expect that this basic usage is very popular. If the file doesn't exist, the normal usage pattern fails in a confusing way:

 from configparser import ConfigParser
 config = ConfigParser()
 config.read('config.txt')
 value = config.getint('section', 'option')
---> configparser.NoSectionError: No section: 'section'

In my opinion, this error isn't very obvious to understand and debug, unless you have read that piece of .read() documentation.

This behavior did also bite me even more, with another usage pattern I've found in a project I maintain:
> config.read('global.txt')
> config.read('local.txt')
Here, both files are expected to exist, with the latter one extending or updating configuration from the first file. If one of the files doesn't exist (eg mistake during deployment), there's no obvious error, but the program will be configured in different way than intended.

Now, I'm aware that all of this can be avoided by simply using `read_file()`:
> with open('file.txt') as f:
>    config.read_file(f)
But again, `.read()` is the one usually mentioned first in both the official documentation, and most independent guides, so it's easy to get wrong.

Due to this, I propose adding an extra parameter to .read():
read(filenames, encoding=None, check_exist=False)
that, when manually set to True, will throw exception if any of input files doesn't exist; and to use this parameter by default in Quick Start section of ConfigParser documentation.
If this is a reasonable idea, I could try and make a PR.

For comparison, the `toml` Python library has the following behavior:
- if argument is a single filename and file doesn't exist, it throws
- if argument is a list of filenames and none exist, it throws
- if argument is a list of filenames and at least one exists, it works, but prints a warning for each nonexistent file.

For the record, seems like this issue was also mentioned in https://bugs.python.org/issue490399
msg331862 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-12-14 20:05
Since the code and doc agree, and since the proposal is to add a call parameter, this would be an enhancement for next release only, not a bug fix.

The proposal seems reasonable.  I might use it for IDLE.

IDLE uses .read within this subclass method.

    def Load(self):
        "Load the configuration file from disk."
        if self.file:  # '' for at least some tests
            self.read(self.file) 

The default config files in idlelib should be present.  (I should see what happens if not.  Does every 'get' pass a seeming redundant and possibly inconsistent backup default?)  I might use the new parameter here.  

User override config files and even the config directory are optional, so the current behavior is fine.
History
Date User Action Args
2018-12-14 20:05:28terry.reedysetversions: + Python 3.8
nosy: + terry.reedy

messages: + msg331862

type: behavior -> enhancement
stage: test needed
2018-12-11 06:40:08hongweipengsetnosy: + hongweipeng
2018-12-10 20:28:32dheibergsetnosy: + dheiberg
2018-12-09 18:02:27xtreaksetnosy: + lukasz.langa
2018-12-09 16:35:30Adrian Wielgosikcreate