classification
Title: ConfigParser does not handle files without sections
Type: enhancement Stage: needs patch
Components: Library (Lib) Versions:
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: lukasz.langa Nosy List: Pedro Lacerda, gvanrossum, johnlinp, kernc, lukasz.langa, martin.panter, paul.moore, serhiy.storchaka, terry.reedy, tshepang
Priority: normal Keywords: patch

Created on 2014-08-22 18:22 by kernc, last changed 2017-07-16 19:44 by Pedro Lacerda.

Files
File name Uploaded Description Edit
nosection.patch Pedro Lacerda, 2016-06-11 20:59 review
Pull Requests
URL Status Linked Edit
PR 2735 open python-dev, 2017-07-16 19:38
Messages (14)
msg225693 - (view) Author: kernc (kernc) Date: 2014-08-22 18:22
ConfigParser does not handle files that specify options in the "global" section (first option specified before any section is). This configuration "behavior" is present in the configuration files in the wild [1, 2], and while the naive workaround is simple, ... really?!

The support for section-less configuration would also play nice with various "POSIX-compatible config files" (think /etc/default/*, /etc/os-release, ...).

Ideally, the support could be enabled by default or only when `strict=False` constructor argument is supplied. The options in this global section could likely be accessed in the empty ('') section, e.g. `config.get('', 'option')`.

Thus, ConfigParser could finally really replace the venerable ConfigObj [3].

[1]: http://stackoverflow.com/a/22501121/
[2]: https://www.google.com/search?q=MissingSectionHeaderError%3A+File+contains+no+section+headers
[3]: http://www.voidspace.org.uk/python/configobj.html
msg225696 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2014-08-22 18:53
That's an interesting feature request. Parsing it only while `strict=False` sounds like a good plan.
msg225715 - (view) Author: kernc (kernc) Date: 2014-08-22 21:41
I, for one, would actually prefer if global options were parsed by default and MissingSectionHeaderError was deprecated instead.
From what little specification available, INI format does **not** require options be in sections [4, 5].

Additionally, "Linux and Unix systems also use a similar file format for system configuration" [6] and allowing global options being a (very sane) default would nicely fill this use case as well.

In general, the format is not well defined [6], so choice of name `strict` for an argument is kind of odd too. What is it conforming to?

It may be my sole opinion that parsing global options by default into a '' (or appropriate) section and deprecating MissingSectionHeaderError would benefit everyone [2, 9] and hinder few if any one at all [8, 9]. YMMV.

[4]: http://en.wikipedia.org/wiki/INI_file#Sections
[5]: http://en.wikipedia.org/wiki/INI_file#Global_properties
[6]: http://en.wikipedia.org/wiki/INI_file
[7]: http://en.wikipedia.org/wiki/INI_file#Varying_features
[8]: http://nullege.com/pages/noresult/MissingSectionHeaderError
[9]: https://github.com/search?l=python&q=MissingSectionHeaderError&type=Code
msg226587 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2014-09-08 16:24
It looks like this feature request tries to change an existing (ancient) module into something it isn't.  At the very least can you point to a spec for the syntax of "POSIX" config files?  I always thought they were essentially shell scripts, which suggests that they might have a more complex (and different) quoting syntax than ConfigParser, so there might be cases where the interpretation of a line in a POSIX config file would be different than the interpretation of the same line in a .ini file.
msg226593 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2014-09-08 17:35
It's not unreasonable as a new feature, but the default behaviour shouldn't change. It matches ini files (like it or not, ConfigParser parses ini-style files - the docs even say so), and sectionless values are not standard ini format.

I'd suggest a new __init__ option, allow_unnamed_section (default False) that permits variables to be placed before the first section header. I'd further suggest that the names be treated as if they were in a section with name '', for consistency of access with other sections.

It's plausible that people might want the defaults section to be the unnamed section. If so, that could be another option, default_is_unnamed (default False, if True this implies allow_unnamed_section). But I'm not sure the additional complexity is worth it.
msg226621 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-09-09 04:29
The MS function GetPrivateProfileString appears to require sections.
http://msdn.microsoft.com/en-us/library/ms724353.aspx
On the other hand, it does not appear to do interpolation, so we have already not restricted ourselves to the MS function.

In looking through the .ini files in my game directory, which includes some old games, I found a couple with no section header.  So such files do exist in the wild.  I am dubious that there are any with a mixture of both sections and additional option lines at the top without a section.

Anyone writing an app and planning to parse a .ini file can add [Start] or [Setup] at the top.  So there is only an issue for 3rd party software parsing a file without.

I think a more useful new configparser feature would be to keep comment lines and write them back out after a configuration is changed.
msg226827 - (view) Author: kernc (kernc) Date: 2014-09-12 16:46
>
> I am dubious that there are any with a mixture of both sections and
> additional option lines at the top without a section.
>

rsyncd.conf [1] is one such example, and I wouldn't say there aren't
countless more in the wild.

> Anyone writing an app and planning to parse a .ini file can add [Start] or
> [Setup] at the top.
>

Indeed. Here lies the problem of this unfortunate issue:
MissingSectionHeaderError is only ever caught [9] to mitigate this **awful
default behavior** and attach a dummy section at the top, as you say. Or
can anyone care to propose another relevant use case for this poorly (un-)
thought through exception?

> I think a more useful new configparser feature would be to keep comment
> lines and write them back out after a configuration is changed.
>

While this is very much off-topic, configobj [3] does too seem to have done
so since ages.
msg226835 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-09-12 18:23
Microsoft Windows INI files, "POSIX-compatible config files", and other formats (e.g. Java properties files) use different methods for escaping, quoting, line continuing, interpolations, etc. Actually there are more differences than similarity between them.
msg226909 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2014-09-15 09:38
I don't like the idea to magically introduce a '' section since this behaviour would be confusing for interpolation and not particularly discoverable by programmers. Let alone bikeshedding if this should rather be a None section.

Using DEFAULTSECT for this purpose is equally wrong since it would silently inject default values to every section, which may or may not be desirable.

All in all, it comes down to the question whether the programmer expects section-less configuration. If not, the '' section will not be helpful anyway. If yes, then it's desirable to be able to specify a section name for global options at *read time*. Symmetrically, the user could specify which section name to omit during configuration writing. I like that since it's explicit and more composable than a blanket global section. It would also be 100% backwards compatible.

I'll prepare a patch for this idea so we can see how good this API looks like in practice.
msg226936 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-09-15 19:41
I read the rsyncd.conf doc at http://linux.die.net/man/5/rsyncd.conf (linked from the StackOverflow question).  These files are not .ini files. However, I believe the parsing rules are mostly compatible with RawConfigParser, or could be made so by using existing customization options, including subclassing.

The sectionless options are called 'global options' for one of two different reasons.  Some, selected from a predefined list of startup options, act as if they were in a [STARTUP] section.  Others, selected from a predefined list of module options, act as if they were in a [DEFAULT] section. The latter provide alternate default value for options in the various [<module>] sections.

We clearly should not directly support the specialized rules of rsyncd.conf.  But if, as kernc requests, RawConfigParser gathered sectionless options into a '' section, users could probably work with these files.  The details would be up to them, or a config_unix module writer.  The same comment applies to other files, including .ini files with sectionless options.

Łukasz, the only one bikeshedding '' is you.  I do not see any need for a new API and I think it just confuses this issue.  Reading and writing sectionless options via a '' section should be sufficient to work with .ini files.

To me, the remaining question is whether to retain configparser.MissingSectionHeaderError.  The problem with piggy-backing its optional use on the 'strict' parameter is that someone might want to reject duplicates while reading sectionless options.  But it ia a plausible idea.

As an aside, the documentation for MissingSectionHeaderError is currently a bit confused.  The docstring correctly says "Raised when a key-value pair is found before any section header."  The doc incorrectly says "Exception raised when attempting to parse a file which has no section headers."  I tested and it is indeed raised even when there is a section header after an initial option line.  The exception message is also wrong: "File contains no section headers."  The latter two should really say that the files does not *start* with a section header (ignoring comment lines), or use the wording of the docstring.
msg227749 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2014-09-28 02:36
See also Issue 549037, about handling files with a single anonymous section
msg268271 - (view) Author: Pedro Lacerda (Pedro Lacerda) * Date: 2016-06-11 20:59
I also never found a mixture of sectionless options followed by sectioned options. So an unnamed section that is also the DEFAULTSECTION will probably work.

In this patch when `default_section=None` is passed to `RawConfigParser` it will parse top level options into the default section and skip writing its title.

As drawback, default options is not showed in `options()` or `has_section()` reducing it usefulness. It works with `items()` and `keys()` however.

> Using DEFAULTSECT for this purpose is equally wrong since it would
> silently inject default values to every section

I disagree with that because I really *never* found in wild a file where it will happen.

> All in all, it comes down to the question whether the programmer
> expects section-less configuration. If not, the '' section will not be 
> helpful anyway. If yes, then it's desirable to be able to specify a
> section name for global options at *read time*.

Pass a name at read time will improve the API as `sections()` and `has_section()` will work as usual and not like a DEFAULTSECTION.

Please look my patch and tell if it's acceptable, if you prefer that a section name must be given at read and write time we can manage it.

It's my first post in this tracker and I'm very glad that I got it working even if not merged!
msg298339 - (view) Author: 林自均 (johnlinp) Date: 2017-07-14 05:21
Hi Pedro Lacerda,

I think you should submit your patch as a GitHub pull request. Please correct me if I'm wrong. Thanks.
msg298448 - (view) Author: Pedro Lacerda (Pedro Lacerda) * Date: 2017-07-16 19:44
Thank you 林自均! I just made a pull-request with the relevant bits.
History
Date User Action Args
2017-07-16 19:44:08Pedro Lacerdasetmessages: + msg298448
2017-07-16 19:38:49python-devsetpull_requests: + pull_request2795
2017-07-14 05:21:32johnlinpsetnosy: + johnlinp
messages: + msg298339
2016-06-11 20:59:23Pedro Lacerdasetfiles: + nosection.patch
versions: - Python 3.5
nosy: + Pedro Lacerda

messages: + msg268271

keywords: + patch
2015-03-19 23:57:20martin.panterlinkissue23711 superseder
2014-09-28 02:36:34martin.pantersetmessages: + msg227749
2014-09-15 19:41:18terry.reedysetmessages: + msg226936
2014-09-15 09:38:53lukasz.langasetmessages: + msg226909
2014-09-12 18:23:54serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg226835
2014-09-12 16:46:15kerncsetmessages: + msg226827
2014-09-09 04:29:42terry.reedysetnosy: + terry.reedy
messages: + msg226621
2014-09-08 17:35:32paul.mooresetnosy: + paul.moore
messages: + msg226593
2014-09-08 16:24:35gvanrossumsetnosy: + gvanrossum
messages: + msg226587
2014-09-05 02:46:46martin.pantersetnosy: + martin.panter
2014-09-02 18:29:44tshepangsetnosy: + tshepang
2014-08-22 21:41:55kerncsetmessages: + msg225715
2014-08-22 18:53:11lukasz.langasetassignee: lukasz.langa
type: behavior -> enhancement
versions: - Python 2.7, Python 3.2
nosy: + lukasz.langa

messages: + msg225696
stage: needs patch
2014-08-22 18:22:48kernccreate