classification
Title: ConfigParser does not handle options without values
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: fdrake Nosy List: dwayne, eric.araujo, fdrake, lukasz.langa, mkindahl
Priority: normal Keywords: patch

Created on 2009-09-27 06:14 by mkindahl, last changed 2010-09-03 04:24 by fdrake. This issue is now closed.

Files
File name Uploaded Description Edit
cfgparser-1.patch mkindahl, 2009-09-27 06:13 Patch #1 for allowing ConfigParser to read configuration files containing options without values
cfgparser-2.patch mkindahl, 2009-09-29 08:35 Patch #2 for allowing ConfigParser to read configuration files containing options without values
cpsample.py fdrake, 2010-09-03 03:25 Diagnostic script
issue-7005.patch fdrake, 2010-09-03 03:50 Patch (Python 2)
Messages (16)
msg93168 - (view) Author: Mats Kindahl (mkindahl) Date: 2009-09-27 06:13
When ConfigParser is used to read in a my.cnf file (MySQL Server
Configuration File), it fails for options that do not have value.

ConfigParser is designed to require a value for each option, but some
systems, such as MySQL option file reader, accepts options without
values. Reading a my.cnf file is almost certain to include options
without values. The server can accept options with values even though
the value is not necessary, but there are some tools that do not allow
values for options that do not require them.

There is an attached patch that optionally will allow options to not
have a value. In order to distinguish options with no value from options
with the empty string, None is assigned to options without values.

The default behavior is to not allow options without values.
msg93183 - (view) Author: Fred L. Drake, Jr. (fdrake) (Python committer) Date: 2009-09-27 20:34
The test "value is not None" in line 620 (of the new version) could be
just "value" and get a little more value from less code.

I don't think I've ever run across a sample .ini-style file that used
unspecified values, though it's frequently done in "flat" configuration
files.

Have you come across an existing configuration format that uses the
[section] markers and unspecified values, or is this really a new use-case?
msg93199 - (view) Author: Mats Kindahl (mkindahl) Date: 2009-09-28 09:27
I replied to the mail, but I don't know if it is attached to the issue,
so I'll repeat here.

I'll change the "value is not None" to just "value" where I used that idiom.

The style of using options without values is quite common in my.cnf
files used by MySQL programs. For example, in the Debian (Ubuntu)
package I have installed is using several options without values.
msg93200 - (view) Author: Mats Kindahl (mkindahl) Date: 2009-09-28 09:32
Fred L. Drake, Jr. wrote:
> Fred L. Drake, Jr. <fdrake@acm.org> added the comment:
> 
> The test "value is not None" in line 620 (of the new version) could be
> just "value" and get a little more value from less code.

OK.

> 
> I don't think I've ever run across a sample .ini-style file that used
> unspecified values, though it's frequently done in "flat" configuration
> files.
> 
> Have you come across an existing configuration format that uses the
> [section] markers and unspecified values, or is this really a new use-case?

It is very common in my.cnf files for MySQL, and has been so "for ever." For
example, the default my.cnf file that is installed on my Debian (Ubuntu) system
have several options without values (e.g., skip-external-locking, quick, and
quote-names).

Best wishes,
Mats Kindahl

> 
> ----------
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue7005>
> _______________________________________
msg93259 - (view) Author: Mats Kindahl (mkindahl) Date: 2009-09-29 08:35
Adding patch with the changes requested by Fred.
msg93345 - (view) Author: Fred L. Drake, Jr. (fdrake) (Python committer) Date: 2009-09-30 03:16
I just looked at some MySQL configuration examples, and wonder if the
guys that made that configuration-style choice were on crack.

MySQL's syntax is sufficient justification for this feature (however
abhorrent I consider it).

(I've not read the patch.)
msg95244 - (view) Author: Mats Kindahl (mkindahl) Date: 2009-11-14 17:11
So, what is the status on this?
Who needs to review it?
Is there anything I can do to get it accepted?
Do I need to make any changes (in addition to those already suggested
and done by fdrake)?
msg99454 - (view) Author: Fred L. Drake, Jr. (fdrake) (Python committer) Date: 2010-02-17 01:22
Assigning to myself for handling.

Bumping to Python 2.7 / 3.2 since support for this syntax variation is a new feature.
msg99558 - (view) Author: Fred L. Drake, Jr. (fdrake) (Python committer) Date: 2010-02-19 05:25
Patch and documentation committed to the trunk (r78232).
msg99559 - (view) Author: Fred L. Drake, Jr. (fdrake) (Python committer) Date: 2010-02-19 06:11
Patch and documentation merged to the py3k branch (r78233).

Work on this is complete.
msg114965 - (view) Author: Dwayne Bailey (dwayne) Date: 2010-08-26 11:29
This is causing a regression in our code.

Previously when we write out our INI file for an entry that has a value of None we saw the following:
value = None

These are now stored as:
value

This is now causing a traceback in our code.

But interestingly I haven't changed anything in our initialisation of ConfigParser, I would have assumed that I need to set allow_no_value for this to work in the new way that MySQL expects.

I would have expected everything to work as it currently does in 2.6 unless I specifically request

You can see the traceback of Virtaal under Python 2. in this bug:
https://bugzilla.redhat.com/show_bug.cgi?id=622061
msg114968 - (view) Author: Fred L. Drake, Jr. (fdrake) (Python committer) Date: 2010-08-26 12:28
Re-opening for investigation.

(The previous message really should have been a new issue.)
msg115407 - (view) Author: Ɓukasz Langa (lukasz.langa) (Python committer) Date: 2010-09-02 22:29
This one is strange. Trying to set a None value through set() raises 'TypeError: option values must be strings' on both Python 3.2 and 2.7. A string value of 'None' behaves as expected (e.g. correctly).
msg115414 - (view) Author: Fred L. Drake, Jr. (fdrake) (Python committer) Date: 2010-09-03 03:25
I've attached a diagnostic script that I ran with Python 2.4..3.2 (current py3k HEAD); there are two output variants:

"old style":

    [section]
    option = None

"new style":

    [section]
    option

This is the output I get when running this script for each of those Python versions:

    2.4.6 RawConfigParser: old-style output
    2.4.6 SafeConfigParser: raised TypeError on set
    2.4.6 ConfigParser: old-style output

    2.5.5 RawConfigParser: old-style output
    2.5.5 SafeConfigParser: raised TypeError on set
    2.5.5 ConfigParser: old-style output

    2.6.5 RawConfigParser: old-style output
    2.6.5 SafeConfigParser: raised TypeError on set
    2.6.5 ConfigParser: old-style output

    2.7 RawConfigParser: new-style output
    2.7 SafeConfigParser: raised TypeError on set
    2.7 ConfigParser: new-style output

    3.1.1 RawConfigParser: old-style output
    3.1.1 SafeConfigParser: raised TypeError on set
    3.1.1 ConfigParser: old-style output

    3.2a1+ RawConfigParser: new-style output
    3.2a1+ SafeConfigParser: raised TypeError on set
    3.2a1+ ConfigParser: new-style output

Essentially: For the RawConfigParser and ConfigParser classes, the output changes in 2.7 and 3.2, and in a way that should be considered incorrect because it conflicts with the allow_no_values setting.

This is a bug and should be fixed in both 2.7 and 3.2.

The TypeError-on-set is consistently raised only for SafeConfigParser, and should remain unchanged.  (Why this was handled differently for SafeConfigParser I don't recall offhand.)
msg115416 - (view) Author: Fred L. Drake, Jr. (fdrake) (Python committer) Date: 2010-09-03 03:50
Attached fix & test for Python 2; adjusting to Python 3 is trivial.

The test could be added to 2.6 as well to protect against regressions there, though that's unlikely to be a significant issue.
msg115417 - (view) Author: Fred L. Drake, Jr. (fdrake) (Python committer) Date: 2010-09-03 04:24
Commited as r84443 (release27-maint), r84444 (py3k)
History
Date User Action Args
2010-09-03 04:24:03fdrakesetstatus: open -> closed

messages: + msg115417
2010-09-03 03:50:44fdrakesetfiles: + issue-7005.patch

messages: + msg115416
2010-09-03 03:25:26fdrakesetfiles: + cpsample.py

messages: + msg115414
2010-09-02 22:29:10lukasz.langasetmessages: + msg115407
2010-08-26 12:32:37fdrakesetnosy: + lukasz.langa
2010-08-26 12:28:44fdrakesetstatus: closed -> open

messages: + msg114968
2010-08-26 11:29:05dwaynesetnosy: + dwayne
messages: + msg114965
2010-02-19 06:11:39fdrakesetstatus: open -> closed
type: enhancement
messages: + msg99559

resolution: accepted
stage: resolved
2010-02-19 05:25:32fdrakesetmessages: + msg99558
2010-02-17 16:17:12eric.araujosetnosy: + eric.araujo
2010-02-17 01:22:38fdrakesetassignee: fdrake
messages: + msg99454
versions: + Python 2.7, Python 3.2, - Python 2.6
2009-11-14 17:11:45mkindahlsetmessages: + msg95244
2009-09-30 03:16:15fdrakesetmessages: + msg93345
2009-09-29 08:35:50mkindahlsetfiles: + cfgparser-2.patch

messages: + msg93259
2009-09-28 09:32:40mkindahlsetmessages: + msg93200
2009-09-28 09:27:34mkindahlsetmessages: + msg93199
2009-09-27 20:34:10fdrakesetmessages: + msg93183
2009-09-27 20:26:31fdrakesetnosy: + fdrake
2009-09-27 06:14:03mkindahlcreate