Title: ConfigParser does not implement "comments need to be preceded by a whitespace character" correctly
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.7
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: DanielFortunov, eric.araujo, lukasz.langa
Priority: normal Keywords:

Created on 2011-08-30 17:30 by DanielFortunov, last changed 2011-09-03 16:00 by eric.araujo. This issue is now closed.

Messages (4)
msg143230 - (view) Author: Daniel Fortunov (DanielFortunov) Date: 2011-08-30 17:30
ConfigParser does not implement "comments need to be preceded by a whitespace character" correctly and in most cases will treat a value beginning with a comment character as a comment, even though it is not preceded by a whitespace character.

The ConfigParser documentation states:
Comments may appear on their own in an otherwise empty line, or may be entered in lines holding values or section names. In the latter case, they need to be preceded by a whitespace character to be recognized as a comment.

This suggests that in the following configuration file, the value of 'password' would be read as ';23bUx1'.


In fact, there appears to be a bug in the following code within RawConfigParser._read():

    if vi in ('=', ':') and ';' in optval:
        # ';' is a comment delimiter only if it follows
        # a spacing character
        pos = optval.find(';')
        if pos != -1 and optval[pos-1].isspace():
            optval = optval[:pos]
    optval = optval.strip()

For the example file above, vi==';' and optval==';23bUx1\r'. pos therefore takes the value 0, and the second part of the compound if statement which checks if the preceding character is a space ends up looking at optval[-1] -- the last character in optval -- which is \r, since it has not yet been stripped.

I think this can be resolved by changing the line:
    if pos != -1 and optval[pos-1].isspace():
    if pos > 0 and optval[pos-1].isspace():

Thus, the "if preceded by a space" case is only considered if the comment character appears *after* the first character in the value. (And if it appears as the very *first* character, then by definition it cannot be preceded by a space!)

A rather fragile workaround (which works only if there is only one single value affected by this bug) would be to ensure that the affected value is defined on the last line of your config file, the file does not end with a carriage return. In this case, the optval[-1] would return a non-whitespace character and prevent the value being considered a comment.

The above analysis pertains to Python 2.7.2, and I see that the implementation has been re-written in python 3 so this bug doesn't seem to exist there.
msg143415 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-09-02 16:55
(Adding 3.x versions because we’ll want to add tests to them.)
msg143437 - (view) Author: Łukasz Langa (lukasz.langa) * (Python committer) Date: 2011-09-02 21:25
Daniel, thank you for your report. Indeed, 3.2+ has an updated configparser where we turned off support for inline comments (e.g. comments after option values). This was decided after we discovered how inconsistent it is. Your report shows this decision was the right one.

As for 2.7 unfortunately it's long in maintenance mode now so no functional changes are possible.

I've prepared a backport of the 3.2+ configparser for 2.6+. It is available on PyPI: Have a look, might solve your problem (and quite a bunch of others I suppose).

Éric, the docs clearly state that currently only full-line comments are enabled by default.
msg143463 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-09-03 16:00
> Éric, the docs clearly state that currently only full-line comments
> are enabled by default.

Wow, I missed that.  I use such comments in some config files or config file examples in documentation :(
Date User Action Args
2011-09-03 16:00:55eric.araujosetmessages: + msg143463
2011-09-02 21:25:35lukasz.langasetstatus: open -> closed
versions: - Python 3.2, Python 3.3
messages: + msg143437

resolution: wont fix
stage: resolved
2011-09-02 16:55:25eric.araujosetnosy: + eric.araujo, lukasz.langa

messages: + msg143415
versions: + Python 3.2, Python 3.3
2011-08-30 17:30:53DanielFortunovcreate