There seems to be no way to add a config item with a value containing a
formatstring without this formatstring beeing handled by configparser.

A possible way to escape the formatstrings could be to double the
%-sign, e.g.:

[foo]
bar = %%(string)s

The value of the bar item should then be "%(string)s".

The formatstring is a Python % formatting code, and doubling the %% is
the way to escape one, so what you suggest is in fact the way it works.
Do you have a test case demonstrating that it doesn't work as
documented (it works for me)?

Afacs it is not documented to work. Here is the testcase:

#!/usr/bin/python
# vim: fileencoding=utf8

import ConfigParser
import tempfile

file = tempfile.TemporaryFile(mode='rw+b')
file.write("[s]\nf=%%(b)s\n")
file.seek(0)
config = ConfigParser.ConfigParser()

config.readfp(file)

print config.get("s", "f")

file.close()

Ah, that's because you aren't using SafeConfigParser (which the docs
recommend that you do unless you have to use ConfigParser for backward
compatibility reasons). Your test case works with SafeConfigParser.

You are correct that it is not explicitly documented in the ConfigParser
docs. It is implied by the fact that the magic substitution is
documented to be a standard format string, and by the recommendation to
use SafeConfigParser because ConfigParser's handling of formatstrings is
broken.

To make this all explicit, I've attached a doc patch to make the
motivation for using SafeConfigParser clearer, using this as the
example. Let me know what you think.

Georg, Raymond, I added you as nosy to ask for a style/content opinion.
Should we instead rewrite the section to put SafeConfigParser first,
and perhaps even deprecate ConfigParser?

It would be nice if you could expand the patch to also use only one name
for the support of format strings.

In the beginning it is introduced as "reference expansion", but later in
the document, several variatons of interpolation are used: "magical
interpolation", "string interpolation", "%" interpolation is used.
Therefore I guess it should also be introduced as interpolation.

Another nice improvement would also be to move the ConfigParser to a
"deprecated classes" section at the end like it is done for some methods
in the string module.

This issue superseeds bpo-8888 because of the patches attached.

Brett, there are two ways we can solve this issue:

1. Rewrite the documentation so it clearly puts more emphasis on the fact that SafeConfigParser is the one to choose.
2. We deprecate ConfigParser in the code and in the documentation.

I personally would go with 1. and maybe just "hint" about the fact that ConfigParser is not for end users. Formal deprecation seems too big for this since the regular ConfigParser still does have its valid use cases.

Please decide on what we're going to do and I'll edit the docs and add the tests here to py3k.

I disagree.

The documentation should promote RawConfigParser, and note
SafeConfigParser and ConfigParser as remaining for backward
compatibility for existing software. Maintainers of legacy software
using ConfigParser should be encouraged to convert to SafeConfigParser
(or even RawConfigParser) if possible.

Documentation changes should be sufficient; deprecation warnings
typically generate more pain than good.

The documentation should promote RawConfigParser, and note
SafeConfigParser and ConfigParser as remaining for backward
compatibility for existing software.

That is another way to go around this. Anyway, ConfigParser is the least predictable of all three for end users and the documentation should be adjusted to emphasize this.

Maintainers of legacy software
using ConfigParser should be encouraged to convert to SafeConfigParser
(or even RawConfigParser) if possible.

That's another comment that should appear in the documentation.

Documentation changes should be sufficient; deprecation warnings
typically generate more pain than good.

Isn't is what I was saying above?

We're in agreement; I was specifically adding weight to *not*
selecting the second optoin Łukasz Langa presented:

2. We deprecate ConfigParser in the code and in the documentation.

This should just be applied. I'll do it shortly unless there is an objection.

It is (very) unfortunate that configparser.ConfigParser should *not* be used and that configparser.SafeConfigParser is the correct class instead.

I would be *in favour* of deprecating ConfigParser and eventually renaming SafeConfigParser back to ConfigParser (leaving SafeConfigParser as an alias).

Now that deprecation warnings are silent by default this should be less of an issue.

You are right, IMO, at least the current doc patch should be applied. Please go ahead, if you want to, I won't have time to get to it for a couple of days.

Maybe you could come up with a new title for this issue that reflects the concerns being addressed here...

Note that the patch doesn't apply cleanly. Łukasz Langa is going to update it.

There is still discussion as to whether we should *also* deprecate ConfigParser (well - PendingDeprecation in 3.2, Deprecation in 3.3 and in 3.4 making ConfigParser an alias for SafeConfigParser).

The 3.2 docs now don't mention ConfigParser prominently anymore (as part of a different patch that added some features). Could be done in other branches as well.

Yes, so the patch part is already solved. The thing that is still open to discussion is whether we should do something like this:

1. Pending-Deprecate naked the ConfigParser class in 3.2.
2. Deprecate it in 3.3.
3. Remove it in 3.4 and rename SafeConfigParser to ConfigParser.

This is controversial but many developers (myself included) don't see the point in running naked ConfigParser because it's basically SafeConfigParser sans its completeness and predictability.

So, please +1 or -1 the deprecation process idea.

+1 from me

+1 for deprecation. Nobody *should* be using ConfigParser anyway, and of those who are 99% either wouldn't notice or would have bugs in their code *fixed* by the rename, so I can't see much of a downside.

If ConfigParser is not documented first, the name “SafeConfigParser” becomes strange—safe compared to what? These names have an historical motivation and could become clearer if renamed, but I don’t know if python-dev will agree with this deprecation. Renaming a class to an existing name with different behavior can be bad.

FTR, in my head RawConfigParser is the config parser, and SafeConfigParser is another thing that I’ll maybe use one day.

If ConfigParser is not documented first, the name “SafeConfigParser” becomes strange—safe compared to what?

The first sentence is "Derived class of ConfigParser that implements a sane variant of the magical interpolation feature." I think it's enough for an explanation.

If this were an encyclopedia, you would be right. But this is more like a Google search results page. Most people will take the first thing that looks like a solution they need.

These names have an historical motivation and could become clearer if renamed

That is the point.

but I don’t know if python-dev will agree with this deprecation.

That would be a shame, essentially it should happen in 3.0 IMO. But it's never too late I think.

Think of the children! One day you will read this comment and think: whoa, this was even BEFORE 3.2! Yeah, ancient history.

Renaming a class to an existing name with different behavior can be bad.

Yes but this is going to be a problem for 3.4. Maybe then we'll come up with something more natural.

FTR, in my head RawConfigParser is the config parser, and SafeConfigParser is another thing that I’ll maybe use one day.

YMMV. FTR, many people I've spoken to treated RawConfigParser as something more low-level and not suitable for consumer use just because of the name AND the presence of a default (=name like the module) ConfigParser class.

Agree with Michael, +1.

The first sentence is "Derived class of ConfigParser that implements
a sane variant of the magical interpolation feature." I think it's
enough for an explanation.

True.

> but I don’t know if python-dev will agree with this deprecation.

I wrote that before seeing Michael’s reply. Since Georg is +1 too, I can only say: Great, let’s do it!

FTR, many people I've spoken to treated RawConfigParser as
something more low-level and not suitable for consumer use just
because of the name AND the presence of a default (=name like the
module) ConfigParser class.

Right. I don’t know if people using ConfigParser really want to use the interpolation. If we want to give the same name as the module to the best class for people who don’t read docs, I’d prefer renaming RawConfigParser to ConfigParser and SafeConfigParser to SomethingConfigParser.

Eric, while I agree that would be nice as well, renaming each and every parser in the module will be more problematic for sure.

*** TO ALL: WHAT DO YOU SAY TO A PATH LIKE THIS ***

1. In 3.2 we add an alias:

InterpolatingConfigParser = SafeConfigParser

1.1) In 3.2 we Pending-Deprecate:

ConfigParser (message about it being removed in 3.4)
SafeConfigParser (message about it being renamed to InterpolatingConfigParser in 3.4, a name you can already use)

2. In 3.3 we Deprecate:

ConfigParser
SafeConfigParser

Same messages.

3. In 3.4 we remove ConfigParser and rename SafeConfigParser to InterpolatingConfigParser (removing the alias). So then we have two to choose from: Raw and Interpolating. As you see there's no default ConfigParser any more.

I like that because it opens a new possibility which I would wait with until 3.5: re-introduce ConfigParser but as a completely new subclass that has better but backwards incompatible defaults. For now most of the new functionality I've added is turned off by default because of backwards compatibility reasons and this is unfortunate.

2010/8/3 Łukasz Langa <report@bugs.python.org>:

1. In 3.2 we add an alias:

InterpolatingConfigParser = SafeConfigParser

I'd rather see the class renamed and SafeConfigParser made the alias in 3.2.

Otherwise, +1 for this plan (msg 112589), as there's no silent breakage.

I'd be happy with aliasing SafeConfigParser to ConfigParser in 3.2. Can we just do this without a deprecation process?

Making ConfigParser an alias for SafeConfigParser creates a silent
behavioral change. An application developer may not realize that
users rely on the "full" ConfigParser anti-glory and end up breaking
their configurations without so much as providing a heads-up in the
application news.

-1

So long as the application developer has to change their code to move
away from ConfigParser, there's a better chance of affected end users
receiving a heads-up.

Unfortunately, I have to agree with Fred here. We'll stick to renaming and the deprecation process.

Sorry - I misunderstood your earlier suggestion Fred.

configparser.ConfigParser is the *natural* name for SafeConfigParser. I'm strongly +1 on moving towards that. (I doubt there would *actually* be any real code breakage if we did it earlier though ;-)

By the way, given that deprecation warnings are silent I am strongly -1 on removing the ConfigParser name altogether. That would cause far more breakage. As ConfigParser should not be used at all, and SafeConfigParser provides its functionality minus bugs, SafeConfigParser (horrible name) should replace it.

Agree on the proposal of Łukasz, with the caveat mentioned by Fred (rename the class and make the old name an alias, for pickle and all). I’ll let Michael and Fred decide if the name ConfigParser has to go or not, I’m happy enough that the class will be deprecated and removed.

Getting *rid* of the name ConfigParser would be annoying and cause *gratuitous* code breakage.

If we are going to keep the name but get rid of the "unsafe" version then we can only replace it with what is now SafeConfigParser - as it is almost entirely compatible with it. (Modulo the bug fixing "behavioural change".)

So the real choices are:

• leave ConfigParser as it is
• deprecate it but not remove it (so as not to needlessly break code)
• deprecate now and replace with SafeConfigParser later

Only the last of these is a positive step forwards... :-)

(Strongly -1 on introducing *yet another name* to refer to these classes by.)

There IS one more option that seems to be better than all of the above:

1. Add an interpolation=True argument to RawConfigParser __init__ and move the interpolating functionality from SafeConfigParser to it.
2. Rename RawConfigParser to ConfigParser leaving the old name in PendingDeprecation with interpolation=False.
3. Make an alias for SafeConfigParser pointing to ConfigParser with PendingDeprecation.

We can do all this for 3.2 I guess without inflicting any actual damage. It will fix more bugs in running code that break config files.

Maybe we shouldn't be so afraid after all and just clean it up.

It doesn't make sense to make any of these changes to Python 2; this
really should have been separate from the documentation issue. That's
probably understood by everyone, but explicit is better.

Merging implementations
-----------------------

• I've no objection to merging RawConfigParser and SafeConfigParser,
  using a constructor argument to control whether interpolation is
  performed. It's not clear this provides any improvement in
  maintainability or usage.

• Isolating the old (broken interpolation) ConfigParser behavior so it's
  not in the middle of the inheritance stack would be good.

Changing ConfigParser behavior
------------------------------

Changing the behavior of the ConfigParser name requires the deprecation
process. We may think nobody in their right mind is using that, but
changing something out from under app developers is really bad. This
should be considered a problem for configparser as outlined in msg 112598.

Whether we can assign new semantics to the ConfigParser name in the
future is questionable. I think Python's compatibility policy allows
it, but that doesn't make it a good idea.

(This seems to be the real sticking point, unfortunately.)

Class names
-----------

I don't understand Michael's objection to adding new names for the
configparser classes; that's one of the few points where we don't run
into backward-compatibility black holes.

In the case of a merged implementation, a new name for the merged class
(possibly "Configuration") may be the best bet; the old names can then
be subclasses of that which generate appropriate deprecation warnings.

If we merge the functionality in a single class with a new name then I guess that is fine as it will simplify the documentation rather than complexify it (good word hey). We still need to *mention* the old names so that people finding them in old code can find an up to date reference on them.

Here's what I don't understand about Fred's difficulty with replacing ConfigParser with the sane implementation.

After we deprecate ConfigParser as it is now we have two choices.

• delete the ConfigParser name - breaking *all* code that uses it and has not been updated
• point the name at what is currently called SafeConfigParser - causing a slight risk of incompatibility but likely *improving* most code that hasn't been updated

I don't see how the first option could *in any way* be preferable to the second.

Hopefully both RawConfigParser and ConfigParser will be successfully deprecated as part of bpo-10499. The discussion goes on there.