Index: Doc/library/configparser.rst =================================================================== --- Doc/library/configparser.rst (revision 83201) +++ Doc/library/configparser.rst (working copy) @@ -15,105 +15,161 @@ single: ini file single: Windows ini file -This module defines the class :class:`ConfigParser`. The :class:`ConfigParser` -class implements a basic configuration file parser language which provides a -structure similar to what you would find on Microsoft Windows INI files. You -can use this to write Python programs which can be customized by end users -easily. +This module defines the classes :class:`RawConfigParser` and +:class:`SafeConfigParser`. The :class:`RawConfigParser` class implements +a basic configuration file parser language which provides a structure similar to +what you would find on Microsoft Windows INI files. You can use this to write +Python programs which can be customized by end users easily. .. note:: This library does *not* interpret or write the value-type prefixes used in the Windows Registry extended version of INI syntax. -The configuration file consists of sections, led by a ``[section]`` header and -followed by ``name: value`` entries, with continuations in the style of -:rfc:`822` (see section 3.1.1, "LONG HEADER FIELDS"); ``name=value`` is also -accepted. Note that leading whitespace is removed from values. The optional -values can contain format strings which refer to other values in the same -section, or values in a special ``DEFAULT`` section. Additional defaults can be -provided on initialization and retrieval. Lines beginning with ``'#'`` or -``';'`` are ignored and may be used to provide comments. +The configuration file consists of sections, each led by a ``[section]`` header, +followed by key/value entries indicated by ``name`` and ``value`` delimited with +a specific substring (``=`` or ``:`` by default). Note that leading whitespace +is removed from values. Values can be ommitted, in which case the key/value +delimiter may also be left out. Values can also span multiple lines, as long as +they are indented deeper than the first line of the value. Depending on the +parser's mode, blank lines may be treated as parts of multiline values or +ignored. +Configuration files may include comments, prefixed by specific characters (``#`` +and ``;`` by default). Comments may appear on their own in an otherwise empty +line or may be entered in lines holding values or spection names. + +On top of the core functionality, :class:`SafeConfigParser` supports +interpolation. This means values can contain format strings which refer to other +values in the same section, or values in a special ``DEFAULT`` section. +Additional defaults can be provided on initialization and retrieval. + For example:: - [My Section] - foodir: %(dir)s/whatever - dir=frob - long: this value continues - in the next line + [Paths] + home_dir: /Users + my_dir: %(home_dir)s/lumberjack + my_pictures: %(my_dir)s/Pictures -would resolve the ``%(dir)s`` to the value of ``dir`` (``frob`` in this case). -All reference expansions are done on demand. + [Multiline Values] + chorus: I'm a lumberjack, and I'm okay + I sleep all night and I work all day + + [No Values] + key_without_value + empty string value here = + + [You can use comments] # after a useful line + ; in an empty line + after: a_value # here's another comment + inside: a ;comment + multiline ;comment + value! ;comment -Default values can be specified by passing them into the :class:`ConfigParser` -constructor as a dictionary. Additional defaults may be passed into the -:meth:`get` method which will override all others. + [Sections Can Be Indented] + can_values_be_as_well = True + does_that_mean_anything_special = False + purpose = formatting for readability + multiline_values = are + handled just fine as + long as they are indented + deeper than the first line + of a value + ; Did I mention we can indent comments, too? + -Sections are normally stored in a built-in dictionary. An alternative dictionary -type can be passed to the :class:`ConfigParser` constructor. For example, if a -dictionary type is passed that sorts its keys, the sections will be sorted on -write-back, as will be the keys within each section. +In the example above, :class:`SafeConfigParser` would resolve ``%(home_dir)s`` +to the value of ``home_dir`` (``/Users`` in this case). ``%(my_dir)s`` in effect +would resolve to ``/Users/lumberjack``. All interpolations are done on demand so +keys used in the chain of references do not have to be specified in any specific +order in the configuration file. +:class:`RawConfigParser` would simply return ``%(my_dir)s/Pictures`` as the +value of ``my_pictures`` and ``%(home_dir)s/lumberjack`` as the value of +``my_dir``. Other features presented in the example are handled in the same +manner by both parsers. -.. class:: RawConfigParser(defaults=None, dict_type=collections.OrderedDict, - allow_no_value=False) +Default values can be specified by passing them into the +:class:`SafeConfigParser` :meth:`__init__` method as a dictionary. Additional +defaults may be passed into the :meth:`get` method which will override all +others. +Sections are normally stored in an :class:`collections.OrderedDict` which +maintains the order of all keys. An alternative dictionary type can be passed to +the :meth:`__init__` method. For example, if a dictionary type is passed that +sorts its keys, the sections will be sorted on write-back, as will be the keys +within each section. + + +.. class:: RawConfigParser(defaults=None, dict_type=collections.OrderedDict, delimiters=('=', ':'), comment_prefixes=('#', ';'), empty_lines_in_values=True, allow_no_value=False) + The basic configuration object. When *defaults* is given, it is initialized - into the dictionary of intrinsic defaults. When *dict_type* is given, it will - be used to create the dictionary objects for the list of sections, for the - options within a section, and for the default values. When *allow_no_value* - is true (default: ``False``), options without values are accepted; the value - presented for these is ``None``. + into the dictionary of intrinsic defaults. When *dict_type* is given, it + will be used to create the dictionary objects for the list of sections, for + the options within a section, and for the default values. When *delimiters* + is given, it will be used as the set of substrings that divide keys from + values. When *comment_prefixes* is given, it will be used as the set of + substrings that prefix comments in a line. When *empty_lines_in_values* is + ``False`` (default: ``True``), each empty line marks the end of an option. + Otherwise, internal empty lines of a multiline option are kept as part of the + value. When *allow_no_value* is true (default: ``False``), options without + values are accepted; the value presented for these is ``None``. - This class does not - support the magical interpolation behavior. + This class does not support the magical interpolation behavior. .. versionchanged:: 3.1 The default *dict_type* is :class:`collections.OrderedDict`. .. versionchanged:: 3.2 - *allow_no_value* was added. + *delimiters*, *comment_prefixes*, *empty_lines_in_values* and + *allow_no_value* were added. -.. class:: ConfigParser(defaults=None, dict_type=collections.OrderedDict, - allow_no_value=False) +.. class:: SafeConfigParser(defaults=None, dict_type=collections.OrderedDict, delimiters=('=', ':'), comment_prefixes=('#', ';'), empty_lines_in_values=True, allow_no_value=False) - Derived class of :class:`RawConfigParser` that implements the magical - interpolation feature and adds optional arguments to the :meth:`get` and - :meth:`items` methods. The values in *defaults* must be appropriate for the - ``%()s`` string interpolation. Note that *__name__* is an intrinsic default; - its value is the section name, and will override any value provided in - *defaults*. + Derived class of :class:`ConfigParser` that implements a sane variant of + the magical interpolation feature. This implementation is more predictable as + it validates the interpolation syntax used within a configuration file. This + class also enables escaping the interpolation character (e.g. a key can have + ``%`` as part of the value by specifying ``%%`` in the file). - All option names used in interpolation will be passed through the - :meth:`optionxform` method just like any other option name reference. For - example, using the default implementation of :meth:`optionxform` (which converts - option names to lower case), the values ``foo %(bar)s`` and ``foo %(BAR)s`` are - equivalent. + Applications that don't require interpolation should use + :class:`RawConfigParser`, otherwise :class:`SafeConfigParser` is the best + option. .. versionchanged:: 3.1 The default *dict_type* is :class:`collections.OrderedDict`. .. versionchanged:: 3.2 - *allow_no_value* was added. + *delimiters*, *comment_prefixes*, *empty_lines_in_values* and + *allow_no_value* were added. -.. class:: SafeConfigParser(defaults=None, dict_type=collections.OrderedDict, - allow_no_value=False) +.. class:: ConfigParser(defaults=None, dict_type=collections.OrderedDict, delimiters=('=', ':'), comment_prefixes=('#', ';'), empty_lines_in_values=True, allow_no_value=False) - Derived class of :class:`ConfigParser` that implements a more-sane variant of - the magical interpolation feature. This implementation is more predictable as - well. New applications should prefer this version if they don't need to be - compatible with older versions of Python. + Derived class of :class:`RawConfigParser` that implements the magical + interpolation feature and adds optional arguments to the :meth:`get` and + :meth:`items` methods. + + :class:`SafeConfigParser` is generally recommended over this class if you + need interpolation. + + The values in *defaults* must be appropriate for the ``%()s`` string + interpolation. Note that *__name__* is an intrinsic default; its value is + the section name, and will override any value provided in *defaults*. - .. XXX Need to explain what's safer/more predictable about it. + All option names used in interpolation will be passed through the + :meth:`optionxform` method just like any other option name reference. For + example, using the default implementation of :meth:`optionxform` (which converts + option names to lower case), the values ``foo %(bar)s`` and ``foo %(BAR)s`` are + equivalent. .. versionchanged:: 3.1 The default *dict_type* is :class:`collections.OrderedDict`. .. versionchanged:: 3.2 - *allow_no_value* was added. + *delimiters*, *comment_prefixes*, *empty_lines_in_values* and + *allow_no_value* were added. .. exception:: NoSectionError @@ -295,11 +351,13 @@ interpolation and output to files) can only be achieved using string values. -.. method:: RawConfigParser.write(fileobject) +.. method:: RawConfigParser.write(fileobject, space_around_delimiters=True) Write a representation of the configuration to the specified file object, which must be opened in text mode (accepting strings). This representation - can be parsed by a future :meth:`read` call. + can be parsed by a future :meth:`read` call. If ``space_around_delimiters`` + is ``True`` (the default), delimiters between keys and values are surrounded + by spaces. .. method:: RawConfigParser.remove_option(section, option) @@ -342,21 +400,24 @@ -------------------- The :class:`ConfigParser` class extends some methods of the -:class:`RawConfigParser` interface, adding some optional arguments. +:class:`RawConfigParser` interface, adding some optional arguments. Whenever you +can, consider using :class:`SafeConfigParser` which adds validation and escaping +for the interpolation. .. method:: ConfigParser.get(section, option, raw=False, vars=None) - Get an *option* value for the named *section*. All the ``'%'`` interpolations - are expanded in the return values, based on the defaults passed into the - constructor, as well as the options *vars* provided, unless the *raw* argument - is true. + Get an *option* value for the named *section*. All the ``'%'`` + interpolations are expanded in the return values, based on the defaults + passed into the :meth:`__init__` method, as well as the options *vars* + provided, unless the *raw* argument is true. .. method:: ConfigParser.items(section, raw=False, vars=None) - Return a list of ``(name, value)`` pairs for each option in the given *section*. - Optional arguments have the same meaning as for the :meth:`get` method. + Return a list of ``(name, value)`` pairs for each option in the given + *section*. Optional arguments have the same meaning as for the :meth:`get` + method. .. _safeconfigparser-objects: @@ -466,8 +527,8 @@ Some configuration files are known to include settings without values, but which otherwise conform to the syntax supported by :mod:`configparser`. The -*allow_no_value* parameter to the constructor can be used to indicate that such -values should be accepted: +*allow_no_value* parameter to the :meth:`__init__` method can be used to +indicate that such values should be accepted: .. doctest:: @@ -476,12 +537,12 @@ >>> sample_config = """ ... [mysqld] - ... user = mysql - ... pid-file = /var/run/mysqld/mysqld.pid - ... skip-external-locking - ... old_passwords = 1 - ... skip-bdb - ... skip-innodb + ... user = mysql + ... pid-file = /var/run/mysqld/mysqld.pid + ... skip-external-locking + ... old_passwords = 1 + ... skip-bdb + ... skip-innodb # we don't need ACID today ... """ >>> config = configparser.RawConfigParser(allow_no_value=True) >>> config.readfp(io.BytesIO(sample_config)) Index: Lib/configparser.py =================================================================== --- Lib/configparser.py (revision 83201) +++ Lib/configparser.py (working copy) @@ -1,6 +1,6 @@ """Configuration file parser. -A setup file consists of sections, lead by a "[section]" header, +A configuration file consists of sections, lead by a "[section]" header, and followed by "name: value" entries, with continuations and such in the style of RFC 822. @@ -24,12 +24,31 @@ methods: - __init__(defaults=None) - create the parser and specify a dictionary of intrinsic defaults. The - keys must be strings, the values must be appropriate for %()s string - interpolation. Note that `__name__' is always an intrinsic default; - its value is the section's name. + __init__(defaults=None, dict_type=_default_dict, + delimiters=('=', ':'), comment_prefixes=('#', ';'), + empty_lines_in_values=True, allow_no_value=False): + create the parser. When `defaults' is given, it is initialized into the + dictionary or intrinsic defaults. The keys must be strings, the values + must be appropriate for %()s string interpolation. Note that `__name__' + is always an intrinsic default; its value is the section's name. + + When `dict_type' is given, it will be used to create the dictionary + objects for the list of sections, for the options within a section, and + for the default values. + + When `delimiters' is given, it will be used as the set of substrings + that divide keys from values. + + When `comment_prefixes' is given, it will be used as the set of + substrings that prefix comments in a line. + When `empty_lines_in_values' is False (default: True), each empty line + marks the end of an option. Otherwise, internal empty lines of + a multiline option are kept as part of the value. + + When `allow_no_value' is True (default: False), options without + values are accepted; the value presented for these is None. + sections() return all the configuration section names, sans DEFAULT @@ -83,8 +102,10 @@ set(section, option, value) set the given option - write(fp) - write the configuration state in .ini format + write(fp, space_around_delimiters=True) + write the configuration state in .ini format. If + `space_around_delimiters' is True (the default), delimiters + between keys and values are surrounded by spaces. """ try: @@ -94,6 +115,7 @@ _default_dict = dict import re +import sys __all__ = ["NoSectionError", "DuplicateSectionError", "NoOptionError", "InterpolationError", "InterpolationDepthError", @@ -115,11 +137,13 @@ def _get_message(self): """Getter for 'message'; needed only to override deprecation in BaseException.""" + return self.__message def _set_message(self, value): """Setter for 'message'; needed only to override deprecation in BaseException.""" + self.__message = value # BaseException.message has been deprecated since Python 2.6. To prevent @@ -136,6 +160,7 @@ __str__ = __repr__ + class NoSectionError(Error): """Raised when no section matches a requested option.""" @@ -144,6 +169,7 @@ self.section = section self.args = (section, ) + class DuplicateSectionError(Error): """Raised when a section is multiply-created.""" @@ -152,6 +178,7 @@ self.section = section self.args = (section, ) + class NoOptionError(Error): """A requested option was not found.""" @@ -162,6 +189,7 @@ self.section = section self.args = (option, section) + class InterpolationError(Error): """Base class for interpolation-related exceptions.""" @@ -171,6 +199,7 @@ self.section = section self.args = (option, section, msg) + class InterpolationMissingOptionError(InterpolationError): """A string substitution required a setting which was not available.""" @@ -185,10 +214,12 @@ self.reference = reference self.args = (option, section, rawval, reference) + class InterpolationSyntaxError(InterpolationError): """Raised when the source text into which substitutions are made does not conform to the required syntax.""" + class InterpolationDepthError(InterpolationError): """Raised when substitutions are nested too deeply.""" @@ -201,6 +232,7 @@ InterpolationError.__init__(self, option, section, msg) self.args = (option, section, rawval) + class ParsingError(Error): """Raised when a configuration file does not follow legal syntax.""" @@ -214,6 +246,7 @@ self.errors.append((lineno, line)) self.message += '\n\t[line %2d]: %s' % (lineno, line) + class MissingSectionHeaderError(ParsingError): """Raised when a key-value pair is found before any section header.""" @@ -227,19 +260,30 @@ self.line = line self.args = (filename, lineno, line) + class RawConfigParser: def __init__(self, defaults=None, dict_type=_default_dict, - allow_no_value=False): + delimiters=('=', ':'), comment_prefixes=('#', ';'), + empty_lines_in_values=True, allow_no_value=False): self._dict = dict_type self._sections = self._dict() self._defaults = self._dict() - if allow_no_value: - self._optcre = self.OPTCRE_NV - else: - self._optcre = self.OPTCRE if defaults: for key, value in defaults.items(): self._defaults[self.optionxform(key)] = value + self._delimiters = tuple(delimiters) + if delimiters == ('=', ':'): + self._optcre = self.OPTCRE_NV if allow_no_value else self.OPTCRE + else: + delim = "|".join((re.escape(d) for d in delimiters)) + if allow_no_value: + self._optcre = re.compile(self._OPT_NV_TMPL.format(delim=delim), + re.VERBOSE) + else: + self._optcre = re.compile(self._OPT_TMPL.format(delim=delim), + re.VERBOSE) + self._comment_prefixes = tuple(comment_prefixes) + self._empty_lines_in_values = empty_lines_in_values def defaults(self): return self._defaults @@ -256,6 +300,7 @@ already exists. Raise ValueError if name is DEFAULT or any of it's case-insensitive variants. """ + if section.lower() == "default": raise ValueError('Invalid section name: %s' % section) @@ -293,6 +338,7 @@ Return list of successfully read files. """ + if isinstance(filenames, str): filenames = [filenames] read_ok = [] @@ -313,8 +359,8 @@ second argument is the `filename', which if not given, is taken from fp.name. If fp has no `name' attribute, `' is used. + """ - """ if filename is None: try: filename = fp.name @@ -374,6 +420,7 @@ def has_option(self, section, option): """Check for the existence of a given option in a given section.""" + if not section or section == DEFAULTSECT: option = self.optionxform(option) return option in self._defaults @@ -386,6 +433,7 @@ def set(self, section, option, value=None): """Set an option.""" + if not section or section == DEFAULTSECT: sectdict = self._defaults else: @@ -395,25 +443,24 @@ raise NoSectionError(section) sectdict[self.optionxform(option)] = value - def write(self, fp): - """Write an .ini-format representation of the configuration state.""" + def write(self, fp, space_around_delimiters=True): + """Write an .ini-format representation of the configuration state. + If `space_around_delimiters' is True (the default), delimiters + between keys and values are surrounded by spaces.""" + + if space_around_delimiters: + d = " {} ".format(self._delimiters[0]) + else: + d = self._delimiters[0] if self._defaults: - fp.write("[%s]\n" % DEFAULTSECT) - for (key, value) in self._defaults.items(): - fp.write("%s = %s\n" % (key, str(value).replace('\n', '\n\t'))) - fp.write("\n") + self._write_section(fp, DEFAULTSECT, self._defaults.items(), d) for section in self._sections: - fp.write("[%s]\n" % section) - for (key, value) in self._sections[section].items(): - if key == "__name__": - continue - if value is not None: - key = " = ".join((key, str(value).replace('\n', '\n\t'))) - fp.write("%s\n" % (key)) - fp.write("\n") + self._write_section(fp, section, + self._sections[section].items(), d) def remove_option(self, section, option): """Remove an option.""" + if not section or section == DEFAULTSECT: sectdict = self._defaults else: @@ -429,6 +476,7 @@ def remove_section(self, section): """Remove a file section.""" + existed = section in self._sections if existed: del self._sections[section] @@ -437,63 +485,97 @@ # # Regular expressions for parsing section headers and options. # - SECTCRE = re.compile( - r'\[' # [ - r'(?P
[^]]+)' # very permissive! - r'\]' # ] - ) - OPTCRE = re.compile( - r'(?P