classification
Title: str.format_map()
Type: enhancement Stage: committed/rejected
Components: Interpreter Core Versions: Python 3.2
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: eric.smith Nosy List: ebehar, eric.araujo, eric.smith, ezio.melotti, flox, gruszczy, jason.coombs, r.david.murray, rhettinger, terry.reedy
Priority: normal Keywords: easy, needs review, patch

Created on 2009-05-21 23:23 by rhettinger, last changed 2011-01-28 19:20 by jason.coombs. This issue is now closed.

Files
File name Uploaded Description Edit
6081_1.patch gruszczy, 2010-03-18 16:17 review
issue6081.diff eric.smith, 2010-03-19 13:16 review
6081_2.diff gruszczy, 2010-03-23 21:27 review
issue6081-1.diff eric.smith, 2010-11-04 00:13
Messages (22)
msg88173 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-05-21 23:23
The old % formatting allowed arbitrary mappings:

  >>> class Default(dict):
  ...     def __missing__(self, key):
  ...         return key
  ...
  >>> s = '%(name)s was born in %(country)s'
  >>> s % Default(name='Guido')
  'Guido was born in country'

But the new str.format() demands mappings be first converted to a
regular dict using **kwargs, so something like the above is not possible.

  >>> s = '{name} was born in {country}'
  >>> s.format(**Default(name='Guido'))
  Traceback (most recent call last):
    File "<pyshell#27>", line 1, in <module>
      s.format(**Default(name='Guido'))
  KeyError: 'country'

There is a work-around using string.vformat() but it is obscure and awkward:

  >>> import string
  >>> string.Formatter().vformat(s, (), Default(name='Guido'))
  'Guido was born in country'

Instead, it would be better to offer an alternate method:

  >>> s.format_from_mapping(Default(name='Guido'))
  'Guido was born in country'
msg88425 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2009-05-27 18:02
I think this would be useful.

I don't fee terribly strongly about it, but I think I'd like the name
str.format_using_mapping(). When I initially saw this, I thought from
the name it was creating a format object (whatever that would be) from a
mapping object.
msg101265 - (view) Author: Filip Gruszczyński (gruszczy) Date: 2010-03-18 16:17
I have created a small patch, that adds method that formats using a dict. It's the first time I have written anything in python implementation, so I would very appreciate any advice. Change allows the following:

>>> m = Mapping(a='b')
>>> '{a} {c}'.format_using_mapping(m)
'b c'
>>>
msg101269 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2010-03-18 18:19
Thanks for the patch. It would be nice if you could include unit tests too.
msg101272 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-03-18 19:11
I don't think this patch satisfies Raymond's request.  It is explicitly checking for a __missing__ attribute, but Raymond was talking about a more general facility whereby you can pass in an arbitrary object that implements the mapping interface.  Using the __missing__ facility was just an example of why this would be useful.
msg101273 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2010-03-18 19:27
I agree with David.

Although it's not clear to my why the code doesn't just work with the addition of do_string_format_using_mapping and without the other code. It's possible the existing code is too dict-specific and should be calling a more generic PyObject interface, like PyMapping_GetItemString instead of PyDict_GetItem.
msg101283 - (view) Author: Filip Gruszczyński (gruszczy) Date: 2010-03-18 21:25
My first intention was simply to push mapping from args to kwargs, just like Eric suggested, but that didn't help with __missing__, only with accepting a dict instead of pushing keyword arguments.

I didn't like explicitly asking for __missing__ either, but since I have little knowledge of what should be called, I didn't know what to use. I too believe something else the PyDict_GetItem should be called, something that would take care of __missing__ and other possibilities (I don't know what exactly and really would like to know what these are) internally.

I am going to check, whether PyMapping_GetItemString is going to help. But can this really be called on a dict (or a subclass of dict)? What about retrieving getitem method from the given object and simply calling it? Wouldn't that do the trick?
msg101324 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2010-03-19 13:10
I believe this patch fixes the issue. Tests and documentation are still needed, of course.
msg101325 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2010-03-19 13:16
Added a comment to explain the change.
msg101342 - (view) Author: Filip Gruszczyński (gruszczy) Date: 2010-03-19 21:39
Could you point me, where to add tests and documentation? I would happily add those.
msg101346 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2010-03-19 21:48
http://docs.python.org/library/stdtypes.html#str.format, for starters. This is in Doc/library/stdtypes.rst.

For tests, probably in Lib/test/test_unicode.py.

I'm not sure if we should add this to 2.7 (or even 3.2, for that matter), but if so, we should start by patching trunk and then porting to py3k.
msg101349 - (view) Author: Filip Gruszczyński (gruszczy) Date: 2010-03-19 22:33
Ok, unfortunately this code won't work for certain tests. Take those:

        self.assertEqual("My name is {0}".format('Fred'), "My name is Fred")

We pass only one argument, which is a dict and this won't satisfy such test. We need to think about a different way of passing those arguments there. We can do one of two thins:
* the last argument of args tuple would be an object that can be subscripted for format values
* keyword with above object

I believe the second version is more explicit and therefore better.
msg101603 - (view) Author: Filip Gruszczyński (gruszczy) Date: 2010-03-23 21:27
I have created a new patch, that should be satisfying now. There is help (though it is quite small, I tried to mimic those that were already in unicode.c) and tests. Right now format_using_mapping is called like this:

format_using_mapping(mapping, *args)

where mapping is a subscriptible object, that will be pushed to kwargs and the following args are used just like in normal format. This should be as similar to the normal format as possible.

There are also tests, including usage presented in the ticket.
msg101609 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2010-03-23 23:12
I'm not sure I'm wild about the *args parameter. Calling "Fred" the 0-th parameter here seems non-intuitive:

"My name is {0}".format_using_mapping({}, 'Fred')

If you're going to have *args, why not **kwargs and then merge/update the dicts? I'm being facetious, but I think even having *args is feature creep.

I think it's time to ask about this on python-dev. I'd vote for not using *args. It can always be added in the future if it's seen as a hole in the API.
msg101692 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2010-03-25 13:00
It occurs to me that Raymond's use case could be satisfied using existing Python, by slightly changing the format string. After all, str.format() supports mapping lookup already:

$ ./python.exe
Python 2.6.5+ (release26-maint:79421, Mar 25 2010, 08:51:39)
[GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class Default(dict):
...  def __missing__(self, key):
...   return key
...
>>> s = '{m[name]} was born in {m[country]}'
>>> s.format(m=Default(name='Guido'))
'Guido was born in country'
>>>

Considering that, maybe the best thing is to do nothing. Or maybe update the documentation with this example.

Plus, you could mix and match this with *args as you'd like.
msg113443 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-08-09 18:39
I believe this is covered by the PEP3003 3.2 change moratorium.
msg113610 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2010-08-11 18:15
This can be done for Py3.2.  It completes needed functionality for string formatting which is something we all want to take hold and is necessary for the 3.x series to succeed.
msg113617 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2010-08-11 19:21
I'll work on cleaning this up for 3.2.

Any comments on the name of the method?
msg113622 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-08-11 19:51
I understand now that new methods, as opposed to changed methods, are allowed.

I agree with Eric that this seems more like a convinience rather than absolute necessity, and that the doc should be augmented.

The doc for vformat (which I admit I had not noticed before) says it is exposed just for this case:

"vformat(format_string, args, kwargs) 
This function does the actual work of formatting. It is exposed as a separate function for cases where you want to pass in a predefined dictionary of arguments, rather than unpacking and repacking the dictionary as individual arguments using the *args and **kwds syntax."

'Dictionary' should be replaced with 'mapping'.

string.Formatter.format is documented as "just a wrapper that calls vformat(). Is the same effectively true of str.format also?

If .format_map (I prefer shorted names) is added as a convenience str method, particularly for matching current %-formatting use, I think it should take just one parameter, mapping. I presume it could implemented as a wrapper for .vformat (or whatever internal function .vformat calls).

str.format_map(map) == string.Format.vformat(formstring, (), map)

More complicated, mixed cases can use the explict lookup with map arg.
msg120361 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2010-11-04 00:13
Updated patch which adds tests and minimal docs. Named changed to format_map. I'll commit this before 3.2b1 unless I hear a complaint.
msg120422 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2010-11-04 17:07
Committed to 3.2 in r86170.
msg127325 - (view) Author: Jason R. Coombs (jason.coombs) * (Python committer) Date: 2011-01-28 19:20
Good work Eric.

When I first heard of new string formatting, I was a little wary. The syntax to supply a dictionary of keyword replacements seemed awkward. It took me a while before I realized why it really bothered me. There's string formatting you can do with the old format operator (%) that you can't do with str.format.

Here's an example.

    import random
    class MyDynamicObject:
        def __getitem__(self, name):
            return name + ' ' + str(random.randint(1,10))

    print("%(foo)s" % MyDynamicObject()) # works!
    print("{foo}".format(**MyDynamicObject())) # can't do that because
MyDynamicObject can't enumerate every possible kwparam

As you can see, the % operator naturally accepts any object that responds to __getitem__ but .format requires that all keyword params be enumerated in advance. This limitation seems to me to be a serious problem to favoring .format over %.

I frequently use % to format the properties of an object... and while
it's true one can use myob.__dict__ or vars(myob) to get a dictionary of
some of the values, that doesn't work for properties and other dynamic
behavior.

format_map addresses this shortcoming nicely. Thanks.
History
Date User Action Args
2011-01-28 19:20:05jason.coombssetnosy: + jason.coombs
messages: + msg127325
2010-11-04 17:07:37eric.smithsetstatus: open -> closed

messages: + msg120422
stage: patch review -> committed/rejected
2010-11-04 17:04:22eric.smithsettitle: str.format_from_mapping() -> str.format_map()
2010-11-04 00:13:03eric.smithsetkeywords: + easy, needs review
files: + issue6081-1.diff
resolution: accepted
messages: + msg120361
2010-09-09 19:20:44floxsetnosy: + flox
2010-08-11 19:51:58terry.reedysetmessages: + msg113622
2010-08-11 19:21:48eric.smithsetmessages: + msg113617
2010-08-11 18:15:58rhettingersetmessages: + msg113610
versions: + Python 3.2, - Python 3.3
2010-08-11 15:00:26eric.araujosetnosy: + eric.araujo
2010-08-09 18:39:07terry.reedysetnosy: + terry.reedy

messages: + msg113443
versions: + Python 3.3, - Python 2.7, Python 3.2
2010-03-25 13:00:19eric.smithsetmessages: + msg101692
2010-03-23 23:12:02eric.smithsetmessages: + msg101609
stage: test needed -> patch review
2010-03-23 21:27:29gruszczysetfiles: + 6081_2.diff

messages: + msg101603
2010-03-19 22:33:13gruszczysetmessages: + msg101349
2010-03-19 21:48:58eric.smithsetmessages: + msg101346
2010-03-19 21:39:01gruszczysetmessages: + msg101342
2010-03-19 13:16:49eric.smithsetfiles: - issue6081.diff
2010-03-19 13:16:40eric.smithsetfiles: + issue6081.diff

messages: + msg101325
2010-03-19 13:10:50eric.smithsetfiles: + issue6081.diff

messages: + msg101324
2010-03-18 21:25:54gruszczysetmessages: + msg101283
2010-03-18 19:27:53eric.smithsetmessages: + msg101273
2010-03-18 19:11:19r.david.murraysetnosy: + r.david.murray
messages: + msg101272
2010-03-18 18:19:42ezio.melottisetpriority: normal

messages: + msg101269
stage: test needed
2010-03-18 16:17:02gruszczysetfiles: + 6081_1.patch

nosy: + gruszczy
messages: + msg101265

keywords: + patch
2010-03-15 09:03:54eric.smithsetassignee: eric.smith
2010-03-14 02:53:09ezio.melottisetnosy: + ezio.melotti
2009-06-02 03:12:46ebeharsetnosy: + ebehar
2009-05-27 18:02:53eric.smithsetmessages: + msg88425
components: + Interpreter Core
2009-05-22 17:56:32eric.smithsetnosy: + eric.smith
2009-05-21 23:23:15rhettingercreate