classification
Title: %-formatting and dicts
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.8
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: ajaksu2, anthonybaxter, benjamin.peterson, eric.araujo, eric.smith, flox, georg.brandl, hdiwan650, jafo, jimjjewett, jwilk, lemburg, loewis, nvetoshkin, rhettinger
Priority: low Keywords: patch

Created on 2006-04-10 19:39 by lemburg, last changed 2018-07-30 06:06 by eric.smith. This issue is now closed.

Files
File name Uploaded Description Edit
format.patch akuchling, 2006-07-25 12:35 Draft of approach (fixes string object only)
format-v2.patch jafo, 2007-08-25 20:15
issue1467929_py3k.diff nvetoshkin, 2010-10-20 19:57 review
Messages (27)
msg28186 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2006-04-10 19:39
This looks like a bug in the way the %-formatting code
works or is it a feature ?

>>> '%s %(a)s' % {'a': 'xyz'}
"{'a': 'xyz'} xyz"

>>> u'%s %(a)s' % {'a': 'xyz'}
u"{'a': 'xyz'} xyz"

Note that both strings and Unicode are affected.

Python 2.3 and 2.4 also show this behavior.

I would have expected an exception or the %-formatter
simply ignoring the first %s.


msg28187 - (view) Author: Hasan Diwan (hdiwan650) Date: 2006-04-14 08:33
Logged In: YES 
user_id=1185570

It looks as though it's a feature... The first %s will print
the whole dictionary as a string, the second, only that
value looked up by the key.
msg28188 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2006-07-24 13:37
Logged In: YES 
user_id=849994

The library ref specifies that if a dict is supplied, the
format specifiers MUST include a mapping key, so the right
thing to do would be to raise an exception.

Is it worth breaking backwards compatibility, Martin?
msg28189 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-07-24 20:07
Logged In: YES 
user_id=21627

IMO, it's correct to break backwards compatibility, as the
current behaviour clearly violates the spec; I'm not sure
whether it's good to break the behaviour *now* (i.e. with no
further betas before the release of 2.5).

Deferring to the release manager.
msg28190 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2006-07-25 12:35
Logged In: YES 
user_id=11375

So, what should '%s' % {} do?  Should it be 1) '{}' or 2) an
error because the argument is a mapping but the format
specifier doesn't have a '(key)'?

I've attached a draft patch that fixes stringobject.c; if
the approach is deemed OK, I'll apply it to unicodeobject.c,
too.  PyString_Format() records the type of argument being
processed (a tuple or a mapping) and raises ValueError if
you mix them, at the cost of two extra comparisons for each
format specifier processed.  This preserves the current
behaviour of '%s' % dictionary.

Questions: 1) is the approach reasonably clear?  2) are the
additional two comparisons unacceptably slow?  3) Is
ValueError the right exception?  4) can someone come up with
a better exception message than "both keyed and unkeyed
format specifiers used"?

msg28191 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2006-08-04 13:21
Logged In: YES 
user_id=44345

Looks okay to me, though why is the FORMAT_TYPE_UNKNOWN test
necessary in the second case but not the first?
msg28192 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2006-08-04 13:26
Logged In: YES 
user_id=38388

The patch looks OK.

I'd make it a TypeError and use "cannot use positional and
named formatting parameters at the same time" as message.

Thanks.
msg28193 - (view) Author: Jim Jewett (jimjjewett) Date: 2006-08-18 22:01
Logged In: YES 
user_id=764593

Just a bit of encouragement for checking consistency like 
this; the explicit error message would have helped with a 
mistake I made earlier today.  For one of several keys, I 
mistyped it as "(%key)s", and a message about "not enough 
values" just didn't make sense.
msg28194 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2006-08-18 22:17
Logged In: YES 
user_id=38388

Should this patch be applied to the 2.5 branch ?

And if so, before or after the release of 2.5 ?
msg28195 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2006-08-19 05:25
Logged In: YES 
user_id=849994

I'd say before 2.5 final...
msg28196 - (view) Author: Anthony Baxter (anthonybaxter) (Python triager) Date: 2007-01-12 09:10
I'm happy for this to be applied for 2.5.1. I don't have time to do it myself for a few days, though, so feel free to beat me to it.
msg28197 - (view) Author: A.M. Kuchling (akuchling) * (Python committer) Date: 2007-01-12 14:57
The patch shouldn't be applied as it stands, though, because it's not complete; similiar changes need to be made to the Unicode type, for a start.

To answer Skip's question: I don't remember the logic of the format code.  I think the FORMAT_TYPE_UNKNOWN check may be unnecessary; the code could just always do format_type = _TUPLE, occasionally doing a redundant assignment (but who cares)?

I don't think I'll have any chance to work on this; PyCon is keeping me busy, and the mailbox bugs will take priority for me.
msg55292 - (view) Author: Sean Reifschneider (jafo) * (Python committer) Date: 2007-08-25 20:15
I'm attaching a new version of this which includes AMK's string patch
ported over to unicode.

Any objections to my committing this to the trunk?  Should it also go in
for the next 2.5 release?
msg66798 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2008-05-13 19:27
This probably won't be important anymore now that we have str.format()...
msg66801 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2008-05-13 20:06
Sean, why don't you just check in the patch ?

Then we can close the bug.

Georg, the fact that we have an alternative method for string formatting
doesn't mean that it's ok for Python to hide error using the prevailing
method of string formatting.
msg66802 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2008-05-13 20:42
I didn't want to imply that, but seeing that nobody cared about it for
so long I hadn't much hope for the future... ;)
msg66813 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2008-05-14 09:39
I guess the patch was just forgotten after the 2.5 release was out.

I've added a 2.6 tag and assigned the patch to Sean.
msg86636 - (view) Author: Daniel Diniz (ajaksu2) (Python triager) Date: 2009-04-27 01:19
Confirmed in trunk and py3k. How about changing this for 3.1?
msg86671 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2009-04-27 15:15
The patch incorrectly raises an exception for:
'%(a)s %%' % {'a':'xyz'}

I'll look into fixing it.
msg115960 - (view) Author: Florent Xicluna (flox) * (Python committer) Date: 2010-09-09 19:35
Too late for 2.x and 3.1.
msg119061 - (view) Author: Vetoshkin Nikita (nvetoshkin) Date: 2010-10-18 18:46
Updated patch against py3k. Fixing last reported issue with
'%(a)s %%' % {'a':'xyz'}
msg119067 - (view) Author: Vetoshkin Nikita (nvetoshkin) Date: 2010-10-18 19:14
Updated patch to capture another patological case:
'%(a)s %' % {'a':'xyz'} - incomplete formatter at the end of the line
msg119232 - (view) Author: Vetoshkin Nikita (nvetoshkin) Date: 2010-10-20 19:57
Updated patch: some tests added.
msg322586 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2018-07-28 18:49
Eric, would you like an update PR for this?
msg322590 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2018-07-28 20:35
I unassigned myself from this a few months ago. If someone wants to fix it, I'm not opposed. But the issue is 12 years old and doesn't cause any real problems as far as I can see, so it's hard to get excited about it. And as we've seen there are a lot of corner cases. I'd hate to make things worse.
msg322613 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2018-07-29 06:52
Ah, sorry Eric, I misread that change as you assigning this to yourself.
msg322650 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2018-07-30 04:16
[Eric V. Smith]
>  But the issue is 12 years old and doesn't cause any real problems 
> as far as I can see, so it's hard to get excited about it. 
> And as we've seen there are a lot of corner cases. I'd hate to 
> make things worse.

I ooncur with Eric and think this ought to be closed. It is a oddity but has proven itself to be harmless in the wild.  At this point in the life cycle of %-formatting it would be prudent to value stability over niggling behavior changes.
History
Date User Action Args
2018-07-30 06:06:05eric.smithsetstatus: open -> closed
resolution: wont fix
stage: patch review -> resolved
2018-07-30 04:16:37rhettingersetnosy: + rhettinger
messages: + msg322650
2018-07-29 06:52:53taleinatsetnosy: - taleinat
2018-07-29 06:52:31taleinatsetnosy: lemburg, loewis, anthonybaxter, georg.brandl, jafo, jimjjewett, taleinat, hdiwan650, eric.smith, ajaksu2, benjamin.peterson, jwilk, eric.araujo, flox, nvetoshkin
messages: + msg322613
versions: + Python 3.8, - Python 3.2, Python 3.3
2018-07-28 20:35:15eric.smithsetmessages: + msg322590
2018-07-28 18:49:54taleinatsetnosy: + taleinat
messages: + msg322586
2018-03-20 01:11:32eric.smithsetassignee: eric.smith ->
2014-12-31 16:19:19akuchlingsetnosy: - akuchling
2014-10-10 11:08:04jwilksetnosy: + jwilk
2011-10-30 19:51:05floxsetstage: needs patch -> patch review
versions: + Python 3.3
2010-10-20 19:57:54nvetoshkinsetfiles: + issue1467929_py3k.diff

messages: + msg119232
2010-10-20 19:57:25nvetoshkinsetfiles: - issue1467929_py3k.diff
2010-10-18 19:14:36nvetoshkinsetfiles: + issue1467929_py3k.diff

messages: + msg119067
2010-10-18 19:13:38nvetoshkinsetfiles: - issue1467929_py3k.diff
2010-10-18 18:46:21nvetoshkinsetfiles: + issue1467929_py3k.diff
nosy: + nvetoshkin
messages: + msg119061

2010-09-11 21:41:35eric.araujosetnosy: + eric.araujo
2010-09-09 19:35:47floxsetnosy: + flox

messages: + msg115960
versions: - Python 3.1, Python 2.7
2010-09-09 15:29:46r.david.murraylinkissue9805 superseder
2010-08-22 09:13:50BreamoreBoysetstage: needs patch
versions: + Python 3.2
2010-05-20 20:29:35skip.montanarosetnosy: - skip.montanaro
2009-04-27 15:15:19eric.smithsetmessages: + msg86671
2009-04-27 12:23:53eric.smithsetassignee: jafo -> eric.smith
versions: + Python 2.7, - Python 2.6
2009-04-27 01:19:26ajaksu2setversions: + Python 3.1, - Python 2.5
nosy: + ajaksu2, eric.smith, benjamin.peterson

messages: + msg86636

keywords: + patch
type: behavior
2008-05-14 09:39:36lemburgsetassignee: anthonybaxter -> jafo
messages: + msg66813
versions: + Python 2.6
2008-05-13 20:43:04georg.brandlsetmessages: + msg66802
2008-05-13 20:06:40lemburgsetmessages: + msg66801
2008-05-13 19:27:41georg.brandlsetpriority: normal -> low
messages: + msg66798
2007-08-30 16:55:28gvanrossumsetpriority: release blocker -> normal
2007-08-25 20:15:12jafosetfiles: + format-v2.patch
nosy: + jafo
messages: + msg55292
2006-04-10 19:39:08lemburgcreate