This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: undefined parsing behavior with the old style string formatting
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Jerry Dimitrov, benjamin.peterson, larry, ned.deily, r.david.murray, rhettinger, serhiy.storchaka, xiang.zhang
Priority: normal Keywords: patch

Created on 2017-02-15 14:53 by Jerry Dimitrov, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
format-percent.patch serhiy.storchaka, 2017-02-15 22:46 review
Pull Requests
URL Status Linked Edit
PR 513 merged serhiy.storchaka, 2017-03-06 11:54
Messages (9)
msg287857 - (view) Author: Jerry Dimitrov (Jerry Dimitrov) Date: 2017-02-15 14:53
Hello everyone,
This is my first bug report to the python project, so please excuse me if the metadata for this particular issue is not 100% accurate.

Today I noticed (with the help from couple of people in IRC) a strange behavior in the python string formatting functionality.

Consider the following code snippets:

```
'%(a)s %(b)' % {'a': '1', 'b': '2'}

# result:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: incomplete format
```


```
'%(a) %(b)s' % {'a': '1', 'b': '2'}

# result:
'%(b)s'

# expected result:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: incomplete format
```

It seems that there is some kind of inconsistent (undefined) behavior, during the parsing of the type character for the formatted string (tested across all major python 2.x/3.x versions).

According to the documentation for string formatting and the relevant PEPs, there is no additional info about this particular case.

I want to say thank you to Yhg1s, JustASlacker, Jerub and lz1irq for discovering this 'bug/feature' and the additional information about it.

Please let me know if this is a bug, since I am not 100% sure if this is the case.
Thanks in advance for your time!

Best Regards,
Jerry
msg287859 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-02-15 15:19
In this example characters between percents are parsed as for any conversion specifier but ignored and '%(a) %' is formatted to '%'. '(a)' specifies a mapping key, ' ' is a conversion flag, the second '%' is a conversion type.

>>> '%(a) d' % {'a': 123}
' 123'
>>> '%(a) %' % {'a': 123}
'%'

This behavior is explicable but looks weird and errorprone. I'm not sure about changing behavior in maintained branches (technically this may be not a bug), but I think it is worth to make this an error in the developed branch.
msg287862 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-02-15 16:13
I don't think we should change the behavior in maintenance branches.  I take it what you want to do is make it so that if the parser ends up thinking it is seeing '%%' but there is stuff between the two %s, that's an error?
msg287864 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-02-15 17:05
Yes, it is what I want. Current behavior is just an artefact of the implementation. I have doubts that any other implementation of printf-like formatting has such behavior.

Bytes formatting starves from the same issue.

>>> b'% %' % {}
b'%'

But there are differences:

>>> '%12%' % ()
'%'
>>> b'%12%' % ()
b'           %'
msg287871 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2017-02-15 17:34
The documentation[1] explicitly states using % to do string format could be error-prone and recommends using str.format(). So +1 on no change in maintenance branches.

[1] https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting
msg287872 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2017-02-15 17:37
Or even not fix it in develop branch.
msg287890 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-02-15 22:46
Here is a patch.
msg287916 - (view) Author: Jerry Dimitrov (Jerry Dimitrov) Date: 2017-02-16 05:48
Thanks for the documentation reference. Can we at least add this link reference [1] as a note or something like that into those documentation sections: [2] https://docs.python.org/2/library/stdtypes.html#string-formatting and  [3] https://docs.python.org/3/library/string.html#string-formatting

I couldn't fine the printf specific documentation at first, so this is why I created the report.

Regards,
Jerry
msg290276 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2017-03-24 22:43
New changeset 9f8ad3f39e0a92ed37d012b9dd237399524f0d51 by Xiang Zhang (Serhiy Storchaka) in branch 'master':
bpo-29568: Disable any characters between two percents for escaped percent "%%"  in the format string for classic string formatting. (GH-513)
https://github.com/python/cpython/commit/9f8ad3f39e0a92ed37d012b9dd237399524f0d51
History
Date User Action Args
2022-04-11 14:58:43adminsetgithub: 73754
2017-03-24 22:43:13xiang.zhangsetmessages: + msg290276
2017-03-17 21:06:07larrysetpull_requests: - pull_request588
2017-03-17 21:00:33larrysetpull_requests: + pull_request588
2017-03-08 03:51:57xiang.zhangsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2017-03-06 11:54:59serhiy.storchakasetpull_requests: + pull_request422
2017-02-16 05:48:36Jerry Dimitrovsetmessages: + msg287916
2017-02-15 22:46:17serhiy.storchakasetfiles: + format-percent.patch
versions: - Python 2.7, Python 3.5, Python 3.6
messages: + msg287890

keywords: + patch
stage: patch review
2017-02-15 17:37:47xiang.zhangsetmessages: + msg287872
2017-02-15 17:34:21xiang.zhangsetnosy: + xiang.zhang
messages: + msg287871
2017-02-15 17:05:56serhiy.storchakasetmessages: + msg287864
2017-02-15 16:13:54r.david.murraysetnosy: + r.david.murray
messages: + msg287862
2017-02-15 15:19:37serhiy.storchakasetnosy: + larry, serhiy.storchaka, rhettinger, benjamin.peterson, ned.deily

messages: + msg287859
versions: - Python 3.3, Python 3.4
2017-02-15 14:53:02Jerry Dimitrovcreate