Title: PyUnicode_FromFormat integer format handling different from printf about zeropad
Type: behavior Stage: resolved
Components: Documentation, Interpreter Core Versions: Python 3.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, eric.smith, louielu, serhiy.storchaka, terry.reedy, vstinner, xiang.zhang, ztane
Priority: normal Keywords: easy

Created on 2016-10-11 08:56 by xiang.zhang, last changed 2017-04-27 03:38 by xiang.zhang. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 885 merged louielu, 2017-03-29 10:19
Messages (7)
msg278467 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-10-11 08:56
Although declared *exactly equivalent* to printf in the doc, PyUnicode_FromFormat could generate different result from printf with the same format.

For example:

from ctypes import pythonapi, py_object, c_int
f = getattr(pythonapi, 'PyUnicode_FromFormat')
f.restype = py_object
f(b'%010.5d', c_int(100))

while printf outputs:

printf("%010.5d\n", 100);

I use both gcc and clang to compile and get the same result. gcc gives me a warning:

warning: '0' flag ignored with precision and ‘%d’ gnu_printf format

I am not sure this should be fixed. It seems the change could break backwards compatibility.
msg278528 - (view) Author: Antti Haapala (ztane) * Date: 2016-10-12 12:36
To be more precise, C90, C99, C11 all say that ~"For d, i, o, u, x and X conversions, if a precision is specified, the 0 flag will be ignored."
msg278666 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-10-14 21:21
I presume that PyUnicode_FromFormat is responsible for the first of the following:
>>> '%010.5d' % 100
>>> b'%010.5d' % 100

I am strongly of the opinion that the behavior should be left alone and the C-API doc changed by either 1) replacing 'exactly' with 'nearly' or 2) adding the following: "except that a 0 conversion flag is not ignored when a precision is given for d, i, o, u, x and X conversion types" (and other exceptions as discovered).

I took the terms 'conversion flag' and 'conversion type' from

I consider the Python behavior to be superior.  The '0' conversion flag, the '.' precision indicator, and the int conversion types are literal characters.  If one does not want the '0' conversion, one should omit it and not write it to be ignored.
>>> '%10.5d' % 100
'     00100'

And I consider the abolition of int 'precision', inr {} formatting even better.  
>>> '{:010.5d}'.format(100)
Traceback (most recent call last):
  File "<pyshell#2>", line 1, in <module>
ValueError: Precision not allowed in integer format specifier

It has always been a source of confusion, and there is hardly any real-world use case for a partial 0 fill.
msg290776 - (view) Author: Louie Lu (louielu) * Date: 2017-03-29 10:20
Add a note block under Py*_FromFormat in unicode.rst and bytes.rst.

Could Xiang or Terry help to review? Thanks.
msg290881 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-03-30 20:45
*Way* too wordy. In msg278666, I suggested minimal and terse alternatives.
a. /exactly/nearly/
b. add "except that a 0 conversion flag is not ignored when a precision is given for d, i, o, u, x and X conversion types"
msg291175 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-04-05 11:00
(Response to what I believe is latest patch.)  In msg278666, my two suggestions were 'either...or', not both.  The list came from Antti's msg278528, but the correct list for Python appears to be different, and different for bytes and unicode.  When I made the suggestion, I did not realize that 'exactly' was repeated for each conversion type in a table.  As a note, I think the following might work. "For <list of> conversion types, the 0-conversion flag has effect even when a precision is given."

I also think that 'exactly could be dropped when it is not exactly true.
msg292392 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2017-04-27 03:36
New changeset 88c38b32b761cb566759b8ad96704bff590a1de9 by Xiang Zhang (Louie Lu) in branch 'master':
bpo-28415: Note 0 conversion different between Python and C (#885)
Date User Action Args
2017-04-27 03:38:33xiang.zhangsetstatus: open -> closed
stage: needs patch -> resolved
resolution: fixed
versions: - Python 2.7, Python 3.5, Python 3.6
2017-04-27 03:36:38xiang.zhangsetmessages: + msg292392
2017-04-05 11:00:54terry.reedysetmessages: + msg291175
2017-03-30 20:45:00terry.reedysetmessages: + msg290881
2017-03-29 10:54:32xiang.zhangsetversions: + Python 2.7
2017-03-29 10:20:21louielusetnosy: + louielu
messages: + msg290776
2017-03-29 10:19:00louielusetpull_requests: + pull_request787
2017-03-17 03:26:42xiang.zhangsetkeywords: + easy
stage: needs patch
2016-10-19 02:57:10josh.rsettitle: PyUnicode_FromFromat interger format handling different from printf about zeropad -> PyUnicode_FromFormat integer format handling different from printf about zeropad
2016-10-14 21:21:57terry.reedysetnosy: + eric.smith, terry.reedy, docs@python
messages: + msg278666

assignee: docs@python
components: + Documentation
2016-10-12 12:36:08ztanesetnosy: + ztane
messages: + msg278528
2016-10-11 08:56:20xiang.zhangcreate