classification
Title: PyUnicode_FromFormat integer format handling different from printf about zeropad
Type: behavior Stage: needs patch
Components: Documentation, Interpreter Core Versions: Python 3.7, Python 3.6, Python 3.5, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: docs@python, eric.smith, haypo, louielu, serhiy.storchaka, terry.reedy, xiang.zhang, ztane
Priority: normal Keywords: easy

Created on 2016-10-11 08:56 by xiang.zhang, last changed 2017-04-05 11:00 by terry.reedy.

Pull Requests
URL Status Linked Edit
PR 885 open louielu, 2017-03-29 10:19
Messages (6)
msg278467 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2016-10-11 08:56
Although declared *exactly equivalent* to printf in the doc, PyUnicode_FromFormat could generate different result from printf with the same format.

For example:

from ctypes import pythonapi, py_object, c_int
f = getattr(pythonapi, 'PyUnicode_FromFormat')
f.restype = py_object
f(b'%010.5d', c_int(100))
'0000000100'

while printf outputs:

printf("%010.5d\n", 100);
     00100

I use both gcc and clang to compile and get the same result. gcc gives me a warning:

warning: '0' flag ignored with precision and ā€˜%dā€™ gnu_printf format

I am not sure this should be fixed. It seems the change could break backwards compatibility.
msg278528 - (view) Author: Antti Haapala (ztane) * Date: 2016-10-12 12:36
To be more precise, C90, C99, C11 all say that ~"For d, i, o, u, x and X conversions, if a precision is specified, the 0 flag will be ignored."
msg278666 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-10-14 21:21
I presume that PyUnicode_FromFormat is responsible for the first of the following:
>>> '%010.5d' % 100
'0000000100'
>>> b'%010.5d' % 100
b'0000000100'

I am strongly of the opinion that the behavior should be left alone and the C-API doc changed by either 1) replacing 'exactly' with 'nearly' or 2) adding the following: "except that a 0 conversion flag is not ignored when a precision is given for d, i, o, u, x and X conversion types" (and other exceptions as discovered).

I took the terms 'conversion flag' and 'conversion type' from
https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting
https://docs.python.org/3/library/stdtypes.html#printf-style-bytes-formatting

I consider the Python behavior to be superior.  The '0' conversion flag, the '.' precision indicator, and the int conversion types are literal characters.  If one does not want the '0' conversion, one should omit it and not write it to be ignored.
>>> '%10.5d' % 100
'     00100'

And I consider the abolition of int 'precision', inr {} formatting even better.  
>>> '{:010.5d}'.format(100)
Traceback (most recent call last):
  File "<pyshell#2>", line 1, in <module>
    '{:010.5d}'.format(100)
ValueError: Precision not allowed in integer format specifier

It has always been a source of confusion, and there is hardly any real-world use case for a partial 0 fill.
msg290776 - (view) Author: Louie Lu (louielu) * Date: 2017-03-29 10:20
Add a note block under Py*_FromFormat in unicode.rst and bytes.rst.

Could Xiang or Terry help to review? Thanks.
msg290881 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-03-30 20:45
*Way* too wordy. In msg278666, I suggested minimal and terse alternatives.
a. /exactly/nearly/
b. add "except that a 0 conversion flag is not ignored when a precision is given for d, i, o, u, x and X conversion types"
msg291175 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2017-04-05 11:00
(Response to what I believe is latest patch.)  In msg278666, my two suggestions were 'either...or', not both.  The list came from Antti's msg278528, but the correct list for Python appears to be different, and different for bytes and unicode.  When I made the suggestion, I did not realize that 'exactly' was repeated for each conversion type in a table.  As a note, I think the following might work. "For <list of> conversion types, the 0-conversion flag has effect even when a precision is given."

I also think that 'exactly could be dropped when it is not exactly true.
History
Date User Action Args
2017-04-05 11:00:54terry.reedysetmessages: + msg291175
2017-03-30 20:45:00terry.reedysetmessages: + msg290881
2017-03-29 10:54:32xiang.zhangsetversions: + Python 2.7
2017-03-29 10:20:21louielusetnosy: + louielu
messages: + msg290776
2017-03-29 10:19:00louielusetpull_requests: + pull_request787
2017-03-17 03:26:42xiang.zhangsetkeywords: + easy
stage: needs patch
2016-10-19 02:57:10josh.rsettitle: PyUnicode_FromFromat interger format handling different from printf about zeropad -> PyUnicode_FromFormat integer format handling different from printf about zeropad
2016-10-14 21:21:57terry.reedysetnosy: + eric.smith, terry.reedy, docs@python
messages: + msg278666

assignee: docs@python
components: + Documentation
2016-10-12 12:36:08ztanesetnosy: + ztane
messages: + msg278528
2016-10-11 08:56:20xiang.zhangcreate