This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: The Python implementation of Decimal does not support the "N" format
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eric.smith, facundobatista, mark.dickinson, rhettinger, serhiy.storchaka
Priority: normal Keywords:

Created on 2021-11-06 13:51 by serhiy.storchaka, last changed 2022-04-11 14:59 by admin.

Messages (10)
msg405861 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-11-06 13:51
The C implementation supports both formats "n" and "N". The Python implementation only supports format "n".

>>> from decimal import Decimal
>>> format(Decimal('1e100'), 'n')
'1e+100'
>>> format(Decimal('1e100'), 'N')
'1E+100'
>>> from _pydecimal import Decimal
>>> format(Decimal('1e100'), 'n')
'1e+100'
>>> format(Decimal('1e100'), 'N')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/serhiy/py/cpython/Lib/_pydecimal.py", line 3766, in __format__
    spec = _parse_format_specifier(specifier, _localeconv=_localeconv)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/serhiy/py/cpython/Lib/_pydecimal.py", line 6194, in _parse_format_specifier
    raise ValueError("Invalid format specifier: " + format_spec)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Invalid format specifier: N
msg405898 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-11-07 10:46
Interesting. I think the behaviour of the Python implementation behaviour is actually more correct here: neither `int` nor `float` supports 'N', and I'm not seeing any indication in tests or documentation that 'N' should be supported. So is this a bug in libmpdec, or a missing feature in the Python implementation? (Either way, it's definitely a bug that the two aren't aligned.)

>>> format(123, 'n')
'123'
>>> format(123, 'N')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Unknown format code 'N' for object of type 'int'
msg405901 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2021-11-07 12:38
I don’t think “N” should be supported. It was never the intention to have it work.
msg407195 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-11-28 09:39
Eric, Serhiy: do you have opinions on the right way forward?

Here are 6 options, on a spectrum of increasing level of acceptance of "N".

-2. Remove "N" support for cdecimal right now (i.e., for Python 3.11), on the basis that there's no need for deprecation warnings, because it was never officially a feature.
-1. Deprecate "N" support for cdecimal, remove it in Python 3.13.
0. Do nothing (the default), leaving _pydecimal and cdecimal inconsistent.
1. Add "N" support to the Python implementation for parity with cdecimal, but don't document it - leave it as an undocumented feature.
2. Officially add "N" support to decimal formatting - add documentation, tests, and fix the Python implementation.
3. Officially add "N" support to all numeric formatting ...
msg407196 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2021-11-28 09:40
I could be persuaded for any of options -1, 1 and 2. I don't much like option 0.
msg407210 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2021-11-28 15:02
I support deprecating "N".
msg407212 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2021-11-28 16:51
I'd support -1.
msg407229 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-11-28 20:30
I have no strong opinion. I found the discrepancy when reviewed one of Mark's PRs. I was surprised because I did not know that "N" is supported.

On other hand, there are reasons for supporting upper-case variant of "n" if we support upper-case variants of "e", "g" and "x" (formats which produce output containing letters).

There is an alternative solution of the problem solved by upper-case variations of formats: add a special converter for converting a string to upper case, so for example f'{x:12.8E}' could be written as f'{x:12.8e!u}'.

I prefer -1 or 2. If we choose deprecation I prefer adding an upper-case converter.
msg407234 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2021-11-28 21:35
While I'd prefer -1 overall, I would also prefer 3 over 2. If we were adding the feature from scratch, we wouldn't have decimal differ from int, float, and complex.

And I'm not in favor of an uppercase converter, no matter what we do here. The other converters work like:

f'{obj!s}' -> format(str(obj))
f'{obj!r}' -> format(repr(obj))

but the proposed !u would be

f'{obj!u}' -> format(obj).upper()

That is, it operates on the result of __format__, not its input.
msg407254 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-11-29 09:28
You are right about an uppercase converter. I forget this. I withdraw this proposition.

It seems that even if we add support of "N" to all numeric formatting it will not cover all use cases. "n" is a locale specific variant of "g", but other formats could have locale specific variants too. In C all numeric formatting is locale specific. Maybe add a flag which makes any numeric formatting locale specific? Then "n" will became obsolete.
History
Date User Action Args
2022-04-11 14:59:52adminsetgithub: 89902
2021-11-29 09:28:44serhiy.storchakasetmessages: + msg407254
2021-11-28 21:35:42eric.smithsetmessages: + msg407234
2021-11-28 20:30:52serhiy.storchakasetmessages: + msg407229
2021-11-28 16:51:59eric.smithsetmessages: + msg407212
2021-11-28 15:02:42rhettingersetmessages: + msg407210
2021-11-28 09:40:49mark.dickinsonsetmessages: + msg407196
2021-11-28 09:39:33mark.dickinsonsetmessages: + msg407195
2021-11-07 12:38:47eric.smithsetmessages: + msg405901
2021-11-07 10:46:57mark.dickinsonsetnosy: + eric.smith
messages: + msg405898
2021-11-06 13:51:18serhiy.storchakacreate