This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Add PyFloat_(Pack|Unpack)(2|4|8) to the public C API
Type: Stage: resolved
Components: C API Versions: Python 3.11
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: lemburg, mark.dickinson, methane, rhettinger, stutzbach, vstinner
Priority: normal Keywords: patch

Created on 2022-03-03 03:09 by methane, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 31649 closed methane, 2022-03-03 03:55
PR 31657 merged vstinner, 2022-03-03 10:46
PR 31832 closed vstinner, 2022-03-12 00:08
PR 31866 merged vstinner, 2022-03-14 09:43
Messages (12)
msg414401 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2022-03-03 03:09
Original issue. https://github.com/msgpack/msgpack-python/issues/497

_PyFloat_(Pack|Unpack)(4|8) is very nice API for serializers like msgpack.
Converting double and float into char[] is not trivial and these APIs do it in very efficient way.

And these APIs don't reveal CPython internal strucutre. It just convert double and float into char[].

So please keep these APIs public for libraries like msgpack.
msg414410 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-03 08:19
I disagree. A function should be either fully public: tested, documented. Or fully internal.

In the past, many functions were in both, in the gray area.

If these functions are useful, please make them *public* and document them. I'm +1 to make these functions public.
msg414411 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2022-03-03 08:25
OK. By quick grepping, I found only msgpack and bitstruct use these API.
It is not enough number to make them public.
msg414416 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-03 10:10
I reopen the issue, I would like to make these functions public :-) I searched for the 6 functions moved to the internal C API and I found 6 projets using it in the top 5000 PyPI projects (at 2022-01-26):

* msgpack
* ddtrace
* bitstruct
* pickle5
* line_profiler
* zodbpickle

The 6 functions are used:

* _PyFloat_Pack2()
* _PyFloat_Pack4()
* _PyFloat_Pack8()
* _PyFloat_Unpack2()
* _PyFloat_Unpack4()
* _PyFloat_Unpack8()
msg414419 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-03 10:48
The pack/unpack functions have been moved to the internal C API by this change:

commit 0a883a76cda8205023c52211968bcf87bd47fd6e
Author: Victor Stinner <vstinner@python.org>
Date:   Thu Oct 14 23:41:06 2021 +0200

    bpo-35134: Add Include/cpython/floatobject.h (GH-28957)
    
    Split Include/floatobject.h into sub-files: add
    Include/cpython/floatobject.h and
    Include/internal/pycore_floatobject.h.
msg414427 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-03 11:24
I prepared a pythoncapi_compat PR to provide these functions to Python 3.10 and older:
https://github.com/pythoncapi/pythoncapi_compat/pull/26

Note: bpo-11734 (commit 7c4e409d075fdb923807513353b18a75a4520eba) added _PyFloat_Pack2() and _PyFloat_Unpack2() to Python 3.6.0b1.
msg414944 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-11 23:10
New changeset 882d8096c262a5945e0cfdd706e5db3ad2b73543 by Victor Stinner in branch 'main':
bpo-46906: Add PyFloat_Pack8() to the C API (GH-31657)
https://github.com/python/cpython/commit/882d8096c262a5945e0cfdd706e5db3ad2b73543
msg414952 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-11 23:58
PR for msgpack: https://github.com/msgpack/msgpack-python/pull/499
msg414953 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-12 00:12
PR for bitstruct: https://github.com/eerimoq/bitstruct/pull/26
msg414954 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-12 00:21
zodbpickle issue: https://github.com/zopefoundation/zodbpickle/issues/67
msg415151 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-14 15:51
New changeset 11c25b87aeed162d422bc18530fe9699f311e586 by Victor Stinner in branch 'main':
bpo-46906: Mention native endian in PyFloat_Pack8() doc (GH-31866)
https://github.com/python/cpython/commit/11c25b87aeed162d422bc18530fe9699f311e586
msg415152 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2022-03-14 15:58
msgpack and bitstruct use the newly added functions: my two PRs got merged. msgpack was my main motivation to add these functions :-)

Thanks to great reviews, the functions got a new better documentation!

I close the issue. Thanks again for reviews!
History
Date User Action Args
2022-04-11 14:59:56adminsetgithub: 91062
2022-03-14 15:58:36vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg415152

stage: patch review -> resolved
2022-03-14 15:51:57vstinnersetmessages: + msg415151
2022-03-14 09:43:38vstinnersetpull_requests: + pull_request29964
2022-03-12 00:21:16vstinnersetmessages: + msg414954
2022-03-12 00:12:53vstinnersetmessages: + msg414953
2022-03-12 00:08:42vstinnersetpull_requests: + pull_request29929
2022-03-11 23:58:18vstinnersetmessages: + msg414952
2022-03-11 23:10:10vstinnersetmessages: + msg414944
2022-03-03 11:24:11vstinnersetnosy: + lemburg, rhettinger, mark.dickinson, stutzbach
2022-03-03 11:24:01vstinnersetmessages: + msg414427
2022-03-03 10:48:28vstinnersetmessages: + msg414419
2022-03-03 10:46:57vstinnersetstage: resolved -> patch review
pull_requests: + pull_request29775
2022-03-03 10:10:38vstinnersetstatus: closed -> open
resolution: rejected -> (no value)
messages: + msg414416

title: Make _PyFloat_(Pack|Unpack)(4|8) cpython API, not internal. -> Add PyFloat_(Pack|Unpack)(2|4|8) to the public C API
2022-03-03 08:26:02methanesetstatus: open -> closed
resolution: rejected
stage: patch review -> resolved
2022-03-03 08:25:54methanesetmessages: + msg414411
2022-03-03 08:19:26vstinnersetmessages: + msg414410
2022-03-03 03:55:39methanesetkeywords: + patch
stage: patch review
pull_requests: + pull_request29769
2022-03-03 03:09:32methanecreate