classification
Title: Add a minimal decimal capsule API
Type: enhancement Stage: resolved
Components: C API Versions: Python 3.10
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: mark.dickinson, mattip, piro, pitrou, rhettinger, skrah
Priority: normal Keywords: patch

Created on 2020-07-17 10:55 by skrah, last changed 2021-05-10 13:51 by piro. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 21519 merged skrah, 2020-07-17 10:56
Messages (12)
msg373823 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-07-17 11:18
This adds a minimal decimal capsule API.  As can be seen from the patch,
adding anything to decimal while doing it properly is quite labor and
testing intensive, so the intent is to *keep* the API minimal!

That said, some functions are really necessary:

  1) PyDec_TypeCheck()         -- for obvious reasons.

  2) PyDec_Alloc()             -- create new decimals.

  3) PyDec_Get()               -- get the mpd_t, enables the use of all libmpdec functions.

  4) PyDec_AsUint128Triple()   -- export the decimal as (sign, hi, lo, exp).

  5) PyDec_FromUint128Triple() -- create a decimal from (sign, hi, lo, exp).


4) and 5) have been requested by Antoine for a real world use case.  (hi, lo)
is the coefficient as a __uint128_t split in two uint64_t.


Antoine, could you verify that this is sufficient for the database use case?
msg373829 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-07-17 11:57
Also as a note for Mark and Raymond: This API is for exact conversions
and avoids the use of the context except for raising ConversionSyntax.

I'll add documentation once Antoine has tried it out for his use case.
msg373831 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-07-17 12:43
Adding Daniele Varrazzo, in case this is useful for the PostgreSQL
adapter.
msg373842 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2020-07-17 17:58
I probably won't try it out explicitly (it's basically cumbersome for us to test with non-release Pythons, because of our dependencies to Numpy and Cython), but reviewing the API should be enough anyway.
msg373910 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-07-18 17:28
It looks like the API would be usable, so the PR now has documentation.
msg374012 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-07-20 15:02
I'm happy with the API, except that --with-system-libmpdec is naturally
broken.  So I've to release a new libmpdec, which I'd rather do soon
because I don't want to revisit this later.

Antoine has looked at the API.  If anyone else has requests or objections,
now would be a good time to raise concerns.  These two functions are
the centerpiece of the PR:

   mpd_uint128_triple_t PyDec_AsUint128Triple(const PyObject *dec)
   PyObject *PyDec_FromUint128Triple(const mpd_uint128_triple_t *triple)
msg375121 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2020-08-10 14:32
New changeset 39042e00ab01d6521548c1b7cc6554c09f4389ff by Stefan Krah in branch 'master':
bpo-41324 Add a minimal decimal capsule API (#21519)
https://github.com/python/cpython/commit/39042e00ab01d6521548c1b7cc6554c09f4389ff
msg388134 - (view) Author: mattip (mattip) * Date: 2021-03-04 22:28
Was expanding the C-API discussed on the mailing list or in any wider forum? It seems vstinner is working hard to reduce the public API exposure while this issue + PR makes it even larger. Please reconsider putting this in for 3.10.
msg388233 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2021-03-07 11:26
@mattip, see issue43422
msg389244 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2021-03-21 16:14
For the record, this is going to be reverted at Stefan's request in issue43422.
msg393395 - (view) Author: Daniele Varrazzo (piro) * Date: 2021-05-10 13:46
FYI, will try to use this API in psycopg3, which supports the PostgreSQL decimal binary format. Thank you very much: it seems exactly what needed.

Binary conversion with Python Decimal currently has disappointing performances and is slower than the text format conversion. This is likely caused by the use of Decimal.as_tuple(), which creates tuples for the return value and the digits and requires Python calls to access their values.

I have considered using libmpdec directly: personally I wouldn't be opposed to that, but the additional dependency on libmpdec-dev would earn the project no new fan :\

Ref. https://github.com/psycopg/psycopg3/issues/54
msg393396 - (view) Author: Daniele Varrazzo (piro) * Date: 2021-05-10 13:51
Ah, just noticed that this has been now reverted. Oh well, I'll see what to do then :(
History
Date User Action Args
2021-05-10 13:51:05pirosetmessages: + msg393396
2021-05-10 13:46:54pirosetmessages: + msg393395
2021-03-21 16:14:34pitrousetmessages: + msg389244
2021-03-07 11:26:20pitrousetassignee: skrah ->
messages: + msg388233
2021-03-04 22:28:09mattipsetnosy: + mattip
messages: + msg388134
2020-08-10 14:34:29skrahsetstatus: open -> closed
resolution: fixed
components: + C API, - Extension Modules
stage: patch review -> resolved
2020-08-10 14:32:29skrahsetmessages: + msg375121
2020-07-20 15:02:56skrahsetmessages: + msg374012
2020-07-19 04:51:49xtreaksettitle: En az ondalık kapsül API ekleme -> Add a minimal decimal capsule API
2020-07-19 04:47:19Abdulkadir Özbudaksettitle: Add a minimal decimal capsule API -> En az ondalık kapsül API ekleme
2020-07-18 17:28:43skrahsetmessages: + msg373910
2020-07-17 17:58:05pitrousetmessages: + msg373842
2020-07-17 12:43:58skrahsetnosy: + piro
messages: + msg373831
2020-07-17 11:57:45skrahsetmessages: + msg373829
2020-07-17 11:25:14skrahsettype: enhancement
2020-07-17 11:24:39skrahlinkissue22194 superseder
2020-07-17 11:18:33skrahsetnosy: + rhettinger, mark.dickinson, pitrou
messages: + msg373823
2020-07-17 10:56:48skrahsetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request20656
2020-07-17 10:55:47skrahcreate