classification
Title: Modules needing PY_SSIZE_T_CLEAN
Type: behavior Stage: resolved
Components: Extension Modules Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: 8675 Superseder:
Assigned To: Nosy List: Arfrever, inada.naoki, loewis, mark.dickinson, pitrou, python-dev, serhiy.storchaka, terry.reedy, vstinner
Priority: normal Keywords: patch

Created on 2010-05-10 19:59 by pitrou, last changed 2019-03-24 05:25 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
zlibmodule_ssize_t_clean.patch vstinner, 2014-06-16 16:04
Pull Requests
URL Status Linked Edit
PR 12434 merged inada.naoki, 2019-03-19 12:41
PR 12464 merged inada.naoki, 2019-03-20 08:45
PR 12466 merged inada.naoki, 2019-03-20 09:26
PR 12467 merged inada.naoki, 2019-03-20 09:27
PR 12469 merged inada.naoki, 2019-03-20 11:11
PR 12473 merged inada.naoki, 2019-03-20 12:54
Messages (25)
msg105461 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-05-10 19:59
This is a list of extension modules making use of one of the "#" format codes ("s#", "y#", etc.) without defining PY_SSIZE_T_CLEAN, and therefore being 64-bit unclean:

Modules/audioop.c
Modules/_cursesmodule.c
Modules/_elementtree.c
Modules/_gdbmmodule.c
Modules/nismodule.c
Modules/ossaudiodev.c
Modules/pyexpat.c
Modules/socketmodule.c
Modules/_ssl.c
Modules/unicodedata.c
msg105462 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-05-10 20:09
The documentation says that sometimes in the future Py_ssize_t will be the default, so we may also need some kind of transition period for non-complying third-party extensions; first with warnings and then with errors perhaps.
msg105464 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-05-10 20:23
I personally don't think that a transition period would be a useful thing to have, in particular not if it goes like this:
- in version A, require use of PY_SSIZE_T_CLEAN
- in version A+1, make use of PY_SSIZE_T_CLEAN redundant

So I'd rather drop PY_SSIZE_T clean altogether from 3.2, and risk any breakage that this may cause.
msg105739 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-05-14 18:51
Mark just fixed audioop in #8675
msg105741 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2010-05-14 18:58
And the curses module was made PY_SSIZE_T_CLEAN in r81085 (a very simple change).
msg105755 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-05-14 20:12
Is there any reason why you reported zlibmodule.c separately in #8650 (other than maybe finding that first), and for 4 versions there versus 1 here? Should this supersede that?
msg105767 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2010-05-14 21:32
> Is there any reason why you reported zlibmodule.c separately in #8650
> (other than maybe finding that first),

Because zlibmodule.c has more 64-bitness issues than just PY_SSIZE_T_CLEAN (see bug description).

> Should this supersede that?

No.
msg220678 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-06-15 21:43
Given the rise of the 64 bit machine I'd guess that this needs completing sooner rather than later.  I'd volunteer myself but I've never heard of the '#' format codes, let alone know anything about them :-(
msg220733 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-06-16 16:04
zlibmodule_ssize_t_clean.patch: Patch to make the zlib module "ssize_t clean". The patch just defines PY_SSIZE_T_CLEAN, no "#" format is used. "./python -m test -v test_zlib" test pass on my 64-bit Linux.
msg220735 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-06-16 16:13
I just created the issue #21780 to make the unicodedata module 64-bit safe.
msg220737 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-06-16 16:19
I created the issue #21781 to make the _ssl module 64-bit clean.
msg222040 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-07-01 14:49
New changeset 691ca1694fe7 by Victor Stinner in branch '3.4':
Issue #8677: make the zlib module "ssize_t clean" for parsing parameters
http://hg.python.org/cpython/rev/691ca1694fe7

New changeset 45dcdd8f3211 by Victor Stinner in branch 'default':
(Merge 3.4) Issue #8677: make the zlib module "ssize_t clean" for parsing
http://hg.python.org/cpython/rev/45dcdd8f3211
msg338349 - (view) Author: Inada Naoki (inada.naoki) * (Python committer) Date: 2019-03-19 13:10
New changeset 29198ea1c6d58f87389136b0ac0b8b2318dbac24 by Inada Naoki in branch 'master':
bpo-8677: use PY_SSIZE_T_CLEAN in sqlite (GH-12434)
https://github.com/python/cpython/commit/29198ea1c6d58f87389136b0ac0b8b2318dbac24
msg338350 - (view) Author: Inada Naoki (inada.naoki) * (Python committer) Date: 2019-03-19 13:11
Modules/_gdbmmodule.c
Modules/socketmodule.c

They use '#' without PY_SSIZE_T_CLEAN yet.
msg338355 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-03-19 13:39
Also PC/winreg.c. In this case winreg.SetValue() needs a length of size DWORD instead of ssize_t.
msg338356 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-19 13:43
Would it be possible to emit a deprecation warning, maybe at runtime, when PY_SSIZE_T_CLEAN is not defined?
msg338443 - (view) Author: Inada Naoki (inada.naoki) * (Python committer) Date: 2019-03-20 09:15
> Also PC/winreg.c. In this case winreg.SetValue() needs a length of size DWORD instead of ssize_t.

As MSDN, cbData is ignored.
https://docs.microsoft.com/en-us/windows/desktop/api/winreg/nf-winreg-regsetvaluew

Can I skip overflow check?

If I can not, what is DWORD_MAX? (INT_MAX?)
msg338447 - (view) Author: Inada Naoki (inada.naoki) * (Python committer) Date: 2019-03-20 10:02
New changeset c5a216e0b97712bf19b4a6b7655c6bf22a367edd by Inada Naoki in branch 'master':
bpo-8677: use PY_SSIZE_T_CLEAN in Modules/_gdbmodule.c (GH-12464)
https://github.com/python/cpython/commit/c5a216e0b97712bf19b4a6b7655c6bf22a367edd
msg338449 - (view) Author: Inada Naoki (inada.naoki) * (Python committer) Date: 2019-03-20 10:02
New changeset e9a1dcb4237cb2be71ab05883d472038ea9caf62 by Inada Naoki in branch 'master':
bpo-8677: use PY_SSIZE_T_CLEAN in socketmodule.c (GH-12467)
https://github.com/python/cpython/commit/e9a1dcb4237cb2be71ab05883d472038ea9caf62
msg338451 - (view) Author: Inada Naoki (inada.naoki) * (Python committer) Date: 2019-03-20 10:10
New changeset d5f18a63cc2dfe8d0adec8bce384a8c569b0f3dc by Inada Naoki in branch 'master':
bpo-8677: use PY_SSIZE_T_CLEAN in PC/winreg.c (GH-12466)
https://github.com/python/cpython/commit/d5f18a63cc2dfe8d0adec8bce384a8c569b0f3dc
msg338456 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-20 11:07
PC/winreg.c:

    if (value_length >= INT_MAX) {
        PyErr_SetString(PyExc_OverflowError,
                        "the value is too long");
        return NULL;
    }

PY_DWORD_MAX should be used here. It's twice larger than INT_MAX ;-)
msg338457 - (view) Author: Inada Naoki (inada.naoki) * (Python committer) Date: 2019-03-20 11:53
New changeset cc60cdd9c44dd15e441603ee5f78e09ea3e76929 by Inada Naoki in branch 'master':
bpo-8677: use PY_DWORD_MAX instead of INT_MAX (GH-12469)
https://github.com/python/cpython/commit/cc60cdd9c44dd15e441603ee5f78e09ea3e76929
msg338467 - (view) Author: Inada Naoki (inada.naoki) * (Python committer) Date: 2019-03-20 12:59
> Would it be possible to emit a deprecation warning, maybe at runtime, when PY_SSIZE_T_CLEAN is not defined?

I created bpo-36381 for it.  Let's close this long living issue.
msg338480 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2019-03-20 15:14
> Let's close this long living issue.

Thanks INADA-san for fixing last issues and for creating the deprecation issue!
msg338715 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2019-03-24 05:25
Thank you for doing this Inada-san!
History
Date User Action Args
2019-03-24 05:25:57serhiy.storchakasetmessages: + msg338715
2019-03-20 15:14:33vstinnersetmessages: + msg338480
2019-03-20 12:59:38inada.naokisetstatus: open -> closed
resolution: fixed
messages: + msg338467

stage: patch review -> resolved
2019-03-20 12:54:06inada.naokisetpull_requests: + pull_request12425
2019-03-20 11:53:17inada.naokisetmessages: + msg338457
2019-03-20 11:11:55inada.naokisetpull_requests: + pull_request12421
2019-03-20 11:07:01vstinnersetmessages: + msg338456
2019-03-20 10:10:21inada.naokisetmessages: + msg338451
2019-03-20 10:02:48inada.naokisetmessages: + msg338449
2019-03-20 10:02:03inada.naokisetmessages: + msg338447
2019-03-20 09:27:12inada.naokisetpull_requests: + pull_request12419
2019-03-20 09:26:50inada.naokisetpull_requests: + pull_request12418
2019-03-20 09:15:49inada.naokisetmessages: + msg338443
2019-03-20 08:45:28inada.naokisetpull_requests: + pull_request12416
2019-03-19 13:43:27vstinnersetmessages: + msg338356
2019-03-19 13:39:39serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg338355
2019-03-19 13:11:59inada.naokisetmessages: + msg338350
versions: + Python 3.8, - Python 3.2
2019-03-19 13:10:24inada.naokisetnosy: + inada.naoki
messages: + msg338349
2019-03-19 12:41:43inada.naokisetstage: needs patch -> patch review
pull_requests: + pull_request12389
2019-03-16 00:10:18BreamoreBoysetnosy: - BreamoreBoy
2014-07-01 14:49:02python-devsetnosy: + python-dev
messages: + msg222040
2014-06-16 16:19:13vstinnersetmessages: + msg220737
2014-06-16 16:13:29vstinnersetmessages: + msg220735
2014-06-16 16:04:38vstinnersetfiles: + zlibmodule_ssize_t_clean.patch

nosy: + vstinner
messages: + msg220733

keywords: + patch
2014-06-15 21:43:39BreamoreBoysetnosy: + BreamoreBoy
messages: + msg220678
2010-05-14 21:32:38pitrousetmessages: + msg105767
2010-05-14 20:12:36terry.reedysetmessages: + msg105755
2010-05-14 18:58:23mark.dickinsonsetmessages: + msg105741
2010-05-14 18:51:09terry.reedysetnosy: + terry.reedy
messages: + msg105739
2010-05-10 20:45:29Arfreversetnosy: + Arfrever
2010-05-10 20:23:29loewissetmessages: + msg105464
2010-05-10 20:09:00pitrousetnosy: + loewis
messages: + msg105462
2010-05-10 20:00:39pitrousetnosy: + mark.dickinson
dependencies: + audioop module needs an int -> Py_ssize_t upgrade
2010-05-10 19:59:39pitroucreate