Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modules needing PY_SSIZE_T_CLEAN #52923

Closed
pitrou opened this issue May 10, 2010 · 25 comments
Closed

Modules needing PY_SSIZE_T_CLEAN #52923

pitrou opened this issue May 10, 2010 · 25 comments
Labels
3.8 only security fixes extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error

Comments

@pitrou
Copy link
Member

pitrou commented May 10, 2010

BPO 8677
Nosy @loewis, @terryjreedy, @mdickinson, @pitrou, @vstinner, @methane, @serhiy-storchaka
PRs
  • bpo-8677: use PY_SSIZE_T_CLEAN in sqlite #12434
  • bpo-8677: use PY_SSIZE_T_CLEAN in Modules/_gdbmodule.c #12464
  • bpo-8677: use PY_SSIZE_T_CLEAN in PC/winreg.c #12466
  • bpo-8677: use PY_SSIZE_T_CLEAN in socketmodule.c #12467
  • bpo-8677: use PY_DWORD_MAX instead of INT_MAX #12469
  • bpo-36381: warn when no PY_SSIZE_T_CLEAN #12473
  • Dependencies
  • bpo-8675: audioop module needs an int -> Py_ssize_t upgrade
  • Files
  • zlibmodule_ssize_t_clean.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-03-20.12:59:38.299>
    created_at = <Date 2010-05-10.19:59:39.449>
    labels = ['extension-modules', 'type-bug', '3.8']
    title = 'Modules needing PY_SSIZE_T_CLEAN'
    updated_at = <Date 2019-03-24.05:25:57.287>
    user = 'https://github.com/pitrou'

    bugs.python.org fields:

    activity = <Date 2019-03-24.05:25:57.287>
    actor = 'serhiy.storchaka'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-03-20.12:59:38.299>
    closer = 'methane'
    components = ['Extension Modules']
    creation = <Date 2010-05-10.19:59:39.449>
    creator = 'pitrou'
    dependencies = ['8675']
    files = ['35656']
    hgrepos = []
    issue_num = 8677
    keywords = ['patch']
    message_count = 25.0
    messages = ['105461', '105462', '105464', '105739', '105741', '105755', '105767', '220678', '220733', '220735', '220737', '222040', '338349', '338350', '338355', '338356', '338443', '338447', '338449', '338451', '338456', '338457', '338467', '338480', '338715']
    nosy_count = 9.0
    nosy_names = ['loewis', 'terry.reedy', 'mark.dickinson', 'pitrou', 'vstinner', 'Arfrever', 'methane', 'python-dev', 'serhiy.storchaka']
    pr_nums = ['12434', '12464', '12466', '12467', '12469', '12473']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue8677'
    versions = ['Python 3.8']

    @pitrou
    Copy link
    Member Author

    pitrou commented May 10, 2010

    This is a list of extension modules making use of one of the "#" format codes ("s#", "y#", etc.) without defining PY_SSIZE_T_CLEAN, and therefore being 64-bit unclean:

    Modules/audioop.c
    Modules/_cursesmodule.c
    Modules/_elementtree.c
    Modules/_gdbmmodule.c
    Modules/nismodule.c
    Modules/ossaudiodev.c
    Modules/pyexpat.c
    Modules/socketmodule.c
    Modules/_ssl.c
    Modules/unicodedata.c

    @pitrou pitrou added extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error labels May 10, 2010
    @pitrou
    Copy link
    Member Author

    pitrou commented May 10, 2010

    The documentation says that sometimes in the future Py_ssize_t will be the default, so we may also need some kind of transition period for non-complying third-party extensions; first with warnings and then with errors perhaps.

    @loewis
    Copy link
    Mannequin

    loewis mannequin commented May 10, 2010

    I personally don't think that a transition period would be a useful thing to have, in particular not if it goes like this:

    • in version A, require use of PY_SSIZE_T_CLEAN
    • in version A+1, make use of PY_SSIZE_T_CLEAN redundant

    So I'd rather drop PY_SSIZE_T clean altogether from 3.2, and risk any breakage that this may cause.

    @terryjreedy
    Copy link
    Member

    Mark just fixed audioop in bpo-8675

    @mdickinson
    Copy link
    Member

    And the curses module was made PY_SSIZE_T_CLEAN in r81085 (a very simple change).

    @terryjreedy
    Copy link
    Member

    Is there any reason why you reported zlibmodule.c separately in bpo-8650 (other than maybe finding that first), and for 4 versions there versus 1 here? Should this supersede that?

    @pitrou
    Copy link
    Member Author

    pitrou commented May 14, 2010

    Is there any reason why you reported zlibmodule.c separately in bpo-8650
    (other than maybe finding that first),

    Because zlibmodule.c has more 64-bitness issues than just PY_SSIZE_T_CLEAN (see bug description).

    Should this supersede that?

    No.

    @BreamoreBoy
    Copy link
    Mannequin

    BreamoreBoy mannequin commented Jun 15, 2014

    Given the rise of the 64 bit machine I'd guess that this needs completing sooner rather than later. I'd volunteer myself but I've never heard of the '#' format codes, let alone know anything about them :-(

    @vstinner
    Copy link
    Member

    zlibmodule_ssize_t_clean.patch: Patch to make the zlib module "ssize_t clean". The patch just defines PY_SSIZE_T_CLEAN, no "#" format is used. "./python -m test -v test_zlib" test pass on my 64-bit Linux.

    @vstinner
    Copy link
    Member

    I just created the issue bpo-21780 to make the unicodedata module 64-bit safe.

    @vstinner
    Copy link
    Member

    I created the issue bpo-21781 to make the _ssl module 64-bit clean.

    @python-dev
    Copy link
    Mannequin

    python-dev mannequin commented Jul 1, 2014

    New changeset 691ca1694fe7 by Victor Stinner in branch '3.4':
    Issue bpo-8677: make the zlib module "ssize_t clean" for parsing parameters
    http://hg.python.org/cpython/rev/691ca1694fe7

    New changeset 45dcdd8f3211 by Victor Stinner in branch 'default':
    (Merge 3.4) Issue bpo-8677: make the zlib module "ssize_t clean" for parsing
    http://hg.python.org/cpython/rev/45dcdd8f3211

    @methane
    Copy link
    Member

    methane commented Mar 19, 2019

    New changeset 29198ea by Inada Naoki in branch 'master':
    bpo-8677: use PY_SSIZE_T_CLEAN in sqlite (GH-12434)
    29198ea

    @methane
    Copy link
    Member

    methane commented Mar 19, 2019

    Modules/_gdbmmodule.c
    Modules/socketmodule.c

    They use '#' without PY_SSIZE_T_CLEAN yet.

    @methane methane added the 3.8 only security fixes label Mar 19, 2019
    @serhiy-storchaka
    Copy link
    Member

    Also PC/winreg.c. In this case winreg.SetValue() needs a length of size DWORD instead of ssize_t.

    @vstinner
    Copy link
    Member

    Would it be possible to emit a deprecation warning, maybe at runtime, when PY_SSIZE_T_CLEAN is not defined?

    @methane
    Copy link
    Member

    methane commented Mar 20, 2019

    Also PC/winreg.c. In this case winreg.SetValue() needs a length of size DWORD instead of ssize_t.

    As MSDN, cbData is ignored.
    https://docs.microsoft.com/en-us/windows/desktop/api/winreg/nf-winreg-regsetvaluew

    Can I skip overflow check?

    If I can not, what is DWORD_MAX? (INT_MAX?)

    @methane
    Copy link
    Member

    methane commented Mar 20, 2019

    New changeset c5a216e by Inada Naoki in branch 'master':
    bpo-8677: use PY_SSIZE_T_CLEAN in Modules/_gdbmodule.c (GH-12464)
    c5a216e

    @methane
    Copy link
    Member

    methane commented Mar 20, 2019

    New changeset e9a1dcb by Inada Naoki in branch 'master':
    bpo-8677: use PY_SSIZE_T_CLEAN in socketmodule.c (GH-12467)
    e9a1dcb

    @methane
    Copy link
    Member

    methane commented Mar 20, 2019

    New changeset d5f18a6 by Inada Naoki in branch 'master':
    bpo-8677: use PY_SSIZE_T_CLEAN in PC/winreg.c (GH-12466)
    d5f18a6

    @vstinner
    Copy link
    Member

    PC/winreg.c:

        if (value_length >= INT_MAX) {
            PyErr_SetString(PyExc_OverflowError,
                            "the value is too long");
            return NULL;
        }

    PY_DWORD_MAX should be used here. It's twice larger than INT_MAX ;-)

    @methane
    Copy link
    Member

    methane commented Mar 20, 2019

    New changeset cc60cdd by Inada Naoki in branch 'master':
    bpo-8677: use PY_DWORD_MAX instead of INT_MAX (GH-12469)
    cc60cdd

    @methane
    Copy link
    Member

    methane commented Mar 20, 2019

    Would it be possible to emit a deprecation warning, maybe at runtime, when PY_SSIZE_T_CLEAN is not defined?

    I created bpo-36381 for it. Let's close this long living issue.

    @methane methane closed this as completed Mar 20, 2019
    @vstinner
    Copy link
    Member

    Let's close this long living issue.

    Thanks INADA-san for fixing last issues and for creating the deprecation issue!

    @serhiy-storchaka
    Copy link
    Member

    Thank you for doing this Inada-san!

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 only security fixes extension-modules C modules in the Modules dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    6 participants