Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

EXT_SUFFIX inconsistent between sysconfig and distutils.sysconfig (Windows) #84006

Closed
pitrou opened this issue Mar 2, 2020 · 14 comments
Closed

EXT_SUFFIX inconsistent between sysconfig and distutils.sysconfig (Windows) #84006

pitrou opened this issue Mar 2, 2020 · 14 comments
Labels
3.8 only security fixes 3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@pitrou
Copy link
Member

pitrou commented Mar 2, 2020

BPO 39825
Nosy @pfmoore, @pitrou, @tjguk, @tarekziade, @merwok, @zware, @mattip, @zooba, @dstufft, @aixtools, @frenzymadness, @miss-islington
PRs
  • bpo-39825: fix sysconfig.get_config_var('EXT_SUFFIX') on windows #22088
  • [3.9] bpo-39825: Fixes sysconfig.get_config_var('EXT_SUFFIX') on Windows to match distutils (GH-22088) #23681
  • [3.8] bpo-39825: Fixes sysconfig.get_config_var('EXT_SUFFIX') on Windows to match distutils (GH-22088) #23682
  • TST: unbreak tests on FREEBSD #23684
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2021-02-03.18:03:19.215>
created_at = <Date 2020-03-02.13:40:07.588>
labels = ['3.8', 'type-bug', 'library', '3.9', '3.10']
title = 'EXT_SUFFIX inconsistent between sysconfig and distutils.sysconfig (Windows)'
updated_at = <Date 2021-02-03.18:03:19.215>
user = 'https://github.com/pitrou'

    bugs.python.org fields:

    activity = <Date 2021-02-03.18:03:19.215>
actor = 'steve.dower'
assignee = 'none'
closed = True
closed_date = <Date 2021-02-03.18:03:19.215>
closer = 'steve.dower'
components = ['Distutils', 'Library (Lib)']
creation = <Date 2020-03-02.13:40:07.588>
creator = 'pitrou'
dependencies = []
files = []
hgrepos = []
issue_num = 39825
keywords = ['patch']
message_count = 14.0
messages = ['363171', '379030', '382639', '382660', '382661', '382663', '382664', '382685', '382686', '382688', '382697', '382720', '382723', '386228']
nosy_count = 12.0
nosy_names = ['paul.moore', 'pitrou', 'tim.golden', 'tarek', 'eric.araujo', 'zach.ware', 'mattip', 'steve.dower', 'dstufft', 'Michael.Felt', 'frenzy', 'miss-islington']
pr_nums = ['22088', '23681', '23682', '23684']
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39825'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']
    @pitrou
    Copy link
    Member Author

    pitrou commented Mar 2, 2020

    On Windows, Python 3.7.6 and 3.8.1:

    >>> import sysconfig
>>> sysconfig.get_config_var('EXT_SUFFIX')
'.pyd'
>>> from distutils import sysconfig
>>> sysconfig.get_config_var('EXT_SUFFIX')
'.cp38-win_amd64.pyd'

    The sysconfig answer is probably wrong (the ABI-qualified extension '.cp38-win_amd64.pyd' should be preferred).

    @pitrou pitrou added 3.7 (EOL) end of life 3.8 only security fixes 3.9 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Mar 2, 2020
    @zooba
    Copy link
    Member

    zooba commented Oct 19, 2020

    Agreed, this should be updated.

    3.7 is out of scope, I'm afraid, but 3.8 onwards should be fine.

    @zooba zooba added 3.10 only security fixes and removed 3.7 (EOL) end of life labels Oct 19, 2020
    @mattip
    Copy link
    Contributor

    mattip commented Dec 7, 2020

    the PR is awaiting approval/merge

    @zooba
    Copy link
    Member

    zooba commented Dec 7, 2020

    New changeset c0afb7f by Matti Picus in branch 'master':
    bpo-39825: Fixes sysconfig.get_config_var('EXT_SUFFIX') on Windows to match distutils (GH-22088)
    c0afb7f

    @zooba
    Copy link
    Member

    zooba commented Dec 7, 2020

    Thanks!

    @zooba zooba closed this as completed Dec 7, 2020
    @zooba zooba closed this as completed Dec 7, 2020
    @miss-islington
    Copy link
    Contributor

    New changeset ca52aa3 by Miss Islington (bot) in branch '3.8':
    bpo-39825: Fixes sysconfig.get_config_var('EXT_SUFFIX') on Windows to match distutils (GH-22088)
    ca52aa3

    @miss-islington
    Copy link
    Contributor

    New changeset e9a6dcd by Miss Islington (bot) in branch '3.9':
    bpo-39825: Fixes sysconfig.get_config_var('EXT_SUFFIX') on Windows to match distutils (GH-22088)
    e9a6dcd

    @vstinner
    Copy link
    Member

    vstinner commented Dec 7, 2020

    This change broke the AMD64 FreeBSD Shared 3.x buildbot:
    https://buildbot.python.org/all/#/builders/483/builds/513

    FAIL: test_EXT_SUFFIX_in_vars (test.test_sysconfig.TestSysConfig)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_sysconfig.py", line 368, in test_EXT_SUFFIX_in_vars
    self.assertEqual(vars['EXT_SUFFIX'], _imp.extension_suffixes()[0])
AssertionError: '.so' != '.cpython-310d.so'
- .so
+ .cpython-310d.so

    @vstinner vstinner reopened this Dec 7, 2020
    @vstinner vstinner reopened this Dec 7, 2020
    @vstinner
    Copy link
    Member

    vstinner commented Dec 7, 2020

    It also broke POWER6 AIX 3.8:
    https://buildbot.python.org/all/#builders/138/builds/125

    Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.8.aixtools-aix-power6/build/Lib/test/test_sysconfig.py", line 366, in test_EXT_SUFFIX_in_vars
    self.assertEqual(vars['EXT_SUFFIX'], _imp.extension_suffixes()[0])
AssertionError: '.so' != '.cpython-38d.so'
- .so
+ .cpython-38d.so

    @mattip
    Copy link
    Contributor

    mattip commented Dec 7, 2020

    I submitted PR 23684 to try to skip tests to fix this. I don't understand why this wasn't caught before merge, and where AIX and FREEBSD are setting EXT_SUFFIX improperly.

    @mattip
    Copy link
    Contributor

    mattip commented Dec 7, 2020

    I moved the new test to a separate test function and added skipIf for freebsd and AIX.

    @aixtools
    Copy link
    Contributor

    aixtools commented Dec 8, 2020

    On Windows - in distutils (?) did they perhaps make a change only for windows, for whatever reason.

    I don't have a python3.7 pr a 3.8 handy atm.

    Not sure what 'distutils' pip updates either - but with py36 and pip 20.2.4 and setuptools 40.6.2 I get:

    Python 3.6.12 (default, Sep 23 2020, 08:27:01) [C] on aix5
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('EXT_SUFFIX')
'.so'
>>> from distutils import sysconfig
>>> sysconfig.get_config_var('EXT_SUFFIX')
'.so'

    I am guessing that distutils has not updated - whatever - for AIX, FreeBSD, and perhaps others.

    @mattip: is the problem perhaps with distutils rather than a particular platform. I also admit, I do not know what 'EXT_SUFFIX' is suppossed to return. From past behavior I took it to mean a file 'tag' for files that are accessed using dlopen().

    Is it - perhaps - that different platforms have different needs or abilities - and the error is not the platform - but the adequacy of tests to differentiate platform behavior. I am guessing now - but maybe that is why distutils has not changed AIX and FreeBSD and maybe more.

    @aixtools
    Copy link
    Contributor

    aixtools commented Dec 8, 2020

    Looking at https://github.com/python/cpython/pull/22088/files

    imho: the patch forced Windows behavior ("""Initialize the module as appropriate for NT""") to be equivalent to vars['SO'] - which is what the test used to compare.

    So, likely the change to the tests would be to have one for NT (per the comment in Lib/sysconfig.py) and the prior test set to be skipped if the platform is NT aka non_posix.

    @zooba
    Copy link
    Member

    zooba commented Feb 3, 2021

    Distutils is now deprecated (see PEP-632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils.

    If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at https://github.com/pypa/setuptools

    @zooba zooba closed this as completed Feb 3, 2021
    @zooba zooba closed this as completed Feb 3, 2021
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @pseudoyim pseudoyim mentioned this issue Sep 22, 2022
    Merged
    @dnicolodi dnicolodi mentioned this issue Nov 15, 2022
    Merged
    maloel added a commit to maloel/librealsense that referenced this issue Nov 27, 2022
    @maloel
    Go back to pybind v2.9.0 because of bug
    e22a46e 
    python/cpython#84006

>>> import sysconfig
>>> sysconfig.get_config_var('EXT_SUFFIX')
'.pyd'
>>> from distutils import sysconfig
>>> sysconfig.get_config_var('EXT_SUFFIX')
'.cp38-win_amd64.pyd'

v2.9.2 takes from the previous, v2.9.0 from the latter.
So with v2.9.2 we get ABI-less extension which we need.

v2.10.1 seems to have fixed this, but we cannot move to it (it
deprecates VS2015)...
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.8 only security fixes 3.9 only security fixes 3.10 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    6 participants
    @vstinner @mattip @zooba @pitrou @aixtools @miss-islington