classification
Title: EXT_SUFFIX inconsistent between sysconfig and distutils.sysconfig (Windows)
Type: behavior Stage: resolved
Components: Distutils, Library (Lib) Versions: Python 3.10, Python 3.9, Python 3.8
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: Michael.Felt, dstufft, eric.araujo, frenzy, mattip, miss-islington, paul.moore, pitrou, steve.dower, tarek, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2020-03-02 13:40 by pitrou, last changed 2021-02-03 18:03 by steve.dower. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 22088 merged mattip, 2020-09-04 11:58
PR 23681 merged miss-islington, 2020-12-07 17:34
PR 23682 merged miss-islington, 2020-12-07 17:34
PR 23684 mattip, 2020-12-07 21:57
Messages (14)
msg363171 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2020-03-02 13:40
On Windows, Python 3.7.6 and 3.8.1:
```
>>> import sysconfig
>>> sysconfig.get_config_var('EXT_SUFFIX')
'.pyd'
>>> from distutils import sysconfig
>>> sysconfig.get_config_var('EXT_SUFFIX')
'.cp38-win_amd64.pyd'
```

The sysconfig answer is probably wrong (the ABI-qualified extension '.cp38-win_amd64.pyd' should be preferred).
msg379030 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-10-19 21:54
Agreed, this should be updated.

3.7 is out of scope, I'm afraid, but 3.8 onwards should be fine.
msg382639 - (view) Author: mattip (mattip) * Date: 2020-12-07 14:37
the PR is awaiting approval/merge
msg382660 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-12-07 17:33
New changeset c0afb7fa0ebd1c0e95c0760bbe75a99a8dd12ea6 by Matti Picus in branch 'master':
bpo-39825: Fixes sysconfig.get_config_var('EXT_SUFFIX') on Windows to match distutils (GH-22088)
https://github.com/python/cpython/commit/c0afb7fa0ebd1c0e95c0760bbe75a99a8dd12ea6
msg382661 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2020-12-07 17:33
Thanks!
msg382663 - (view) Author: miss-islington (miss-islington) Date: 2020-12-07 17:50
New changeset ca52aa3ddd949ce2d259b4263344339b56db00b7 by Miss Islington (bot) in branch '3.8':
bpo-39825: Fixes sysconfig.get_config_var('EXT_SUFFIX') on Windows to match distutils (GH-22088)
https://github.com/python/cpython/commit/ca52aa3ddd949ce2d259b4263344339b56db00b7
msg382664 - (view) Author: miss-islington (miss-islington) Date: 2020-12-07 17:57
New changeset e9a6dcdefabb6c19074566f4ee0e02daaf57be18 by Miss Islington (bot) in branch '3.9':
bpo-39825: Fixes sysconfig.get_config_var('EXT_SUFFIX') on Windows to match distutils (GH-22088)
https://github.com/python/cpython/commit/e9a6dcdefabb6c19074566f4ee0e02daaf57be18
msg382685 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-12-07 21:49
This change broke the AMD64 FreeBSD Shared 3.x buildbot:
https://buildbot.python.org/all/#/builders/483/builds/513

FAIL: test_EXT_SUFFIX_in_vars (test.test_sysconfig.TestSysConfig)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_sysconfig.py", line 368, in test_EXT_SUFFIX_in_vars
    self.assertEqual(vars['EXT_SUFFIX'], _imp.extension_suffixes()[0])
AssertionError: '.so' != '.cpython-310d.so'
- .so
+ .cpython-310d.so
msg382686 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-12-07 21:50
It also broke POWER6 AIX 3.8:
https://buildbot.python.org/all/#builders/138/builds/125

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.8.aixtools-aix-power6/build/Lib/test/test_sysconfig.py", line 366, in test_EXT_SUFFIX_in_vars
    self.assertEqual(vars['EXT_SUFFIX'], _imp.extension_suffixes()[0])
AssertionError: '.so' != '.cpython-38d.so'
- .so
+ .cpython-38d.so
msg382688 - (view) Author: mattip (mattip) * Date: 2020-12-07 21:57
I submitted PR 23684 to try to skip tests to fix this. I don't understand why this wasn't caught before merge, and where AIX and FREEBSD are setting EXT_SUFFIX improperly.
msg382697 - (view) Author: mattip (mattip) * Date: 2020-12-07 23:38
I moved the new test to a separate test function and added `skipIf` for freebsd and AIX.
msg382720 - (view) Author: Michael Felt (Michael.Felt) * Date: 2020-12-08 10:11
On Windows - in distutils (?) did they perhaps make a change only for windows, for whatever reason.

I don't have a python3.7 pr a 3.8 handy atm.

Not sure what 'distutils' pip updates either - but with py36 and pip 20.2.4 and setuptools 40.6.2 I get:

Python 3.6.12 (default, Sep 23 2020, 08:27:01) [C] on aix5
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_config_var('EXT_SUFFIX')
'.so'
>>> from distutils import sysconfig
>>> sysconfig.get_config_var('EXT_SUFFIX')
'.so'

I am guessing that distutils has not updated - whatever - for AIX, FreeBSD, and perhaps others.

@mattip: is the problem perhaps with distutils rather than a particular platform. I also admit, I do not know what 'EXT_SUFFIX' is suppossed to return. From past behavior I took it to mean a file 'tag' for files that are accessed using dlopen().

Is it - perhaps - that different platforms have different needs or abilities - and the error is not the platform - but the adequacy of tests to differentiate platform behavior. I am guessing now - but maybe that is why distutils has not changed AIX and FreeBSD and maybe more.
msg382723 - (view) Author: Michael Felt (Michael.Felt) * Date: 2020-12-08 10:25
Looking at https://github.com/python/cpython/pull/22088/files

imho: the patch forced Windows behavior ("""Initialize the module as appropriate for NT""") to be equivalent to vars['SO'] - which is what the test used to compare.

So, likely the change to the tests would be to have one for NT (per the comment in Lib/sysconfig.py) and the prior test set to be skipped if the platform is `NT` aka non_posix.
msg386228 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-02-03 18:03
Distutils is now deprecated (see PEP 632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils.

If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at https://github.com/pypa/setuptools
History
Date User Action Args
2021-02-03 18:03:19steve.dowersetstatus: open -> closed
resolution: out of date
messages: + msg386228

stage: patch review -> resolved
2020-12-08 15:14:03vstinnersetnosy: - vstinner
2020-12-08 10:25:43Michael.Feltsetmessages: + msg382723
2020-12-08 10:11:29Michael.Feltsetnosy: + Michael.Felt
messages: + msg382720
2020-12-07 23:38:11mattipsetmessages: + msg382697
2020-12-07 21:57:54mattipsetstage: resolved -> patch review
messages: + msg382688
pull_requests: + pull_request22549
2020-12-07 21:50:33vstinnersetmessages: + msg382686
2020-12-07 21:49:07vstinnersetstatus: closed -> open

nosy: + vstinner
messages: + msg382685

resolution: fixed -> (no value)
2020-12-07 17:57:00miss-islingtonsetmessages: + msg382664
2020-12-07 17:50:58miss-islingtonsetmessages: + msg382663
2020-12-07 17:34:17miss-islingtonsetpull_requests: + pull_request22546
2020-12-07 17:34:09miss-islingtonsetnosy: + miss-islington

pull_requests: + pull_request22545
2020-12-07 17:33:58steve.dowersetstatus: open -> closed
resolution: fixed
messages: + msg382661

stage: patch review -> resolved
2020-12-07 17:33:36steve.dowersetmessages: + msg382660
2020-12-07 14:37:15mattipsetmessages: + msg382639
2020-10-23 11:16:57frenzysetnosy: + frenzy
2020-10-19 21:54:02steve.dowersetmessages: + msg379030
versions: + Python 3.10, - Python 3.7
2020-09-04 11:58:03mattipsetkeywords: + patch
nosy: + mattip

pull_requests: + pull_request21174
stage: needs patch -> patch review
2020-03-02 13:40:07pitroucreate