classification
Title: Remove Lib/plat-*/* files
Type: Stage: resolved
Components: Versions: Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: doko Nosy List: doko, gvanrossum, koobs, ned.deily, python-dev, vstinner, xdegaye, zach.ware
Priority: normal Keywords:

Created on 2016-09-08 18:38 by zach.ware, last changed 2017-10-02 09:52 by vstinner. This issue is now closed.

Messages (30)
msg275106 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-09-08 18:38
After discussion at the core dev sprints, Guido has decreed that the files in Lib/plat-* can and should be removed.
msg275107 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-09-08 18:39
New changeset 14d89359a389 by Zachary Ware in branch 'default':
Issue #28027: Remove Lib/plat-* files
https://hg.python.org/cpython/rev/14d89359a389
msg275108 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-09-08 18:40
Will close this after the buildbots have confirmed they don't hate it.
msg275110 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-09-08 18:43
New changeset e256583e6062 by Zachary Ware in branch 'default':
Issue #28027: Mention the names of the removed modules in Misc/NEWS
https://hg.python.org/cpython/rev/e256583e6062
msg275121 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-08 19:21
> Issue #28027: Mention the names of the removed modules in Misc/NEWS

Thanks :-)
msg275189 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-09-08 22:33
None of the buildbots have complained about this (they've been too busy complaining about other things), so I'm closing it.
msg275252 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-09 04:49
Yeah! Thanks Zach :-)
msg275253 - (view) Author: Kubilay Kocak (koobs) (Python triager) Date: 2016-09-09 04:51
YES, finally. Thanks :)
msg275435 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-09 20:49
FYI I first proposed to remove plat-* in 2011:
https://mail.python.org/pipermail/python-dev/2011-October/114105.html

That's why I added os.RTDL_* constants:
https://docs.python.org/dev/library/os.html#os.RTLD_LAZY
msg275443 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2016-09-09 21:15
the removal of these modules should follow the normal deprecation process, for 3.6, then be removed for 3.7.  There is no harm for this approach.
msg275444 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-09-09 21:17
Why? Do you have a dependency on these? None of them are documented.
Do you propose to document them just so we can mark them deprecated?
msg275449 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2016-09-09 21:26
Since the 3.6 cycle, these modules are regenerated for the build, so they are not outdated anymore.  They are not a maintenance burden either.  So I'd like to see these available in 3.6, and documented as deprecated.  Then removed in 3.7.

Apparently the proposers for this removal don't have the time for an analysis before the beta, I don't have the time either. So pretty please let's revert that change, and instead have the deprecation documented.
msg275450 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-09-09 21:27
No, you need to demonstrate that they are used (despite undocumented).

On Fri, Sep 9, 2016 at 2:26 PM, Matthias Klose <report@bugs.python.org> wrote:
>
> Matthias Klose added the comment:
>
> Since the 3.6 cycle, these modules are regenerated for the build, so they are not outdated anymore.  They are not a maintenance burden either.  So I'd like to see these available in 3.6, and documented as deprecated.  Then removed in 3.7.
>
> Apparently the proposers for this removal don't have the time for an analysis before the beta, I don't have the time either. So pretty please let's revert that change, and instead have the deprecation documented.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue28027>
> _______________________________________
msg275451 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-09-09 21:29
I have been searching sources.debian.net, and have found no evidence of anything that would be broken by this.  Every use of these modules that I have found has been already wrapped in try/except ImportError, or is in code that is fairly obviously not meant for Python 3.
msg275454 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2016-09-09 21:37
whatever you searched, there are regressions:

https://codesearch.debian.net/search?q=import+DLFCN

some of these have conditional imports, however even boost uses unconditional imports, and is often used as an embedded copy.
msg275455 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-09-09 21:40
Those boost imports will never be triggered in Python 3, because sys.platform will never be 'linux2'.
msg275456 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-09 21:42
> https://codesearch.debian.net/search?q=import+DLFCN

I see 8 projects: boost, trilinos, vtk6, openbabel, gdcm, paraview, ecasound, yade. It doesn't like a giant project to update them.


> boost1.58_1.58.0+dfsg-5.1/libs/mpi/build/__init__.py
>
> import sys
> if sys.platform == 'linux2':
>    import DLFCN as dl

It's not like this code already works on Python 3 :-) sys.platform was replace with "linux" on Python 3.

It's easy to replace this import with:

try:
   import DLFCN as dl
except ImportError
   import os as dl
msg275457 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2016-09-09 21:44
these projects are the ones packaged by Debian; there are surely others.  Now again, why can't the actual removal for 3.7, following the usual deprecation rules?
msg275465 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-09-09 22:00
Because in every release of Python they have been undocumented, in any recent release of Python they have been inaccurate (at best) and probably unnecessary, they have been unmaintained for years, the presence of the plat-* dir on sys.path slows down imports from any directory after it on sys.path, the problem they purport to solve has other much better solutions, and this discussion started 5 years ago with only the the slightest of defenses from MvL.
msg275466 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-09-09 22:14
> Since the 3.6 cycle, these modules are regenerated for the build, so they are not outdated anymore.  They are not a maintenance burden either.

Indeed, they are regenerated by the changes made in issue 23968 ("rename the platform directory from plat-$(MACHDEP) to plat-$(PLATFORM_TRIPLET)"). And they are a maintenance burden as these changes fail to regenerate these modules on Android cross-builds as explained in msg269360, the generation being done with the headers of the build system instead of the target host (Android) system. The problem is not limited to Android cross-compilation of course.
msg275468 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-09 22:15
The FreeBSD packager told me that he is *very* happy to see these modules go away, it was also a pain for him to maintain a patch for them. The maintaince burden is even wider than Python source code.
msg275469 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-09-09 22:18
With no solid evidence against the change and much joy with the change, I'm reclosing the issue.  doko, if people complain to you about being broken by this, feel free to refer them to me.
msg275482 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2016-09-09 22:55
you asked for evidence, you got evidence. re-opening the issue again.  feel free to close it again without any evidence, and stating that you ignore the usual deprecation rules.
msg275494 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-09 23:34
Matthias Klose: Maybe you didn't understand something: breaking the backward compatibility was a deliberate choice. It's a matter of maintenance burden versus the number of broken applications.

According to your search, there are only 8 projects in the wild which will break on Python 3.6. Please compare this number to the number of Python modules on PyPI: 88 298 packages, according to https://pypi.python.org/pypi

Moreover, it seems like some of these 8 projects are not compatible with Python 3 yet, so it's not like this specific change broke them...

Finally, I don't like repeating myself, but: fixing these project requires to modify A SINGLE LINE OF CODE. I know well that Debian likes patching packages, it's not like it is something technically impossible.

Please don't reopen the issue again without a very strong argument.
msg275514 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2016-09-10 00:44
"It's a matter of maintenance burden". No I don't think so.

As agreed on IRC with zware, I'll restore the removed modules, including a deprecation notice.
msg275515 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2016-09-10 00:55
"Agreed" is a bit strong, I still don't agree that the change was bad or should be reverted.  I'm just not ruling out the possibility of a partial (because there is absolutely no reason to re-add the files for any platform but Linux and *maybe* OSX) reversion if you put forth the effort to do it after #28046 and add deprecation notice.
msg275517 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-09-10 00:59
Sorry, I'm leaving this discussion now.
msg275537 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2016-09-10 01:44
The modules should remain deleted for 3.6b1 so we can see if there are
actually any complaints -- if we get any we can consider restoring
them for b2. I fail to be convinced by the arguments against deletion
brought up so far -- the "evidence" produced is way too thin to
matter.
msg303445 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2017-10-01 02:13
Since we've reached 3.6.3 with no complaints that have made it to me, I'm going to go ahead and close the issue.
msg303520 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-10-02 09:52
Thank you Zachary for this removal ;-)
History
Date User Action Args
2017-10-02 09:52:51vstinnersetmessages: + msg303520
2017-10-01 02:13:44zach.waresetstatus: open -> closed
resolution: fixed
messages: + msg303445
2016-09-10 01:44:14gvanrossumsetmessages: + msg275537
2016-09-10 00:59:46vstinnersetnosy: gvanrossum, doko, vstinner, ned.deily, xdegaye, python-dev, zach.ware, koobs
messages: + msg275517
2016-09-10 00:55:30zach.waresetmessages: + msg275515
2016-09-10 00:44:09dokosetstatus: closed -> open
assignee: zach.ware -> doko
resolution: fixed -> (no value)
messages: + msg275514
2016-09-09 23:34:07vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg275494
2016-09-09 22:55:28dokosetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg275482
2016-09-09 22:18:39zach.waresetstatus: open -> closed
resolution: fixed
messages: + msg275469
2016-09-09 22:15:39vstinnersetmessages: + msg275468
2016-09-09 22:14:05xdegayesetmessages: + msg275466
2016-09-09 22:00:46zach.waresetmessages: + msg275465
2016-09-09 21:45:47xdegayesetnosy: + xdegaye
2016-09-09 21:44:30dokosetmessages: + msg275457
2016-09-09 21:42:26vstinnersetmessages: + msg275456
2016-09-09 21:40:11zach.waresetmessages: + msg275455
2016-09-09 21:37:30dokosetmessages: + msg275454
2016-09-09 21:29:00zach.waresetmessages: + msg275451
2016-09-09 21:27:33gvanrossumsetmessages: + msg275450
2016-09-09 21:26:22dokosetmessages: + msg275449
2016-09-09 21:21:37zach.warelinkissue27986 superseder
2016-09-09 21:17:45gvanrossumsetmessages: + msg275444
2016-09-09 21:15:51dokosetstatus: closed -> open

nosy: + doko
messages: + msg275443

resolution: fixed -> (no value)
2016-09-09 20:49:56vstinnersetmessages: + msg275435
2016-09-09 16:48:07zach.warelinkissue19554 superseder
2016-09-09 16:46:51zach.warelinkissue3990 superseder
2016-09-09 16:41:58zach.warelinkissue1565071 superseder
2016-09-09 16:41:42zach.warelinkissue12619 superseder
2016-09-09 04:51:38koobssetnosy: + koobs
messages: + msg275253
2016-09-09 04:49:39vstinnersetmessages: + msg275252
2016-09-08 22:33:53zach.waresetstatus: open -> closed
resolution: fixed
messages: + msg275189

stage: resolved
2016-09-08 19:21:04vstinnersetmessages: + msg275121
2016-09-08 18:43:24python-devsetmessages: + msg275110
2016-09-08 18:40:02zach.waresetmessages: + msg275108
2016-09-08 18:39:12python-devsetnosy: + python-dev
messages: + msg275107
2016-09-08 18:38:19zach.warecreate