classification
Title: Deprecate some of the module file formats
Type: performance Stage: resolved
Components: Interpreter Core, Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, barry, brett.cannon, dmalcolm, eric.araujo, eric.snow, loewis, nadeem.vawda, neologix, pitrou, python-dev, schmir
Priority: normal Keywords: patch

Created on 2012-02-17 13:36 by pitrou, last changed 2012-02-20 18:54 by pitrou. This issue is now closed.

Files
File name Uploaded Description Edit
impsuffix.patch pitrou, 2012-02-17 13:36 review
impsuffix2.patch pitrou, 2012-02-18 18:12 review
removesuffixes.patch pitrou, 2012-02-19 17:15 review
Messages (10)
msg153544 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-02-17 13:36
Stating module files of the form "xxxmodule.so" consumes all 1/3 of stat calls at startup. Looking at the site-packages folders on my machine, both for 2.7 and 3.2, reveals no C extension that follows such naming.

This patch deprecates such module namings, so that they can be removed in a later version.
msg153546 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-02-17 14:08
Patch looks fine, although there aren't any tests. At least for importlib you can simply use a finder test for extension modules to verify the warning is triggered. That way you can create an empty module with the expected naming rather than worry about compiling an extension module.
msg153550 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2012-02-17 14:56
On Feb 17, 2012, at 01:36 PM, Antoine Pitrou wrote:

>Stating module files of the form "xxxmodule.so" consumes all 1/3 of stat
>calls at startup. Looking at the site-packages folders on my machine, both
>for 2.7 and 3.2, reveals no C extension that follows such naming.
>
>This patch deprecates such module namings, so that they can be removed in a
>later version.

+1

In Debian/Ubuntu I have seen some third party extensions (not built with
distutils) that still use this naming convention in their Python 3 extension
module, but it's usually pretty easy to rename them to PEP 3149 style names in
the platform build scripts.  I think it's a great idea to eventually get rid
of the untagged names altogether, and saving some stat calls is added benefit.
msg153651 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-02-18 18:12
Here is a patch with tests.
msg153652 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-02-18 18:23
In this case, I propose to drop the feature without deprecation. It is very easy to adjust the build process of packages that still use the feature, and even end users can rename the files.

If you want to improve ease-of-use, you could still do the stat calls, and record whether a file would have been imported. If then the import fails, put a message into the import error saying the /some/where/somemodule.so was ignored because the file name is no longer supported.
msg153662 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2012-02-19 00:52
I like Martin's suggestion of simply throwing an error. But then again I also really like his idea of simply not warning since it's easier. =)
msg153712 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-02-19 17:15
Here is a patch simply removing the suffixes.
msg153727 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-02-19 21:04
I'm in favour of the removesuffixes.patch. However, it will need to be accompanied with a whatsnew change.
msg153795 - (view) Author: Roundup Robot (python-dev) Date: 2012-02-20 18:44
New changeset 42f61304f77d by Antoine Pitrou in branch 'default':
Issue #14040: Remove rarely used file name suffixes for C extensions (under POSIX mainly).
http://hg.python.org/cpython/rev/42f61304f77d
msg153797 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-02-20 18:54
I've now committed the suffix removal patch with a what's new entry. Thanks!
History
Date User Action Args
2012-02-20 18:54:02pitrousetstatus: open -> closed
resolution: fixed
messages: + msg153797

stage: patch review -> resolved
2012-02-20 18:44:38python-devsetnosy: + python-dev
messages: + msg153795
2012-02-19 22:42:04eric.araujosetnosy: + eric.araujo
2012-02-19 21:04:06loewissetmessages: + msg153727
2012-02-19 17:15:25pitrousetfiles: + removesuffixes.patch

messages: + msg153712
2012-02-19 00:52:26brett.cannonsetmessages: + msg153662
2012-02-18 18:37:28eric.snowsetnosy: + eric.snow
2012-02-18 18:28:37Arfreversetnosy: + Arfrever
2012-02-18 18:23:28loewissetmessages: + msg153652
2012-02-18 18:12:48pitrousetfiles: + impsuffix2.patch

messages: + msg153651
2012-02-17 14:59:23schmirsetnosy: + schmir
2012-02-17 14:56:34barrysetmessages: + msg153550
2012-02-17 14:08:05brett.cannonsetmessages: + msg153546
2012-02-17 13:41:43nadeem.vawdasetnosy: + nadeem.vawda
2012-02-17 13:36:36pitroucreate