classification
Title: test_distutils warning: initfunc exported twice on Windows
Type: behavior Stage: resolved
Components: Distutils, Distutils2 Versions: Python 3.4
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: skrah Nosy List: brian.curtin, eric.araujo, jkloth, loewis, mhammond, python-dev, santoso.wijaya, skrah, tarek, terry.reedy, thorsten.behrens
Priority: normal Keywords: patch

Created on 2010-08-29 17:10 by skrah, last changed 2014-01-28 14:09 by skrah. This issue is now closed.

Files
File name Uploaded Description Edit
initfunc.patch skrah, 2010-11-09 11:10 review
initfunc2.patch skrah, 2010-11-28 11:52 review
Messages (20)
msg115181 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2010-08-29 17:10
On Windows, the initfunc of a C extension is exported twice, as
seen here:

test_distutils
xxmodule.c
xxmodule.obj : warning LNK4197: export 'initxx' specified multiple times; using first specification


First export: pyport.h: #define PyMODINIT_FUNC __declspec(dllexport) void

Second export: Specified on the command line with /EXPORT

The code responsible for adding the initfunc name to ext.export_symbols
is in build_ext.py:get_export_symbols. I'm not sure if it could be
removed, since older extensions might not use PyMODINIT_FUNC.

If it can't be removed, perhaps PyMODINIT_FUNC could be specified
simply as void.
msg115194 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2010-08-29 20:32
Tarek, I forgot to add you, sorry. I was thinking mainly about distutils,
not distutils2.
msg120589 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-11-06 06:45
I can review a patch for this bug but not write it, lacking knowledge in C.
msg120859 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2010-11-09 11:10
This is the history of the issue:

In r16757 the current version of get_export_symbols() was added to
distutils. This function adds /EXPORT initfunc to the command line.

In 27697 PyMODINIT_FUNC was defined as __declspec(dllexport) void.


I'd suggest to remove get_export_symbols(). This would only affect
extensions that do not use PyMODINIT_FUNC, but that can be easily
repaired.
msg122615 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-11-28 03:41
I don’t think we can remove a method.  Can you make a patch with your other suggestion?  Thanks.
msg122653 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2010-11-28 11:52
Thinking about it, declaring PyMODINIT_FUNC as void would be a huge
step backwards. Using PyMODINIT_FUNC is the correct way of doing things.
The current situation punishes developers who actually read the docs.

If a C module stops working because of the change, _all_ the maintainer
has to do is change a single line in the sources. PyMODINIT_FUNC is
nearly 10 years old, so I think this is not too much to ask.


May I suggest deprecating get_export_symbols() for these reasons:

  1) The docstring is incorrect: The function does not 'either
     use export_symbols or "PyInit_" + module_name'. It forcibly
     adds PyInit_" + module_name', if it isn't already present in
     export_symbols.

  2) PyMODINIT_FUNC - which is nearly 10 years old - has obsoleted
     this function.

  3) Developers who do the right thing are punished, since they have
     to investigate the warning.



It is also possible to leave the function (see initfunc2.patch). Then
again, on its own get_export_symbols() is practically useless.
msg122883 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-11-30 05:24
The policy for distutils is: no changes, only bug fixes.  We cannot remove a function, even useless.  Deprecating does not seem useful if we aren’t going to remove.  Changing the docstring, the docs or the output of the code is okay.  Therefore, initfunc2.patch is the way to go.  Can you add a test for its behavior?
msg122897 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2010-11-30 10:20
Without the patch, you see the warning if test_build_ext is run in
verbose mode. With the patch, the warning disappears.
msg123178 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-12-03 04:45
Sounds like a “+1 for the patch” to me :)

I get no warning on linux2 after the patch.  This is still a behavior change, and there might be code out there relying on self.get_export_symbols being called.  My understanding of extension modules build is too small for me to have confidence, so I will defer the decision to Tarek.
msg124616 - (view) Author: Thorsten Behrens (thorsten.behrens) Date: 2010-12-24 22:28
Thank you for that patch, Stefan. I am currently tinkering with bringing pycrypto to 3.x and ran into this issue. initfunc2.patch resolves the issue on Win7-64, python31-64.

I don't feel comfortable releasing code that requires the user to manually patch Python to build. What are my options here? Go back to void instead of PyMODINIT_FUNC? Is there aught else I can do to make sure pycrypto will build well "in the wild"?
msg124685 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-12-26 23:47
Thorsten: my recommendation is to ignore this issue in your software. It's just a warning.
msg132991 - (view) Author: Santoso Wijaya (santoso.wijaya) * Date: 2011-04-04 23:52
A workaround would be to define an arbitrary macro when building with distutils. For example, in setup.cfg:

    [build_ext]
    define = _DISTUTILS

Then in some main header file(s) in your project:

    #ifdef _DISTUTILS
    #undef PyMODINIT_FUNC
    #define PyMODINIT_FUNC void
    #endif

And you won't get the warning anymore.
msg196746 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-09-01 22:35
This warning still appears on buildbots running with warnings turned on.
msg196774 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-09-02 08:24
Is the distutils freeze still in place? If not, I'll commit initfunc2.patch.
msg196786 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2013-09-02 15:43
The feature freeze was lifted at PyCon 2013.  This patch can be committed in the default branch.  Could you add a test?
msg196787 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2013-09-02 15:44
Ignore my comment about a test, you already replied to that :)  Please commit.
msg205115 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2013-12-03 12:55
New changeset 97fb852c5c26 by Stefan Krah in branch 'default':
Issue #9709: Stop adding PyInit_" + module_name' to export_symbols.  This is
http://hg.python.org/cpython/rev/97fb852c5c26
msg205117 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2013-12-03 13:16
Éric, thanks for taking a look.
msg209546 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-01-28 14:05
New changeset 69827c2ab9d0 by Stefan Krah in branch 'default':
Issue #9709: Revert 97fb852c5c26. Many extensions are not using PyMODINIT_FUNC.
http://hg.python.org/cpython/rev/69827c2ab9d0
msg209547 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2014-01-28 14:09
Too many extensions are not using PyMODINIT_FUNC (See #20166).
Closing as WONT_FIX.
History
Date User Action Args
2014-01-28 14:13:20skrahlinkissue20166 superseder
2014-01-28 14:09:25skrahsetresolution: fixed -> wont fix
messages: + msg209547
2014-01-28 14:05:39python-devsetmessages: + msg209546
2013-12-03 13:16:19skrahsetstatus: open -> closed
resolution: fixed
messages: + msg205117

stage: commit review -> resolved
2013-12-03 12:55:54python-devsetnosy: + python-dev
messages: + msg205115
2013-09-02 15:44:22eric.araujosetassignee: eric.araujo -> skrah
messages: + msg196787
stage: needs patch -> commit review
2013-09-02 15:43:32eric.araujosetassignee: tarek -> eric.araujo
messages: + msg196786
versions: - 3rd party, Python 2.7, Python 3.3
2013-09-02 08:24:24skrahsetmessages: + msg196774
2013-09-01 22:35:18terry.reedysetnosy: + terry.reedy

messages: + msg196746
versions: + Python 3.4, - Python 3.1, Python 3.2
2012-12-26 14:00:20skrahlinkissue16779 superseder
2012-12-26 14:00:12skrahsetnosy: + jkloth
2011-04-05 00:01:29santoso.wijayasetversions: + 3rd party
2011-04-04 23:52:36santoso.wijayasetnosy: + santoso.wijaya

messages: + msg132991
versions: + Python 3.3, - 3rd party
2010-12-26 23:47:36loewissetnosy: loewis, mhammond, tarek, eric.araujo, brian.curtin, skrah, thorsten.behrens
messages: + msg124685
2010-12-24 22:28:36thorsten.behrenssetnosy: + thorsten.behrens
messages: + msg124616
2010-12-03 04:45:44eric.araujosetassignee: eric.araujo -> tarek
messages: + msg123178
2010-11-30 10:20:44skrahsetmessages: + msg122897
2010-11-30 05:24:14eric.araujosetmessages: + msg122883
2010-11-28 11:52:30skrahsetfiles: + initfunc2.patch

messages: + msg122653
2010-11-28 03:41:58eric.araujosetmessages: + msg122615
2010-11-09 11:10:42skrahsetfiles: + initfunc.patch

nosy: + loewis, mhammond
messages: + msg120859

keywords: + patch
2010-11-06 06:45:36eric.araujosetmessages: + msg120589
2010-09-30 00:01:04eric.araujosetversions: + 3rd party
2010-08-29 20:32:55skrahsetnosy: + tarek
messages: + msg115194
2010-08-29 17:10:37skrahcreate