classification
Title: Possibly out of date C extension documentation
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Romuald, amaury.forgeotdarc, christian.heimes, docs@python, erik.bray, lazka, r.david.murray, skrah
Priority: normal Keywords: patch

Created on 2017-09-13 09:03 by Romuald, last changed 2017-09-22 17:39 by skrah. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 3688 merged skrah, 2017-09-21 16:51
PR 3697 merged skrah, 2017-09-22 16:10
Messages (17)
msg302038 - (view) Author: Romuald Brunet (Romuald) * Date: 2017-09-13 09:03
In the "Defining New Types documentation" basics about tp_new -> PyType_GenericNew, the doc states:

> We’d like to just assign this to the tp_new slot, but we can’t, for
> portability sake, On some platforms or compilers, we can’t statically
> initialize a structure member with a function defined in another C
> module, so, instead, we’ll assign the tp_new slot in the module
> initialization function just before calling PyType_Ready():


But looking a python C code itself [1] does seem to do this. So I'm guessing that part of the documentation is now irrelevant and the example could just assign PyType_GenericNew to tp_new.

[1] https://github.com/python/cpython/blob/2ebc5ce42a8a9e047e790aefbf9a94811569b2b6/Objects/listobject.c#L2733
msg302074 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2017-09-13 13:24
On Windows at least: I think "another C module" should be understood as "another DLL or executable".

A user extension module is a different DLL from the core Python, so the advice is still valid.
msg302364 - (view) Author: Romuald Brunet (Romuald) * Date: 2017-09-17 12:55
I'm not sure this is relevant, but I've just made a simple test on Windows (7) with my C extension using { … .tp_new = PyType_GenericNew } and the compiler did not complain (also the code worked as expected)
msg302365 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2017-09-17 13:12
PyType_GenericNew() should be in libpython, so indeed the example in the docs seems a bit odd to me.
msg302386 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2017-09-17 21:32
FWIW, I've been using

  https://github.com/python/cpython/blob/master/Modules/_decimal/_decimal.c#L689


the static initialization on problematic platforms like Windows and
AIX for years, without any problems.


I'm sure this is valid C, and people here agree:

https://stackoverflow.com/questions/6989107/cant-initialize-static-structure-with-function-pointer-from-another-translation


Perhaps this was an issue on very old IRIX systems or similar?
msg302388 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2017-09-17 22:31
Christian, do you remember which compiler was the reason for the commit

cbf3b5cb76906fba15dbf59a1e83c540a447b907 ?

+       /* Due to cross platform compiler issues the slots must be filled
+        * here. It's required for portability to Windows without requiring
+        * C++. */
        Null_Type.tp_base = &PyBaseObject_Type;
+       Null_Type.tp_new = PyType_GenericNew;
        Str_Type.tp_base = &PyUnicode_Type;



I've never had problems with VS 2008 or greater.
msg302460 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-18 14:53
I think it's svnmerge of this commit, which talks about VS 2005. It might have been required to support VS 2003 and 2005.

```
 r59290 | christian.heimes | 2007-12-03 14:47:29 +0100 (Mon, 03 Dec 2007) | 3 lines

  Applied my patch #1455 with some extra fixes for VS 2005
  The new msvc9compiler module supports VS 2005 and VS 2008. I've also fixed build_ext to support PCbuild8 and PCbuild9 and backported my fix for xxmodule.c from py3k. The old code msvccompiler is still in place in case somebody likes to build an extension with VS 2003 or earlier.
  I've also updated the cygwin compiler module for VS 2005 and VS 2008. It works with VS 2005 but I'm unable to test it with VS 2008. We have to wait for a new version of cygwin.
```
msg302642 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2017-09-20 17:47
Thanks, Christian. -- I found that in the docs the culprit is Cygwin:

   db6a569de7ae595ada53b618fce6bbbd1c98d350
msg302643 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2017-09-20 17:48
Do we still support cygwin?
msg302646 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2017-09-20 18:03
I've just asked on python-dev if Cygwin is still broken. Not sure
if we support it.
msg302650 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2017-09-20 18:38
We do not currently officially support cygwin.  There are people working on getting it working again (and having a buildbot) so we can support it, so cygwin support is a goal, but not currently a requirement.  I've nosied Erik Brey, who is one of the main people interested in and working on cygwin.  The note might well be out of date even with regards to cygwin.
msg302711 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2017-09-21 16:58
Erik, if you are interested in Cygwin, could you please check that xxmodule.c builds on Cygwin with the patch?  You need to uncomment
a couple of lines in setup.py to build 'xx'.
msg302713 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2017-09-21 17:05
In fact, building _decimal should also fail on Cygwin if this
were still an issue.
msg302746 - (view) Author: Christoph Reiter (lazka) Date: 2017-09-22 14:26
Building the following with gcc from msys2 (cygwin) worked fine here: https://bpaste.net/show/0cafd5fa8211
msg302750 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2017-09-22 15:32
Okay, thanks.

I found that bz2module.c has also used direct initialization for ages.
I'm going to commit the change.
msg302752 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2017-09-22 15:45
New changeset ca72589bfabe2fd0e12eebfeb770b6c7a499b3e6 by Stefan Krah in branch 'master':
bpo-31443: Formulate the type slot initialization rules in terms of C99. (#3688)
https://github.com/python/cpython/commit/ca72589bfabe2fd0e12eebfeb770b6c7a499b3e6
msg302753 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2017-09-22 16:14
New changeset b1558a0368949714f5765702a8d83a2d163eaacf by Stefan Krah in branch 'master':
bpo-31443: Update included code. (#3697)
https://github.com/python/cpython/commit/b1558a0368949714f5765702a8d83a2d163eaacf
History
Date User Action Args
2017-09-22 17:39:48skrahsetstatus: open -> closed
stage: patch review -> resolved
resolution: fixed
versions: + Python 3.7
2017-09-22 16:14:15skrahsetmessages: + msg302753
2017-09-22 16:10:05skrahsetpull_requests: + pull_request3681
2017-09-22 15:45:00skrahsetmessages: + msg302752
2017-09-22 15:32:44skrahsetmessages: + msg302750
2017-09-22 14:26:41lazkasetnosy: + lazka
messages: + msg302746
2017-09-21 17:05:06skrahsetmessages: + msg302713
2017-09-21 16:58:33skrahsetmessages: + msg302711
2017-09-21 16:51:48skrahsetkeywords: + patch
stage: patch review
pull_requests: + pull_request3678
2017-09-20 18:38:10r.david.murraysetnosy: + erik.bray, r.david.murray
messages: + msg302650
2017-09-20 18:03:50skrahsetmessages: + msg302646
2017-09-20 17:48:31christian.heimessetmessages: + msg302643
2017-09-20 17:47:20skrahsetmessages: + msg302642
2017-09-18 14:53:20christian.heimessetmessages: + msg302460
2017-09-17 22:31:48skrahsetnosy: + christian.heimes
messages: + msg302388
2017-09-17 21:32:15skrahsetmessages: + msg302386
2017-09-17 13:12:15skrahsetnosy: + skrah
messages: + msg302365
2017-09-17 12:55:49Romualdsetmessages: + msg302364
2017-09-13 13:24:39amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg302074
2017-09-13 09:03:10Romualdcreate