classification
Title: MinGW built extensions do not load (specified procedure cannot be found)
Type: Stage:
Components: Distutils Versions: Python 3.0, Python 2.6, 3rd party
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: amaury.forgeotdarc, loewis, rogerbinns
Priority: normal Keywords:

Created on 2008-07-07 00:16 by rogerbinns, last changed 2008-07-08 05:04 by loewis. This issue is now closed.

Files
File name Uploaded Description Edit
localtime.zip rogerbinns, 2008-07-07 02:26 Trivial extension showing issue with localtime(), msvc9 and Python 2.6
Messages (12)
msg69361 - (view) Author: Roger Binns (rogerbinns) Date: 2008-07-07 00:16
My extension (apsw) builds and runs just fine on Linux, Mac and Windows
for Python 2.3, 2.4 and 2.5.  For Linux and Mac Python 2.6 beta 1 and
Python 3.0 beta 1 also work just fine.  However on Windows using MinGW
and Python 2.6 beta 1 and Python 3.0 beta 1 the pyd fails to load
claiming the "specified procedure cannot be found".  The compile/link is
just fine and pexports shows the procedure is present.  

# Compile lines
c:/python26/python setup.py build --compile=mingw32 install
c:\mingw\bin\gcc.exe -mno-cygwin -mdll -O -Wall -DSQLITE_THREADSAFE=1
-DNDEBUG=1 -DEXPERIMENTAL=1
-DAPSW_USE_SQLITE_AMALGAMATION=\"C:\apsw\sqlite3.c\"
-Ic:\python26\include -Ic:\python26\PC -c apsw.c -o
build\temp.win32-2.6\Release\apsw.o
writing build\temp.win32-2.6\Release\apsw.def
c:\mingw\bin\gcc.exe -mno-cygwin -shared -s
build\temp.win32-2.6\Release\apsw.o
build\temp.win32-2.6\Release\apsw.def -Lc:\python26\libs
-Lc:\python26\PCbuild -lpython26 -lmsvcr90 -o build\lib.win32-2.6\apsw.pyd
# Running
ImportError: DLL load failed: The specified procedure could not be found.
# Pexports
C:\apsw>pexports build\lib.win32-2.6\apsw.pyd
LIBRARY apsw.pyd
EXPORTS
initapsw

When using Python 3.0 things are substantially the same except the init
function is PyInit_apsw.

MinGW was installed using the default configuration with the automated
installer http://sourceforge.net/forum/forum.php?forum_id=817299  The
libmsvcr90.a is part of MinGW found in c:\MinGW\lib
msg69364 - (view) Author: Roger Binns (rogerbinns) Date: 2008-07-07 01:16
I figured maybe it was something to do with MSVC 90 dlls.  

<pre>
C:\apsw>dir \*msvc*90* /s
 Volume in drive C has no label.
 Volume Serial Number is F4A5-1661

 Directory of C:\MinGW\lib

12/27/2007  08:23 AM           554,136 libmsvcr90.a
12/27/2007  08:23 AM           555,910 libmsvcr90d.a
               2 File(s)      1,110,046 bytes

 Directory of
C:\WINDOWS\WinSxS\x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_x-ww_d08d0375

11/06/2007  08:23 PM           224,768 msvcm90.dll
11/07/2007  01:19 AM           568,832 msvcp90.dll
11/07/2007  01:19 AM           655,872 msvcr90.dll
               3 File(s)      1,449,472 bytes

 Directory of
C:\WINDOWS\WinSxS\x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.30411.0_x-ww_71382c73

04/10/2008  10:52 PM           225,280 msvcm90.dll
04/11/2008  04:32 AM           572,928 msvcp90.dll
04/11/2008  04:32 AM           655,872 msvcr90.dll
               3 File(s)      1,454,080 bytes

     Total Files Listed:
               8 File(s)      4,013,598 bytes
               0 Dir(s)  22,924,242,944 bytes free
</pre>

Process Explorer shows Python 2.6 exe using MSVCR90.dll from the
71382c73 WinSxS directory.  Using Process Monitor for the 'import apsw'
invocation shows the cp437 encoding being looked for and found, then the
apsw.pyd being looked for, found, opened, LoadImage, ReadFile and
CloseFile.  Then a file named angle bracket stdin angle bracket it
looked for and not found (presumably to show the error message).
msg69367 - (view) Author: Roger Binns (rogerbinns) Date: 2008-07-07 02:26
I can't prove it since Python gives no further information than a
procedure cannot be found, but using a bunch of other tools I think this
may be due at least to the use of localtime() and it not being present
in the msvcr90.dll but is in the vc version 7 dlls which is why earlier
versions of Python have no issue.

If the above is the case then I have no idea where the actual underlying
cause lies.  Is MinGW missing header information that should mangle
localtime() to one of the other variants used by msvc (_s suffix, _
prefix, 32/64 in there somewhere)?  Is the Python header causing issues?

Attached is a trivial extension providing the issue with localtime.
msg69370 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-07-07 04:29
Can you please check the event log at the point of failure, and report
any events it may have recorded?
msg69373 - (view) Author: Roger Binns (rogerbinns) Date: 2008-07-07 04:44
I cleared all event categories, and then ran Python followed by the
import (which fails).  No events in any category appeared.
msg69380 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-07-07 05:17
Do you have a copy of depends.exe? If so, how does it resolve the DLLs
referenced in apsw.pyd?
msg69381 - (view) Author: Roger Binns (rogerbinns) Date: 2008-07-07 05:57
I didn't have a copy of depends.exe since it doesn't appear to come with
MinGW.  System is basically VirtualBox VM with fresh install of XP Pro
SP2, upgraded to SP3 and TortoiseSVN, Firefox, Xemacs, MinGW and Python
versions installed.

I found a GUI program named depends which is what I used to figure out
that the issue appeared to be localtime() missing in msvcr90.dll.  BTW
did you notice msg69367 and the attached file that shows the problem in
a trivial test case?

The gui depends program found the following issues:

- No localtime in msvcr90.dll
- DWMAPI.DLL not found (not mentioned on link line so no idea why
depends wants it)
- MPR.DLL & SHLWAPI.DLL are red but I don't know why (again not
mentioned on link line)

I think that the localtime() symbol is the underlying cause.  However
this isn't exactly Python's fault (Python's headers don't do anything to
the localtime symbol but there may be defines that cause "secure crt" to
be used which wants localtime_s to be used instead).  Maybe MinGW is
supposed to be redefining localtime to one of the _/32/64/s variants but
isn't.

In any event the net result is that if a Python extension uses current
MinGW and localtime() then it will compile and link correctly on Python
2.3, 2.4, 2.5, 2.6b1 & 3.0b1.  However it won't run on 2.6b1 or 3.0b1
due to some sort of mismatch between those versions of Python using new
MSVC runtime and/or MinGW.
msg69382 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2008-07-07 12:36
It seems that mingw is unable to compile the simplest program containing
a call to localtime():

$ echo "#include <time.h>
> int main() { localtime(NULL); }" > t.c
$ gcc -mno-cygwin t.c -lmsvcr90

Then starting "a.exe" displays a modal box with the message: "The
procedure entry point localtime could not be located in the dynamic link
library msvcr90.dll."

So the problem is not python-specific, and should be reported to Mingw.

However, I discovered that it works if you include time.h this way:

   #define __MSVCRT_VERSION__ 0x0700    /* whatever above 0x0601 */
   #include <time.h>
   #define time_t __time64_t
   #define localtime _localtime64
   #define time _time64
msg69393 - (view) Author: Roger Binns (rogerbinns) Date: 2008-07-07 19:18
In MinGW's defense the MSVC 7 dll does include localtime.  Since
Python/distutils only says that MSVC 9 dll is being used at link time,
how exactly is MinGW (or any other code) supposed to know that localtime
should be provided some other way?  Maybe distutils should be defining 
__MSVCRT_VERSION__ to 0x0900 for the compile phase as well for Python
2.6/3.0.  If that was the case then at least I could work around the
issue (the localtime call isn't actually in my code but a library I
statically link in) and still work correctly on earlier Pythons which
don't need any workarounds.
msg69394 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-07-07 20:19
> BTW
> did you notice msg69367 and the attached file that shows the problem in
> a trivial test case?

I didn't understand it, so I ignored it. What use of localtime, by whom,
what tools? The mentioning of localtime seems to come out of nowhere.
(later messages clarified these, apparently: if localtime is called in
an extension, then the extension won't load, otherwise, it will load)

> The gui depends program found the following issues:

That's likely the one that I was asking for.

> In any event the net result is that if a Python extension uses current
> MinGW and localtime() then it will compile and link correctly on Python
> 2.3, 2.4, 2.5, 2.6b1 & 3.0b1.  However it won't run on 2.6b1 or 3.0b1
> due to some sort of mismatch between those versions of Python using new
> MSVC runtime and/or MinGW.

That sounds rather like a bug in MinGW. Can you ask on one of their
lists?
msg69404 - (view) Author: Roger Binns (rogerbinns) Date: 2008-07-07 23:39
I will ask on the MinGW lists.  I am still curious as to how MinGW is
supposed to know which MSVC library will be used at compile time since
distutils doesn't tell it until link time.

As a seperate issue Python isn't too helpful when an extension doesn't
load :-)
msg69406 - (view) Author: Roger Binns (rogerbinns) Date: 2008-07-08 01:48
I guess you can close this now.  Unfortunately SourceForge goes out of
its way to not make an easy link for the MinGW mailing list but you can
see the messages on 8th July 2008:
 
http://sourceforge.net/mailarchive/forum.php?forum_name=mingw-users&viewmonth=200807&viewday=8

Basically MinGW erroneously ships a .lib saying localtime is in
MSVCR90.DLL when it isn't.  That means there is no link failure but the
pyd will fail to load without information as to why.  Fixing the .lib
means that localtime is picked up from MSVCRT.DLL (ie no version in
name) and everything appears to work well.

The MinGW project also claims they only support MSVCRT.DLL and not any
of the numbered ones, so basically it is luck they work.  Noone pointed
out any compile time directives to direct versioning of MSVCRT.

I guess this is more fuel for the semi-regular flame war about free
software and free compilers on Windows, but not in this ticket!
History
Date User Action Args
2008-07-08 05:04:44loewissetstatus: open -> closed
resolution: wont fix
versions: + 3rd party
2008-07-08 01:48:45rogerbinnssetmessages: + msg69406
2008-07-07 23:39:03rogerbinnssetmessages: + msg69404
2008-07-07 20:19:33loewissetmessages: + msg69394
2008-07-07 19:18:51rogerbinnssetmessages: + msg69393
2008-07-07 12:36:05amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg69382
2008-07-07 05:57:24rogerbinnssetmessages: + msg69381
2008-07-07 05:17:18loewissetmessages: + msg69380
2008-07-07 04:44:06rogerbinnssetmessages: + msg69373
2008-07-07 04:29:42loewissetnosy: + loewis
messages: + msg69370
2008-07-07 02:26:58rogerbinnssetfiles: + localtime.zip
messages: + msg69367
2008-07-07 01:16:05rogerbinnssetmessages: + msg69364
2008-07-07 00:16:19rogerbinnscreate