classification
Title: Compilation on MINGW64 fails (CODESET,wcstok,...)
Type: compile error Stage:
Components: Build, Interpreter Core, Windows Versions: Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: eamanu, erikjanss, eryksun, lazka, paul.moore, steve.dower, tim.golden, vengelson, zach.ware
Priority: normal Keywords:

Created on 2019-08-09 11:53 by vengelson, last changed 2019-08-17 14:02 by erikjanss.

Messages (6)
msg349282 - (view) Author: Vadim Engelson (vengelson) Date: 2019-08-09 11:53
Compilation on MINGW64 fails (CODESET,wcstok,...)
I am using the latest MINGW64 (http://repo.msys2.org/distrib/x86_64/msys2-x86_64-20190524.exe)
Versions: Python-3.7.2, Python-3.8.0b3

$ gcc -v
Using built-in specs.
COLLECT_GCC=C:\msys64\mingw64\bin\gcc.exe
COLLECT_LTO_WRAPPER=C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.1.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
gcc version 9.1.0 (Rev3, Built by MSYS2 project)

Result of make:

Python/initconfig.c: In function 'config_get_locale_encoding':
Python/initconfig.c:1427:28: error: implicit declaration of function 'nl_langinfo' [-Werror=implicit-function-declaration]
 1427 |     const char *encoding = nl_langinfo(CODESET);
      |                            ^~~~~~~~~~~
Python/initconfig.c:1427:40: error: 'CODESET' undeclared (first use in this function); did you mean 'ECONNRESET'?
 1427 |     const char *encoding = nl_langinfo(CODESET);
      |                                        ^~~~~~~
      |                                        ECONNRESET
Python/initconfig.c:1427:40: note: each undeclared identifier is reported only once for each function it appears in
Python/initconfig.c: In function 'config_init_env_warnoptions':
Python/initconfig.c:1992:18: error: too many arguments to function 'wcstok'
 1992 | #  define WCSTOK wcstok
      |                  ^~~~~~
Python/initconfig.c:2015:20: note: in expansion of macro 'WCSTOK'
 2015 |     for (warning = WCSTOK(env, L",", &context);
      |                    ^~~~~~
In file included from ./Include/Python.h:30,
                 from Python/initconfig.c:1:
C:/msys64/mingw64/x86_64-w64-mingw32/include/string.h:147:20: note: declared here
  147 |   wchar_t *__cdecl wcstok(wchar_t * __restrict__ _Str,const wchar_t * __restrict__ _Delim) __MINGW_ATTRIB_DEPRECATED_SEC_WARN;
      |                    ^~~~~~
Python/initconfig.c:1992:18: error: too many arguments to function 'wcstok'
 1992 | #  define WCSTOK wcstok
      |                  ^~~~~~
Python/initconfig.c:2017:20: note: in expansion of macro 'WCSTOK'
 2017 |          warning = WCSTOK(NULL, L",", &context))
      |                    ^~~~~~
In file included from ./Include/Python.h:30,
                 from Python/initconfig.c:1:
C:/msys64/mingw64/x86_64-w64-mingw32/include/string.h:147:20: note: declared here
  147 |   wchar_t *__cdecl wcstok(wchar_t * __restrict__ _Str,const wchar_t * __restrict__ _Delim) __MINGW_ATTRIB_DEPRECATED_SEC_WARN;
      |                    ^~~~~~
cc1.exe: some warnings being treated as errors
make: *** [Makefile:1703: Python/initconfig.o] Error 1
msg349283 - (view) Author: Vadim Engelson (vengelson) Date: 2019-08-09 11:54
(code from Python-3.8.0b3, but Python-3.7.2 had similar issues)
msg349294 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-08-09 16:43
Guessing it needs an extra header file that is implicitly included in the Windows headers?
msg349312 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2019-08-09 19:53
I wasn't aware that CPython builds for MSYS2 out of the box, since it's a POSIX-on-Windows platform like Cygwin. Apparently there are patches that enable it to build, since MSYS2 has Python available.

For Windows, WCSTOK expands to wcstok_s, which takes a context pointer that allows concurrently parsing multiple strings in a single thread. The old function lacks this parameter and instead uses a per-thread static buffer. They're declared as follows:

    wchar_t *wcstok(wchar_t *strToken, const wchar_t *strDelimit);
    wchar_t *wcstok_s(wchar_t *str, const wchar_t *delimiters,
                      wchar_t **context);

Otherwise the WCSTOK macro expands to wcstok, which assumes that POSIX systems use the standard definition [1]:

    wchar_t *wcstok(wchar_t *restrict ws1, const wchar_t *restrict ws2,
                    wchar_t **restrict ptr);

Apparently the version of wcstok declared in your build environment takes only two arguments, like the old insecure function in Windows: 

    wchar_t *__cdecl wcstok(wchar_t * __restrict__ _Str,
                            const wchar_t * __restrict__ _Delim)
    __MINGW_ATTRIB_DEPRECATED_SEC_WARN;

[1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/wcstok.html
msg349337 - (view) Author: Christoph Reiter (lazka) * Date: 2019-08-10 08:16
> I wasn't aware that CPython builds for MSYS2 out of the box, since it's a POSIX-on-Windows platform like Cygwin. Apparently there are patches that enable it to build, since MSYS2 has Python available.

MSYS2 consists of a cygwin like environment and a mingw one. OP is trying to build with mingw.

MSYS2 contains a somewhat heavily patched Python in both environments so it's expected that building plain CPython doesn't work. There are plans to upstream smaller non-MSYS2 specific patches but nothing has come of it so far.
msg349900 - (view) Author: Erik Janssens (erikjanss) * Date: 2019-08-17 14:02
fyi 1 : this issue pops up in multiple places, cfr :

 * bpo-35890
 * bpo-20596

the selection of the wcstok function is based on MS_WINDOWS being
defined, rather than eg. an autoconf check on which function is
available. 

fyi 2 : I've been able to cross compile 3.8 with mingw64 (gcc 7.3), but with a custom meson script instead of autoconf
History
Date User Action Args
2019-08-17 14:02:17erikjansssetnosy: + erikjanss
messages: + msg349900
2019-08-10 08:16:47lazkasetnosy: + lazka
messages: + msg349337
2019-08-09 19:53:33eryksunsetnosy: + eryksun
messages: + msg349312
2019-08-09 16:43:53steve.dowersetmessages: + msg349294
2019-08-09 12:26:57eamanusetnosy: + eamanu
2019-08-09 11:54:58vengelsonsetmessages: + msg349283
2019-08-09 11:53:21vengelsoncreate