classification
Title: 3.7.1 compile failure on CentOS 6.10; _ctypes did not build
Type: compile error Stage: resolved
Components: Build, ctypes, Installation Versions: Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: izbyshev, lana.deere, zach.ware
Priority: normal Keywords:

Created on 2018-11-05 20:03 by lana.deere, last changed 2018-11-06 23:55 by vstinner. This issue is now closed.

Files
File name Uploaded Description Edit
Log.python.install.xz lana.deere, 2018-11-05 20:03 Output captured during the attempted build
Messages (4)
msg329317 - (view) Author: (lana.deere) Date: 2018-11-05 20:03
When I try to compile 3.7.1 on CentOS6.10 it fails to build _ctypes, but I can't find any indication of why.  There are several mentions of _ctypes during compiles,

building '_ctypes_test' extension
creating build/temp.linux-x86_64-3.7/home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Modules/_ctypes
/usr/bin/gcc -pthread -fPIC -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I./Include -I. -I/usr/local/include -I/home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Include -I/home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1 -c /home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Modules/_ctypes/_ctypes_test.c -o build/temp.linux-x86_64-3.7/home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Modules/_ctypes/_ctypes_test.o
/usr/bin/gcc -pthread -shared build/temp.linux-x86_64-3.7/home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Modules/_ctypes/_ctypes_test.o -L/usr/local/lib -lm -o build/lib.linux-x86_64-3.7/_ctypes_test.cpython-37m-x86_64-linux-gnu.so

After that, it gets to listing modules later and says:

INFO: Could not locate ffi libs and/or headers

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_ssl
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc                  atexit                pwd
time


Failed to build these modules:
_ctypes               _uuid


Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381


I can see the problem with _uuid, it's a conflict between headers.


In file included from /home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Modules/_uuidmodule.c:8:
/usr/include/uuid.h:94: error: conflicting types for ‘uuid_t’
/usr/include/uuid/uuid.h:44: note: previous declaration of ‘uuid_t’ was here
/usr/include/uuid.h:107: error: conflicting types for ‘uuid_compare’
/usr/include/uuid/uuid.h:73: note: previous declaration of ‘uuid_compare’ was here
/home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Modules/_uuidmodule.c: In function ‘py_uuid_generate_time_safe’:
/home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Modules/_uuidmodule.c:15: error: storage size of ‘uuid’ isn’t known
/home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Modules/_uuidmodule.c:15: warning: unused variable ‘uuid’


However, I see nothing indicating building the _ctypes extension other than the ctypes_test I mentioned already above.  Eventually the build aborts because of the missing _ctypes.


Generating grammar tables from /home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1-install/lib/python3.7/lib2to3/PatternGrammar.txt
Writing grammar tables to /home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1-install/lib/python3.7/lib2to3/PatternGrammar3.7.1.final.0.pickle
if test "xupgrade" != "xno"  ; then \
        case upgrade in \
            upgrade) ensurepip="--upgrade" ;; \
            install|*) ensurepip="" ;; \
        esac; \
         ./python -E -m ensurepip \
            $ensurepip --root=/ ; \
    fi
Traceback (most recent call last):
  File "/home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Lib/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Lib/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Lib/ensurepip/__main__.py", line 5, in <module>
    sys.exit(ensurepip._main())
  File "/home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Lib/ensurepip/__init__.py", line 204, in _main
    default_pip=args.default_pip,
  File "/home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Lib/ensurepip/__init__.py", line 117, in _bootstrap
    return _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Lib/ensurepip/__init__.py", line 27, in _run_pip
    import pip._internal
  File "/tmp/tmpz6ocn29e/pip-10.0.1-py2.py3-none-any.whl/pip/_internal/__init__.py", line 42, in <module>
  File "/tmp/tmpz6ocn29e/pip-10.0.1-py2.py3-none-any.whl/pip/_internal/cmdoptions.py", line 16, in <module>
  File "/tmp/tmpz6ocn29e/pip-10.0.1-py2.py3-none-any.whl/pip/_internal/index.py", line 25, in <module>
  File "/tmp/tmpz6ocn29e/pip-10.0.1-py2.py3-none-any.whl/pip/_internal/download.py", line 39, in <module>
  File "/tmp/tmpz6ocn29e/pip-10.0.1-py2.py3-none-any.whl/pip/_internal/utils/glibc.py", line 3, in <module>
  File "/home/twk/twkpers/opensrc/python-3.7.1/Python-3.7.1/Lib/ctypes/__init__.py", line 7, in <module>
    from _ctypes import Union, Structure, Array
ModuleNotFoundError: No module named '_ctypes'
make: *** [install] Error 1


I have attached the complete log captured from the build to this report.  Any suggestions about the cause of this would be appreciated.  Note that I had no problem building 3.6.1 on the same system.

$ python3
Python 3.6.1 (default, Apr 21 2017, 11:56:37) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-18)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import _ctypes
>>> help(_ctypes)
Help on module _ctypes:
[...]
msg329384 - (view) Author: Alexey Izbyshev (izbyshev) * (Python triager) Date: 2018-11-06 20:36
The in-tree copy of libffi was removed in #27979 (between 3.6 and 3.7) for all platforms except Windows and macOS. Therefore, you need libffi development package installed in your system to build CPython 3.7. 

As for PIP, personally, I think that's an overkill to require ctypes, especially at CPython build time. If you don't want to build ctypes, I believe you may use '--without-ensurepip' configure option as a workaround.
msg329396 - (view) Author: (lana.deere) Date: 2018-11-06 22:59
I did 'yum install libffi-devel' on my machine as suggested and that cured the ctypes issue.  This issue can be closed; if I run into any
other problems I can open other issues for them.
msg329397 - (view) Author: (lana.deere) Date: 2018-11-06 22:59
I did 'yum install libffi-devel' on my machine as suggested and that cured the ctypes issue.  This issue can be closed; if I run into any
other problems I can open other issues for them.
History
Date User Action Args
2018-11-06 23:55:13vstinnersetstatus: closed
resolution: fixed
stage: resolved
2018-11-06 22:59:53lana.deeresetmessages: + msg329397
2018-11-06 22:59:11lana.deeresetstatus: open -> (no value)

messages: + msg329396
2018-11-06 20:36:09izbyshevsetnosy: + izbyshev, zach.ware
messages: + msg329384
2018-11-05 20:03:36lana.deerecreate