classification
Title: AIX: ModuleNotFoundError: No module named '_ctypes' - make install fails
Type: Stage: resolved
Components: Build Versions: Python 3.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: make install fails: no module _ctypes
View: 31652
Assigned To: Nosy List: Michael.Felt, zach.ware
Priority: normal Keywords:

Created on 2017-11-20 20:38 by Michael.Felt, last changed 2017-11-22 09:03 by Michael.Felt. This issue is now closed.

Messages (6)
msg306581 - (view) Author: Michael Felt (Michael.Felt) * Date: 2017-11-20 20:38
after git clone from master:
make install fails to complete with:

Important - python is not installed already.
FYI: also note - build completes successfully with 2.7.14 (will download and check other python3 versions)

        if test "xupgrade" != "xno"  ; then  case upgrade in  upgrade) ensurepip="--upgrade" ;;  install|*) ensurepip="" ;;  esac;   ./python -E -m ensurepip  $ensurepip --root=/ ;  fi
Traceback (most recent call last):
  File "/data/prj/python/git/src/python3-3.7.0.1/Lib/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/data/prj/python/git/src/python3-3.7.0.1/Lib/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/data/prj/python/git/src/python3-3.7.0.1/Lib/ensurepip/__main__.py", line 5, in <module>
    sys.exit(ensurepip._main())
  File "/data/prj/python/git/src/python3-3.7.0.1/Lib/ensurepip/__init__.py", line 204, in _main
    default_pip=args.default_pip,
  File "/data/prj/python/git/src/python3-3.7.0.1/Lib/ensurepip/__init__.py", line 117, in _bootstrap
    return _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/data/prj/python/git/src/python3-3.7.0.1/Lib/ensurepip/__init__.py", line 27, in _run_pip
    import pip
  File "/tmp/tmpg0jc_9p6/pip-9.0.1-py2.py3-none-any.whl/pip/__init__.py", line 28, in <module>
  File "/tmp/tmpg0jc_9p6/pip-9.0.1-py2.py3-none-any.whl/pip/vcs/mercurial.py", line 9, in <module>
  File "/tmp/tmpg0jc_9p6/pip-9.0.1-py2.py3-none-any.whl/pip/download.py", line 36, in <module>
  File "/tmp/tmpg0jc_9p6/pip-9.0.1-py2.py3-none-any.whl/pip/utils/glibc.py", line 4, in <module>
  File "/data/prj/python/git/src/python3-3.7.0.1/Lib/ctypes/__init__.py", line 7, in <module>
    from _ctypes import Union, Structure, Array
ModuleNotFoundError: No module named '_ctypes'
make: 1254-004 The error code from the last command is 1.
msg306582 - (view) Author: Michael Felt (Michael.Felt) * Date: 2017-11-20 20:55
FYI: version 3.6.3 compiles and builds without this issue.

+ make install DESTDIR=/var/aixtools/python/Python/3.6.3.0 > .buildaix/install.out
+ mkinstallp.ksh /var/aixtools/python/Python/3.6.3.0 > .buildaix/mkinstallp.out
msg306583 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2017-11-20 20:58
Do you have libffi (and its development headers) installed where the compiler can find it?  Python 3.7 no longer bundles a copy of libffi (see #27979).
msg306586 - (view) Author: Michael Felt (Michael.Felt) * Date: 2017-11-20 21:32
no. I'll install what I have atm.

Too bad - as now it will have an install dependency.

re: https://bugs.python.org/issue27976#msg274628 -- seems a warning that libffi is not available might be useful after all.

Question: is ffi still bundled for Darwin - even in 3.7? If yes, maybe it should also be bundled for AIX.

In any case, just installing a libffi.a library fixed the install (i.e., make; make install)
msg306688 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2017-11-21 21:01
You removed me from the nosy list, so I didn't see your reply until now when I happened to refresh the tab that happened to still be open :)

The warning I was talking about in that message was a warning that libffi was found but is outdated, which wouldn't be any help to you here.  There are already messages in the build log about libffi being missing and/or _ctypes not being built.

We do still bundle libffi for OSX and Windows, but we're trying to remove them.  I definitely don't want to add another bundled version.

You already have some install dependencies, or are missing some optional modules.  In particular, the lzma module depends on the xz package, readline depends on libreadline, _ssl depends on OpenSSL, _bz2 depends on bzip2, etc.  There is a message at the end of `make` about which modules were not built and why.

Another workaround you could use (at the expense of not having pip installed) is to configure with `./configure --without-ensurepip`.

Finally, since we've nailed down that lack of libffi was in fact the source of the problem, this is a duplicate of #31652.
msg306703 - (view) Author: Michael Felt (Michael.Felt) * Date: 2017-11-22 09:03
If I removed you from nosy - purely accidental - probably because you answered and I still had a dialog open with you not in it.

My apologies in any case. I'll be more careful should something like that happen again.

I am aware of the other dependencies - the only one I really care about for "master" is _ssl. Surprised that it would be missing. Will need some research on my side.

One of my "packaging" goals for python is to have it install by itself.
So I'll look at linking libffi statically so another fileset is not needed. (And remember to check for new libffi code when a new python version is released).

And - sorry for the duplicate!
History
Date User Action Args
2017-11-22 09:03:53Michael.Feltsetmessages: + msg306703
2017-11-21 21:01:54zach.waresetstatus: open -> closed

superseder: make install fails: no module _ctypes
versions: - Python 3.8
nosy: + zach.ware

messages: + msg306688
resolution: duplicate
stage: resolved
2017-11-20 21:32:32Michael.Feltsetnosy: - zach.ware

messages: + msg306586
versions: + Python 3.8
2017-11-20 20:58:09zach.waresetnosy: + zach.ware

messages: + msg306583
versions: - Python 3.8
2017-11-20 20:55:25Michael.Feltsetmessages: + msg306582
versions: + Python 3.7
2017-11-20 20:38:04Michael.Feltcreate