classification
Title: _ctypes on ppc64: libffi/src/powerpc/linux64.o: ABI version 1 is not compatible with ABI version 2 output
Type: Stage:
Components: Tests Versions: Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: doko, meador.inge, snehal, vstinner
Priority: normal Keywords:

Created on 2014-08-06 12:37 by snehal, last changed 2014-08-17 08:17 by doko. This issue is now closed.

Messages (12)
msg224930 - (view) Author: snehal (snehal) Date: 2014-08-06 12:37
On ppc64le architecture I see following error with 
TAR :  https://www.python.org/ftp/python/3.4.1/Python-3.4.1.tar.xz


======================================================================
FAIL: test_with_pip (test.test_venv.EnsurePipTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ubuntu/python/Python-3.4.1/Lib/test/test_venv.py", line 352, in test_with_pip
    with_pip=True)
subprocess.CalledProcessError: Command '['/tmp/tmpge5lfrua/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/python/Python-3.4.1/Lib/test/test_venv.py", line 358, in test_with_pip
    self.fail(msg.format(exc, details))
AssertionError: Command '['/tmp/tmpge5lfrua/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1

**Subprocess Output**
Traceback (most recent call last):
  File "/home/ubuntu/python/Python-3.4.1/Lib/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/ubuntu/python/Python-3.4.1/Lib/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/ubuntu/python/Python-3.4.1/Lib/ensurepip/__main__.py", line 4, in <module>
    ensurepip._main()
  File "/home/ubuntu/python/Python-3.4.1/Lib/ensurepip/__init__.py", line 209, in _main
    default_pip=args.default_pip,
  File "/home/ubuntu/python/Python-3.4.1/Lib/ensurepip/__init__.py", line 116, in bootstrap
    _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/home/ubuntu/python/Python-3.4.1/Lib/ensurepip/__init__.py", line 40, in _run_pip
    import pip
  File "/tmp/tmpn9zaabpk/pip-1.5.6-py2.py3-none-any.whl/pip/__init__.py", line 9, in <module>
  File "/tmp/tmpn9zaabpk/pip-1.5.6-py2.py3-none-any.whl/pip/log.py", line 9, in <module>
  File "/tmp/tmpn9zaabpk/pip-1.5.6-py2.py3-none-any.whl/pip/_vendor/colorama/__init__.py", line 2, in <module>
  File "/tmp/tmpn9zaabpk/pip-1.5.6-py2.py3-none-any.whl/pip/_vendor/colorama/initialise.py", line 5, in <module>
  File "/tmp/tmpn9zaabpk/pip-1.5.6-py2.py3-none-any.whl/pip/_vendor/colorama/ansitowin32.py", line 6, in <module>
  File "/tmp/tmpn9zaabpk/pip-1.5.6-py2.py3-none-any.whl/pip/_vendor/colorama/winterm.py", line 2, in <module>
  File "/tmp/tmpn9zaabpk/pip-1.5.6-py2.py3-none-any.whl/pip/_vendor/colorama/win32.py", line 7, in <module>
  File "/home/ubuntu/python/Python-3.4.1/Lib/ctypes/__init__.py", line 7, in <module>
    from _ctypes import Union, Structure, Array
ImportError: No module named '_ctypes'


----------------------------------------------------------------------
Ran 13 tests in 0.561s

FAILED (failures=1)
test test_venv failed
make: *** [test] Error
msg224931 - (view) Author: snehal (snehal) Date: 2014-08-06 12:39
I have libffi installed on system

root@beta1:/home/ubuntu/python/Python-3.4.1# dpkg -l | grep libffi
ii  libffi-dev:ppc64el                                          3.1~rc1+r3.0.13-12                         ppc64el      Foreign Function Interface library (development files)
ii  libffi6:ppc64el                                             3.1~rc1+r3.0.13-12                         ppc64el      Foreign Function Interface library runtime
ii  libffi6-dbg:ppc64el                                         3.1~rc1+r3.0.13-12                         ppc64el      Foreign Function Interface library runtime (debug symbols)
msg224959 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-08-06 18:50
When you built 3.4.1, did the _ctypes module build correctly?  What happens when you try to import ctypes?

    python3.4 -c 'import ctypes'
msg224993 - (view) Author: snehal (snehal) Date: 2014-08-07 06:38
When I gave following command, I see no error  

root@beta1:/home/ubuntu/python/Python-3.4.1# python3.4 -c 'import ctypes'
root@beta1:/home/ubuntu/python/Python-3.4.1# 

I do see ctypes in root@beta1:/home/ubuntu/python/Python-3.4.1/Lib# folder
msg224994 - (view) Author: snehal (snehal) Date: 2014-08-07 06:39
Also ctypes contains following 

root@beta1:/home/ubuntu/python/Python-3.4.1/Lib/ctypes# ls
_endian.py  __init__.py  macholib  __pycache__  test  util.py  wintypes.py
msg224996 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-08-07 07:37
OK. Does test_ctypes run successfully?

  cd /home/ubuntu/python/Python-3.4.1
  ./python -E -S -m test -v test_ctypes

and then does test_venv run successfully in isolation?

  cd /home/ubuntu/python/Python-3.4.1
  ./python -E -S -m test -v test_venv

Also, what ./configure command arguments did you use to build it?

It's hard to tell what might be failing otherwise other than the _ctypes extension build failing.
msg225060 - (view) Author: snehal (snehal) Date: 2014-08-08 05:59
1oot@beta1:/home/ubuntu/python/Python-3.4.1#  cd /home/ubuntu/python/Python-3.4. 
esot@beta1:/home/ubuntu/python/Python-3.4.1# ./python -E -S -m test -v test_ctyp 
== CPython 3.4.1 (default, Aug 7 2014, 02:08:49) [GCC 4.8.2]
==   Linux-3.13.0-32-generic-ppc64le-with-debian-jessie-sid little-endian
==   hash algorithm: siphash24 64bit
==   /home/ubuntu/python/Python-3.4.1/build/test_python_1245
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=1, ignore_environment=1, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
[1/1] test_ctypes
test_ctypes skipped -- No module named '_ctypes'
1 test skipped:
    test_ctypes

-----------------------------

== CPython 3.4.1 (default, Aug 7 2014, 02:08:49) [GCC 4.8.2]-m test -v test_venv 
==   Linux-3.13.0-32-generic-ppc64le-with-debian-jessie-sid little-endian
==   hash algorithm: siphash24 64bit
==   /home/ubuntu/python/Python-3.4.1/build/test_python_1248
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=1, ignore_environment=1, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
[1/1] test_venv
test_defaults (test.test_venv.BasicTest) ... ok
test_executable (test.test_venv.BasicTest) ... ok
test_executable_symlinks (test.test_venv.BasicTest) ... ok
test_isolation (test.test_venv.BasicTest) ... ok
test_overwrite_existing (test.test_venv.BasicTest) ... ok
test_prefixes (test.test_venv.BasicTest) ... ok
test_symlinking (test.test_venv.BasicTest) ... ok
test_unoverwritable_fails (test.test_venv.BasicTest) ... ok
test_upgrade (test.test_venv.BasicTest) ... ok
test_devnull_exists_and_is_empty (test.test_venv.EnsurePipTest) ... ok
test_explicit_no_pip (test.test_venv.EnsurePipTest) ... ok
test_no_pip_by_default (test.test_venv.EnsurePipTest) ... 
ok
test_with_pip (test.test_venv.EnsurePipTest) ... FAIL

======================================================================
FAIL: test_with_pip (test.test_venv.EnsurePipTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ubuntu/python/Python-3.4.1/Lib/test/test_venv.py", line 352, in test_with_pip
    with_pip=True)
subprocess.CalledProcessError: Command '['/tmp/tmp1_atkzms/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/python/Python-3.4.1/Lib/test/test_venv.py", line 358, in test_with_pip
    self.fail(msg.format(exc, details))
AssertionError: Command '['/tmp/tmp1_atkzms/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1

**Subprocess Output**
Traceback (most recent call last):
  File "/home/ubuntu/python/Python-3.4.1/Lib/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/ubuntu/python/Python-3.4.1/Lib/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/ubuntu/python/Python-3.4.1/Lib/ensurepip/__main__.py", line 4, in <module>
    ensurepip._main()
  File "/home/ubuntu/python/Python-3.4.1/Lib/ensurepip/__init__.py", line 209, in _main
    default_pip=args.default_pip,
  File "/home/ubuntu/python/Python-3.4.1/Lib/ensurepip/__init__.py", line 116, in bootstrap
    _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/home/ubuntu/python/Python-3.4.1/Lib/ensurepip/__init__.py", line 40, in _run_pip
    import pip
  File "/tmp/tmplglzwy2e/pip-1.5.6-py2.py3-none-any.whl/pip/__init__.py", line 9, in <module>
  File "/tmp/tmplglzwy2e/pip-1.5.6-py2.py3-none-any.whl/pip/log.py", line 9, in <module>
  File "/tmp/tmplglzwy2e/pip-1.5.6-py2.py3-none-any.whl/pip/_vendor/colorama/__init__.py", line 2, in <module>
  File "/tmp/tmplglzwy2e/pip-1.5.6-py2.py3-none-any.whl/pip/_vendor/colorama/initialise.py", line 5, in <module>
  File "/tmp/tmplglzwy2e/pip-1.5.6-py2.py3-none-any.whl/pip/_vendor/colorama/ansitowin32.py", line 6, in <module>
  File "/tmp/tmplglzwy2e/pip-1.5.6-py2.py3-none-any.whl/pip/_vendor/colorama/winterm.py", line 2, in <module>
  File "/tmp/tmplglzwy2e/pip-1.5.6-py2.py3-none-any.whl/pip/_vendor/colorama/win32.py", line 7, in <module>
  File "/home/ubuntu/python/Python-3.4.1/Lib/ctypes/__init__.py", line 7, in <module>
    from _ctypes import Union, Structure, Array
ImportError: No module named '_ctypes'


----------------------------------------------------------------------
Ran 13 tests in 0.574s

FAILED (failures=1)
test test_venv failed
1 test failed:
    test_venv
msg225061 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-08-08 06:06
Thanks. So apparently the _ctypes module failed to build.  You should rerun "make" in your build directory and find out why _ctypes.so fails to build.  If necessary, show the results here.
msg225260 - (view) Author: snehal (snehal) Date: 2014-08-13 07:05
/home/python/Python-3.4.1/Modules/_ctypes/libffi/config.guess: unable to guess system type

This script, last modified 2012-12-29, has failed to recognize
the operating system you are using. It is advised that you
download the most up to date version of the config scripts from

  http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD
and
  http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.sub;hb=HEAD

If the version you run (/home/python/Python-3.4.1/Modules/_ctypes/libffi/config.guess) is already up to date, please
send the following data and any information you think might be
pertinent to <config-patches@gnu.org> in order to provide the needed
information to handle your system.

config.guess timestamp = 2012-12-29

uname -m = ppc64le
uname -r = 3.13-1-powerpc64le
uname -s = Linux
uname -v = #1 SMP Debian 3.13.4-1 (2014-02-22)

/usr/bin/uname -p = 
/bin/uname -X     = 

hostinfo               = 
/bin/universe          = 
/usr/bin/arch -k       = 
/bin/arch              = 
/usr/bin/oslevel       = 
/usr/convex/getsysinfo = 

UNAME_MACHINE = ppc64le
UNAME_RELEASE = 3.13-1-powerpc64le
UNAME_SYSTEM  = Linux
UNAME_VERSION = #1 SMP Debian 3.13.4-1 (2014-02-22)
configure: error: cannot guess build type; you must specify one

----------------------
I tried modifying config.guess for ppc65le , and did run configure and make again... after which I got following 


Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
/usr/bin/ld: build/temp.linux-ppc64le-3.4/home/python/Python-3.4.1/Modules/_ctypes/libffi/src/powerpc/linux64.o: ABI version 1 is not compatible with ABI version 2 output
/usr/bin/ld: failed to merge target specific data of file build/temp.linux-ppc64le-3.4/home/python/Python-3.4.1/Modules/_ctypes/libffi/src/powerpc/linux64.o
/usr/bin/ld: build/temp.linux-ppc64le-3.4/home/python/Python-3.4.1/Modules/_ctypes/libffi/src/powerpc/linux64_closure.o: ABI version 1 is not compatible with ABI version 2 output
/usr/bin/ld: failed to merge target specific data of file build/temp.linux-ppc64le-3.4/home/python/Python-3.4.1/Modules/_ctypes/libffi/src/powerpc/linux64_closure.o
msg225393 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-08-16 12:22
This issue is unrelated to venv, it's an issue between libffi and ctypes.
msg225397 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-08-16 12:45
This issue may be related to the issue #22176.
msg225429 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2014-08-17 08:17
this is fixed in the 3.4 branch (post 3.4.1) and on all active branches. you should be able to use --with-system-libffi in your current environment (maybe after running: apt-get build-dep python3.4.
History
Date User Action Args
2014-08-17 08:17:19dokosetstatus: open -> closed
resolution: fixed
messages: + msg225429
2014-08-16 20:12:04ned.deilysetnosy: + doko, meador.inge, - ned.deily
2014-08-16 12:45:46vstinnersetmessages: + msg225397
2014-08-16 12:22:29vstinnersetnosy: + vstinner
messages: + msg225393
2014-08-16 12:22:07vstinnersettitle: FAIL: test_with_pip (test.test_venv.EnsurePipTest) -> _ctypes on ppc64: libffi/src/powerpc/linux64.o: ABI version 1 is not compatible with ABI version 2 output
2014-08-13 07:05:55snehalsetmessages: + msg225260
2014-08-08 06:06:31ned.deilysetmessages: + msg225061
2014-08-08 05:59:58snehalsetmessages: + msg225060
2014-08-07 07:37:33ned.deilysetmessages: + msg224996
2014-08-07 06:39:54snehalsetmessages: + msg224994
2014-08-07 06:38:47snehalsetmessages: + msg224993
2014-08-06 18:50:53ned.deilysetnosy: + ned.deily
messages: + msg224959
2014-08-06 12:39:27snehalsetmessages: + msg224931
2014-08-06 12:37:48snehalcreate