Issue29439
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2017-02-04 05:07 by yan12125, last changed 2022-04-11 14:58 by admin. This issue is now closed.
Files | ||||
---|---|---|---|---|
File name | Uploaded | Description | Edit | |
decimal.patch | yan12125, 2017-02-04 05:07 |
Messages (10) | |||
---|---|---|---|
msg286906 - (view) | Author: (yan12125) * | Date: 2017-02-04 05:07 | |
Just like issue21668, _decimal requires -lm on Android. This wasn't fixed because _decimal didn't build before issue26846 lands. More specificially, log10 is called https://hg.python.org/cpython/file/tip/Modules/_decimal/libmpdec/mpdecimal.c#l7862 Added _decimal and Android experts |
|||
msg286960 - (view) | Author: Stefan Krah (skrah) * | Date: 2017-02-04 13:41 | |
Thanks. Strange that on other systems the compilers don't complain (usually they do). |
|||
msg286961 - (view) | Author: Roundup Robot (python-dev) | Date: 2017-02-04 13:59 | |
New changeset b60b46ad8751 by Stefan Krah in branch '3.6': Issue29439: _decimal on Android requires linking with libm. https://hg.python.org/cpython/rev/b60b46ad8751 |
|||
msg286962 - (view) | Author: Roundup Robot (python-dev) | Date: 2017-02-04 14:00 | |
New changeset aa26a5712a80849e968171b71b6bc8d9da3ac163 by Stefan Krah in branch '3.6': Issue29439: _decimal on Android requires linking with libm. https://github.com/python/cpython/commit/aa26a5712a80849e968171b71b6bc8d9da3ac163 |
|||
msg286963 - (view) | Author: Roundup Robot (python-dev) | Date: 2017-02-04 14:00 | |
New changeset aa26a5712a80849e968171b71b6bc8d9da3ac163 by Stefan Krah in branch 'master': Issue29439: _decimal on Android requires linking with libm. https://github.com/python/cpython/commit/aa26a5712a80849e968171b71b6bc8d9da3ac163 |
|||
msg286975 - (view) | Author: Xavier de Gaye (xdegaye) * | Date: 2017-02-04 16:42 | |
> This wasn't fixed because _decimal didn't build before issue26846 lands. This is misleading, the problem went unnoticed when not linking with '-lm' because: * The _decimal extension module builds without any warning. * test_decimal does not fail. One way to demonstrate the problem in the interactive interpreter on Android: >>> import decimal >>> import _decimal Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: dlopen failed: cannot locate symbol "log10" referenced by "_decimal.cpython-37dm.so"... |
|||
msg286978 - (view) | Author: Stefan Krah (skrah) * | Date: 2017-02-04 17:10 | |
Perhaps test_decimal should fail for CPython if _decimal can't be imported. All compilers that I tested seem to link fine without -lm in this case. |
|||
msg287167 - (view) | Author: Xavier de Gaye (xdegaye) * | Date: 2017-02-06 19:41 | |
With the following module named dlsym.py, the command 'python dlsym.py log10' produces the same successful output on linux and on Android API 21 (when _decimal is not explicitly linked with libm.so, i.e.without changeset b60b46ad8751): 'The symbol "log10" is resolved.' ---------------------- dlsym.py --------------------------- import sys, os from ctypes import * if len(sys.argv) != 2: sys.exit('Error: the symbol name is required.') symbol = sys.argv[1] libdl = CDLL('libdl.so') libdl.dlopen.restype = c_void_p libdl.dlsym.restype = c_void_p hdle = libdl.dlopen(None, os.RTLD_NOW | RTLD_GLOBAL) if hdle is None: print('Cannot get a handle on the global symbol object.') else: v = libdl.dlsym(c_void_p(hdle), symbol.encode()) _not = 'not ' if v is None else '' print('The symbol "%s" is %sresolved.' % (symbol, _not)) ------------------------------------------------------------------- This is as expected since the python executable is linked with the math library. However on Android API 21 importing _decimal fails with the error message reported earlier: 'dlopen failed: cannot locate symbol "log10"'. So this seems to be a bug with Android dlopen() at API 21 since the 'log10' symbol does exist in the process image relocation tables as shown by dlsym.py. This is confirmed by the fact that the import does not fail anymore at API 24 (still without changeset b60b46ad8751). This change of behavior may be a side effect of the changes reported in this bionic (Android libc) changelog entry: https://android.googlesource.com/platform/bionic/+show/refs/heads/master/android-changes-for-ndk-developers.md#36 For completness, my (possibly wrong) interpretation of why changeset b60b46ad8751 fixes the problem at API 21: Changeset b60b46ad8751 adds the '-lm' command line option to the linker and this adds [1] a new DT_NEEDED entry to the .dynamic section of the shared library ELF file as shown by the command 'readelf -d build/lib.linux-x86_64-3.7-pydebug/_decimal.cpython-37dm-x86_64-linux-gnu.so': Dynamic section at offset 0x54d30 contains 26 entries: Tag Type Name/Value 0x0000000000000001 (NEEDED) Shared library: [libm.so.6] 0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] ... The Android loader lookup for 'log10' is now successful at API 21 and the symbol is found in the libm.so.6 library listed in the DT_NEEDED lists of needed libraries. [1] See option '--as-needed' in the 'ld' man pages or see the ELF specification. Note that the Android toolchains use the clang compiler but continue to use the GNU binutils tools including the linker 'ld'. |
|||
msg287168 - (view) | Author: Xavier de Gaye (xdegaye) * | Date: 2017-02-06 20:00 | |
> Perhaps test_decimal should fail for CPython if _decimal can't be imported. This is not a problem specific to _decimal, the same problem exists for third-party extension modules, for example pyephem (https://github.com/brandon-rhodes/pyephem/issues/114) and existed with some other Python extension modules (issue 21668). |
|||
msg287233 - (view) | Author: (yan12125) * | Date: 2017-02-07 14:02 | |
Actually my device is 6.0. Seems there's nothing interesting between API 23 and 24 on android-changes-for-ndk-developers.md :) Anyway, _decimal should not depend on the whether the python binary references to libm.so or not. Instead, _decimal should explicitly have an explicit DT_NEEDED entry to libm.so if it uses symbols from libm. > Strange that on other systems the compilers don't complain (usually they do). I guess on most system python is built with --enable-shared, so _decimal.so has an entry to libpython3.7m.so, and the latter references libm.so. If a linker supports recursive search, it can find the symbol. For future references, here's the partial output of logcat when I run python with `LD_DEBUG=3 python3.7m` and import _decimal: 02-07 19:43:25.697 10266 10266 linker D DEBUG: /data/local/tmp/python3/usr/lib/python3.7/lib-dynload/_decimal.cpython-37m.so: looking up log10 in python3.7m (from global group) 02-07 19:43:25.697 10266 10266 linker I SEARCH log10 in python3.7m@0x55634f3000 h=735a40(elf) 452 02-07 19:43:25.697 10266 10266 linker I NOT FOUND log10 in python3.7m@0x55634f3000 735a40 452 02-07 19:43:25.698 10266 10266 linker D DEBUG: /data/local/tmp/python3/usr/lib/python3.7/lib-dynload/_decimal.cpython-37m.so: looking up log10 in /system/vendor/lib64/libNimsWrap.so (from global group) 02-07 19:43:25.698 10266 10266 linker I SEARCH log10 in /system/vendor/lib64/libNimsWrap.so@0x7f7c5fb000 (gnu) 02-07 19:43:25.698 10266 10266 linker I NOT FOUND log10 in /system/vendor/lib64/libNimsWrap.so@0x7f7c5fb000 02-07 19:43:25.698 10266 10266 linker D DEBUG: /data/local/tmp/python3/usr/lib/python3.7/lib-dynload/_decimal.cpython-37m.so: looking up log10 in /data/local/tmp/python3/usr/lib/python3.7/lib-dynload/_decimal.cpython-37m.so (from local group) 02-07 19:43:25.698 10266 10266 linker I SEARCH log10 in /data/local/tmp/python3/usr/lib/python3.7/lib-dynload/_decimal.cpython-37m.so@0x7f7c157000 h=735a40(elf) 49 02-07 19:43:25.698 10266 10266 linker I NOT FOUND log10 in /data/local/tmp/python3/usr/lib/python3.7/lib-dynload/_decimal.cpython-37m.so@0x7f7c157000 735a40 49 02-07 19:43:25.698 10266 10266 linker D DEBUG: /data/local/tmp/python3/usr/lib/python3.7/lib-dynload/_decimal.cpython-37m.so: looking up log10 in libdl.so (from local group) 02-07 19:43:25.698 10266 10266 linker I SEARCH log10 in libdl.so@0x0 h=735a40(elf) 0 02-07 19:43:25.698 10266 10266 linker I NOT FOUND log10 in libdl.so@0x0 735a40 0 02-07 19:43:25.698 10266 10266 linker D DEBUG: /data/local/tmp/python3/usr/lib/python3.7/lib-dynload/_decimal.cpython-37m.so: looking up log10 in /system/lib64/libc.so (from local group) 02-07 19:43:25.698 10266 10266 linker I SEARCH log10 in /system/lib64/libc.so@0x7f7c4f6000 (gnu) 02-07 19:43:25.698 10266 10266 linker I NOT FOUND log10 in /system/lib64/libc.so@0x7f7c4f6000 02-07 19:43:25.698 10266 10266 linker D DEBUG: cannot locate symbol "log10" referenced by "/data/local/tmp/python3/usr/lib/python3.7/lib-dynload/_decimal.cpython-37m.so"... And BioniC's linker only checks HASH table of ELF: https://android.googlesource.com/platform/bionic/+/master/linker/linker_soinfo.cpp#277, so log10 is not found in python3.7m > This is misleading Maybe it is. I mean "I didn't write a patch for _decimal at issue21668 as _decimal didn't build then" > Perhaps test_decimal should fail for CPython if _decimal can't be imported. I create a simple script to ensure all installed dynamic modules can be imported: https://github.com/yan12125/python3-android/blob/master/devscripts/import_all.py. It may be worth to transform it into a proper unittest and put it into Lib/test/ |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:58:42 | admin | set | github: 73625 |
2017-02-07 14:02:42 | yan12125 | set | messages: + msg287233 |
2017-02-06 20:00:20 | xdegaye | set | messages: + msg287168 |
2017-02-06 19:41:23 | xdegaye | set | messages: + msg287167 |
2017-02-04 17:10:26 | skrah | set | messages: + msg286978 |
2017-02-04 16:42:22 | xdegaye | set | messages: + msg286975 |
2017-02-04 14:04:35 | skrah | set | status: open -> closed stage: resolved resolution: fixed versions: + Python 3.6 |
2017-02-04 14:00:27 | python-dev | set | messages: + msg286963 |
2017-02-04 14:00:25 | python-dev | set | messages: + msg286962 |
2017-02-04 13:59:42 | python-dev | set | nosy:
+ python-dev messages: + msg286961 |
2017-02-04 13:41:00 | skrah | set | assignee: skrah messages: + msg286960 |
2017-02-04 05:07:30 | yan12125 | create |