classification
Title: The select and time modules uses libm functions without linking against it
Type: crash Stage:
Components: Build, Cross-Build Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Chi Hsuan Yen, WanderingLogic, fornwall, freakboy3742, haypo, python-dev, xdegaye
Priority: normal Keywords: patch

Created on 2014-06-05 12:56 by fornwall, last changed 2016-04-20 08:02 by haypo. This issue is now closed.

Files
File name Uploaded Description Edit
time_and_select_module_link_with_libm.patch fornwall, 2014-06-05 12:56 review
audioop_ctypes_test_link_with_libm.patch WanderingLogic, 2014-10-27 15:46 Additional patch to deal with missing -lm flag for audioop and _ctypes_test.
time_select_audioop_ctypes_test_link_with_libm.patch WanderingLogic, 2014-10-27 17:34 Combined patches to add '-lm' flag. review
python-hg-modules-link-libm.patch Chi Hsuan Yen, 2015-11-26 11:25 review
ext_modules_libm.patch xdegaye, 2016-04-19 07:54 review
Messages (15)
msg219813 - (view) Author: Fredrik Fornwall (fornwall) * Date: 2014-06-05 12:56
The select and time modules use functions from libm, but do not link against it.

* selectmodule.c calls ceil(3) in pyepoll_poll()
* timemodule.c calls fmod(3) and floor(3) in floatsleep()
msg219814 - (view) Author: Fredrik Fornwall (fornwall) * Date: 2014-06-05 12:57
Note: This causes problems at least when running on android, where the system is unable to find the symbols when loading the modules at runtime.
msg230076 - (view) Author: Matt Frank (WanderingLogic) * Date: 2014-10-27 15:54
Additionally,

* audioop calls floor()
* _ctypes_test calls sqrt()

Patch attached.
msg230077 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-10-27 15:58
>  audioop_ctypes_test_link_with_libm.patch 

+                             libraries=['m'])

Why not using math_libs here? It would also be nice to add a comment explaining why libm is needed in each module.

Can someone please combine both patches?
msg230080 - (view) Author: Matt Frank (WanderingLogic) * Date: 2014-10-27 17:34
>>  audioop_ctypes_test_link_with_libm.patch 
> +                             libraries=['m'])
> Why not using math_libs here?

math_libs is defined in detect_modules().  But the _ctypes_test
extension is defined in a different function: detect_ctypes().

The other option, would be to define math_libs=['m'] directly above this line and then use it once.  I didn't think that added clarity, but I'd be happy to do it that way if it fits better with standard style.

> It would also be nice to add a comment explaining why libm is needed in each module.

Done.

> Can someone please combine both patches?

Done.
msg255410 - (view) Author: Chi Hsuan Yen (Chi Hsuan Yen) * Date: 2015-11-26 11:25
Bump. For rev c6930661599b timemodule.c and selectmodule.c no longer calls libm functions, while audioop.c and _ctypes_test.c still do. I have my updated patch here based on previous patches. Note that the introduction of detect_math_libs() is my naive try. Its naming and position in setup.py may need more consideration.

Here's the result before and after my patch.

Before:
shell@GT-N7000:/data/local/tmp $ python3
Python 3.6.0a0 (default:6a8fbb97c8d8+, Nov 26 2015, 18:42:29) 
[GCC 4.9 20140827 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import select 
>>> import time
>>> import audioop
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen failed: cannot locate symbol "floor" referenced by "audioop.cpython-36m-arm-linux-gnueabi.so"...
>>> import _ctypes_test
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen failed: cannot locate symbol "sqrt" referenced by "_ctypes_test.cpython-36m-arm-linux-gnueabi.so"...
>>> 

After:

shell@GT-N7000:/data/local/tmp $ python3
Python 3.6.0a0 (default:c6930661599b+, Nov 26 2015, 19:10:15) 
[GCC 4.9 20140827 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> import selectr
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'selectr'
>>> import select 
>>> import audioop
>>> import _ctypes_test
>>> 

Tested on Samsung Galaxy Note GT-N7000 with my custom build of CyanogenMod 11.0
msg255411 - (view) Author: Chi Hsuan Yen (Chi Hsuan Yen) * Date: 2015-11-26 11:27
Well, the hg revision of Python in two tests are different, because I `hg pull -u` each time I build.
msg263716 - (view) Author: Xavier de Gaye (xdegaye) * Date: 2016-04-19 07:54
The _datetime module also calls functions in libm: delta_new() uses
round() and accum() uses modf().

The attached patch updates setup.py for all the modules that use libm.
msg263718 - (view) Author: Chi Hsuan Yen (Chi Hsuan Yen) * Date: 2016-04-19 08:03
The approach with a helper function is better. See https://github.com/yan12125/python3-android/blob/038271d/mk/python/modules-link-libm.patch.

By the way, I have verified there are no libraries referencing libm functions with https://github.com/yan12125/python3-android/blob/cpython-hg/devscripts/import_all.py
msg263738 - (view) Author: Roundup Robot (python-dev) Date: 2016-04-19 13:58
New changeset f92fea23161d by Victor Stinner in branch '3.5':
setup.py: add missing libm dependency
https://hg.python.org/cpython/rev/f92fea23161d

New changeset 3a9b47b062b9 by Victor Stinner in branch 'default':
Merge 3.5: Issue #21668
https://hg.python.org/cpython/rev/3a9b47b062b9
msg263739 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2016-04-19 14:00
I pushed ext_modules_libm.patch with the helper function proposed by Chi Hsuan Yen in https://github.com/yan12125/python3-android/blob/038271d/mk/python/modules-link-libm.patch.

Tell me if it's not enough. I hope that it will make CPython a little bit more portable ;-)
msg263803 - (view) Author: Xavier de Gaye (xdegaye) * Date: 2016-04-20 07:14
The patch should be acknowledged in Misc/NEWS to Chi Hsuan Yen and not to me.
msg263811 - (view) Author: Roundup Robot (python-dev) Date: 2016-04-20 07:58
New changeset f76753f26982 by Victor Stinner in branch 'default':
Issue #21668: Fix author of the patch.
https://hg.python.org/cpython/rev/f76753f26982
msg263812 - (view) Author: Roundup Robot (python-dev) Date: 2016-04-20 08:02
New changeset 7530caa5ed1a by Victor Stinner in branch 'default':
Issue #21668: Add also Chi Hsuan Yen to Misc/ACKS
https://hg.python.org/cpython/rev/7530caa5ed1a
msg263813 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2016-04-20 08:02
> Xavier de Gaye: "The patch should be acknowledged in Misc/NEWS to Chi Hsuan Yen and not to me."

Oh sorry. I see 4 authors in attached patches, I picked the latest. Thank you to all authors for your contribution on this issue ;-) I fixed the author in Misc/NEWS.
History
Date User Action Args
2016-04-20 08:02:15hayposetmessages: + msg263813
2016-04-20 08:02:13python-devsetmessages: + msg263812
2016-04-20 07:58:21python-devsetmessages: + msg263811
2016-04-20 07:14:16xdegayesetmessages: + msg263803
2016-04-19 14:00:07hayposetstatus: open -> closed
resolution: fixed
messages: + msg263739

versions: + Python 3.6, - Python 3.4
2016-04-19 13:58:47python-devsetnosy: + python-dev
messages: + msg263738
2016-04-19 08:03:57Chi Hsuan Yensetmessages: + msg263718
2016-04-19 07:54:18xdegayesetfiles: + ext_modules_libm.patch
nosy: + xdegaye
messages: + msg263716

2015-11-26 11:27:38Chi Hsuan Yensetmessages: + msg255411
2015-11-26 11:25:35Chi Hsuan Yensetfiles: + python-hg-modules-link-libm.patch
nosy: + Chi Hsuan Yen
messages: + msg255410

2015-02-23 14:38:36haypolinkissue23496 dependencies
2014-10-27 17:34:48WanderingLogicsetfiles: + time_select_audioop_ctypes_test_link_with_libm.patch

messages: + msg230080
2014-10-27 15:58:49hayposetmessages: + msg230077
2014-10-27 15:57:51WanderingLogicsetnosy: + freakboy3742
2014-10-27 15:54:51WanderingLogicsetmessages: + msg230076
2014-10-27 15:46:24WanderingLogicsetfiles: + audioop_ctypes_test_link_with_libm.patch
nosy: + WanderingLogic
2014-06-05 13:18:30hayposetnosy: + haypo
2014-06-05 12:57:24fornwallsetmessages: + msg219814
2014-06-05 12:56:33fornwallcreate