This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ctypes._dlopen should not force RTLD_NOW
Type: Stage:
Components: ctypes Versions: Python 3.3, Python 3.4, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Albert.Zeyer, amaury.forgeotdarc, belopolsky, meador.inge, ronaldoussoren, vstinner
Priority: normal Keywords:

Created on 2014-01-16 01:20 by Albert.Zeyer, last changed 2022-04-11 14:57 by admin.

Messages (3)
msg208226 - (view) Author: Albert Zeyer (Albert.Zeyer) * Date: 2014-01-16 01:20
On MacOSX, when you build an ARC-enabled Dylib with backward compatibility for e.g. MacOSX 10.6, some unresolved functions like `_objc_retainAutoreleaseReturnValue` might end up in your Dylib.

Some reference about the issue:
1. http://stackoverflow.com/q/21119425/133374>.
2. http://osdir.com/ml/python.ctypes/2006-10/msg00029.html
3. https://groups.google.com/forum/#!topic/comp.lang.python/DKmNGwyLl3w

Thus, RTLD_NOW is often not an option for MacOSX.

This affects mostly `py_dl_open()` from ctypes.
But it is also related how you set `dlopenflags` in `PyInterpreterState_New()`.

I suggest to make RTLD_LAZY the default on MacOSX (or is there a good reason not to do?).
Also, ctypes should have options for both RTLD_NOW and RTLD_LAZY so that both can be used.

This is also consistent with the behavior of the [dl module](http://docs.python.org/2/library/dl.html).
msg208242 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2014-01-16 07:06
The big question is why ctypes uses RTLD_NOW in the first place. There's probably a good reason for it because it's added unconditionally to the user provided flags.

BTW. I'd prefer to keep the dlopen flags on OSX the same as on other platforms (that is, either keep the current behavior, or drop RTLD_NOW everywhere) because RTLD_NOW has an impact on the functionality and now using the same flags everywhere could cause subtle changes in functionality between platforms. 

BTW2. The workaround mentioned in message 2 would be new functionality (adding new symbols) and as such could probably not be added to 2.7.

I've added 3.3 and 3.4 to the list of versions because those also use RTLD_NOW and hence behave the same as 2.7 in this respect.
msg208257 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2014-01-16 08:24
FYI, dlopen() flags are configurable:
-  sys.setdlopenflags(n): default=RTLD_NOW (if available, or RTLD_LAZY otherwise)
- mode parameter of dl.dlopen(): default=RTLD_LAZY
- mode parameter of ctypes.CDLL: default=RTLD_LOCAL (but RTLD_GLOBAL on Mac OS X <= 10.3); but RTLD_NOW is always added

See also:
"How to disable RTLD_NOW for Python 2.7.x dlopen() in Mac OS X Mavericks?"
https://groups.google.com/forum/#!msg/comp.lang.python/DKmNGwyLl3w/1tuxcnwBAw8J

"how to handle library interdependencies"
http://osdir.com/ml/python.ctypes/2006-10/msg00022.html

"Need RTLD_GLOBAL for linux?"
http://code.google.com/p/ctypesgen/issues/detail?id=3
History
Date User Action Args
2022-04-11 14:57:57adminsetgithub: 64475
2014-01-16 08:24:45vstinnersetmessages: + msg208257
2014-01-16 07:06:40ronaldoussorensetnosy: + amaury.forgeotdarc, belopolsky, meador.inge

messages: + msg208242
versions: + Python 3.3, Python 3.4
2014-01-16 02:46:37ned.deilysetnosy: + ronaldoussoren
2014-01-16 01:28:56vstinnersetnosy: + vstinner
2014-01-16 01:20:50Albert.Zeyercreate