classification
Title: Interpreter doesn't start when dynamic loading is disabled
Type: compile error Stage: resolved
Components: Build Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: larry Nosy List: Jeffrey.Armstrong, brett.cannon, eric.snow, larry, ncoghlan, petr.viktorin, python-dev, rbcollins, termim
Priority: release blocker Keywords: patch

Created on 2015-08-01 03:17 by Jeffrey.Armstrong, last changed 2015-08-25 21:35 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
issue24769.patch petr.viktorin, 2015-08-10 13:16 review
Messages (15)
msg247797 - (view) Author: Jeffrey Armstrong (Jeffrey.Armstrong) * Date: 2015-08-01 03:17
When attempting to build Python without dynamic loading (HAVE_DYNAMIC_LOADING is not defined), the module "_imp" will not have the function "exec_dynamic."  However, Lib/bootstrap.py seems to assume that "_imp.exec_dynamic" exists, causing the error:

----
./python -E -S -m sysconfig --generate-posix-vars ;\
if test $? -ne 0 ; then \
	echo "generate-posix-vars failed" ; \
	rm -f ./pybuilddir.txt ; \
	exit 1 ; \
fi
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1134, in _install
  File "<frozen importlib._bootstrap>", line 1114, in _setup
  File "<frozen importlib._bootstrap>", line 1082, in _builtin_from_name
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 748, in exec_module
AttributeError: module '_imp' has no attribute 'exec_dynamic'
Fatal Python error: Py_Initialize: importlib install failed

Current thread 0x00000000 (most recent call first):
ABNORMAL TERMINATION
generate-posix-vars failed
----

when trying to build.  This error means that Python 3.5 will not be able to build in a purely static (no dynamic loading whatsoever) form.

This error was encountered in Python 3.5b4.
msg248033 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2015-08-05 12:45
This is an interesting find - thanks.

Adding Larry as 3.5 release manager to the cc here, as I think the right fix actually involves an API adjustment inside _imp.

One of the consequences of PEP 489 (multi-phase initialisation) was that builtin imports and extension module imports now share a lot more code, and that includes the fact that while we have both _imp.create_builtin and _imp.create_dynamic, they currently share a module level exec implementation in _imp.exec_dynamic.

However, the definition and implementation of _imp.exec_dynamic is still behind the same preprocessor guard as _imp.create_dynamic, hence the error reported here.

As far as a fix goes, rather than defining _imp.exec_dynamic unconditionally, my inclination is to say that merging the Python level APIs in addition to the underyling C level implementation represents a design mistake when we implemented the PEP, and there should be a separate _imp.exec_builtin function.

The fact that _imp.exec_dynamic just called _imp.exec_builtin internally would then be a pure CPython implementation detail, rather than something importlib specifically depended on.
msg248359 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2015-08-10 13:16
Apologies for the delay; I was on vacation.

This was indeed a mistake in PEP 489 implementation. I agree with Nick on the solution.
Here is a patch that adds exec_builtin, with implementation shared with exec_dynamic.
msg248370 - (view) Author: Eric Snow (eric.snow) * (Python committer) Date: 2015-08-10 21:23
Looks fine to me.  Nick had suggested calling exec_builtin from exec_dynamic (to the same effect as your patch), but I don't consider that much of an issue. :)
msg248374 - (view) Author: Petr Viktorin (petr.viktorin) * (Python committer) Date: 2015-08-10 21:44
Right. I think a common helper is cleaner than calling a clinic-generated wrapper.
msg248403 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-08-11 11:13
Uh, Nick?  You didn't add me to this bug.
msg248754 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2015-08-18 02:30
Patch looks good to me too. I think this needs to be put forward as a PR to bitbucket right? It looks Release Critical to me.
msg248776 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2015-08-18 16:15
It's ultimately Larry's call, but I think it should go into 3.5.0.
msg249085 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-08-25 00:09
Yes, I'll accept this into 3.5.0, please send a pull request.
msg249101 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-08-25 02:55
I wanted to get this in to Python 3.5.0rc2, so I checked it in myself.  Petr, I gave you credit in the checkin comment and Misc/NEWS.  Hope that's okay!
msg249109 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2015-08-25 06:08
Thanks Larry! Sorry I didn't get to this as soon as I got back from the US.

Will you add it to 3.5.1 and 3.6.0, or would you prefer I take care of that?
msg249122 - (view) Author: Jeffrey Armstrong (Jeffrey.Armstrong) * Date: 2015-08-25 13:31
I pulled the 3.5 branch a few minutes ago, and the patch isn't present.  Has it not been pushed to hg.python.org?
msg249123 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-08-25 13:51
That's correct, it's just in 3.5.0 at the moment.
msg249159 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2015-08-25 21:23
I have merged it forward into 3.5.1 and 3.6.
msg249162 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-08-25 21:35
New changeset 931593401e3e by Larry Hastings in branch '3.5':
Issue #24769: Interpreter now starts properly when dynamic loading
https://hg.python.org/cpython/rev/931593401e3e
History
Date User Action Args
2015-08-25 21:35:24python-devsetnosy: + python-dev
messages: + msg249162
2015-08-25 21:23:58larrysetmessages: + msg249159
2015-08-25 13:51:23larrysetmessages: + msg249123
2015-08-25 13:31:29Jeffrey.Armstrongsetmessages: + msg249122
2015-08-25 06:08:37ncoghlansetmessages: + msg249109
2015-08-25 02:55:23larrysetstatus: open -> closed
versions: + Python 3.6
messages: + msg249101

resolution: fixed
stage: commit review -> resolved
2015-08-25 00:09:52larrysetmessages: + msg249085
2015-08-18 16:15:16brett.cannonsetpriority: deferred blocker -> release blocker
assignee: larry
messages: + msg248776
2015-08-18 02:30:49rbcollinssetnosy: + rbcollins
messages: + msg248754
2015-08-14 21:01:32termimsetnosy: + termim
2015-08-11 11:13:29larrysetnosy: + larry
messages: + msg248403
2015-08-10 21:44:42petr.viktorinsetmessages: + msg248374
2015-08-10 21:23:20eric.snowsetmessages: + msg248370
stage: commit review
2015-08-10 13:16:47petr.viktorinsetfiles: + issue24769.patch
keywords: + patch
messages: + msg248359
2015-08-05 12:45:36ncoghlansetnosy: + petr.viktorin
messages: + msg248033
2015-08-02 04:39:23brett.cannonsetpriority: normal -> deferred blocker
2015-08-02 02:10:03ned.deilysetnosy: + brett.cannon, ncoghlan, eric.snow
2015-08-01 03:17:42Jeffrey.Armstrongcreate