classification
Title: Pass context information into the extension module init function
Type: enhancement Stage:
Components: Extension Modules, Interpreter Core Versions: Python 3.3
process
Status: closed Resolution: postponed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, jcea, loewis, scoder
Priority: normal Keywords:

Created on 2011-11-19 09:16 by scoder, last changed 2011-11-20 21:55 by Arfrever. This issue is now closed.

Messages (10)
msg147931 - (view) Author: Stefan Behnel (scoder) * Date: 2011-11-19 09:16
This is a follow-up to issue 13429, which deals with setting __file__ on newly created extension modules earlier than it is currently the case.

Currently, the module init function of extension modules lacks a way to find out the context in which it is being imported, e.g. its package path or its location in the file system. This makes it tricky for extension modules to do things like loading package resources or using relative imports at init time.

This can be fixed by allowing the init function to take a context struct as argument, which would contain object pointers to the FQ package name and file path, and potentially other information.

I think this would be backwards compatible to existing code, because C should allow the caller of the init function to pack additional arguments on the stack that the called function simply doesn't care about. From CPython 3.3 on, however, new and updated code could benefit from this feature.
msg147933 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-11-19 10:16
This approach is not correct C. You may pass an "incorrect" number of arguments only if the function is declared with an ellipsis, else the behavior is undefined (6.5.5.2p6). Your proposed approach works with most implementations of C, but Python shouldn't deliberately rely on undefined behavior.
msg147934 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-11-19 10:17
FWIW, I think that a PEP will be needed to redesign the module initialization for Python 4. Hopefully, this time people will then identify all requirements before the PEP is accepted (which unfortunately didn't happen for PEP 3121).
msg147935 - (view) Author: Stefan Behnel (scoder) * Date: 2011-11-19 10:47
Yes, that's unfortunate. I found the same paragraph in section 6.5.2.2p6 of the C99 standard now, so it seems that this idea isn't suitable for the Py3.x series.

There's no Python 4 target version in the bug tracker, BTW. :)
msg147970 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-11-19 20:31
I really do propose to close this issue as "won't fix". By the time Python 4 comes along, things will have changed a lot; it's not clear that the value of keeping it open will outweigh the cost of keeping it open.
msg147976 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2011-11-20 05:38
What about providing a function that "init" would call to get this information, instead of a "hidden" parameter?.
msg147979 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-11-20 09:40
> What about providing a function that "init" would call to get this information, instead of a "hidden" parameter?.

How would the function that init calls itself get the information,
without a hidden parameter?
msg147998 - (view) Author: Jesús Cea Avión (jcea) * (Python committer) Date: 2011-11-20 15:55
Python interpreter already have that information internally. The initial proposal was to pass it to the init as an additional parameter. My suggestion is to export it as a new function that the init can call if VERSION >=3.3.
msg148000 - (view) Author: Stefan Behnel (scoder) * Date: 2011-11-20 16:29
The problem with having that information "internally" is that it's currently stored in local variables in the call chain from the dynamic library loader. Passing that information on into a callable function, without passing it as an argument into the init function, means, that it needs to get stored away in some global place, with all the drawbacks that this induces. That's what Martin was referring to.

I agree with Martin that the idea of adding a parameter to the module init function is not worth pursuing before Python 4, so I'm closing this bug.
msg148009 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2011-11-20 21:29
> Python interpreter already have that information internally. The
> initial proposal was to pass it to the init as an additional
> parameter. My suggestion is to export it as a new function that the
> init can call if VERSION >=3.3.

Jesus: Please try to come up with a patch if you don't see the problem.
Trust Stefan and me that there actually *is* a problem with your
approach.
History
Date User Action Args
2011-11-20 21:55:27Arfreversettitle: Pass context information into the extension module init function -> Pass context information into the extension module init function
2011-11-20 21:29:18loewissetmessages: + msg148009
title: Pass context information into the extension module init function -> Pass context information into the extension module init function
2011-11-20 16:29:15scodersetstatus: open -> closed
resolution: postponed
messages: + msg148000

title: Pass context information into the extension module init function -> Pass context information into the extension module init function
2011-11-20 15:55:17jceasetmessages: + msg147998
2011-11-20 09:40:32loewissetmessages: + msg147979
title: Pass context information into the extension module init function -> Pass context information into the extension module init function
2011-11-20 05:38:37jceasetnosy: + jcea
messages: + msg147976
2011-11-19 20:31:13loewissetmessages: + msg147970
2011-11-19 14:27:09Arfreversetnosy: + Arfrever
2011-11-19 10:47:21scodersetmessages: + msg147935
2011-11-19 10:17:49loewissetmessages: + msg147934
2011-11-19 10:16:08loewissetnosy: + loewis
messages: + msg147933
2011-11-19 09:16:48scodercreate