msg92210 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2009-09-03 14:24 |
Note: this is mostly a reminder for myself to clean up the pythonw stub
executable
The current implementation of pythonw on OSX uses exec to start an
executable inside the framework, this is needed to be able to use GUI
functionality from the command-line without resorting to undocumented
and unsupported system APIs. To deal with selection between 32-bit and
64-bit the framework contains a number of python executables.
Using "posix_spawnattr_setbinpref_np", "posix_spawnattr_setflags" and
"posix_spawn" it is possible to do away with the additional executables,
leaving a simpler situation.
Nice to have features:
* python(1) on SnowLeopard has a system preference to select between 32-
bit and 64-bit:
$ defaults read com.apple.versioner.python
{
"Prefer-32-Bit" = 1;
}
(The "versioner" appears to be a private Apple library/tool,
reimplementing the functionality would be fairly trivial)
* It would be nice to have a command-line switch as well
* It would be nice if the stub executable could be reused by tools like
virtualenv without recompilation
|
msg92215 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2009-09-03 18:43 |
Can you kindly report how architecture selection works? Is there a
separate binary which execs? Some other magic?
Asking primarily out of curiosity, but if it's a launcher, then
(sym)linking it into a virtualenv might be sufficient.
|
msg92232 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2009-09-04 06:10 |
Let me first explain in more detail why pythonw is needed in the first
place: MacOSX only allows access to a large number of GUI-related APIs
from application bundles (the symbols are of course always available,
but the APIs fail when you are calling them from an executable that is
not in an application bundle)
This is why we introduced pythonw in framework builds of Python. Pythonw
is a small executable that does nothing else but execv the real python
interpreter that is in and .app bundle inside the Python framework
(Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Pytho
n). The full path to the execv-ed interpreter is hardcoded into the
pythonw executable.
To be able to switch between 32-bit and 64-bit versions of Python a 4-
way universal build of Python creates 3 variations of the Python
executable inside the .app bundle: Python-32, Python-64 and Python (the
latter being 4-way universal). There are also 3 variations of the
pythonw executable (such as pythonw-32) that execv the proper version of
the interpreter.
Using posix_spawn and the other API's I mention it should be fairly easy
to create a simpler situation where we need only one copy of Python in
the .app bundle, with the pythonw wrapper selecting the required
architecture(s) before executing the real interpreter.
Making pythonw fully reusable by virtualenv requires some more work, and
requires more support code in virtualenv itself as well. One fairly easy
way to accomplish easy reusability without requiring a C compiler is to
use a large buffer for storing the path to the real interpreter,
virtualenv could then patch the executable.
Another option is to link the pythonw executable to Python.framework and
then use dyld APIs to find the path to the Python framework and from
that the executable.
As an aside, virtualenv copies the shared library in Python.framework
into virtual environments, this is needed because the framework build
uses the location of that shared library to determine sys.prefix.
|
msg92981 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2009-09-22 10:48 |
The attached file 'pythonw.c' is a first version of a better pythonw
executable.
This version uses posix_spawn rather than execv to start the real
interpreter. The main advantage of the new implementation is that 'arch
-ppc pythonw' works as expected, with the current version of pythonw the
'arch' command only affects the pythonw executable and not the real
interpreter.
Todo:
* I'm not use if the '-X32bit' option is a good idea or not.
The basic idea of this is to provide an easy way to force python to
start in 32bit mode (for use in #! lines) for scripts that need it
(basicly anything that needs to access Carbon GUI APIs, including most
current Python GUI libraries)
* The implementation of '-X32bit' is sucky, it contains a copy of the
getopt format string from Py_Main in Modules/main.c
* The path to the real executable is hardcoded for a standard framework
install of 2.7 (easily changed to the same mechanism as used by
the current edition of pythonw)
What I'd like to do is link pythonw to the framework and use dyld
introspection to deduce the path to the real executable. That's
slightly more complicated, but would provide a clean way to reuse
the executable in tools like virtualenv without recompiling.
|
msg93057 - (view) |
Author: Martin v. Löwis (loewis) * |
Date: 2009-09-24 07:30 |
> This version uses posix_spawn rather than execv to start the real
> interpreter.
In what way is that better? It creates a new process (IIUC); therefore,
I think that using it is worse than using execv. Anybody killing the
pythonw process would only kill the wrapper.
> The main advantage of the new implementation is that 'arch
> -ppc pythonw' works as expected, with the current version of pythonw the
> 'arch' command only affects the pythonw executable and not the real
> interpreter.
I suppose this would also be possible through execv?
|
msg94634 - (view) |
Author: Ned Deily (ned.deily) * |
Date: 2009-10-28 17:58 |
For people searching the bug tracker, I've modified the title of the
issue to make it clearer that there is a problem here on OS X 10.6 Snow
Leopard with multiple architecture builds.
As Ronald mentions above, the effect of using the pythonw "launcher" on
10.6 as it currently stands is to always prefer x86_64 (64-bit) over
i386 (32-bit) when both are available. arch -i386 only forces the
launcher to run as 32-bit; the execv runs the interpreter in 64-bit mode
(if available). To get a multi-arch (32/64) interpreter to run in 32-
bit, one workaround is to arch -i386 directly to the interpreter binary
in the framework app bundle, typically:
/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/C
ontents/MacOS/Python
But then, presumably, the GUI functionality no longer works.
A more robust workaround would be to build a 32-bit-only Python (but
ensure that -arch is forced on the build - see Issue7184).
|
msg94638 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2009-10-28 20:16 |
Ned: I'm planning to use the attached version of pythonw, or a slightly
updated one, for 2.7 and 3.2. This version will not be used for 2.6.5 or
3.1.2 due to backward compatibility constraints.
I will look into the OSX launching issues though, it was my intention that
universal builds had a way to select a 32-bit or 64-bit python but that
code may be broken.
|
msg95947 - (view) |
Author: Ned Deily (ned.deily) * |
Date: 2009-12-03 21:07 |
Another reminder: when implementing, make sure platform.architecture()
always returns the correct results for bits. It seems the Apple patches
in 10.6 don't handle that (it seems to always report 64-bit), probably
because the code in platform.architecture() doesn't look quite robust
enough. (I'll open a bug with Apple.)
|
msg96184 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2009-12-09 21:46 |
FWIW: I now have a 2.7 tree with the new pythonw on my machine, open
issues are:
* Ensure IDLE.app gets build in such a way that the GUI works for
all supported universal binaries (including a 4-way build on 10.5,
where the system Tk doesn't work in 64-bit mode)
* Update NEWS file
* Port to 3.2
|
msg96857 - (view) |
Author: Ronald Oussoren (ronaldoussoren) * |
Date: 2009-12-24 14:04 |
I've committed the completed patch as r77031 (trunk) and r77032 (py3k)
I will not backport to 2.6 and 3.1 because this is not a bugfix.
|
msg97398 - (view) |
Author: Ned Deily (ned.deily) * |
Date: 2010-01-08 09:08 |
Note, r77031 and r77032 cause compile errors for OS X deployment targets of 10.4 or earlier. See Issue7658 for details and patch.
|
|
Date |
User |
Action |
Args |
2022-04-11 14:56:52 | admin | set | github: 51083 |
2010-01-08 09:08:34 | ned.deily | set | messages:
+ msg97398 |
2009-12-24 14:04:10 | ronaldoussoren | set | status: open -> closed resolution: accepted messages:
+ msg96857
stage: needs patch -> resolved |
2009-12-09 21:46:52 | ronaldoussoren | set | messages:
+ msg96184 |
2009-12-03 21:07:54 | ned.deily | set | messages:
+ msg95947 |
2009-10-28 20:16:55 | ronaldoussoren | set | messages:
+ msg94638 title: use different mechanism for pythonw on osx: universal builds always run 64-bit on OS X 10.6 -> use different mechanism for pythonw on osx |
2009-10-28 18:25:04 | robince | set | nosy:
+ robince
|
2009-10-28 18:09:04 | ned.deily | set | versions:
+ Python 2.6, Python 3.1 |
2009-10-28 17:58:45 | ned.deily | set | title: use different mechanism for pythonw on osx -> use different mechanism for pythonw on osx: universal builds always run 64-bit on OS X 10.6 nosy:
+ ned.deily
messages:
+ msg94634
type: enhancement -> behavior |
2009-09-24 07:30:16 | loewis | set | messages:
+ msg93057 |
2009-09-22 10:48:54 | ronaldoussoren | set | files:
+ pythonw.c
messages:
+ msg92981 |
2009-09-04 06:10:53 | ronaldoussoren | set | messages:
+ msg92232 |
2009-09-03 18:43:20 | loewis | set | nosy:
+ loewis messages:
+ msg92215
|
2009-09-03 14:24:43 | ronaldoussoren | create | |