I found a problem with the handling of manifest files in the msvc9compiler. Distutils removes the reference to the MSVC runtime from the manifest resource, to enable installing the runtime system locally (i.e. to allow non-admins to install Python in their own directory, without needing admin access to the winSxS directory). Embedders of Python are suggested to the manifest for the MSCV runtime to their main program (i.e. the program embedding Python).

This is all a good idea, except for one use case: if you are including Python into a product that is itself a plugin to another product. And, of course, if you need to extend this Python with an extension module. In this case, you have no control over the manifest of the main program (which is the outer embedder). You have control over your own plugin (the one incorporating Python), but this intermediate manifest file is not taken into account when loading the distutils-generated .pyd file.

(imagining the amount of time it took me before I got to this analysis is left as an exercise to the reader:-)

I would suggest an option to distutils to skip the _remove_visual_c_ref() step.

I thought this shouldn't be a problem - that as pythonxx.dll contains a manifest with the references and also contains hoops to ensure its "activation context" is used when loading dynamic modules, that things should work correctly.  The scenario you outline isn't that different to Python being used as a COM server - where a program like "cscript.exe", which comes with Windows but I'm pretty sure has no reference to the CRT manifest we care about, loads up a Python COM object, causing Python to be loaded and it to load extension modules - and that works correctly.

That said though, I agree it would be nice if the manifest handling was optional, but it isn't immediately clear how that could be done given how distutils is structured.  Indeed, there isn't even an obvious way to do it programatically other than by monkey-patching the compiler class.  At least if you do take the monkey-patch route, you should find it necessary to only patch one fairly small method in the class.

Hmm, the cscript.exe point is a valid one. I can think of two things that make my situation different, I will investigate (and post here):

- I started getting the problem when the hosting application (Ambulant multimedia player) switched from VS2008 to VS2010. So maybe cscript.exe is built with 2008, and there the problem is masked?

- My outer plugin (the one for Ambulant) also links against the .pyd (actually, to the .lib corresponding to the .pyd, which results in the .dll having a reference to the .pyd) because I need the various _New and _Convert routines when I call out to Python. I think this is a situation that wouldn't occur in the cscript.exe case.

Indeed, my situation is different from the cscript.exe situation because in my case Python needs to call back into the hosting application.

Further experiments have shown that this is indeed the problem, and moreover that this is a problem that cannot be solved (at least, I am not smart enough to solve it:-). The Python extension module that does the callbacks would have to live in two different runtime worlds: if it allocates python objects it would have to use the VS2008 runtime, if it allocates Ambulant objects it would  have to use 2010 runtime.

This makes the whole original topic of this bug report of academic interest only. Fell free to close/won't fix.