classification
Title: reload() is broken for C extension objects
Type: behavior Stage: test needed
Components: Interpreter Core Versions: Python 2.6
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, georg.brandl, knepley, logistix, mwh
Priority: normal Keywords:

Created on 2005-02-19 09:20 by knepley, last changed 2009-04-05 17:41 by georg.brandl. This issue is now closed.

Messages (6)
msg60666 - (view) Author: Matthew G. Knepley (knepley) Date: 2005-02-19 09:20
1) A C extension module (foo.so) is imported

      import foo

2) The library foo.so is rebuilt with changes

3) We reload the module

      foo = reload(foo)

The reload() method calls imp.load_dynamic() which
eventually gets down to _PyImport_GetDynLoadFunc().
This just calls dlopen(), which returns the old filehandle.

This problem can be fixed by augmenting imp with
unload_dynamic(), which could easily be implemented in
a _PyImport_GetDynUnloadFunc(), which would just
consult its handles[] array, and call dlclose() on the
appropriate handle. This will work if Python was the
only program to dlopen() foo.so.
msg60667 - (view) Author: Michael Hudson (mwh) (Python committer) Date: 2005-02-19 21:27
Logged In: YES 
user_id=6656

I don't believe this is quite as easy as you make out.  It might be 
possible to make something that works for platforms that use dlopen 
though...
msg60668 - (view) Author: Matthew G. Knepley (knepley) Date: 2005-02-19 21:37
Logged In: YES 
user_id=58554

I am only interested in fixing it for the dlopen() case,
which I thnk subsumes every architecture likely to have this
problem appear. I have no problem fixing it myself, but I
need to get all the CVS stuff setup. And I am not sure how
to easily generate the patch to send (I'm used to BK).
msg60669 - (view) Author: Matthew G. Knepley (knepley) Date: 2005-02-20 04:34
Logged In: YES 
user_id=58554

I have coded the proposed solution (changed import.c,
importdl.c, and dynload_shlib.c). I will assemble a test
case for it next week.
msg60670 - (view) Author: Grant Olson (logistix) Date: 2005-03-08 20:59
Logged In: YES 
user_id=699438

I don't know if this is a viable general-purpose fix.  Keep in 
mind that python tracks everything by references, and 
reloading doesn't correct existing bindings.  If you recompile it 
is going to screw up existing function pointers in 
PyCFunction objects.  In the example below calling a() will 
probably end up executing random code or generting a 
memory fault.

>>> import select
>>> a = select.select

... imaginary recompile ...

>>> reload(select)
<module 'select' from 'C:\Python24\DLLs\select.pyd'>
>>> b = select.select
>>> id(a)
18165472
>>> id(b)
18165476
>>> a() #BOOM!
msg85537 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-04-05 17:41
I agree with Grant.  There seems to be no way this can be sanely
implemented with today's infrastructure.
History
Date User Action Args
2009-04-05 17:41:15georg.brandlsetstatus: open -> closed

nosy: + georg.brandl
messages: + msg85537

resolution: wont fix
2009-02-15 23:44:17ajaksu2setnosy: + brett.cannon
stage: test needed
type: behavior
versions: + Python 2.6, - Python 2.4
2005-02-19 09:20:13knepleycreate