Author ncoghlan
Recipients brett.cannon, christoph.wiedemann, cschramm, eric.snow, ncoghlan, petr.viktorin, scoder
Date 2018-08-04.16:06:27
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1533398787.68.0.56676864532.issue34309@psf.upfronthosting.co.za>
In-reply-to
Content
As others have noted, dynamically reloading CPython extension modules is akin to dynamically reloading any other C/C++ shared library, so it has enough opportunities for things to go wrong that we consider allowing the shared state to persist across initialize/finalize cycles the less problematic of two problematic options (at least for now).

Reliable hot reloading support is typically a property of pure Python modules (and even at that higher level, inter-module dependencies can still cause problems at runtime).

(FWIW, this problem is currently also the main reason we don't offer an in-REPL package installation command - while PEP 489 offers significant opportunities for improvement, it's likely to be years before we see widespread adoption of that, so we prefer to advise folks to run an installer outside the REPL, then restart and replay their interactive session)

If subinterpreters are an option though, then yeah, that has far more potential to be viable. It wouldn't be trivial, as we'd need to add dlmopen support (thanks Stack Overflow [1]) to give the subinterpreter a truly independent copy of the shared library (and also work out whatever the equivalent to dlmopen might be on other platforms), but going down that path could also potentially provide a way around the known problems with global state leaking between subinterpreters via extension modules.

[1] https://stackoverflow.com/questions/48864659/loading-shared-library-twice/48888598#48888598
History
Date User Action Args
2018-08-04 16:06:27ncoghlansetrecipients: + ncoghlan, brett.cannon, scoder, petr.viktorin, eric.snow, cschramm, christoph.wiedemann
2018-08-04 16:06:27ncoghlansetmessageid: <1533398787.68.0.56676864532.issue34309@psf.upfronthosting.co.za>
2018-08-04 16:06:27ncoghlanlinkissue34309 messages
2018-08-04 16:06:27ncoghlancreate