Currently pickling functions and types stores modules by their name.

So I believe it is possible to support pickling module objects with the following code (based on the logic in PyPy, which supports pickling modules):


import copyreg
import types
import pickle
import sys

def picklemod(mod):
    if mod.__name__ in sys.modules:  # real modules
        return (__import__, (mod.__name__, None, None, ('',)))

    # module objects created manually:
    return (types.ModuleType, (mod.__name__,), mod.__dict__)
copyreg.pickle(types.ModuleType, picklemod)

pickle.loads(pickle.dumps(sys))  # works
import http.server
pickle.loads(pickle.dumps(http.server))  # works for nested modules

fakemod = types.ModuleType('fakemod')
fakemod.field1 = 'whatever'

# This fake type is used instead of types.ModuleType in order to re-confuse pickle back on track.
# Should not have been necessary in the first place,
# but types.ModuleType has misconfigured fields according to pickle
# (and they are read-only).
class _types_ModuleType(types.ModuleType):
    __module__ = 'types'
    __name__ = __qualname__ = 'ModuleType'

_orig_types_ModuleType = types.ModuleType
# bad monkey-patching, but needed for the confusion to work
types.ModuleType = _types_ModuleType
dump = pickle.dumps(fakemod)
# not necessary, but to show unpickling is regardless correct
types.ModuleType = _orig_types_ModuleType

pickle.loads(dump).field1  # works


Disclaimer: I do not see any specific use for this, I was just surprised while trying to port snakeoil library to PyPy, which (snakeoil) uses sys module as an example of an unpicklable object (they should switch to a traceback for instance, but that is not the scope of this issue).

Sorry, I forgot to state what is my actual goal: for the module objects to be pickleable in native CPython (possibly from C layer) without the need to add this to every code that wants to pickle module objects.

The point is that they can already be unpickled from the representation generated by the code, so it should only be necessary to implement __reduce__ in moduleobject.c, and probably to change the object's qualname / module attributes. Or to introduce a helper function, like PyPy did, for the fake module case; I just found supporting existing unpickling paths more elegant (and working across all Python versions since 2.0.0), as much as unelegant is the monkey-patching done in my example. It is possible that (for the actual module case) _compat_pickle.REVERSE_IMPORT_MAPPING should be considered as well for the old protocols (which would probably imply using __reduce_ex__ instead), but I did not explore that.

I would also like to see this added, and I can also provided a usecase where I naturally came across this.

I am helping on a parsing library which uses regular expressions internally. We allow the user to select the stdlib re module or the third party regex module and store the selected module as an attribute in multiple different objects, since those modules (by design) have pretty much compatible interfaces. This however makes it unpicklable, unless one adds a custom __reduce__ function for each of those classes.

Most notably, this issue also extends to deepcopy and the multiprocessing module.