In Python 3.0 the atexit module was translated from Python code to C code.

Prior to Python 3.0, because it was implemented at C code level, the list of 
callbacks was specific to each sub interpreter. In Python 3.0 that appears to 
no longer be the case, with the list of callbacks being maintained globally as 
static C variable across all interpreters.

The end result is that if a sub interpreter registers an atexit callback, that 
callback will be called within the context of the main interpreter on call of 
Py_Finalize(), and not in context of the sub interpreter against which it was 
registered.

Various problems could ensue from this depending on whether or not the sub 
interpreter had also since been destroyed.

Still need to validate the above, but from visual inspection looks to be a 
problem. Issue found because mod_wsgi will trigger atexit callbacks for a sub 
interpreter when the sub interpreter is being destroyed. This is because Python 
prior to 3.0 only did it from the main interpreter. In Python 3.0, this all 
seems to get messed up now as no isolation between callbacks for each 
interpreter.

For mod_wsgi case, since it is explicitly triggering atexit callbacks for sub 
interpreter, in doing so it is actually calling all registered atexit callbacks 
across all interpreters for each sub interpreter being destroyed. They then 
again get called for Py_Finalize().

Even if mod_wsgi weren't doing this, still a problem that Py_Finalize() calling 
atexit callbacks in context of main interpreter which were actually associated 
with sub interpreter.

For mod_wsgi, will probably end up installing own 'atexit' module variant in 
sub interpreters to ensure separation.

Ouch! We totally forgot about sub interpreters during the
reimplementation of the atexit module. 

In my opinion it's a big bug for systems like mod_python and mod_wsgi.
However it's too late to fix it for 3.0.0. Perhaps we can fix it in
3.0.1. We always knew that Python 3.0 is going to have some bugs that
can only be revealed by 3rd party code. I've added Barry to the nosy
list in order to get his opinion as release manager. 

Do you have a suggestion how the problem should be solved? I'd store the
atexit_callback information in PyInterpreterState instead of the module.
Maybe Martin's PEP 3121 could come into play to rescue us.

By the way PEP 3121 ... does the new module initialization code play
nice with multiple sub interpreters?

I wouldn't be concerned about mod_python as likelihood that it will be ported is 
very low and even if it was it would be a long long way in the future. There is too 
much in mod_python itself unrelated to Python 3.0 that needs to be fixed before a 
port to Python 3.0 should be considered.

As to mod_wsgi, I can work around it in the short term by installing an 'atexit' 
module replacement in sub interpreters and avoid the problem.

As to a solution, yes, using PyInterpreterState would seem the most logical place, 
however there is a lot more to it than that.

Prior to Python 3.0, any callbacks registered with atexit module in sub interpreters 
weren't called anyway. This is because Py_EndInterpreter() didn't trigger them, nor 
did Py_Finalize(). The latter is in Python 3.0 at the moment, but as pointed out 
that is a problem in itself.

So, although one may register sub interpreter atexit callbacks against PyInterpreterState, what would be done with them. A decision would need to be made 
as to whether Py_EndInterpreter() should trigger them, or whether the status quo be 
maintained and nothing done with them.

In the short term, ie., for Python 3.0.0, the simplest thing to do may be to have 
functions of atexit module silently not actually do anything for sub interpreters.

The only place this would probably cause a problem would be for mod_wsgi where it 
was itself calling sys.exitfunc() on sub interpreters to ensure they were run. Since 
mod_wsgi has to change for Python 3.0 anyway, to call atexit._run_exitfuncs, with a 
bit more work mod_wsgi can just replace atexit module altogether in sub interpreter 
context and have mod_wsgi track the callback functions and trigger them.

By having atexit module ignore stuff for sub interpreters, at least for now avoid 
problem of callbacks against sub interpreters being execute by Py_Finalize() in main 
interpreter context.

And no I haven't looked at how PEP 3121 has changed things in Python 3.0. Up till 
now I hadn't seen any problems to suggest I may need to look at it.

I'm adding Martin to the nosy list as well. He is the author of PEP 3121
and he might be able to give more insight in the problem.

It would certainly be possible to produce per-interpreter callback lists
through the module state. You declare a per-interpreter structure, add
its size into atexitmodule, and refer to it through PyModule_GetDef,
passing the self argument of the module-level functions.

Reviewing the code, I have two unrelated remarks:
- METH_NOARGS functions still take two arguments.
- I would drop the double-indirection of the callback list. Instead,
allocate a single block of atexit_callback, and use func==NULL as an
indicator that the block is free.

According to the documentation of the atexit module, it seems clear that
it is a per-interpreter thing.

I'm trying to come up with a patch. It's a little bit trickier than I
thought. The atexit module registers "atexit_callfuncs()" with
_Py_PyAtExit(). The "atexit_callfuncs()" function is called by
Python/pythonrun.c when the interpreter exits. The function is cleared
as "static void atexit_callfuncs(void)" so it doesn't get the module
through self. However PyModule_GetDef requires self.

In order to use PEP 3121 we have to add a mechanism to pass the module
instance to atexit_callfuncs().

Christian Heimes wrote:
> However PyModule_GetDef requires self.

(I actually meant PyModule_GetState).

You can use PyState_FindModule to quickly lookup a module.

Ah, thanks. I'll try PyState_FindModule().

Can you review the patch, Martin? I've moved the three state variables
into a module state struct. The patch also fixes the METH_NOARGS functions.

Martin, can you please review and comment on my patch?

Graham, does the patch solve your problem?

By visual inspection the intent looks correct, but can't actually test it 
until I can checkout Python code from source repository and apply patch as 
patch doesn't apply cleanly to 3.0rc1.

With #3723 and #4213 now also having patches, will need to sit down and 
look at all of them and see if find any new issues. May take me a couple 
of days to get time.

This patch looks fine.

In conjunction with #3723 and #4213, the attached atexit_modulestate.patch 
appears to fix issue for mod_wsgi.

Fixed in r67054

I know this issue is closed, but for this patch, the code:

+    modstate = get_atexitmodule_state(module);
+
+    if (modstate->ncallbacks == 0)
+        return;

was added.

Is there any condition under which modstate could be NULL.

Haven't touched Python 3.0 support in mod_wsgi for a long time and when 
revisiting code with final Python 3.0, I find that I get Py_Finalize() 
crashing on process shutdown. It is crashing because modstate above is 
NULL.

2009/7/19 Graham Dumpleton <report@bugs.python.org>:
>
> Graham Dumpleton <Graham.Dumpleton@gmail.com> added the comment:
>
> I know this issue is closed, but for this patch, the code:
>
> +    modstate = get_atexitmodule_state(module);
> +
> +    if (modstate->ncallbacks == 0)
> +        return;
>
> was added.
>
> Is there any condition under which modstate could be NULL.

The module is probably being deallocated before the callbacks are being called.

>
> Haven't touched Python 3.0 support in mod_wsgi for a long time and when
> revisiting code with final Python 3.0, I find that I get Py_Finalize()
> crashing on process shutdown. It is crashing because modstate above is
> NULL.

This new problem I am seeing looks like it may be linked to where the 
'atexit' module is initialised/imported in a sub interpreter but never 
in the main interpreter. I can avoid the crash by having:

    PyImport_ImportModule("atexit");

    Py_Finalize();

At a guess, this is because:

    module = PyState_FindModule(&atexitmodule);
    if (module == NULL)
        return;

still returns a module for case where imported in a sub interpreter but 
not in main interpreter, but then:

    modstate = GET_ATEXIT_STATE(module);

    if (modstate->ncallbacks == 0)
        return;

returns NULL for modstate for the main interpreter as PyInit_atexit() 
had never been called for the main interpreter.

The fix would appear to be to check modstate for being NULL and return. 
Ie.,

    module = PyState_FindModule(&atexitmodule);
    if (module == NULL)
        return;
    modstate = GET_ATEXIT_STATE(module);

    if (modstate == NULL)
        return;

    if (modstate->ncallbacks == 0)
        return;

Does that make sense to anyone? If it does and I am correct, I'll create 
a new issue for it as original fix seems deficient.

Have created issue6531 for my new issue related to this patch.