The frame object has a number of fields which belong to the generator object.
By creating a CoState struct, these fields can be moved to generator/threadstate where they belong.

The interpreter no longer has to swap around exception state whenever generators are entered/exited.
The frame object is made more compact, reducing allocation/deallocation overhead (although benchmarking showed no significant difference) 

The attached patch reduces the code base by about 50 LOC overall.

- Why is it called CoState?  is it related to coroutines?
- I'm a bit worried about the removal of tstate->exc_value, and the changes in sys.exc_info().  "tstate->exc_value" is used by many extension modules, do you really want to replace this?

Extensions modules should really be using PyErr_Occurred/Fetch etc.

The division of responsibilities between generator objects and the eval loop is currently a little messy. The eval loop deals almost entirely with frame objects and also handles swapping exception states around on behalf of generators, which is why the generator specific exception state currently lives on frame objects (i.e. to avoid the eval loop needing to know too much about the internal structure of generator objects).

Before generator related state can reasonably be moved out of the frame objects, those responsibilities should be divided more cleanly. Ron Adam has made an initially attempt at tackling that problem in issue 13607.

Once the responsibilities are divided appropriately, *then* we can look at the fields that ceval no longer touches and see if they should be moved somewhere else (whether that's directly into the generator struct or into a new struct referenced from the generator struct).

Nick Coghlan wrote:
> Nick Coghlan <ncoghlan@gmail.com> added the comment:
> 
> The division of responsibilities between generator objects and the eval loop is currently a little messy. The eval loop deals almost entirely with frame objects and also handles swapping exception states around on behalf of generators, which is why the generator specific exception state currently lives on frame objects (i.e. to avoid the eval loop needing to know too much about the internal structure of generator objects).

In my patch the exception state does not need swapping at all, so the 
question of whose responsibility it is become mute.
Neither generator nor ceval need to do it.

> 
> Before generator related state can reasonably be moved out of the frame objects, those responsibilities should be divided more cleanly. Ron Adam has made an initially attempt at tackling that problem in issue 13607.
> 
> Once the responsibilities are divided appropriately, *then* we can look at the fields that ceval no longer touches and see if they should be moved somewhere else (whether that's directly into the generator struct or into a new struct referenced from the generator struct).

This patch is largely orthogonal to that proposed for issue 13607.
In fact the new CoState struct would be a better place to pass data 
between PyEval_EvalFrame() and gen_send(), rather than adding yet 
another field to the frame.

Amaury Forgeot d'Arc wrote:
> 
> - Why is it called CoState?  is it related to coroutines?

Yes it is related to coroutines, threads and generators *are*
(a limited form of) asymmetric coroutines, even if we don't usually 
think of them that way).

Perhaps GenState would be a better name
(treating the threadstate as a sort of top level pseudo-generator)?

IMO "tstate->exc_value" has nothing to do with generators.  Changing its name seems gratuitous breakage to me.

The important part is not the change of name, but wrapping them in a 
struct which can be embedded in both PyThreadState and PyGenObject.

The state->exc_XXX trio of values are the currently handled exception
(sys.exc_info()) and are shadowed by generator exception handlers.
My patch models that shadowing rather than swapping the values in and 
out. This allows me to eliminate save_exc_state(), swap_exc_state()
and restore_and_clear_exc_state() completely.

This issue was too broad. The new patch is focussed on sys.exc_info.
All hints of coroutines have been removed and f_yieldfrom is untouched.

New patch reduces code by 65 lines and does not conflict with issue 13607.