This issue comes from #5990.

Currently, O& doesn't accept general cleanup function. This causes
memory leak sometimes. Here is an experimental patch to create new
format O&& similar to O& but have general cleanup function.

I would propose a different approach: the same function can be both
argument parser and cleanup function. In parsing, the PyObject* points
to the parameter being converted. In cleanup, it points to NULL.

This would allow the O& to continue to be used even if cleanup is
needed. As not all converters would need or support cleanup, converters
that do need cleanup could be required to return Py_CLEANUP_SUPPORTED
(which would be, say, 131072).

> the same function can be both
> argument parser and cleanup function

Here is an another experimental patch to implement this. (Is this right
usage of PyCObject_FromVoidPtrAndDesc?)

> As not all converters would need or support cleanup, converters
> that do need cleanup could be required to return Py_CLEANUP_SUPPORTED

Well, this is not implemented yet. (I introduced another format
temporary) Does this mean converter should return Py_CLEANUP_SUPPORTED
instead of 1 when cleanup is needed?

> Well, this is not implemented yet. (I introduced another format
> temporary) Does this mean converter should return Py_CLEANUP_SUPPORTED
> instead of 1 when cleanup is needed?

If you are introducing a new character ($), then this isn't actually
needed. I was thinking of reusing O&, literally.

Modifying convert_to_unicode is incorrect; this function is not an O&
converter. Instead, PyUnicode_FSConverter needs to change.

Attached is a patch that does that, and also uses the O& approach. It
also adjusts the patch to use capsules.

> Modifying convert_to_unicode is incorrect; this function is not an O&
> converter. Instead, PyUnicode_FSConverter needs to change.

Well, convert_to_unicode used to use O& before, and I fixed memory leak
with O formatter in current way. I used this function because I know
about it more than PyUnicode_FSConverter. Sorry for confusion. ;-)

> Attached is a patch that does that, and also uses the O& approach. It
> also adjusts the patch to use capsules.

Oh, this is how to use new capsule functions.... I noticed capsule.c was
added, but I didn't know how to use it.

> Well, convert_to_unicode used to use O& before

I don't understand. Where does it use it? Before
your patch, convert_to_unicode was a regular one-argument
function, not a ParseTuple O& function. Only your patch
makes it one.

Well, please see r65745. That was O& before.

So: any opinions what approach would be better? A new converter O$ or a
change to the existing O&, selected by return value from the conversion
function?

Reusing O& looks better to me (we already have too many type specifiers). 
Why have you chosen 0x20000 for Py_CLEANUP_SUPPORTED? Are there other
possible values below that?

> Reusing O& looks better to me (we already have too many type specifiers). 
> Why have you chosen 0x20000 for Py_CLEANUP_SUPPORTED? Are there other
> possible values below that?

It's essentially random. I want it to be a power of two, in case we make
it a bit mask some day, and significantly different from 1. 0x20000 was
the smallest number that qualified :-)

> Reusing O& looks better to me

Me too.

Patch review:

- It needs docs and tests.
- In addcleanup_convert, Py_FatalError calls abort, so there isn't
really a point of returning -1;
- Also in the function, you need to call destr() in the case that
PyList_Append fails.

I have now committed this patch with the proposed modifications as r73014.

I don't believe that the additional destr call is necessary, as
releasing the capsule will invoke destr already.