Title: Make documentation about macros in C API explicit about rvalue vs statement
Type: enhancement Stage: needs patch
Components: C API, Documentation Versions: Python 3.4
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Arfrever, Mark.Shannon, amaury.forgeotdarc, docs@python, georg.brandl, larry, loewis, mark.dickinson, pitrou
Priority: normal Keywords:

Created on 2013-03-31 19:39 by larry, last changed 2020-06-25 09:41 by vstinner.

Messages (7)
msg185646 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-03-31 19:39
CPython API "functions" implemented as macros can expand into either
rvalues or statements.  Most expand into statements (often using the
do {} while (0) syntactic sugar trick), but occasionally they're legal
as rvalues.

As of this writing Py_INCREF works as an rvalue.  But discussion on
another tracker issue (#17206) proposes changing the implementation
in such a way that it will only be usable as a statement.  Although
it's mildly unlikely, it's certainly possible that this will break
somebody's code.

I propose that the documentation make an explicit ruling on whether
macros are usable as rvalues or as statements.  Perhaps a blanket
statement would suffice, "all macros are only supported for use as
statements except where explicitly noted", then annotate specific
macros that are supported for use as rvalues.  Though that raises the
question of acknowledging in the documentation that some things are
macros--I think the documentation glosses over that fact right now.

Note: I added you three (Georg, Mark, Martin) as I thought you might
have an opinion on this one way or the other.  If you're not interested,
my apologies.
msg185667 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2013-03-31 20:25
There are also some macros usable as lvalues, such as Py_REFCNT or Py_SIZE (they aren't documented at all currently).

Anyway, documenting as statement-only unless explicitly stated differently is fine with me.
msg185668 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-03-31 20:40
Py_INCREF usable as an rvalue sounds more like an accident than a deliberate feature, and it would be IMO counter-productive to codify this behaviour in the docs.

As for the lvalue usage of Py_REFCNT and Py_SIZE, I think it would be better if it were limited to CPython core. But arguably writers of extension types may want to mutate the size field of a varsize object.
msg185711 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-04-01 04:55
> Py_INCREF usable as an rvalue sounds more like an accident
> than a deliberate feature

I'd go with "misfeature", but in no way is it accidental.  The coding deliberately preserves the rvalue-ness of it, c.f. _Py_REF_DEBUG_COMMA.
msg186317 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2013-04-08 16:59
There are some extension modules (pytables) that do
  return Py_INREF(x), x;
and Py_RETURN_NONE is also defined with a comma expression.

Oh, and Cython:
#define __Pyx_PyBool_FromLong(b) ((b) ? (Py_INCREF(Py_True), Py_True) : (Py_INCREF(Py_False), Py_False))
msg186367 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-04-09 02:09
Amaury: I'd appreciate it if you'd bring those examples up on bug 17206.  If we're going to change the semantics of Py_INCREF I'd prefer we did it with our eyes wide open.
msg186369 - (view) Author: Larry Hastings (larry) * (Python committer) Date: 2013-04-09 02:15
Oh, and, yes, it's true that Py_RETURN_NONE currently takes advantage of Py_INCREF being an rvalue, and changing Py_INCREF to a statement would break the existing implementation.  But Py_RETURN_NONE itself is of necessity a statement.   We would simply change Py_RETURN_NONE's implementation to multiple statements, probably with the do { ... } while(0) trick, so it worked again.  I'd be shocked if that change broke any existing code.  So that's no big deal.

Having external code that depends on Py_INCREF being an rvalue is my concern, and what I hoped you'd bring up on bug #17206.
Date User Action Args
2020-06-25 09:41:19vstinnersetcomponents: + C API
2013-05-16 12:54:40Arfreversetnosy: + Arfrever
2013-04-09 02:15:15larrysetmessages: + msg186369
2013-04-09 02:09:28larrysetmessages: + msg186367
2013-04-08 17:10:52mark.dickinsonsetnosy: + mark.dickinson
2013-04-08 16:59:08amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg186317
2013-04-01 04:55:13larrysetmessages: + msg185711
2013-03-31 20:40:45pitrousetnosy: + pitrou
messages: + msg185668
2013-03-31 20:25:39georg.brandlsetmessages: + msg185667
2013-03-31 19:39:15larrycreate