Author gregory.p.smith
Recipients Mark.Shannon, carljm, corona10, dino.viehland, eelizondo, eric.snow, gregory.p.smith, methane, nascheme, pablogsal, pitrou, remi.lapeyre, shihai1991, steve.dower, tim.peters, vstinner
Date 2022-01-14.19:39:46
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1642189186.43.0.839342202949.issue40255@roundup.psfhosted.org>
In-reply-to
Content
[data] I finally dug up the old YouTube doc at work with their findings. Mostly just posting this here for future public reference if anyone wants. Nothing surprising.

When youtube experimented with a modified 2.7 adding "eternal refcounts" in 2015, they saw a 3-5% CPU performance regression. (not the 10% I had in my mind)

Their version of this simply set a high bit on the refcount as the indicator and added the obvious conditional into the Py_INCREF/Py_DECREF macros.

Unsurprisingly in line with what others have since found. For their preforked server and decision of what to mark eternal before forking, it saved them 10% ram (fewer copy on writes). The -ram vs +cpu +maintenance cost tradeoff wound up not being worthwhile to them though. Their motivation for trying was entirely COW memory savings.

=== CPython 2.7 object.h modification they used:

```
+#ifdef GOOGLE_ETERNAL_REFCOUNT_SUPPORT
+
+#define PY_ETERNAL_REFCOUNT (PY_SSIZE_T_MAX / 2)
+
+#define Py_IS_ETERNAL(op) (                       \
+  ((PyObject*)(op))->ob_refcnt >= PY_ETERNAL_REFCOUNT)
+
+#define Py_SET_ETERNAL(op)                                \
+  do {                                                    \
+      ((PyObject*)(op))->ob_refcnt = PY_ETERNAL_REFCOUNT; \
+      if (PyObject_IS_GC(op)) {                           \
+        PyObject_GC_UnTrack(op);                          \
+      }                                                   \
+  } while (0)
+
+#define Py_INCREF(op) (                           \
+      Py_IS_ETERNAL(op)                           \
+        ?  PY_ETERNAL_REFCOUNT                    \
+        :  (_Py_INC_REFTOTAL  _Py_REF_DEBUG_COMMA \
+           ((PyObject*)(op))->ob_refcnt++)        \
+  )
+
+#define Py_DECREF(op)                                   \
+    do {                                                \
+        if (Py_IS_ETERNAL(op)) break;                   \
+        if (_Py_DEC_REFTOTAL  _Py_REF_DEBUG_COMMA       \
+        --((PyObject*)(op))->ob_refcnt != 0)            \
+            _Py_CHECK_REFCNT(op)                        \
+        else                                            \
+        _Py_Dealloc((PyObject *)(op));                  \
+    } while (0)
+
+#else
```
History
Date User Action Args
2022-01-14 19:39:46gregory.p.smithsetrecipients: + gregory.p.smith, tim.peters, nascheme, pitrou, vstinner, carljm, dino.viehland, methane, Mark.Shannon, eric.snow, steve.dower, corona10, pablogsal, eelizondo, remi.lapeyre, shihai1991
2022-01-14 19:39:46gregory.p.smithsetmessageid: <1642189186.43.0.839342202949.issue40255@roundup.psfhosted.org>
2022-01-14 19:39:46gregory.p.smithlinkissue40255 messages
2022-01-14 19:39:46gregory.p.smithcreate