Message344469
opcode cache for LOAD_GLOBAL introduced false alarm in memory leak hunting (python3 -m test -R 3:3 ...).
=> opcache: bpo-26219.
Before the change:
$ git checkout 91234a16367b56ca03ee289f7c03a34d4cfec4c8^
$ make && ./python -m test -R 3:3 test_pprint
...
Tests result: SUCCESS
After the change:
$ git checkout 91234a16367b56ca03ee289f7c03a34d4cfec4c8
$ make && ./python -m test -R 3:3 test_pprint
...
test_pprint leaked [4, 2, 4] memory blocks, sum=10
...
The problem is that at each iteration of regrtest -R 3:3 (6 iterations), a few more code objects get this opcache allocated.
There are different solutions to fix regrtest -R 3:3.
(*) Always optimize:
diff --git a/Python/ceval.c b/Python/ceval.c
index 411ba3d73c..6cd148efba 100644
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -103,7 +103,7 @@ static long dxp[256];
#endif
/* per opcode cache */
-#define OPCACHE_MIN_RUNS 1024 /* create opcache when code executed this time */
+#define OPCACHE_MIN_RUNS 1 /* create opcache when code executed this time */
#define OPCACHE_STATS 0 /* Enable stats */
#if OPCACHE_STATS
$ make && ./python -m test -R 3:3 test_pprint
...
Tests result: SUCCESS
(*) Never optimmize: disable opcache until a better fix can be found
diff --git a/Python/ceval.c b/Python/ceval.c
index 411ba3d73c..3c85df6fea 100644
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -1230,6 +1230,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
f->f_stacktop = NULL; /* remains NULL unless yield suspends frame */
f->f_executing = 1;
+#if 0
if (co->co_opcache_flag < OPCACHE_MIN_RUNS) {
co->co_opcache_flag++;
if (co->co_opcache_flag == OPCACHE_MIN_RUNS) {
@@ -1244,6 +1245,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
#endif
}
}
+#endif
#ifdef LLTRACE
lltrace = _PyDict_GetItemId(f->f_globals, &PyId___ltrace__) != NULL;
$ make && ./python -m test -R 3:3 test_pprint
...
Tests result: SUCCESS
(*) Find a way to explicitly deoptimize all code objects
Modules/gcmodule.c has a clear_freelists() function called by collect() if generation == NUM_GENERATIONS-1: on when gc.collect() is collected explicitly for example.
Lib/test/libregrtest/refleak.py also has a dash_R_cleanup() function which clears many caches.
Problem: currently, code objects are not explicitly tracked (for example, they are not tracked in a double linked list).
(*) Add way more warmup iterations to regrtest in buildbots.
I dislike this option. A build on a refleak buildbot worker already takes 2 to 3 hours. Adding more warmup would make a build even way more slower. |
|
Date |
User |
Action |
Args |
2019-06-03 20:34:43 | vstinner | set | recipients:
+ vstinner, methane, yselivanov |
2019-06-03 20:34:43 | vstinner | set | messageid: <1559594083.93.0.475089830437.issue37146@roundup.psfhosted.org> |
2019-06-03 20:34:43 | vstinner | link | issue37146 messages |
2019-06-03 20:34:43 | vstinner | create | |
|