New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
opcode cache for LOAD_GLOBAL emits false alarm in memory leak hunting #81327
Comments
opcode cache for LOAD_GLOBAL introduced false alarm in memory leak hunting (python3 -m test -R 3:3 ...). => opcache: bpo-26219. Before the change: $ git checkout 91234a16367b56ca03ee289f7c03a34d4cfec4c8^
$ make && ./python -m test -R 3:3 test_pprint
...
Tests result: SUCCESS After the change: $ git checkout 91234a16367b56ca03ee289f7c03a34d4cfec4c8
$ make && ./python -m test -R 3:3 test_pprint
...
test_pprint leaked [4, 2, 4] memory blocks, sum=10
... The problem is that at each iteration of regrtest -R 3:3 (6 iterations), a few more code objects get this opcache allocated. There are different solutions to fix regrtest -R 3:3. (*) Always optimize: diff --git a/Python/ceval.c b/Python/ceval.c
index 411ba3d73c..6cd148efba 100644
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -103,7 +103,7 @@ static long dxp[256];
#endif
/* per opcode cache */
-#define OPCACHE_MIN_RUNS 1024 /* create opcache when code executed this time */
+#define OPCACHE_MIN_RUNS 1 /* create opcache when code executed this time */
#define OPCACHE_STATS 0 /* Enable stats */
#if OPCACHE_STATS $ make && ./python -m test -R 3:3 test_pprint
...
Tests result: SUCCESS (*) Never optimmize: disable opcache until a better fix can be found diff --git a/Python/ceval.c b/Python/ceval.c
index 411ba3d73c..3c85df6fea 100644
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -1230,6 +1230,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
f->f_stacktop = NULL; /* remains NULL unless yield suspends frame */
f->f_executing = 1;
+#if 0
if (co->co_opcache_flag < OPCACHE_MIN_RUNS) {
co->co_opcache_flag++;
if (co->co_opcache_flag == OPCACHE_MIN_RUNS) {
@@ -1244,6 +1245,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
#endif
}
}
+#endif
#ifdef LLTRACE
lltrace = _PyDict_GetItemId(f->f_globals, &PyId___ltrace__) != NULL; $ make && ./python -m test -R 3:3 test_pprint
...
Tests result: SUCCESS (*) Find a way to explicitly deoptimize all code objects Modules/gcmodule.c has a clear_freelists() function called by collect() if generation == NUM_GENERATIONS-1: on when gc.collect() is collected explicitly for example. Lib/test/libregrtest/refleak.py also has a dash_R_cleanup() function which clears many caches. Problem: currently, code objects are not explicitly tracked (for example, they are not tracked in a double linked list). (*) Add way more warmup iterations to regrtest in buildbots. I dislike this option. A build on a refleak buildbot worker already takes 2 to 3 hours. Adding more warmup would make a build even way more slower. |
INADA-san wrote PR 13787 to disable opcache when Python is compiled in debug mode. Pablo and me approved the change, so I merged it. Pablo wrote a more robust solution, PR 13789, to disable opcache only in regrtest, to look for memory leaks. But INADA-san had a good argument against this approach: "The code object will be optimized only when ++co->co_opcache_flag == opcacheminruns. So decreasing min_runs by _setopcacheminruns()will cause some hot codes will be not optimized forever. I don't want to expose such switch." I would prefer to keep this issue until a long term approach is designed. |
It seems like nobody came up with a solution for the debug mode since June. I close the issue. Reopen it if you would like to propose a solution. |
I think the only solution is to have a flag to disable optimizations, inlcluding this one. |
Please revert this and use a separate build flag (e.g. DISABLE_INLINE_CACHES) for the refleaks run. (Or perhaps provide an -X flag to disable it without the need to recompile.) I am developing new inline cache ideas and of course I need to run in debug mode, so I regularly end up wasting some time until I remember to add a patch that changes OPCACHE_MIN_RUNS back. |
I reopen the issue (but I'm not interested by trying to fix it). |
Giving the ability to control the cache size, at least at Python startup, is one option. Or maybe need a generic -X flag to tell Python that libregrtest is tracking leaks, since we might need to change other parameters than only the opcode cache size. |
I'd really prefer not to allow users to control cache sizes. There's basically no point in that; the only practically useful thing is to enable or disable it. |
I think we should go this way, maybe even less "public". If we add a -X, then it will be too visible. Maybe some env variable? |
dans cette version pour securité maximale SSLContext.wrap_socket()
import socket
import ssl
hostname = 'www.python.org'
context = ssl.create_default_context()
with socket.create_connection((hostname, 443)) as sock:
with context.wrap_socket(sock, server_hostname=hostname) as ssock:
print(ssock.version()) |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: