New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
./configure --with-pydebug should use -O0 rather than -Og #82531
Comments
In Python 2.7, when using ./configure --with-pydebug, Python is built using -O0 compiler optimization level: disable all optimizations. The comment is quite explicit: "Optimization messes up debuggers, so turn it off for debug builds". if test "$Py_DEBUG" = 'true' ; then In Python 3, -Og is preferred over -O0 for pydebug, if -Og is available: if test "$Py_DEBUG" = 'true' ; then Problem: in my experience, gdb traceback doesn't make sense sometimes, and gdb fails to inspect some function arguments and some variables which are displayed as <optimized out>. See a very concrete example with a test_gdb failure on x86-64 when Python is compiled using gcc -Og: My colleague who is working on gdb suggests to use -O0: Since I started contributing to Python, I always forced gcc -O0 because any other optimization level caused me many issues in gdb. I'm using -O0 for 10 years with sucess. The GCC documentation says "It is a better choice than -O0 for producing debuggable code because some compiler passes that collect debug information are disabled at -O0." But my experience says the opposite. Note: instead of -g, we could use -g3 to include debug information for macros and defines. -- I'm proposing to change the *default* compiler flags from -Og to -O0. Obviously, Linux distributions and developers are free to override the compiler optimization level. For example: ./configure --with-pydebug CFLAGS="-Og" ensures that Python is always built using -Og. I propose to modify 3.7, 3.8 and master branches. |
I basically propose to revert bpo-23445 change: commit 3d6c784
|
test_gdb is failing in different architectures and different operating systems:
I'm quite sure that most issues are caused by the compiler optimization level which is not -O0. See also bpo-37631: the debug build of the RHEL8 package is supposed to be compiled using -O0 using EXTRA_CFLAGS variable, but -O0 in EXTRA_CFLAGS variable is overriden by -02 in CFLAGS_NODIST. |
My use case is to debug a crash using a Python compiled in debug module, like python3-debug program in Fedora. Since the debug ABI is now compatible with the release build, the idea is to attempt to reproduce a crash in gdb using python3-debug instead of python3, and then use gdb to see what's going on. With -Og, the call stack is wrong sometimes, and some function arguments and local variables cannot be read (displayed as <optimized out>). On Travis CI, a few months ago, Python was built in debug mode using -O3. But it was a side effect of OpenSSL flags if I recall correctly. With my PR 16544, Travis CI now uses -O0. |
Do note though that if the -D_FORTIFY_SOURCE=2 hardening flag is used, the compilation will fail with an optimization level less than -Og. Haven't tried yet with -D_FORTIFY_SOURCE=1 to see if it works with -O0. |
Correction, not fail, just a ton of warnings. The same is true for -D_FORTIFY_SOURCE=1 |
If -Og is breaking debugging, isn't that a compiler bug? The GCC manpage claims " -Og enables optimizations that do not interfere with debugging." |
It seems that it's still being worked on from gcc's side: |
Does it make any sense to use a debug build in production? Does it make sense to enable hardening on a debug buil?d |
The purpose of using -Og is that it significantly speeds up tests. You can always pass CFLAGS="-O0" if you debug particular debugger issues, but in normal cases -Og looks more beneficial. It saves hours of time of common developers. |
There consensus seems to be towards the status quo: -Og is a good trade off between debug and performance for most core developers usage. For the Fedora package, it may make sense to use -O0, but that should be discussed in Fedora ;-) So I close the issue. Thanks for the interesting discussion :-) |
See also bpo-40019: "test_gdb should better detect when Python is optimized". |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: