classification
Title: Add -fexception to ppc64le build
Type: enhancement Stage: resolved
Components: Build Versions: Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, brunoalr, gut, haypo, pitrou
Priority: normal Keywords:

Created on 2017-07-12 16:13 by brunoalr, last changed 2017-08-17 12:11 by gut. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 2682 closed brunoalr, 2017-07-12 16:16
Messages (12)
msg298231 - (view) Author: Bruno Rosa (brunoalr) * Date: 2017-07-12 16:13
To correctly throw and catch exceptions in code that is written in both C and C++, the -fexception is needed.

This issue was also reported downstream on Debian and Ubuntu: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=862925
msg300342 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-08-16 09:26
Can you please elaborate?

I don't know the compiler flag -fexceptions. Why is it specific to ppc64le? What about ppc64? What about other platforms?

Is this issue specific to Python 2.7? If not, please tag add also 3.6 and 3.7 versions.

I never used Python with C++. I didn't work on PPC64 recently.
msg300352 - (view) Author: Bruno Rosa (brunoalr) * Date: 2017-08-16 12:20
Hi there, I'm delegating this issue to @gut. He will elaborate it with more detail.

Thanks a lot,
Bruno Rosa
msg300355 - (view) Author: Gustavo Serra Scalet (gut) * Date: 2017-08-16 12:47
-fexceptions documentation:
https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#index-fexceptions

Highlight to: "you may need to enable this option when compiling C code that needs to interoperate properly with exception handlers written in C++"

And this issue was detected when cpython is used like the following:
.---------------------.
|     A C++ code      | <= Designed to be the catcher
:---------------------:
|       cpython       | <= Should only pass exception up
:---------------------:
| called C++ function | <= Raises a C++ exception
'---------------------'

This is already working for X64, but not on PPC64LE (and possibly ARM64 and other architectures as well).

For some reason the C++ unwinder can unwind C frames without special unwind information on X64. Maybe it can assume frame information on this architectures and it succeeds, but that's just pure luck in my opinion.

ps: this patch is not v2.7 specific. It should be applied globally to other versions as well. It was merely reported to v2.7 as it was the python version that our c++ library was using. Maybe it'll be easier to backport this patch to other versions once it's in.
msg300357 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-08-16 12:54
I'm not sure why we should apply this patch to CPython.  We don't have any tests checking for interoperability with C++ exceptions.  We don't have any PPC64LE expertise in-house.  It's probably fine for third-party distributors to apply the patch themselves.

It's also rather unexpected that a flag is needed for this at all.  Why doesn't gcc make it the default behaviour?
msg300362 - (view) Author: Gustavo Serra Scalet (gut) * Date: 2017-08-16 13:33
> Why doesn't gcc make it the default behaviour?

Because it "can produce significant data size overhead, although it does not affect execution" for C code.

However it'd improves interoperability with C++, which I think it makes sense for a universal software like cpython.

Or isn't interoperability with c++ a concern? I noticed you said that there are not tests for it currently, but I wonder the reason behind it.
msg300363 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-08-16 13:37
> Or isn't interoperability with c++ a concern? I noticed you said that there are not tests for it currently, but I wonder the reason behind it.

I suppose nobody cared enough, and/or those tests are difficult to write (you need a cross-platform way to invoke a C++ compiler with exceptions enabled).
msg300376 - (view) Author: Gustavo Serra Scalet (gut) * Date: 2017-08-16 16:51
well, now I'm confused: what should we do with this change? Do we need to ask any other expert?
msg300381 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-08-16 17:25
Several comments:
- we don't want to risk bugfix branches for this, so this should only go into the master branch (future 3.7)
- I'm not sure it's a reasonable policy to fix shortcomings of all compilers in our configure script; though of course we already have such code for common setups
- regarding the PR, OPT does not look like the right variable to use (it's not an optimization setting)
msg300400 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2017-08-17 07:32
Throwing exceptions through CPython is totally unsupported, -fexceptions or not. Like C++ code that's not exception-aware, CPython lacks any of the catch handlers to properly clean up resources on unwind. Thus, throwing exceptions through CPython will lead to, at best, resource leaks and, at worst, subtle corruption of interpreter data structures.

I'm not sure why Debian would take this patch.
msg300407 - (view) Author: Gustavo Serra Scalet (gut) * Date: 2017-08-17 11:30
> Throwing exceptions through CPython is totally unsupported, -fexceptions or not. Like C++ code that's not exception-aware, CPython lacks any of the catch handlers to properly clean up resources on unwind. 

wait wait. It's not expected that CPython would handle the exception at all! The only thing it's needed for CPython is to let the compiler generate extra information about how each function looks like in the stack (how the frame is structured, in DWARF language) in order for the unwinder to do his job.

As documented, the code would not change its current behavior (no machine code is added or removed, so when debugging your function looks the same) but extra information is given in other for unwind to happen through those frames.
msg300408 - (view) Author: Gustavo Serra Scalet (gut) * Date: 2017-08-17 12:11
Sorry, I didn't take a deeper analysis on what you said, I guess you wanted to point out this problem:

.---------------------.
|     A C++ code      | <= Designed to be the catcher
:---------------------:
|       cpython       | <= Malloc'd something but unwind will not free it
:---------------------:
| called C++ function | <= Raises a C++ exception
'---------------------'

I guess memory leak would be a consequence, yes. *Sigh*
History
Date User Action Args
2017-08-17 12:11:28gutsetmessages: + msg300408
2017-08-17 11:30:35gutsetmessages: + msg300407
2017-08-17 07:32:33benjamin.petersonsetstatus: open -> closed

nosy: + benjamin.peterson
messages: + msg300400

resolution: not a bug
stage: resolved
2017-08-16 17:25:56pitrousettype: enhancement
messages: + msg300381
components: + Build
versions: + Python 3.7, - Python 2.7
2017-08-16 16:51:35gutsetmessages: + msg300376
2017-08-16 13:37:03pitrousetmessages: + msg300363
2017-08-16 13:33:42gutsetmessages: + msg300362
2017-08-16 12:54:28pitrousetmessages: + msg300357
2017-08-16 12:47:57gutsetmessages: + msg300355
2017-08-16 12:20:59brunoalrsetmessages: + msg300352
2017-08-16 12:20:33gutsetnosy: + gut
2017-08-16 09:26:33hayposetnosy: + haypo, pitrou
messages: + msg300342
2017-07-12 16:16:10brunoalrsetpull_requests: + pull_request2748
2017-07-12 16:13:57brunoalrcreate