This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: _scproxy and Python Launcher fail to build wtih GNU gcc on macOS 10.13+
Type: compile error Stage: patch review
Components: Build, macOS Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: ned.deily Nosy List: Arfrever, Jeffrey.Kintscher, Robert P Fischer, corona10, ned.deily, python-dev, ronaldoussoren, terry.reedy, vstinner
Priority: high Keywords: patch

Created on 2016-02-09 04:36 by Robert P Fischer, last changed 2022-04-11 14:58 by admin.

Files
File name Uploaded Description Edit
log Robert P Fischer, 2016-02-09 04:35 Log file from attempt to build Python
Pull Requests
URL Status Linked Edit
PR 13306 closed Jeffrey.Kintscher, 2019-05-14 05:24
PR 19681 closed python-dev, 2020-04-23 16:04
PR 20176 merged ned.deily, 2020-05-18 12:53
PR 20177 vstinner, 2020-05-18 14:51
PR 20182 merged ned.deily, 2020-05-18 15:11
Messages (27)
msg259914 - (view) Author: Robert P Fischer (Robert P Fischer) Date: 2016-02-09 04:35
Changes to OS X 10.11 render GCC's Objective-C compiler useless.  However, I want to compile the main part of Python in GCC (because my C++ / Fortran Cython modules use GCC).

I tried to build Python (via MacPorts) using Clang for Objective-C and GCC for C/C++.  The environment upon running ./configure included:

  CC='/Users/rpfische/macports/mpgompi-4.9.3/bin/gcc-mp-4.9'
  CXX='/Users/rpfische/macports/mpgompi-4.9.3/bin/gcc-mp-4.9'
  OBJC='/usr/bin/clang'
  OBJCXX='/usr/bin/clang++'

HOWEVER... the build still tried to use GCC to compile Objective-C, and failed miserably:



:info:destroot /Users/rpfische/macports/mpgompi-4.9.3/bin/gcc-mp-4.9 -pipe -Os -arch x86_64 -Wno-unused-result -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -o FileSettings.o -c ./FileSettings.m
:info:destroot /usr/include/objc/NSObject.h:22:4: error: unknown type name 'instancetype'
:info:destroot  - (instancetype)self;
:info:destroot     ^
...



Log file attached.
msg259927 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2016-02-09 11:39
The build fails while building the "Python Launcher" application. This is an Objective-C project. 

If GCC doesn't support the system headers on OSX 10.11 there's nothing the Python project can do about that.
msg259929 - (view) Author: Robert P Fischer (Robert P Fischer) Date: 2016-02-09 12:26
I'm trying to compile the Objective-C portions of Python (the Python
Launcher app) with clang, not GCC.  Why does Python insist on using GCC,
even when I set the OBJC env variable to clang?  That seems to be a problem
that the Python project CAN fix.

On Tue, Feb 9, 2016 at 6:39 AM, Ronald Oussoren <report@bugs.python.org>
wrote:

>
> Ronald Oussoren added the comment:
>
> The build fails while building the "Python Launcher" application. This is
> an Objective-C project.
>
> If GCC doesn't support the system headers on OSX 10.11 there's nothing the
> Python project can do about that.
>
> ----------
> nosy: +ronaldoussoren
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue26317>
> _______________________________________
>
msg260113 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2016-02-11 16:05
The makefiles use CC throughout and don't look at and OBJC variable. Is that variable a standard way to specify an ObjC compiler in makefiles? 

Make's default ruleset for compiling .m files uses $(CC) and doesn't look at $(OBJC).

I wouldn't be opposed to a patch that makes it possible to specify an ObjC compiler, could you write such a patch?  BTW. The patch should also update the autoconf script (use AC_PROG_OBJC).
msg260162 - (view) Author: Robert P Fischer (Robert P Fischer) Date: 2016-02-12 04:45
> The makefiles use CC throughout and don't look at and OBJC variable. Is
> that variable a standard way to specify an ObjC compiler in makefiles?

I believe that OBJC and OBJCFLAGS are standard for Autoconf/Automake:

https://www.gnu.org/software/automake/manual/html_node/Objective-C-Support.html

https://www.gnu.org/software/autoconf/manual/autoconf-2.61/html_node/Output-Variable-Index.html#Output-Variable-Index

https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=using%20objective%20c%20with%20autoconf%20automake

I wouldn't be opposed to a patch that makes it possible to specify an ObjC
> compiler, could you write such a patch?  BTW. The patch should also update
> the autoconf script (use AC_PROG_OBJC).

Sorry, I don't do Autotools --- a software package that came to us directly
from the seventh level of Hell.  CMake's main claim to fame is it's not as
bad.
msg260194 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-02-12 19:34
3.4 only gets security fixes
msg260195 - (view) Author: Robert P Fischer (Robert P Fischer) Date: 2016-02-12 19:40
Could this be fixed on 3.5 and 3.6?

On Fri, Feb 12, 2016 at 2:34 PM, Terry J. Reedy <report@bugs.python.org>
wrote:

>
> Terry J. Reedy added the comment:
>
> 3.4 only gets security fixes
>
> ----------
> nosy: +terry.reedy
> stage:  -> needs patch
> type:  -> compile error
> versions: +Python 3.5, Python 3.6 -Python 3.4
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue26317>
> _______________________________________
>
msg260215 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-02-12 23:47
When you reply by email to an email, please snip off the quote, except for possible a line that you are responding to.  The disposition of this issue for 3.5+ is up to Ned or Ronald.
msg331768 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2018-12-13 19:58
3.5 is and 3.6 will soon be on security fix only status.  Robert, please verify that this is still an issue with 3.7 and the current build files.
msg342037 - (view) Author: Jeffrey Kintscher (Jeffrey.Kintscher) * Date: 2019-05-10 02:00
This is still an issue with the 3.7 and the master branches. The Objective-C portion of the build still fails with gcc 8.3.0 on OS X Mojave (14.x). Specifying OBJC and OBJC++ on the configure command line does nothing because there is nothing in the script using the options.
msg342055 - (view) Author: Jeffrey Kintscher (Jeffrey.Kintscher) * Date: 2019-05-10 05:41
Module/_scproxy.c compiles cleanly with clang 10.0.0 and fails with GNU gcc 8.3.0. Below is the relevant compiler error:

building '_scproxy' extension
gcc-8 -Wno-unused-result -Wsign-compare -g -Og -Wall -pipe -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I../cpython/Include/internal -I../cpython/Include -IObjects -IPython -I. -I/usr/local/opt/zlib/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/ossp-uuid/1.6.2_2/include -I/usr/local/include -I/Users/jeff/sandbox/src/python3.7/cpython/Include -I/Users/jeff/sandbox/src/python3.7/build -c /Users/jeff/sandbox/src/python3.7/cpython/Modules/_scproxy.c -o build/temp.macosx-10.14-x86_64-3.8-pydebug/Users/jeff/sandbox/src/python3.7/cpython/Modules/_scproxy.o
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Security.framework/Headers/AuthSession.h:32,
                 from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Security.framework/Headers/Security.h:42,
                 from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/SystemConfiguration.framework/Headers/SCPreferences.h:35,
                 from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/SystemConfiguration.framework/Headers/SystemConfiguration.h:126,
                 from /Users/jeff/sandbox/src/python3.7/cpython/Modules/_scproxy.c:6:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Security.framework/Headers/Authorization.h:193:7:error: variably modified 'bytes' at file scope
  char bytes[kAuthorizationExternalFormLength];
       ^~~~~
building 'zlib' extension
msg342081 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-05-10 17:23
THe _scproxy.c compile error is a separate issue.  The problem arises in an Apple-supplied include file and there are many reports on the web of clang vs gcc differences like this.  I'm not sure what we could or should do about it other than forcing _scproxy to always be compiled with clang or figure out a way to avoid use of that include file hierarchy.
msg342139 - (view) Author: Jeffrey Kintscher (Jeffrey.Kintscher) * Date: 2019-05-10 21:53
My mistake was not adding

--enable-framework=${HOME}/sandbox/src/python3.7/inst

to the configure script and then running

make
make install

(I know... read the README file)

gcc 8.3.0 doesn't like the Xcode system headers included by the .m files. I will dig into this now that I am using the proper build steps.

As a side note, I find it counter-intuitive that Mac/Makefile only gets invoked by 'make install' to build the Objective-C source files. IMHO they should be built by "make" and then installed by "make install". Forcing a build and install in one step can be dangerous and accidentally leave the system in an undesirable state when the build process fails with a partial install. Also, --enable-framework should use a subpath of --prefix by default for the framework installation instead of defaulting to /Library/Frameworks regardless of the --prefix setting.
msg342179 - (view) Author: Jeffrey Kintscher (Jeffrey.Kintscher) * Date: 2019-05-11 10:18
I got it to build PythonLauncher using clang and everything else using gcc 8.3.0 by making a few configure script and makefile changes. Unfortunately, "make install" still fails because a python script that requires the unbuilt _scproxy module crashes. I'll work on fixing that next.
msg342432 - (view) Author: Jeffrey Kintscher (Jeffrey.Kintscher) * Date: 2019-05-14 05:26
I submitted a pull request that fixes both the PythonLauncher build issue and the _scproxy module build issue. They are related because _scproxy is required to install PythonLauncher.
msg359893 - (view) Author: Dong-hee Na (corona10) * (Python committer) Date: 2020-01-13 10:00
Any progress?
msg359894 - (view) Author: Dong-hee Na (corona10) * (Python committer) Date: 2020-01-13 10:03
I met this issue during I compile with gcc9 on my local mac machine.
This issue should be fixed..
msg359922 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2020-01-13 18:16
The progress was the PR.  I just fixed the merge conflicts, I believe correctly.  Please test and review.
msg359923 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-01-13 18:19
Thanks, Terry. I'll look at it shortly.
msg369213 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-05-18 13:17
New changeset 0da546665075aefbb476e192ed64122d340164f4 by Ned Deily in branch 'master':
bpo-26317: Support OBJC and OBJCXX configure command line variables (GH-20176)
https://github.com/python/cpython/commit/0da546665075aefbb476e192ed64122d340164f4
msg369218 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-05-18 13:38
Thanks everyone for their comments and reviews and a very big thank you to Jeffrey for working through the details and providing a solid PR. The change will appear in the 3.9.0b1 pre-release.
msg369224 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-18 14:06
This change broke the Python compilation on FreeBSD:
https://buildbot.python.org/all/#/builders/152/builds/855

Either revert it (and re-apply a fixed change after beta1), or fix it. I suggest to block Python 3.9 beta1 release until it's resolved. I consider that FreeBSD is a supported platform.
msg369227 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-05-18 14:08
Sorry about that! Looking now.
msg369246 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-05-18 15:31
New changeset 951ab58024de9b5a21f0b979cdbea51e1049d781 by Ned Deily in branch 'master':
Revert "bpo-26317: Support OBJC and OBJCXX configure command line variables (GH-20176)" (GH-20182)
https://github.com/python/cpython/commit/951ab58024de9b5a21f0b979cdbea51e1049d781
msg369247 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2020-05-18 15:32
I'm not sure yet exactly what the problem is but it's almost certainly a difference between GNU make and BSD make and something that needs to be checked on FreeBSD. With the 3.9.0b1 cutoff coming very soon, I likely won't be able to test on FreeBSD before then so I have reverted the change for now. Sigh!
msg369248 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-05-18 15:32
I tested manually and I confirm that FreeBSD builds again, thanks :-)
msg369325 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2020-05-19 08:11
Also there was FLAFS typo in this line, which is probably not needed at all:
OBJCFLAFS=@OBJCFLAGS@


AC_SUBST(OBJCXX) was added, but there was no usage of @OBJCXX@ anywhere...
History
Date User Action Args
2022-04-11 14:58:27adminsetgithub: 70505
2020-05-19 08:11:12Arfreversetnosy: + Arfrever
messages: + msg369325
2020-05-18 15:32:49vstinnersetmessages: + msg369248
2020-05-18 15:32:22ned.deilysetpriority: release blocker -> high

messages: + msg369247
2020-05-18 15:31:36ned.deilysetmessages: + msg369246
2020-05-18 15:11:46ned.deilysetpull_requests: + pull_request19483
2020-05-18 14:51:54vstinnersetstage: resolved -> patch review
pull_requests: + pull_request19481
2020-05-18 14:08:34ned.deilysetmessages: + msg369227
2020-05-18 14:06:45vstinnersetstatus: closed -> open
priority: normal -> release blocker

nosy: + vstinner
messages: + msg369224

resolution: fixed ->
2020-05-18 13:38:16ned.deilysetstatus: open -> closed
versions: + Python 3.9, - Python 3.7, Python 3.8
title: Build Problem with GCC + Macintosh OS X 10.11 El Capitain -> _scproxy and Python Launcher fail to build wtih GNU gcc on macOS 10.13+
messages: + msg369218

resolution: fixed
stage: patch review -> resolved
2020-05-18 13:17:25ned.deilysetmessages: + msg369213
2020-05-18 12:53:36ned.deilysetpull_requests: + pull_request19474
2020-04-23 16:04:12python-devsetnosy: + python-dev
pull_requests: + pull_request19002
2020-01-13 18:19:58ned.deilysetassignee: ned.deily

messages: + msg359923
nosy: - vstinner
2020-01-13 18:16:58terry.reedysetnosy: + vstinner
messages: + msg359922
2020-01-13 18:05:21vstinnersetnosy: + corona10, - vstinner
2020-01-13 18:04:44vstinnersetnosy: - corona10
2020-01-13 10:08:40corona10setnosy: + vstinner
2020-01-13 10:03:07corona10setmessages: + msg359894
2020-01-13 10:00:37corona10setnosy: + corona10
messages: + msg359893
2019-05-14 05:26:57Jeffrey.Kintschersetmessages: + msg342432
2019-05-14 05:24:48Jeffrey.Kintschersetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request13216
2019-05-11 10:18:48Jeffrey.Kintschersetmessages: + msg342179
2019-05-10 21:53:03Jeffrey.Kintschersetmessages: + msg342139
2019-05-10 17:23:12ned.deilysetmessages: + msg342081
2019-05-10 05:41:39Jeffrey.Kintschersetmessages: + msg342055
2019-05-10 02:00:17Jeffrey.Kintschersetnosy: + Jeffrey.Kintscher

messages: + msg342037
versions: + Python 3.8
2018-12-13 19:58:50terry.reedysetnosy: + terry.reedy

messages: + msg331768
versions: + Python 3.7, - Python 3.5, Python 3.6
2016-02-12 23:47:31terry.reedysetnosy: - terry.reedy
2016-02-12 23:47:16terry.reedysetmessages: + msg260215
2016-02-12 19:40:22Robert P Fischersetmessages: + msg260195
2016-02-12 19:34:30terry.reedysetversions: + Python 3.5, Python 3.6, - Python 3.4
nosy: + terry.reedy

messages: + msg260194

type: compile error
stage: needs patch
2016-02-12 04:45:53Robert P Fischersetmessages: + msg260162
2016-02-11 16:05:05ronaldoussorensetnosy: + ned.deily
messages: + msg260113
components: + macOS
2016-02-09 12:26:51Robert P Fischersetmessages: + msg259929
2016-02-09 11:39:38ronaldoussorensetnosy: + ronaldoussoren
messages: + msg259927
2016-02-09 04:36:24Robert P Fischercreate