Author andrewfg1992
Recipients andrewfg1992, ned.deily
Date 2020-06-26.18:11:05
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1593195068.14.0.719266681703.issue41129@roundup.psfhosted.org>
In-reply-to
Content
Quick repoduction steps:
1) Log into a mac with macOS version 10.15.1 (10.15.x may work)
2) For build tools, use Xcode11. A minimal xcode command-tools installation also reproduced for me.
3) Download and decompress the latest python 3.8.2 source
4) run "./configure" in the top-level source folder
5) run "make" in the top-level source folder

I believe this error may also occur when using python 2.7.x, 3.7.x, and others. Curiously, the errors do not occur when I use the latest 3.6.10 source.

A text file containing the output of "./configure" and "make" is attached.

Main Description:
On Mac 10.15.1 with Xcode11 I encounter compilation errors when building the latest Python 3.8.2 source. When the build reaches the "build_ext" section in setup.py, none of the extension modules build, with each compilation attempt producing an error. For example, when the _struct extension module attempts compilation I get:

--- Error Syndrome ---
building '_struct' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -I./Include/internal -I./Include -I. -I/usr/local/include -I/System/Volumes/Data/mathworks/devel/sandbox/aflewell/python38_source/Python-3.8.2/Include -I/System/Volumes/Data/mathworks/devel/sandbox/aflewell/python38_source/Python-3.8.2 -c _struct.c -o build/temp.macosx-10.15-x86_64-3.8/_struct.o
clang: error: no such file or directory: '_struct.c'
clang: error: no input files
------

After many similar errors, we get to the main build report where we see the failed extension modules:

--- Snippet of build report ---
Python build finished successfully!
The necessary bits to build these optional modules were not found:
_gdbm                 _hashlib              _ssl               
ossaudiodev           spwd                                     
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc                  atexit                pwd                
time                                                           


Failed to build these modules:
_asyncio              _bisect               _blake2            
_bz2                  _codecs_cn            _codecs_hk         
_codecs_iso2022       _codecs_jp            _codecs_kr         
_codecs_tw            _contextvars          _crypt             
_csv                  _ctypes               _ctypes_test       
_curses               _curses_panel         _datetime          
_dbm                  _decimal              _elementtree       
_heapq                _json                 _lsprof            
_lzma                 _md5                  _multibytecodec    
_multiprocessing      _opcode               _pickle            
_posixshmem           _posixsubprocess      _queue             
_random               _scproxy              _sha1              
_sha256               _sha3                 _sha512            
_socket               _sqlite3              _statistics        
_struct               _testbuffer           _testcapi          
_testimportmultiple   _testinternalcapi     _testmultiphase    
_tkinter              _uuid                 _xxsubinterpreters 
_xxtestfuzz           array                 audioop            
binascii              cmath                 fcntl              
grp                   math                  mmap               
nis                   parser                pyexpat            
readline              resource              select             
syslog                termios               unicodedata        
xxlimited             zlib
------

The part "-c _struct.c" in the error syndrome stands out to me because there is not a prefix like the other modules that get built successfully before the "build_ext" section. In the successful cases, we see a prefix like "-c ./Modules/faulthandler.c". This can be seen in my attached log.

Also, when I use the Python 3.6.10 source, the build for _struct looks like this and goes fine on Mac 10.15.1. This seems to be the only version of python that works for me on 10.15.1 that I know of.

-------
building '_struct' extension
creating build/temp.macosx-10.15-x86_64-3.6/System
creating build/temp.macosx-10.15-x86_64-3.6/System/Volumes
creating build/temp.macosx-10.15-x86_64-3.6/System/Volumes/Data
creating build/temp.macosx-10.15-x86_64-3.6/System/Volumes/Data/my_company
creating build/temp.macosx-10.15-x86_64-3.6/System/Volumes/Data/my_company/devel
creating build/temp.macosx-10.15-x86_64-3.6/System/Volumes/Data/my_company/devel/sandbox
creating build/temp.macosx-10.15-x86_64-3.6/System/Volumes/Data/my_company/devel/sandbox/my_username
creating build/temp.macosx-10.15-x86_64-3.6/System/Volumes/Data/my_company/devel/sandbox/my_username/python36_source
creating build/temp.macosx-10.15-x86_64-3.6/System/Volumes/Data/my_company/devel/sandbox/my_username/python36_source/Python-3.6.10
creating build/temp.macosx-10.15-x86_64-3.6/System/Volumes/Data/my_company/devel/sandbox/my_username/python36_source/Python-3.6.10/Modules
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -I./Include -I. -I/usr/local/include -I/System/Volumes/Data/my_company/devel/sandbox/my_username/python36_source/Python-3.6.10/Include -I/System/Volumes/Data/my_company/devel/sandbox/my_username/python36_source/Python-3.6.10 -c /System/Volumes/Data/my_company/devel/sandbox/my_username/python36_source/Python-3.6.10/Modules/_struct.c -o build/temp.macosx-10.15-x86_64-3.6/System/Volumes/Data/my_company/devel/sandbox/my_username/python36_source/Python-3.6.10/Modules/_struct.o
gcc -bundle -undefined dynamic_lookup build/temp.macosx-10.15-x86_64-3.6/System/Volumes/Data/my_company/devel/sandbox/my_username/python36_source/Python-3.6.10/Modules/_struct.o -L/usr/local/lib -o build/lib.macosx-10.15-x86_64-3.6/_struct.cpython-36m-darwin.so
-----

In this successful case, you can see absolute paths are used to compile the .c source, and some temp folders are created for the .o output. I wonder if there is a workaround where we pass a flag to the configure script to produce the same effects? Obviously it would be nice if a plain build worked though.

Also, on Mac 10.14.5, the builds of any python succeeds similarly with no such errors. I am wondering why I am seeing this new pathing and filing behavior on mac 10.15.1 for most versions of Python. Are there any viable workarounds? Thanks for reading!
History
Date User Action Args
2020-06-26 18:11:08andrewfg1992setrecipients: + andrewfg1992, ned.deily
2020-06-26 18:11:08andrewfg1992setmessageid: <1593195068.14.0.719266681703.issue41129@roundup.psfhosted.org>
2020-06-26 18:11:08andrewfg1992linkissue41129 messages
2020-06-26 18:11:07andrewfg1992create