This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Compilation errors on macOS
Type: compile error Stage: resolved
Components: macOS Versions: Python 3.11
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: Nosy List: ned.deily, ronaldoussoren, sandipshah
Priority: normal Keywords:

Created on 2022-01-03 19:36 by sandipshah, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (5)
msg409611 - (view) Author: Sandip Shah (sandipshah) Date: 2022-01-03 19:36
macOS Monterey 12.1
Python 3.10.1

Trying to compile libplist, libimobiledevice, libimobiledevice_glue generates the error
"Could not link test program to Python. Maybe the main Python library has been
  installed in some non-standard library path. If so, pass it to configure,
  via the LDFLAGS environment variable."

The fix is change the following lines (570-571) in /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/_sysconfigdata__darwin_darwin.py

from:
'LINKFORSHARED': '-Wl,-stack_size,1000000  -framework CoreFoundation '
                  'Python.framework/Versions/3.10/Python',
to:
'LINKFORSHARED': '-Wl,-stack_size,1000000  -framework CoreFoundation '
                  '/Library/Frameworks/Python.framework/Versions/3.10/Python',

Also, change the following line (110) in /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/config-3.10-darwin

from:
LINKFORSHARED=  -Wl,-stack_size,1000000  -framework CoreFoundation $(PYTHONFRAMEWORKDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK)
to:
LINKFORSHARED=  -Wl,-stack_size,1000000  -framework CoreFoundation $(PYTHONFRAMEWORKINSTALLDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK)

It seems the problem got created from - https://bugs.python.org/issue15298 - and most likely it exists in all the versions of Python (it exists in the native Python 2.7.18 that comes with macOS too.

Thanks.
msg409616 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2022-01-03 20:26
Thanks for the report. Without more details, we can only guess at what you are trying to accomplish. I would assume it involves embedding Python in another library or application. If so, you should not be using LINKFORSHARED at all which is used to build Python itself (see, for example, Issue36508). Moreover, when embedding on macOS, we do not recommend trying to use the -framework argument to compile with or link to Python: it's too easy to get wrong. Instead, as shown in the use the "Extending and Embedding the Python Interpreter" section of the docset, use the output from the appropriate python3.x-config command, for example:

$ python3.10-config --cflags
-I/Library/Frameworks/Python.framework/Versions/3.10.1_11/include/python3.10 -I/Library/Frameworks/Python.framework/Versions/3.10.1_11/include/python3.10 -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g

$ python3.10-config --ldflags
-L/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/config-3.10-darwin -ldl -framework CoreFoundation

https://docs.python.org/3/extending/embedding.html#compiling-and-linking-under-unix-like-systems
msg409683 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2022-01-04 14:30
Part of the problem here is that most keys in the result of sysconfig.get_config_vars() are copied as-is from the CPython Makefile and pyconfig.h header file.

The keys copied from the Makefile are in a lot of cases not useful for users of python because they contain values that are only useful while building CPython itself. 

Without knowing your use case it is hard to point you to a better solution, but as Ned writes in general just follow the instructions from the extending and embedding documentation.
msg409704 - (view) Author: Sandip Shah (sandipshah) Date: 2022-01-04 19:51
Hi,

Like I mentioned in the ticket, I am trying to compile libplist,
libimobiledevice , and libimobiledevice-glue-1.0 on my mac.  These are
needed to do any development on a mobile device (specifically for testing /
automation).  I use Python for a lot of developmental work, and hence would
rather stick to the same framework to work on mobile devices too.

https://github.com/libimobiledevice

The project is not to "embed" Python, but to link to it, so that it can be
used for testing the app on the mobile (the tools rely on Cython).

The three tools have their own "autogen.sh" which fails with the error
mentioned in the ticket.

While researching, I had found this solution on stackexchange -
https://stackoverflow.com/questions/6490513/vim-failing-to-compile-with-python-on-os-x

Which led me to this after looking more in the python distribution files:

---
In file
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/sysconfig.py
On line 410 - fn _generate_posix_vars

439-448
# There's a chicken-and-egg situation on OS X with regards to the
# _sysconfigdata module after the changes introduced by #15298:
# get_config_vars() is called by get_platform() as part of the
# `make pybuilddir.txt` target -- which is a precursor to the
# _sysconfigdata.py module being constructed.  Unfortunately,
# get_config_vars() eventually calls _init_posix(), which attempts
# to import _sysconfigdata, which we won't have built yet.  In order
# for _init_posix() to work, if we're on Darwin, just mock up the
# _sysconfigdata module manually and populate it with the build vars.
# This is more than sufficient for ensuring the subsequent call to
---

Let me know if you need any other information.

Thanks,

Sandip

On Tue, Jan 4, 2022 at 6:30 AM Ronald Oussoren <report@bugs.python.org>
wrote:

>
> Ronald Oussoren <ronaldoussoren@mac.com> added the comment:
>
> Part of the problem here is that most keys in the result of
> sysconfig.get_config_vars() are copied as-is from the CPython Makefile and
> pyconfig.h header file.
>
> The keys copied from the Makefile are in a lot of cases not useful for
> users of python because they contain values that are only useful while
> building CPython itself.
>
> Without knowing your use case it is hard to point you to a better
> solution, but as Ned writes in general just follow the instructions from
> the extending and embedding documentation.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue46248>
> _______________________________________
>
msg409769 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2022-01-05 13:51
The libraries you mention are 3th party projects.  Please check with the project first.

I've look at the GitHub repository for libmobiledevice and have no idea how they determine the compiler flags for Python.   As mentioned in an earlier version "python-config" is the documented way to fetch such flags.
History
Date User Action Args
2022-04-11 14:59:54adminsetgithub: 90406
2022-01-05 18:05:48ned.deilysetstatus: open -> closed
resolution: third party
stage: resolved
2022-01-05 13:51:23ronaldoussorensetmessages: + msg409769
2022-01-04 19:51:00sandipshahsetmessages: + msg409704
2022-01-04 14:30:05ronaldoussorensetmessages: + msg409683
2022-01-03 20:26:26ned.deilysetmessages: + msg409616
2022-01-03 19:36:05sandipshahcreate