This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: freeze.py not working properly with OS X framework builds
Type: enhancement Stage: needs patch
Components: macOS Versions: Python 3.4, Python 3.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: ned.deily Nosy List: ned.deily, ronaldoussoren
Priority: normal Keywords:

Created on 2014-05-14 00:11 by yjiangnan, last changed 2022-04-11 14:58 by admin.

Messages (2)
msg218493 - (view) Author: Jiang-Nan Yang (yjiangnan) Date: 2014-05-14 00:11
Command:
python3.4 /Library/Frameworks/Python.framework/Versions/3.4/share/doc/python3.4/examples/Tools/freeze/freeze.py okcointrade.py

Error:
Reporting needed directory %s not found

Basically the problem is on lines 235~239: config should be config-3.4m, python%s on two lines should be python%sm

After I corrected these errors, the command works, asking me to run make, except a warning about "unknown modules remains".

So I run make, another error occurs:
gcc-4.2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m -I/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m -c M_difflib.c
gcc-4.2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m -I/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m -c M_ftplib.c
gcc-4.2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m -I/Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m -c M_pydoc.c
gcc-4.2  -Wl,-stack_size,1000000  -framework CoreFoundation Python.framework/Versions/3.4/Python config.o frozen.o M___future__.o M___main__.o M__bootlocale.o M__collections_abc.o M__dummy_thread.o M__osx_support.o M__sitebuiltins.o M__strptime.o M__sysconfigdata.o M__threading_local.o M__weakrefset.o M_abc.o M_argparse.o M_ast.o M_base64.o M_bdb.o M_bisect.o M_bz2.o M_calendar.o M_cmd.o M_code.o M_codecs.o M_codeop.o M_collections.o M_contextlib.o M_copy.o M_copyreg.o M_datetime.o M_difflib.o M_dis.o M_distutils.o M_distutils__errors.o M_distutils__log.o M_distutils__spawn.o M_distutils__sysconfig.o M_distutils__text_file.o M_doctest.o M_dummy_threading.o M_email.o M_email___encoded_words.o M_email___header_value_parser.o M_email___parseaddr.o M_email___policybase.o M_email__base64mime.o M_email__charset.o M_email__contentmanager.o M_email__encoders.o M_email__errors.o M_email__feedparser.o M_email__generator.o M_email__header.o M_email__headerregistry.o M_email__iterators.o M_email__message.o M_email__parser.o M_email__policy.o M_email__quoprimime.o M_email__utils.o M_encodings.o M_encodings__aliases.o M_encodings__latin_1.o M_encodings__utf_8.o M_enum.o M_fnmatch.o M_ftplib.o M_functools.o M_genericpath.o M_getopt.o M_getpass.o M_gettext.o M_glob.o M_gzip.o M_hashlib.o M_heapq.o M_html.o M_html__entities.o M_http.o M_http__client.o M_http__cookiejar.o M_http__server.o M_imp.o M_importlib.o M_importlib___bootstrap.o M_importlib__machinery.o M_importlib__util.o M_inspect.o M_io.o M_json.o M_json__decoder.o M_json__encoder.o M_json__scanner.o M_keyword.o M_linecache.o M_locale.o M_logging.o M_lzma.o M_mimetypes.o M_ntpath.o M_nturl2path.o M_opcode.o M_operator.o M_optparse.o M_os.o M_pdb.o M_pkgutil.o M_platform.o M_plistlib.o M_posixpath.o M_pprint.o M_py_compile.o M_pydoc.o M_pydoc_data.o M_pydoc_data__topics.o M_quopri.o M_random.o M_re.o M_reprlib.o M_rlcompleter.o M_selectors.o M_shlex.o M_shutil.o M_site.o M_socket.o M_socketserver.o M_sre_compile.o M_sre_constants.o M_sre_parse.o M_ssl.o M_stat.o M_string.o M_struct.o M_subprocess.o M_sysconfig.o M_tarfile.o M_tempfile.o M_textwrap.o M_threading.o M_token.o M_tokenize.o M_traceback.o M_tty.o M_types.o M_unittest.o M_unittest__case.o M_unittest__loader.o M_unittest__main.o M_unittest__result.o M_unittest__runner.o M_unittest__signals.o M_unittest__suite.o M_unittest__util.o M_urllib.o M_urllib__error.o M_urllib__parse.o M_urllib__request.o M_urllib__response.o M_uu.o M_warnings.o M_weakref.o M_webbrowser.o M_xml.o M_xml__parsers.o M_xml__parsers__expat.o M_zipfile.o /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/config-3.4m/libpython3.4.a   -ldl  -framework CoreFoundation   -o okcointrade
clang: error: no such file or directory: 'Python.framework/Versions/3.4/Python'
make: *** [okcointrade] Error 1

I guess that 'Python.framework/Versions/3.4/Python' using a relative location might be a problem, so I go into the Makefile, changed PYTHONFRAMEWORKDIR=Python.framework
into
PYTHONFRAMEWORKDIR=/Library/Frameworks/Python.framework

Now the above problem disappeared, but a new appeared:
gcc-4.2  -Wl,-stack_size,1000000  -framework CoreFoundation /Library/Frameworks/Python.framework/Versions/3.4/Python config.o frozen.o M___future__.o M___main__.o M__bootlocale.o M__collections_abc.o M__dummy_thread.o M__osx_support.o M__sitebuiltins.o M__strptime.o M__sysconfigdata.o M__threading_local.o M__weakrefset.o M_abc.o M_argparse.o M_ast.o M_base64.o M_bdb.o M_bisect.o M_bz2.o M_calendar.o M_cmd.o M_code.o M_codecs.o M_codeop.o M_collections.o M_contextlib.o M_copy.o M_copyreg.o M_datetime.o M_difflib.o M_dis.o M_distutils.o M_distutils__errors.o M_distutils__log.o M_distutils__spawn.o M_distutils__sysconfig.o M_distutils__text_file.o M_doctest.o M_dummy_threading.o M_email.o M_email___encoded_words.o M_email___header_value_parser.o M_email___parseaddr.o M_email___policybase.o M_email__base64mime.o M_email__charset.o M_email__contentmanager.o M_email__encoders.o M_email__errors.o M_email__feedparser.o M_email__generator.o M_email__header.o M_email__headerregistry.o M_email__iterators.o M_email__message.o M_email__parser.o M_email__policy.o M_email__quoprimime.o M_email__utils.o M_encodings.o M_encodings__aliases.o M_encodings__latin_1.o M_encodings__utf_8.o M_enum.o M_fnmatch.o M_ftplib.o M_functools.o M_genericpath.o M_getopt.o M_getpass.o M_gettext.o M_glob.o M_gzip.o M_hashlib.o M_heapq.o M_html.o M_html__entities.o M_http.o M_http__client.o M_http__cookiejar.o M_http__server.o M_imp.o M_importlib.o M_importlib___bootstrap.o M_importlib__machinery.o M_importlib__util.o M_inspect.o M_io.o M_json.o M_json__decoder.o M_json__encoder.o M_json__scanner.o M_keyword.o M_linecache.o M_locale.o M_logging.o M_lzma.o M_mimetypes.o M_ntpath.o M_nturl2path.o M_opcode.o M_operator.o M_optparse.o M_os.o M_pdb.o M_pkgutil.o M_platform.o M_plistlib.o M_posixpath.o M_pprint.o M_py_compile.o M_pydoc.o M_pydoc_data.o M_pydoc_data__topics.o M_quopri.o M_random.o M_re.o M_reprlib.o M_rlcompleter.o M_selectors.o M_shlex.o M_shutil.o M_site.o M_socket.o M_socketserver.o M_sre_compile.o M_sre_constants.o M_sre_parse.o M_ssl.o M_stat.o M_string.o M_struct.o M_subprocess.o M_sysconfig.o M_tarfile.o M_tempfile.o M_textwrap.o M_threading.o M_token.o M_tokenize.o M_traceback.o M_tty.o M_types.o M_unittest.o M_unittest__case.o M_unittest__loader.o M_unittest__main.o M_unittest__result.o M_unittest__runner.o M_unittest__signals.o M_unittest__suite.o M_unittest__util.o M_urllib.o M_urllib__error.o M_urllib__parse.o M_urllib__request.o M_urllib__response.o M_uu.o M_warnings.o M_weakref.o M_webbrowser.o M_xml.o M_xml__parsers.o M_xml__parsers__expat.o M_zipfile.o /Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/config-3.4m/libpython3.4.a   -ldl  -framework CoreFoundation   -o okcointrade
Undefined symbols for architecture x86_64:
  "_PyInit__imp", referenced from:
      __PyImport_Inittab in config.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [okcointrade] Error 1

Now it is out of my comprehension. What is wrong? How can I get freeze.py to work? I have a program doing very very heavy computation, so I really hope I can compile it into binary so that it could run much faster.

Thanks.
msg235528 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2015-02-07 15:29
Sorry, I've finally gotten around to taking a longer look at this and it seems that freeze support is kind of a mess.  First, it looks like some of the issues you ran into were fixed in 3.4.1 (changes associated with Issue11824 and Issue16047). Unfortunately, one of them introduced a new problem that I've documented in Issue23405 (and which can be worked around by editing the freeze Makefile to use $(LIBRARY) instead of $(LDLIBRARY)).  But there appear to be still more basic problems when trying to use freeze with OS X framework builds such as provided by the python.org installers for OS X.

1. freeze makes use of the LINKFORSHARED configuration variable when building the frozen executable.  For OS X non-framework Python builds, it typically looks like this:
  LINKFORSHARED="-Wl,-stack_size,1000000  -framework CoreFoundation"
For framework builds, it also includes the relative framework file path:
  LINKFORSHARED="-Wl,-stack_size,1000000  -framework CoreFoundation Python.framework/Versions/3.4/Python"
The path is not useful as-is outside of the Python build itself and, by having a path here, the OS X framework configuration seems to be an exception to most other platforms, causing problems here with freeze and elsewhere (there have been other complaints about this usage).  As noted, one can workaround this problem by editing the freeze-produced Makefile.  But that leads to another problem.

2. Unlike for non-shared and shared Python builds, OS X Python framework builds do not include the Python library as a static archive file in the config-* directory (or anywhere else for that matter).  Instead, symbolic links are created in the directory to the framework shared library.

shared build
$ ls -l ./lib/python3.4/config-3.4dm/lib*
-rw-r-----+ 1 nad  pyd  9514552 Feb  7 16:57 libpython3.4dm.a

framework build
$ ls -l ./lib/python3.4/config-3.4dm/lib*
lrwxr-x---  1 nad  pyd     21 Feb  7 17:41 libpython3.4.a -> ../../../Python
lrwxr-x---  1 nad  pyd     21 Feb  7 17:41 libpython3.4.dylib -> ../../../Python
lrwxr-x---  1 nad  pyd     21 Feb  7 17:41 libpython3.4dm.a -> ../../../Python
lrwxr-x---  1 nad  pyd     21 Feb  7 17:41 libpython3.4dm.dylib -> ../../../Python

This difference causes problems when linking the frozen executable: because the framework build configuration presents a dynamic shared lib, the resulting executable has a runtime dependency on the shared lib:

% otool -L hello
hello:
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1152.0.0)
	/Library/Frameworks/Python.framework/Versions/3.4/Python (compatibility version 3.4.0, current version 3.4.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)

and the frozen executable cannot be run on another system which does not have a compatible version of Python installed.  I don't know of an easy freeze build workaround for this.  I think the correct solution for freeze is to always only provide the static library in the config* directory for all Python build configurations.  But I'm not sure whether there are any compelling reasons to provide shared lib links as well for other users of the config* directory (e.g. for embedding Python); if so, the static and dynamic libs may have to have distinct names.

Ronald, do you have other thoughts on this?

At the moment, I think the safest solution for using freeze on OS X is to use a non-shared, non-framework build of Python, which probably means building Python 3.4 yourself.
History
Date User Action Args
2022-04-11 14:58:03adminsetgithub: 65701
2015-02-07 15:33:25ned.deilysetversions: + Python 3.5, - Python 3.3
2015-02-07 15:29:18ned.deilysetnosy: + ronaldoussoren
messages: + msg235528
2014-05-14 04:12:09ned.deilysetnosy: + ned.deily
title: freeze.py not working properly on Mac OSX 10.9 -> freeze.py not working properly with OS X framework builds
assignee: ronaldoussoren -> ned.deily
type: compile error -> enhancement
stage: needs patch
2014-05-14 01:15:43yjiangnansetnosy: - ronaldoussoren, yjiangnan
-> (no value)
versions: + Python 3.3
2014-05-14 00:11:32yjiangnancreate