Issue38705
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2019-11-05 20:58 by dend, last changed 2022-04-11 14:59 by admin. This issue is now closed.
Messages (7) | |||
---|---|---|---|
msg356061 - (view) | Author: Den Delimarsky (dend) | Date: 2019-11-05 20:58 | |
I have a project, where I am working inside a virtual environment (on macOS Catalina). Inside said project, I am attempting to create a new virtual environment with: venv.create(virtual_environment_directory, with_pip=True) This, however, errors out: File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/Users/user1/Documents/repos/adg/src/adg/__main__.py", line 35, in <module> CommandProcessor.validate(args, current_os) File "/Users/user1/Documents/repos/adg/src/adg/helpers/commandprocessor.py", line 21, in validate LibraryProcessor.process_libraries(command.library, command.platform, command.out) File "/Users/user1/Documents/repos/adg/src/adg/helpers/coreutil.py", line 79, in process_libraries LibraryInstaller.create_environment() File "/Users/user1/Documents/repos/adg/src/adg/helpers/coreutil.py", line 40, in create_environment venv.create(virtual_environment_directory, with_pip=True) File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/venv/__init__.py", line 363, in create builder.create(env_dir) File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/venv/__init__.py", line 68, in create self._setup_pip(context) File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/venv/__init__.py", line 261, in _setup_pip subprocess.check_output(cmd, stderr=subprocess.STDOUT) File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/subprocess.py", line 395, in check_output **kwargs).stdout File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/subprocess.py", line 487, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['/Users/user1/Documents//repos/adg/src/dtemp/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' died with <Signals.SIGABRT: 6>. Seems like attempting to run this both from inside and outside the virtual environment results in the error. |
|||
msg356064 - (view) | Author: Den Delimarsky (dend) | Date: 2019-11-05 21:10 | |
Should also mention that this is working as expected in Python 3.8.0 |
|||
msg356066 - (view) | Author: Ned Deily (ned.deily) * | Date: 2019-11-05 21:28 | |
Thanks for the report but without a supplying a reproducible test case it's really difficult to say what issue you might be seeing. FWIW, I was not able to reproduce a failure but I was just guessing what your project code really does. Also be aware that you appear to be using the Apple-supple Python 3.7.3 that is included with Apple's Xcode 11 Command Line Tools for 10.15 Catalina and is not part of a standard macOS installation. I believe that Python3 is there only to support other parts of the Command Line Tools and it is not recommended that you use it for your own projects. |
|||
msg356086 - (view) | Author: Den Delimarsky (dend) | Date: 2019-11-06 01:47 | |
@ned.deily actually you bring up a very good point, I did not notice that it was using the default Catalina Python 3 install. I've chatted with @brett.cannon about this earlier and it seemed like the issue was in that the Python version I was using is no longer supported, with the fix likely being in one of the later branches. I've updated my installation from python.org and everything works as expected now. Let me know if you want me to close this issue, and thank you for your insights! |
|||
msg379724 - (view) | Author: Nikhil Benesch (benesch) | Date: 2020-10-27 03:24 | |
This issue continues to exist with the system Python on macOS. Here is how to reproduce on macOS Catalina 10.15.7 and Xcode 12.1. $ /usr/bin/python3 --version Python 3.8.2 $ /usr/bin/python3 -c 'import venv; venv.create("venv");' $ venv/bin/python3 dyld: Library not loaded: @executable_path/../Python3 Referenced from: /Users/benesch/venv/bin/python3 Reason: image not found Abort trap: 6 Weird, isn't it? The thing that gets copied into the venv is just a totally different file than /usr/bin/python3: $ ls -lah venv/bin/python3 -rwxr-xr-x 1 benesch staff 148K Oct 26 23:16 venv/bin/python3 $ ls -lah /usr/bin/python3 -rwxr-xr-x 1 root wheel 31K Sep 21 20:29 /usr/bin/python3 I assume this is Apple's bug, but I'll be damned if I have to file another report into the blackhole that is radar. If someone needs a quick workaround, just create your venvs with symlinks, which works just fine: $ /usr/bin/python3 -c 'import venv; venv.create("venv2");' $ venv2/bin/python3 Python 3.8.2 (default, Sep 24 2020, 19:37:08) [Clang 12.0.0 (clang-1200.0.32.21)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> Incidentally this is why `/usr/bin/python3 -m venv` works just fine, since it defaults to `symlinks=True` on macOS. |
|||
msg379726 - (view) | Author: Nikhil Benesch (benesch) | Date: 2020-10-27 03:25 | |
Oops, sorry, I forgot to actually include the correct command in the workaround. Here it is: $ /usr/bin/python3 -c 'import venv; venv.create("venv2", symlinks=True);' |
|||
msg379730 - (view) | Author: Nikhil Benesch (benesch) | Date: 2020-10-27 03:42 | |
So it looks like /usr/bin/python3 is just a shim that immediately execs the real Python binary: /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/bin/python3 This binary is identical to the one that ends up in the venv. Problematically, this binary is dynamically linked, and links the "Python3" shared library using a relative path: $ otool -L /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/bin/python3 /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8/bin/python3: @executable_path/../Python3 (compatibility version 3.8.0, current version 3.8.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.0.0) So obviously copying that binary to a different path breaks the @executable_path-relative link. Ah, well. Definitely seems like Apple's bug. Not sure there is anything that even could be done on the CPython side. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:22 | admin | set | github: 82886 |
2020-10-27 03:42:44 | benesch | set | messages: + msg379730 |
2020-10-27 03:25:35 | benesch | set | messages: + msg379726 |
2020-10-27 03:24:41 | benesch | set | nosy:
+ benesch messages: + msg379724 |
2019-11-06 07:35:21 | ned.deily | set | status: open -> closed resolution: third party stage: resolved |
2019-11-06 01:47:20 | dend | set | messages: + msg356086 |
2019-11-05 21:28:45 | ned.deily | set | messages: + msg356066 |
2019-11-05 21:10:03 | dend | set | messages: + msg356064 |
2019-11-05 20:58:06 | dend | create |