This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Title: entry_points/console_scripts is too slow
Type: Stage: resolved
Components: Versions:
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: Nosy List: barry, ericvw, gwk
Priority: normal Keywords:

Created on 2018-06-19 14:34 by gwk, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (5)
msg319971 - (view) Author: George King (gwk) * Date: 2018-06-19 14:34
On my newish macOS laptop using Python 3.6 or 3.7, a no-op script takes 3 times as long to invoke using the entry_points machinery as it does to invoke directly. Here are some exemplary times (best times after several tries).

$ time python3.6 

real	0m0.047s
user	0m0.033s
sys	0m0.010s

$ time python3.6 /Library/Frameworks/Python.framework/Versions/3.6/bin/entrypoint

real	0m0.153s
user	0m0.129s
sys	0m0.020s

In this example, consists of `def main(): pass`; the second script is that generated by `pip3 install ...`. I have confirmed that the `-e` flag to pip is not the culprit.

I look at a cprofile trace of the invocation and I do not see an obvious single culprit, but I'm not an expert at reading such traces either.

I know that there has been some previous debate about "how slow is slow", whether to try to improve import times, etc. I'm not trying to fan any flames here, but from a practical perspective, this discrepancy is showing up in developer tools that I'm writing, and is driving me to abandon entry_points in favor of less standard, less cross-platform install hackery in my scripts. An extra 0.1s per invocation is visibly detrimental to shell scripts calling out to installed python programs.
msg319989 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-06-19 17:54
This really isn't enough information to know exactly what's going on in your case, but there are things that are known to be slow for CLI startups.  Probably the biggest contributor is pkg_resources.  Of course, that's a third party library that's popular for handling entry points, so it's not really a Python problem.  There are some other third party libraries on PyPI that may be more performant for you.  Entry point handling is something that I'd like to try to address in 3.8 in a manner similar to importlib.resources.
msg319994 - (view) Author: George King (gwk) * Date: 2018-06-19 19:21
Thanks Barry. My question then is, what relationship does cpython have with pip, setuptools, distutils and pkg_resources? Since pip comes bundled with Python now it seems a little bit closer than "3rd party". Thanks!
msg319995 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-06-19 20:00
"It's complicated" :)  But technically speaking they are all separate projects, so while there is synergy, CPython itself *imports* some of those to provide various tasks, but it doesn't lead their development.  You'll find a lot of the same players in all those projects though.

This might be a good resource:

I hope you don't mind, but I am going to close this issue since it's not directly related to Python itself.  If you feel otherwise and/or can provide a reproducible test case, please do reopen it.

Thanks for your contribution to Python!
msg319996 - (view) Author: George King (gwk) * Date: 2018-06-19 20:07
OK, thanks. I agree that this is best pursued with the developers of the relevant modules. I appreciate your quick and detailed responses!
Date User Action Args
2022-04-11 14:59:01adminsetgithub: 78083
2018-06-19 20:07:46gwksetmessages: + msg319996
2018-06-19 20:00:40barrysetstatus: open -> closed
resolution: third party
messages: + msg319995

stage: resolved
2018-06-19 19:21:48gwksetmessages: + msg319994
2018-06-19 17:54:20barrysetnosy: + barry
messages: + msg319989
2018-06-19 15:15:36ericvwsetnosy: + ericvw
2018-06-19 14:34:17gwkcreate