classification
Title: After installing python 3.x on Mac pip doesn't work at all
Type: crash Stage: resolved
Components: macOS Versions: Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: miss-islington, ned.deily, peteje66, ronaldoussoren
Priority: normal Keywords: patch

Created on 2020-08-04 10:34 by peteje66, last changed 2020-10-20 07:18 by ronaldoussoren. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8689 peteje66, 2020-08-04 10:34
PR 22762 merged ronaldoussoren, 2020-10-19 10:31
PR 22773 merged miss-islington, 2020-10-19 18:16
PR 22774 merged miss-islington, 2020-10-19 18:16
Messages (7)
msg374800 - (view) Author: Jens Petersen (peteje66) Date: 2020-08-04 10:34
Using pip fails with the same error messages independent of version an user. Also su or sudo -H doesn't change anything:
How to Reproduce
install python 3.x on your Mac and try pip

After a quite long search I found that it is a problem with the proxy config. You need to remove all settings in network settings/proxy even those defining which domains shouldn't use a proxy. Proxy settings work fine with everything except python.

xcode-select install
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew install pyenv
pyenv install 3.7.3
pip list --outdated or pip install something
Exception
xxx-MBP:~ xxx$ pip list --outdated
Exception:
Traceback (most recent call last):
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 179, in main
    status = self.run(options, args)
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip/_internal/commands/list.py", line 144, in run
    packages = self.get_outdated(packages, options)
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip/_internal/commands/list.py", line 152, in get_outdated
    dist for dist in self.iter_packages_latest_infos(packages, options)
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip/_internal/commands/list.py", line 152, in <listcomp>
    dist for dist in self.iter_packages_latest_infos(packages, options)
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip/_internal/commands/list.py", line 179, in iter_packages_latest_infos
    all_candidates = finder.find_all_candidates(dist.key)
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip/_internal/index.py", line 610, in find_all_candidates
    for page in self._get_pages(url_locations, project_name):
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip/_internal/index.py", line 743, in _get_pages
    page = _get_html_page(location, session=self.session)
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip/_internal/index.py", line 229, in _get_html_page
    resp = _get_html_response(url, session=session)
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip/_internal/index.py", line 177, in _get_html_response
    "Cache-Control": "max-age=0",
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip/_internal/download.py", line 403, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 524, in request
    prep.url, proxies, stream, verify, cert
  File "/Users/xxxx.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip/_vendor/requests/sessions.py", line 700, in merge_environment_settings
    env_proxies = get_environ_proxies(url, no_proxy=no_proxy)
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip/_vendor/requests/utils.py", line 761, in get_environ_proxies
    if should_bypass_proxies(url, no_proxy=no_proxy):
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip/_vendor/requests/utils.py", line 745, in should_bypass_proxies
    bypass = proxy_bypass(parsed.hostname)
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/urllib/request.py", line 2610, in proxy_bypass
    return proxy_bypass_macosx_sysconf(host)
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/urllib/request.py", line 2587, in proxy_bypass_macosx_sysconf
    return _proxy_bypass_macosx_sysconf(host, proxy_settings)
  File "/Users/xxx/.pyenv/versions/3.7.3/lib/python3.7/urllib/request.py", line 2573, in _proxy_bypass_macosx_sysconf
    if (hostIP >> mask) == (base >> mask):
ValueError: negative shift count
msg374805 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-08-04 13:15
I don't quite understand what the issue is.

* What proxy settings do you have?

* Is the problem reproducible without using pip?

  - By using the requests API to fetch a web page
  - By using the stdlib http.client API to do the same
msg378937 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-10-19 10:17
The error is due to invalid proxy settings on the system.

I can reproduce the error by enabling a HTTP proxy in system settings, while adding "172.19.0.0/40" to the proxy exclude list in the HTTP proxy settings.
msg378938 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-10-19 10:33
I've tested the same settings with Safari, and that browser ignores excludes with an invalid prefix length.

I've created a PR that does the same for urllib.request.
msg378976 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-10-19 18:16
New changeset 93a1ccabdede416425473329b8c718d507c55e29 by Ronald Oussoren in branch 'master':
bpo-41471: Ignore invalid prefix lengths in system proxy settings on macOS (GH-22762)
https://github.com/python/cpython/commit/93a1ccabdede416425473329b8c718d507c55e29
msg379092 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-10-20 07:17
New changeset aa7b03b31bf09b21a012b24e21977538e66b0172 by Miss Skeleton (bot) in branch '3.9':
bpo-41471: Ignore invalid prefix lengths in system proxy settings on macOS (GH-22762) (GH-22773)
https://github.com/python/cpython/commit/aa7b03b31bf09b21a012b24e21977538e66b0172
msg379093 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2020-10-20 07:18
New changeset e7c5a43984f82ef9634cb0b2b8c4750b2fd431a0 by Miss Skeleton (bot) in branch '3.8':
bpo-41471: Ignore invalid prefix lengths in system proxy settings on macOS (GH-22762) (GH-22774)
https://github.com/python/cpython/commit/e7c5a43984f82ef9634cb0b2b8c4750b2fd431a0
History
Date User Action Args
2020-10-20 07:18:20ronaldoussorensetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-10-20 07:18:01ronaldoussorensetmessages: + msg379093
2020-10-20 07:17:30ronaldoussorensetmessages: + msg379092
2020-10-19 18:16:52miss-islingtonsetpull_requests: + pull_request21736
2020-10-19 18:16:37miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request21735
2020-10-19 18:16:26ronaldoussorensetmessages: + msg378976
2020-10-19 10:33:27ronaldoussorensetmessages: + msg378938
2020-10-19 10:31:26ronaldoussorensetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request21726
2020-10-19 10:17:30ronaldoussorensetstage: needs patch
messages: + msg378937
versions: + Python 3.9, Python 3.10, - Python 3.8
2020-08-04 13:15:06ronaldoussorensetmessages: + msg374805
2020-08-04 10:34:27peteje66create