Title: test_site fails in macOS-PR VSTS builds for 3.7 branch
Type: Stage: resolved
Components: macOS, Tests Versions: Python 3.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: methane, ned.deily, pablogsal, ronaldoussoren, vstinner, xtreak
Priority: normal Keywords: patch

Created on 2018-07-21 07:32 by xtreak, last changed 2018-08-10 07:17 by methane. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8430 merged methane, 2018-07-24 05:42
PR 8433 closed methane, 2018-07-24 06:35
PR 8491 merged miss-islington, 2018-07-27 07:55
PR 8493 merged methane, 2018-07-27 08:07
Messages (10)
msg322082 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-07-21 07:32
Mac OS builds fail due to test_site in VSTS for 3.7 branches only

Ref :

2018-07-21T07:01:08.5912930Z Traceback (most recent call last):
2018-07-21T07:01:08.5927520Z   File "/Users/vsts/agent/2.136.1/work/1/s/Lib/test/", line 199, in test_s_option
2018-07-21T07:01:08.5942120Z     self.assertIn(usersite, sys.path)
2018-07-21T07:01:08.5959870Z AssertionError: '/Users/vsts/.local/lib/python3.7/site-packages' not found in ['/Users/vsts/agent/2.136.1/work/1/s', '/usr/local/lib/', '/Users/vsts/agent/2.136.1/work/1/s/Lib', '/Users/vsts/agent/2.136.1/work/1/s/build/lib.macosx-10.13-x86_64-3.7-pydebug', '/usr/local/lib/python3.7/site-packages']
2018-07-21T07:01:08.5985860Z ======================================================================
2018-07-21T07:01:08.6000880Z FAIL: test_startup_imports (test.test_site.StartupImportTests)
2018-07-21T07:01:08.6017050Z ----------------------------------------------------------------------
2018-07-21T07:01:08.6033630Z Traceback (most recent call last):
2018-07-21T07:01:08.6048270Z   File "/Users/vsts/agent/2.136.1/work/1/s/Lib/test/", line 487, in test_startup_imports
2018-07-21T07:01:08.6063180Z     self.assertFalse(modules.intersection(collection_mods), stderr)
2018-07-21T07:01:08.6079990Z AssertionError: {'reprlib', 'heapq', 'collections', 'keyword', 'operator', 'functools', 'types'} is not false : import _frozen_importlib # frozen

I couldn't find any related issues in search and hence opened this. Feel free to close this if it's a duplicate or fixed in some other branch.

msg322083 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-07-21 07:44
It seems that the last successful build in 3.7 was for and all PRs after it had the same failure.

Sorry for not adding the relevant components during ticket creation.

msg322207 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-07-23 12:40
Adding @vstinner as a friendly reminder since buildbot related failures are notified. This issue causes all the PRs for 3.7 on Mac OS VSTS builds to fail since July 19 though other Mac related builds are passing.

Feel free to remove yourself or close this if it's irrelevant.

msg322283 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2018-07-24 09:33
Maybe, vsts machine installs Homebrew's Python.
Homebrew's Python installes /usr/local/lib/python3.7/

When running tests, it is imported.
It causes test_startup_imports fail.

Maybe, we can use random prefix instead of default /usr/local to avoid this conflict.
msg322473 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2018-07-27 09:03
New changeset 777cdd94b9bebd5b22df0fc293aa078d5537b988 by INADA Naoki in branch '3.6':
bpo-34177: vsts: Avoid conflict with Homebrew Python (GH-8430)
msg322508 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-27 16:35
Thank you for the fix Naokia!
msg322936 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-08-02 10:06
This error is happening in the x86-64 High Sierra 3.7 buildbot:

FAIL: test_startup_imports (test.test_site.StartupImportTests)
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.7.billenstein-sierra/build/Lib/test/", line 487, in test_startup_imports
    self.assertFalse(modules.intersection(collection_mods), stderr)
AssertionError: {'collections', 'heapq', 'types', 'functools', 'keyword', 'reprlib', 'operator'} is not false : import _frozen_importlib # frozen
msg322940 - (view) Author: Inada Naoki (methane) * (Python committer) Date: 2018-08-02 10:15
Same fix can be applied to buildbot.

This is caused by (a) Homebrew installs Python to /usr/local with (hacky) sitecustomize, and (b) Python's default prefix is /usr/local.

I think test should be run without conflicting with another installation.
msg322941 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-08-02 10:37
>  Same fix can be applied to buildbot.

Sure. Go ahead :-) The buildbot configuration can be found at:

Send a pull request, and Zachary Ware (or me) will  review it. Zach is the expert here ;-)
msg323274 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-08-08 11:15
I think this can be closed with The buildbots are also green now with the failure reported in fixed with the commit.

Date User Action Args
2018-08-10 07:17:02methanesetstatus: open -> closed
resolution: fixed
2018-08-08 11:15:24xtreaksetmessages: + msg323274
2018-08-02 10:37:28vstinnersetresolution: fixed -> (no value)
messages: + msg322941
2018-08-02 10:15:21methanesetmessages: + msg322940
2018-08-02 10:06:56pablogsalsetstatus: closed -> open
2018-08-02 10:06:12pablogsalsetnosy: + pablogsal
messages: + msg322936
2018-07-27 16:35:42vstinnersetmessages: + msg322508
2018-07-27 09:04:15methanesetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2018-07-27 09:03:58methanesetmessages: + msg322473
2018-07-27 08:07:30methanesetpull_requests: + pull_request8014
2018-07-27 07:55:58miss-islingtonsetpull_requests: + pull_request8013
2018-07-24 09:33:33methanesetnosy: + methane
messages: + msg322283
2018-07-24 06:35:17methanesetpull_requests: + pull_request7958
2018-07-24 05:42:34methanesetkeywords: + patch
stage: patch review
pull_requests: + pull_request7956
2018-07-23 13:54:11vstinnersetcomponents: + Tests, - Build
2018-07-23 13:54:04vstinnersetnosy: + ronaldoussoren, ned.deily
components: + macOS
2018-07-23 12:40:41xtreaksetnosy: + vstinner
messages: + msg322207
2018-07-21 07:44:49xtreaksetmessages: + msg322083
components: + Build
versions: + Python 3.7
2018-07-21 07:32:04xtreakcreate