This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Test_venv fails when _ctypes is not available.
Type: behavior Stage: resolved
Components: Library (Lib), Tests Versions: Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: dstufft Nosy List: dstufft, ned.deily, terry.reedy, vinay.sajip
Priority: normal Keywords:

Created on 2014-10-19 00:37 by terry.reedy, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (12)
msg229667 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-10-19 00:37
For whatever reason, AMD 64 Open Indiana ('stable' buildbot #3) is not building ctypes. test_venv fails with
FAIL: test_with_pip (test.test_venv.EnsurePipTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/export/home/buildbot/64bits/3.4.cea-indiana-amd64/build/Lib/test/test_venv.py", line 356, in test_with_pip
    with_pip=True)
subprocess.CalledProcessError: Command '['/tmp/tmp644tt7bi/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/export/home/buildbot/64bits/3.4.cea-indiana-amd64/build/Lib/test/test_venv.py", line 362, in test_with_pip
    self.fail(msg.format(exc, details))
AssertionError: Command '['/tmp/tmp644tt7bi/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1

**Subprocess Output**
Traceback (most recent call last):
  File "/export/home/buildbot/64bits/3.4.cea-indiana-amd64/build/Lib/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/export/home/buildbot/64bits/3.4.cea-indiana-amd64/build/Lib/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/export/home/buildbot/64bits/3.4.cea-indiana-amd64/build/Lib/ensurepip/__main__.py", line 4, in <module>
    ensurepip._main()
  File "/export/home/buildbot/64bits/3.4.cea-indiana-amd64/build/Lib/ensurepip/__init__.py", line 209, in _main
    default_pip=args.default_pip,
  File "/export/home/buildbot/64bits/3.4.cea-indiana-amd64/build/Lib/ensurepip/__init__.py", line 116, in bootstrap
    _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/export/home/buildbot/64bits/3.4.cea-indiana-amd64/build/Lib/ensurepip/__init__.py", line 40, in _run_pip
    import pip
  File "/tmp/tmp7od_thjv/pip-1.5.6-py2.py3-none-any.whl/pip/__init__.py", line 9, in <module>
  File "/tmp/tmp7od_thjv/pip-1.5.6-py2.py3-none-any.whl/pip/log.py", line 9, in <module>
  File "/tmp/tmp7od_thjv/pip-1.5.6-py2.py3-none-any.whl/pip/_vendor/colorama/__init__.py", line 2, in <module>
  File "/tmp/tmp7od_thjv/pip-1.5.6-py2.py3-none-any.whl/pip/_vendor/colorama/initialise.py", line 5, in <module>
  File "/tmp/tmp7od_thjv/pip-1.5.6-py2.py3-none-any.whl/pip/_vendor/colorama/ansitowin32.py", line 6, in <module>
  File "/tmp/tmp7od_thjv/pip-1.5.6-py2.py3-none-any.whl/pip/_vendor/colorama/winterm.py", line 2, in <module>
  File "/tmp/tmp7od_thjv/pip-1.5.6-py2.py3-none-any.whl/pip/_vendor/colorama/win32.py", line 7, in <module>
  File "/export/home/buildbot/64bits/3.4.cea-indiana-amd64/build/Lib/ctypes/__init__.py", line 7, in <module>
    from _ctypes import Union, Structure, Array
ImportError: No module named '_ctypes'

Since this is not a failure of ensurepip, I think this should instead be a test skip.  The easiest thing would be to do what you did with ssl.
msg229668 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-10-19 01:05
This is a duplicate of Issue22611.
msg229669 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-10-19 06:02
I think this was improperly closed.  #22611 was about pip.  This one is about test_venv failing today, 8 days after 22611 was closed, when optional module _ctypes is not built.  The resolution of this issue should be that either the buildbots start using the new version of pip that will not fail or that test_venv is patched to work with version of pip that is being used.
msg229670 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-10-19 06:31
Issue22611 documents exactly the same failure so if this issue was improperly closed than Issue22611 was, too.  So I would think the earlier issue would be the place to discuss this.  But, in any case, I think the issue is that the latest version of pip hasn't been released yet and/or not yet vendored into the 3.4 and default source trees.  Donald would be the person to discuss that.
msg229671 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2014-10-19 06:39
It hasn't been released yet, test_venv could be temp disabled if ctypes isn't available though.
msg229672 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2014-10-19 06:47
As I understand it, the reason test_venv is failing is because of a defect in the pip vendored colorama.  The test has found the unexpected dependency of the vendored package with ctypes.  It's good that the test failed.  The right solution is to provide the updated pip with the fixed colorama and that should be happening shortly.  And when that happens, there should no longer be a test failure.  Is my understanding correct?  If so, I would argue that we should not be touching test_venv.
msg229673 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2014-10-19 06:52
More or less. You could argue whether it's a defect or not but it unconditionally imports ctypes and apparently that is an optional module (I had no idea it was personally, and I can't find any documentation to say that it is). colorama doesn't really *need* ctypes on Linux it only needs it on Windows so it fixes this by just conditionally importing it.
msg231443 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2014-11-20 17:29
Assigning this to myself, it'll get fixed when pip 6.0 is released and I upgrade ensurepip to it.
msg233045 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2014-12-23 14:22
This should be fixed now as of https://hg.python.org/cpython/rev/651e1862dbed, https://hg.python.org/cpython/rev/651e1862dbed, and https://hg.python.org/cpython/rev/9f60d024e586.
msg233058 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-12-23 22:32
test_venv passed on Open Indiana in the 14:10-14:22 buildbot runs triggered by the patch.  It failed with a different error on a later default run, but that looked like a different issue, if permanent.
msg233059 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2014-12-23 22:33
I'm not sure how to get a link to that, can you link it and I can see if I think it's a permanent error or not?
msg233060 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-12-23 22:42
http://buildbot.python.org/all/builders/AMD64%20OpenIndiana%203.x/builds/9012/steps/test/logs/stdio

Start with buildbot.python.org, select set of (all/stable) buildbots for version.  'Build ####' like gives commit(s) that triggered build.  Test stdio link, as above, gives test results.  Above was triggered by Serhiy's commit after yours. "No module named pip.__main__;"
History
Date User Action Args
2022-04-11 14:58:09adminsetgithub: 66859
2014-12-23 22:42:14terry.reedysetmessages: + msg233060
2014-12-23 22:33:51dstufftsetmessages: + msg233059
2014-12-23 22:32:56terry.reedysetmessages: + msg233058
2014-12-23 14:22:39dstufftsetstatus: open -> closed
resolution: fixed
messages: + msg233045
2014-11-20 17:29:32dstufftsetassignee: vinay.sajip -> dstufft
messages: + msg231443
2014-10-19 06:52:21dstufftsetmessages: + msg229673
2014-10-19 06:47:29ned.deilysetmessages: + msg229672
2014-10-19 06:39:01dstufftsetmessages: + msg229671
2014-10-19 06:31:02ned.deilysetmessages: + msg229670
2014-10-19 06:02:04terry.reedysetstatus: closed -> open
superseder: pip needs ctypes ->
resolution: duplicate -> (no value)
messages: + msg229669
2014-10-19 01:05:00ned.deilysetstatus: open -> closed

superseder: pip needs ctypes

nosy: + ned.deily, dstufft
messages: + msg229668
resolution: duplicate
stage: needs patch -> resolved
2014-10-19 00:37:48terry.reedycreate