classification
Title: _contextvars missing in xmaster branch Windows build?
Type: behavior Stage: resolved
Components: Installation Versions: Python 3.8, Python 3.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: steve.dower Nosy List: helena.marina, miss-islington, ossdev07, paul.moore, steve.dower, steven.winfield, tim.golden, tim.peters, willingc, yselivanov, zach.ware
Priority: normal Keywords: patch, patch, patch

Created on 2018-09-15 03:03 by tim.peters, last changed 2019-02-03 00:06 by steve.dower. This issue is now closed.

Files
File name Uploaded Description Edit
Captura de Tela (12).png helena.marina, 2018-09-26 21:50
Pull Requests
URL Status Linked Edit
PR 11741 merged steve.dower, 2019-02-02 21:33
PR 11741 merged steve.dower, 2019-02-02 21:33
PR 11741 merged steve.dower, 2019-02-02 21:33
PR 11743 merged miss-islington, 2019-02-02 22:36
PR 11743 merged miss-islington, 2019-02-02 22:36
PR 11743 merged miss-islington, 2019-02-02 22:36
Messages (11)
msg325419 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2018-09-15 03:02
Using Visual Studio 2017 to build the current master branch of Python (something I'm trying for the first time in about two years - maybe I'm missing something obvious!), with the x64 target, under both the Release and Debug builds I get a Python that can't even start to run the test suite.  Here's output from the Release build - the Debug build output is nearly identical.  The problem does not appear with the Win32 target.  Of course I'm doing this on a 64-bit box (under Win 10 Pro):

C:\Code\Python\PCbuild>rt -q -x64

C:\Code\Python\PCbuild>"C:\Code\Python\PCbuild\amd64\python.exe"  -u -Wd -E -bb -m test
Traceback (most recent call last):
  File "C:\Code\Python\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Code\Python\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Code\Python\lib\test\__main__.py", line 1, in <module>
    from test.libregrtest import main
  File "C:\Code\Python\lib\test\libregrtest\__init__.py", line 4, in <module>
    from test.libregrtest.cmdline import _parse_args, RESOURCE_NAMES, ALL_RESOURCES
  File "C:\Code\Python\lib\test\libregrtest\cmdline.py", line 4, in <module>
    from test import support
  File "C:\Code\Python\lib\test\support\__init__.py", line 6, in <module>
    import asyncio.events
  File "C:\Code\Python\lib\asyncio\__init__.py", line 8, in <module>
    from .base_events import *
  File "C:\Code\Python\lib\asyncio\base_events.py", line 39, in <module>
    from . import events
  File "C:\Code\Python\lib\asyncio\events.py", line 14, in <module>
    import contextvars
  File "C:\Code\Python\lib\contextvars.py", line 1, in <module>
    from _contextvars import Context, ContextVar, Token, copy_context
ModuleNotFoundError: No module named '_contextvars'


_contextvarsmodule.c is in the list of Modules under the pythoncore project, and it appears to compile without incident when the Solution is built.

The executable appears to be fine in other respects I checked; e.g.,

C:\Code\Python\PCbuild>amd64\python.exe
Python 3.8.0a0 (heads/master:24f6846920, Sep 14 2018, 18:55:33) [MSC v.1915 64 bit (AMD64)] on win32

Type "help", "copyright", "credits" or "license" for more information.
>>> import math
>>> from collections import Counter
>>> Counter("abab")
Counter({'a': 2, 'b': 2})
>>> import _contextvars
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named '_contextvars'

Clue?
msg325555 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-09-17 16:58
It looks like _contextvars has both its own project and is also built as part of pythoncore.vcxproj.

However, _contextvars.vcxproj is not in PCbuild.sln, which means when you build through Visual Studio (rather than with build.bat via pcbuild.proj), you don't get the separate extension module.

And even though it's built into pythoncore.vcxproj, it hasn't been added to PC/config.c, and so it isn't initialised as a regular builtin module.

Yury - is there a specific reason for it to be a standalone module? How much do you think it hurts to have it always loaded into memory, even if it hasn't truly been imported or used?
msg325556 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-09-17 17:00
> How much do you think it hurts to have it always loaded into memory, even if it hasn't truly been imported or used?

I'd be fine with that.  Would you be able to do the necessary change (I don't have VS at hand)?
msg325560 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-09-17 17:22
Shouldn't need VS for this change (updating the SLN file generally does need it). I'm happy to do it though.

In case someone else gets to it before me:
* delete _contextvars.vcxproj and remove all references from PCbuild and Tools/msi
* add _contextvars entries to PC/config.c

The CI builds will tell you pretty quickly if it worked or not.
msg325602 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2018-09-18 00:45
FYI, I bet I didn't see a problem with the Win32 target because I followed instructions ;-) and did my first build using build.bat.  Using that for the x64 too target makes the problem go away.
msg326513 - (view) Author: Helena Centanin (helena.marina) Date: 2018-09-26 21:50
Good night I installed the python program at the start it appears this message
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:06:47) [MSC v.1914 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>>
msg333822 - (view) Author: ossdev (ossdev07) * Date: 2019-01-17 05:49
I am building python for windows on ARM64,
I am getting "No module named '_contextvars'" when trying to import contextvars:
>>> import contextvars
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\ps-emb\Downloads\cpython-master\cpython-master\lib\contextvars.py", line 1, in <module>
    from _contextvars import Context, ContextVar, Token, copy_context
ModuleNotFoundError: No module named '_contextvars'

thanks
msg333823 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2019-01-17 05:57
Precisely _how_ are you building it?  As noted above, if you're using Visual Studio, try using build.bat instead for the first time.
msg334763 - (view) Author: Steven Winfield (steven.winfield) Date: 2019-02-02 18:53
(Just updating the issue to note that Python 3.7 is similarly affected)
msg334772 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-02-02 22:36
New changeset 4c70d9f79c9b371990c8e054ccde53f7ff15946b by Steve Dower in branch 'master':
bpo-34691: Compile _contextvars module into main Python library (GH-11741)
https://github.com/python/cpython/commit/4c70d9f79c9b371990c8e054ccde53f7ff15946b
msg334775 - (view) Author: miss-islington (miss-islington) Date: 2019-02-02 22:54
New changeset 81eda28382168bfea48bb1a352954ea90aadd4ca by Miss Islington (bot) in branch '3.7':
bpo-34691: Compile _contextvars module into main Python library (GH-11741)
https://github.com/python/cpython/commit/81eda28382168bfea48bb1a352954ea90aadd4ca
History
Date User Action Args
2019-02-03 00:06:06steve.dowersetkeywords: patch, patch, patch
status: open -> closed
resolution: fixed
stage: patch review -> resolved
2019-02-02 22:54:44miss-islingtonsetnosy: + miss-islington
messages: + msg334775
2019-02-02 22:36:52miss-islingtonsetpull_requests: + pull_request11662
2019-02-02 22:36:50miss-islingtonsetpull_requests: + pull_request11663
2019-02-02 22:36:48miss-islingtonsetpull_requests: + pull_request11661
2019-02-02 22:36:27steve.dowersetmessages: + msg334772
2019-02-02 21:33:55steve.dowersetkeywords: + patch
stage: patch review
pull_requests: + pull_request11656
2019-02-02 21:33:39steve.dowersetkeywords: + patch
stage: (no value)
pull_requests: + pull_request11655
2019-02-02 21:33:23steve.dowersetkeywords: + patch
stage: (no value)
pull_requests: + pull_request11654
2019-02-02 21:33:04steve.dowersetassignee: steve.dower
2019-02-02 18:53:57steven.winfieldsetnosy: + steven.winfield

messages: + msg334763
versions: + Python 3.7
2019-01-25 23:47:32willingcsetnosy: + willingc
2019-01-17 05:57:14tim.peterssetmessages: + msg333823
2019-01-17 05:49:04ossdev07setnosy: + ossdev07

messages: + msg333822
versions: + Python 3.8, - Python 3.7
2018-09-26 21:50:14helena.marinasetfiles: + Captura de Tela (12).png


components: + Installation, - Build, Library (Lib), Windows
versions: + Python 3.7, - Python 3.8
nosy: + helena.marina
title: _contextvars missing in x64 master branch Windows build? -> _contextvars missing in xmaster branch Windows build?
messages: + msg326513
2018-09-18 00:45:58tim.peterssetmessages: + msg325602
2018-09-17 17:22:22steve.dowersetmessages: + msg325560
2018-09-17 17:00:23yselivanovsetmessages: + msg325556
2018-09-17 16:58:28steve.dowersetnosy: + yselivanov
messages: + msg325555
2018-09-15 03:03:00tim.peterscreate