This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: test_winconsoleio fails on Windows 11
Type: behavior Stage: resolved
Components: Tests, Windows Versions: Python 3.11, Python 3.10, Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: eryksun, jkloth, miss-islington, paul.moore, steve.dower, tim.golden, vstinner, zach.ware
Priority: normal Keywords: patch

Created on 2021-10-03 14:42 by jkloth, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 28712 merged jkloth, 2021-10-03 17:46
PR 28736 merged miss-islington, 2021-10-05 12:18
PR 28737 merged miss-islington, 2021-10-05 12:18
Messages (14)
msg403090 - (view) Author: Jeremy Kloth (jkloth) * Date: 2021-10-03 14:42
It appears there have been some console related changes in Windows 11

======================================================================
ERROR: test_open_name (test.test_winconsoleio.WindowsConsoleIOTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Jeremy\source\repos\cpython\lib\test\test_winconsoleio.py", line 95, in test_open_name
    f = open('C:/con', 'rb', buffering=0)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'C:/con'

======================================================================
FAIL: test_conout_path (test.test_winconsoleio.WindowsConsoleIOTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Jeremy\source\repos\cpython\lib\test\test_winconsoleio.py", line 118, in test_conout_path
    self.assertIsInstance(f, ConIO)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: <_io.FileIO name='C:\\Users\\Jeremy\\AppData\\Local\\Temp\\tmpoqx235b0\\CONOUT$' mode='wb' closefd=True> is not an instance of <class '_io._WindowsConsoleIO'>

----------------------------------------------------------------------
msg403091 - (view) Author: Jeremy Kloth (jkloth) * Date: 2021-10-03 14:45
Note that I have a pending PR for adding a Windows 11 build worker that will, once merged, help in testing a solution.
msg403143 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-10-04 15:16
This will be the change to make "special" filenames only be special when used on their own, and not as part of a path (e.g. you shouldn't have any trouble with "aux" or "com" filenames either). I had expected it to roll out sooner, but I guess Windows 11 is when it arrives.

We'll have to go through our regression tests that rely on this behaviour. Happily, I don't think we merged any major "fixes" for the old behaviour (I've been discouraging them, but haven't been able to say that the fix was coming :( ).
msg403146 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-10-04 18:52
> This will be the change to make "special" filenames only be 
> special when used on their own, and not as part of a path

For some reason, Windows 11 still reserves case-insensitive "nul" in qualified paths, but none of the other DOS device names. Thankfully it's just "nul", and not "nul.txt", since they changed it to never reserve a name with an extension. So "./nul" -> "\\.\nul", but "./nul.txt" resolve to the filename in the current directory.

For cases such as "./con" vs "con", this is another example in favor of changing ntpath.normpath() and pathlib.Path to preserve an initial leading dot component. Other cases include alternate data streams on single-letter filenames (e.g. accessing "C:stream" as "./C:stream") and executable paths that should explicitly refer to the current directory (e.g. "./spam.exe").
msg403233 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-10-05 12:17
New changeset de4052fe0633e3a053e66c8477f13677054d6ede by Jeremy Kloth in branch 'main':
bpo-45354: Skip obsolete device name tests on Windows 11 (GH-28712)
https://github.com/python/cpython/commit/de4052fe0633e3a053e66c8477f13677054d6ede
msg403234 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-10-05 12:19
Thanks, Jeremy! And thanks for being on top of getting the Win11 buildbot set up!
msg403237 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-10-05 12:38
New changeset d0d0909a3a0b553826d1ddbb04a676fdabb61359 by Miss Islington (bot) in branch '3.10':
bpo-45354: Skip obsolete device name tests on Windows 11 (GH-28712)
https://github.com/python/cpython/commit/d0d0909a3a0b553826d1ddbb04a676fdabb61359
msg403238 - (view) Author: miss-islington (miss-islington) Date: 2021-10-05 12:39
New changeset 63c9a6cc8b48740c88199b5150c9948b1a61756b by Miss Islington (bot) in branch '3.9':
bpo-45354: Skip obsolete device name tests on Windows 11 (GH-28712)
https://github.com/python/cpython/commit/63c9a6cc8b48740c88199b5150c9948b1a61756b
msg406009 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-11-09 10:57
I reopen the issue: test_winconsoleio fails on the "AMD64 Windows11 3.x" buildbot, whereas the purpose of this issue was to get test_winconsoleio pass on Windows 11.

First AMD64 Windows11 3.x failure:
https://buildbot.python.org/all/#/builders/733/builds/243
msg406014 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-11-09 12:23
The embedded manifest in python[_d].exe is missing, so the process is running with Windows 8 compatibility mode (i.e. 6.2.9200), and thus the check for Windows 11+ (10.0.22000+) is false. It's disappointing that compatibility mode doesn't restore the old DOS device behavior, but whatever.

bpo-45220 removed the winuser.h include from the PC\*.rc files, so the RT_MANIFEST resource type macro isn't defined. It needs to be defined in the file or a common header. For example:

    #define RT_MANIFEST 24
    1 RT_MANIFEST "python.manifest"
msg406016 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-11-09 12:42
It would also work to add RT_MANIFEST=24 to the ResourceCompile PreprocessorDefinitions in PCbuild\pyproject.props. Though to me, adding the definition there is obscure compared to including a private header, such as "resourcetype.h".
msg406018 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-11-09 13:10
> bpo-45220 removed the winuser.h include from the PC\*.rc files, so the RT_MANIFEST resource type macro isn't defined. It needs to be defined in the file or a common header. For example:
> 
>      #define RT_MANIFEST 24
>      1 RT_MANIFEST "python.manifest"

I vote for this approach, since apparently RC doesn't complain about 
undefined macros. Otherwise I'd put it in the build scripts.

It's weird that the WinSDK breaks RC with the winuser.h header, but 
since it does, we don't really have a choice but to hard-code the value.
msg406020 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2021-11-09 13:18
> #define RT_MANIFEST 24

If possible, add a comment with a reference to this issue (bpo-45354) to help future readers ;-)
msg406043 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-11-09 19:35
FYI, adding a fix for this on issue45220
History
Date User Action Args
2022-04-11 14:59:50adminsetgithub: 89517
2022-04-05 17:38:36jklothsetstatus: open -> closed
resolution: fixed
2021-11-09 19:35:56steve.dowersetmessages: + msg406043
2021-11-09 13:18:28vstinnersetmessages: + msg406020
2021-11-09 13:10:54steve.dowersetmessages: + msg406018
2021-11-09 12:42:31eryksunsetmessages: + msg406016
2021-11-09 12:23:44eryksunsetmessages: + msg406014
2021-11-09 10:57:11vstinnersetstatus: closed -> open

nosy: + vstinner
messages: + msg406009

resolution: fixed -> (no value)
2021-10-05 12:39:22miss-islingtonsetmessages: + msg403238
2021-10-05 12:39:19steve.dowersetstatus: open -> closed
stage: backport needed -> resolved
2021-10-05 12:38:54steve.dowersetmessages: + msg403237
2021-10-05 12:19:52steve.dowersetresolution: fixed
stage: patch review -> backport needed
messages: + msg403234
versions: + Python 3.9, Python 3.10, Python 3.11
2021-10-05 12:18:31miss-islingtonsetpull_requests: + pull_request27085
2021-10-05 12:18:27miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request27084
2021-10-05 12:17:20steve.dowersetmessages: + msg403233
2021-10-04 18:52:20eryksunsetnosy: + eryksun
messages: + msg403146
2021-10-04 15:16:10steve.dowersetmessages: + msg403143
2021-10-03 17:46:38jklothsetkeywords: + patch
stage: patch review
pull_requests: + pull_request27062
2021-10-03 14:45:01jklothsetmessages: + msg403091
2021-10-03 14:42:28jklothcreate