classification
Title: Unformatted “Windows Error 0x%X” exception message on Wine
Type: behavior Stage: resolved
Components: Windows Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, eryksun, martin.panter, python-dev, serhiy.storchaka, steve.dower, tim.golden, vstinner, zach.ware
Priority: normal Keywords: easy, patch

Created on 2014-12-02 11:19 by martin.panter, last changed 2015-04-06 17:40 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
win-error-format.patch martin.panter, 2015-01-12 03:48 review
win-error-format-v2.patch martin.panter, 2015-01-13 01:42 review
win-error-format-v3.patch martin.panter, 2015-03-23 04:36 review
Messages (21)
msg231987 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2014-12-02 11:19
The following code generates a connection reset error on Wine (Windows emulator, because I don’t have actual Windows to test on). Probably only a minor issue, but the error message isn’t quite right:

>>> s = create_connection(("localhost", 8181))
>>> # Server end accepts connection and then closes it
>>> s.sendall(b"3" * 3000000)                 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ConnectionResetError: [WinError 10054] Windows Error 0x%X

I’m just guessing, but looking at Python/errors.c, there are two PyUnicode_FromFormat("Windows Error 0x%X", ...) calls. The documentation for that function says only a lower-case %x is supported, so that would explain the behaviour I am seeing.
msg232046 - (view) Author: Eryk Sun (eryksun) * Date: 2014-12-02 19:46
This also affects SEH-related exceptions raised by ctypes. For example, VC++ uses exception code 0xE06D7363 (i.e. b'\xe0msc'), so unhandled VC++ exceptions leak into Python like  this:

    >>> ctypes.windll.kernel32.RaiseException(0xe06d7363, 0, 0, None)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OSError: [WinError -529697949] Windows Error 0x%X

The ctypes SEH handler defaults to calling PyErr_SetFromWindowsErr(code). Since this isn't actually a Windows error code, Win32 FormatMessageW fails. Then Python uses the following default: PyUnicode_FromFormat("Windows Error 0x%X", err). 

Normally (i.e. not under Wine) the OP's error number formats correctly:

    >>> ctypes.windll.kernel32.RaiseException(10054, 0, 0, None)     
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

PyErr_SetExcFromWindowsErrWithFilenameObjects
https://hg.python.org/cpython/file/ab2c023a9432/Python/errors.c#l553

PyErr_SetFromErrnoWithFilenameObjects
https://hg.python.org/cpython/file/ab2c023a9432/Python/errors.c#l416
msg233873 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-01-12 03:48
Here’s a simple patch which should fix it, although I have not verified this because I don’t have a Windows compiler (and MINGW cross compiling sounds too tricky)
msg233876 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-01-12 09:07
The attached patch lacks an unit test. When I will be able to build CPython again, I will try the patch.
msg233899 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-01-13 01:42
This patch includes a test case, based on Eryksun’s exception code
msg238978 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-03-23 04:36
V3 patch with suggested changes to the test case, though still completely untested by me.
msg238986 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-23 07:38
LGTM.
msg238989 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-03-23 07:52
There are other occurrences of %X in the code. Do you want provide a patch for them Martin?
msg239875 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-04-02 04:42
All the other occurrences of capitalized %X that I can find are not using Python’s string formatting functions. Please point them out if you can, but all I can see are some using a Microsoft vfwprintf_s() API, some calling standard C sprintf(), sscanf(), fprintf(), strftime() and strptime() APIs, and of course native Python 2-style string formatting, which I assume should all support %X.
msg239877 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-02 06:39
Yes, you are right.
msg239878 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-02 06:50
New changeset 7907746baa0d by Serhiy Storchaka in branch '3.4':
Issue #22977: Fixed formatting Windows error messages on Wine.
https://hg.python.org/cpython/rev/7907746baa0d

New changeset cf0cac11813d by Serhiy Storchaka in branch 'default':
Issue #22977: Fixed formatting Windows error messages on Wine.
https://hg.python.org/cpython/rev/cf0cac11813d
msg239882 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-04-02 07:35
I'm not sure that Windows appreciate your change :-)

http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/6011/steps/test/logs/stdio

[216/393] test_exceptions
Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\PCbuild\..\lib\test\regrtest.py", line 1589, in <module>
    main_in_temp_cwd()
  File "C:\buildbot.python.org\3.x.kloth-win64\build\PCbuild\..\lib\test\regrtest.py", line 1564, in main_in_temp_cwd
    main()
  File "C:\buildbot.python.org\3.x.kloth-win64\build\PCbuild\..\lib\test\regrtest.py", line 738, in main
    raise Exception("Child error on {}: {}".format(test, result[1]))
Exception: Child error on test_exceptions: Exit code 3765269347
msg239885 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-04-02 07:46
I don’t pretend to know what is going on, or the best way to fix it. That exit code is the same code that my test passes to RaiseException. Perhaps it would be best to disable the test until someone with more knowledge or a Windows compiler can investigate.
msg239888 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-04-02 08:04
On Python 3.4, windll.kernel32.RaiseException(2, 0, 0, None) raised a FileNotFound error.

On Python 3.5, it displays a popup and the program exit.

It looks like the behaviour of RaiseException() changed in Python 3.5. I tested in debug and release mode.

@Steve: Any idea?

--

Instead of RaiseException, you can use ctypes.pythondll.PyErr_SetFromWindowsErr(code).

> code = int.from_bytes(b"\xE0msc", "big")

Why not writing directly code = 3765269347?

The unit test should also check the exception message.
msg239891 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015-04-02 08:19
Oh, and the ctypes module (import) must be optional, just skip the test if ctypes is missing.

http://buildbot.python.org/all/builders/AMD64%20OpenIndiana%203.x/builds/9555/steps/test/logs/stdio

test test_exceptions crashed -- Traceback (most recent call last):
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/test/regrtest.py", line 1267, in runtest_inner
    the_module = importlib.import_module(abstest)
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/importlib/__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2222, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2205, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2194, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1153, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1431, in exec_module
  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/test/test_exceptions.py", line 9, in <module>
    import ctypes
  File "/export/home/buildbot/64bits/3.x.cea-indiana-amd64/build/Lib/ctypes/__init__.py", line 7, in <module>
    from _ctypes import Union, Structure, Array
ImportError: No module named '_ctypes'
msg239892 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-02 08:23
Could you provide a patch if you can test it Victor?
msg239910 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-02 12:23
New changeset dafae2b3c257 by Victor Stinner in branch '3.4':
Issue #22977: Fix test_exceptions
https://hg.python.org/cpython/rev/dafae2b3c257
msg239949 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-04-02 23:32
Thanks for fixing the test Victor. The ctypes.pythonapi trick looks like a much better way. :)
msg240164 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2015-04-06 17:13
Unconditional 'import ctypes' in Lib/test/test_exceptions.py was not yet deleted.
msg240166 - (view) Author: Roundup Robot (python-dev) Date: 2015-04-06 17:37
New changeset 5cc0a090829a by Serhiy Storchaka in branch '3.4':
Issue #22977: Remove unconditional import of ctypes.
https://hg.python.org/cpython/rev/5cc0a090829a

New changeset f46454229cf5 by Serhiy Storchaka in branch 'default':
Issue #22977: Remove unconditional import of ctypes.
https://hg.python.org/cpython/rev/f46454229cf5
msg240168 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2015-04-06 17:40
Thanks for your outsight Arfrever.
History
Date User Action Args
2015-04-06 17:40:25serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg240168

stage: resolved
2015-04-06 17:37:56python-devsetmessages: + msg240166
2015-04-06 17:13:10Arfreversetstatus: closed -> open

nosy: + Arfrever
messages: + msg240164

resolution: fixed -> (no value)
stage: resolved -> (no value)
2015-04-03 05:14:28serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: needs patch -> resolved
2015-04-02 23:32:18martin.pantersetmessages: + msg239949
2015-04-02 12:23:10python-devsetmessages: + msg239910
2015-04-02 08:23:36serhiy.storchakasetassignee: serhiy.storchaka ->
messages: + msg239892
stage: resolved -> needs patch
2015-04-02 08:19:13vstinnersetmessages: + msg239891
2015-04-02 08:04:27vstinnersetmessages: + msg239888
2015-04-02 07:46:29martin.pantersetmessages: + msg239885
2015-04-02 07:35:34vstinnersetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg239882
2015-04-02 06:51:04serhiy.storchakasetstatus: open -> closed
resolution: fixed
stage: commit review -> resolved
2015-04-02 06:50:34python-devsetnosy: + python-dev
messages: + msg239878
2015-04-02 06:39:42serhiy.storchakasetmessages: + msg239877
2015-04-02 04:42:32martin.pantersetmessages: + msg239875
2015-03-23 07:52:19serhiy.storchakasetmessages: + msg238989
2015-03-23 07:38:41serhiy.storchakasetmessages: + msg238986
stage: patch review -> commit review
2015-03-23 04:36:35martin.pantersetfiles: + win-error-format-v3.patch

messages: + msg238978
2015-03-21 09:45:50serhiy.storchakasetassignee: serhiy.storchaka

nosy: + serhiy.storchaka
2015-01-13 01:42:38martin.pantersetfiles: + win-error-format-v2.patch

messages: + msg233899
2015-01-12 16:36:32berker.peksagsetstage: patch review
versions: + Python 3.4, Python 3.5, - Python 3.3
2015-01-12 09:07:03vstinnersetmessages: + msg233876
2015-01-12 03:48:02martin.pantersetfiles: + win-error-format.patch
keywords: + patch
messages: + msg233873
2014-12-02 21:34:24vstinnersettitle: Unformatted “Windows Error 0x%X” exception message -> Unformatted “Windows Error 0x%X” exception message on Wine
2014-12-02 19:46:05eryksunsetnosy: + eryksun
messages: + msg232046
2014-12-02 11:35:00vstinnersetkeywords: + easy
nosy: + vstinner
2014-12-02 11:19:44martin.pantercreate