Based on a bug report (https://bitbucket.org/openpyxl/openpyxl/issues/1266/locale) from a user of the openpyxl library I've identified a bug in the zipfile module that causes the Python process to crash on Windows. Currently tested with Python 3.7.3 (32-bit on Windows 10).

Sample code

import faulthandler
import locale
from zipfile import ZipFile

faulthandler.enable()

locale.setlocale(locale.LC_ALL, 'de_DE')
out = open("out.zip", "wb") 
archive = ZipFile(out, "w")
archive.writestr("properties.xml", b"<workbookPr/>")

faulthandler fingers line 1757 as the culprit but running this line locally does not cause the crash. The issue seems to be limited to Windows.

I can't reproduce this error on my system by using the same code.

Could you share the system info with us?

Or would you share the Exception with us ?

I guess it's caused by system setting

Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Intel)] on win32

Same environment.

But I still can not reproduce this exception. I guess maybe it's about the local time or timezone problem. I will find a way to figure it out

That's what we thought when we looked at it, but as I said, I couldn't reproduce it with just the `time` call or the `ZInfo` instantiation, so something odd is happening. I do have a German version of Windows as I suspect the original reporter does. You'd think that sort of thing wouldn't matter…

I can reproduce it on Python 3.7.3 german Windows10 enterprise

Windows fatal exception: code 0xc0000374

Current thread 0x00003bc8 (most recent call first):
  File "C:\Python37\lib\zipfile.py", line 1757 in writestr

Hi  Dominik Geldmacher

May I get your system version? 1809 or 1903?

I guess maybe it's microsoft error

Enterprise 1809 and Professional 1803 (german localized) both reproduce the issue

copy that

I will reset my locale setting to figure it out

winver tells me I have 1809. I'm only using Windows in a VM so I'm not that familiar with its innards.

Also get the error with WinPython 3.6:

Windows fatal exception: code 0xc0000374

Current thread 0x000010c0 (most recent call first):
  File "C:\Users\charlieclark\WPy64-3680\python-3.6.8.amd64\lib\zipfile.py", line 1658 in writestr
  File "<stdin>", line 1 in <module>

Related to issue bpo-36319

I can confirm the error is caused by time.localtime(time.time()) as indicated by the related bug.

I've also found the crash log. It's in C:\ProgramData\Microsoft\Windows\WER\ReportQueue on my machine. Well, at least that's all I can find.

So does that mean that simply doing:

import locale, time
locale.setlocale(locale.LC_ALL, 'de_DE')
time.localtime(time.time())

is enough to trigger the heap corruption?  If yes, then what is the output of:

import locale, time
locale.setlocale(locale.LC_ALL, 'de_DE')
time.strftime('%Z')

import time, locale
locale.setlocale(locale.LC_ALL, 'de_DE')
'de_DE'
time.strftime("%Z")
aborted (disconnected)

Ok, now let's try it using the C runtime directly:

import ctypes, struct
libc = ctypes.cdll.msvcrt
buf = ctypes.create_string_buffer(1024)
tm = struct.pack('9i', 2019, 5, 6, 9, 50, 4, 0, 126, 1)
print('count:', libc.strftime(buf, 1024, b'%Z', tm))
print('value:', buf.value)

import ctypes, struct
libc = ctypes.cdll.msvcrt
buf = ctypes.create_string_buffer(1024)
tm = struct.pack('9i', 2019, 5, 6, 9, 50, 4, 0, 126, 1)
print('count:', libc.strftime(buf, 1024, b'%Z', tm))
print('value:', buf.value)

count: 28
value: b'Mitteleurop\xe4ische Sommerzeit'

Oops, I forgot to add in my snippet, the setlocale() call prior to calling the C strftime() function.  So an updated test:

import locale
locale.setlocale(locale.LC_ALL, 'de_DE')
import ctypes, struct
libc = ctypes.cdll.msvcrt
buf = ctypes.create_string_buffer(1024)
tm = struct.pack('9i', 2019, 5, 6, 9, 50, 4, 0, 126, 1)
print('count:', libc.strftime(buf, 1024, b'%Z', tm))
print('value:', buf.value)
wbuf = ctypes.create_unicode_buffer(1024)
print('count:', libc.wcsftime(wbuf, 1024, '%Z', tm))
print('value:', wbuf.value)
print('count:', libc.mbstowcs(wbuf, buf, 1024))
print('value:', wbuf.value)

import ctypes, struct
libc = ctypes.cdll.msvcrt
buf = ctypes.create_string_buffer(1024)
tm = struct.pack('9i', 2019, 5, 6, 9, 50, 4, 0, 126, 1)
print('count:', libc.strftime(buf, 1024, b'%Z', tm))
print('value:', buf.value)
wbuf = ctypes.create_unicode_buffer(1024)
print('count:', libc.wcsftime(wbuf, 1024, '%Z', tm))
print('value:', wbuf.value)
print('count:', libc.mbstowcs(wbuf, buf, 1024))
print('value:', wbuf.value)

count: 28
value: b'Mitteleurop\xe4ische Sommerzeit'
count: 28
value: Mitteleuropäische Sommerzeit
count: 28
value: Mitteleuropäische Sommerzeit

> libc = ctypes.cdll.msvcrt

That's the private CRT of Windows, not the Universal CRT for applications. In a release build (python.exe), use ctypes.CDLL('ucrtbase', use_errno=True). In a debug build (python_d.exe), use ctypes.CDLL('ucrtbased', use_errno=True).

I suppose we should use API sets [1] for the release build, such as "api-ms-win-crt-locale-l1-1-0" and 
"api-ms-win-crt-time-l1-1-0". But they resolve to "ucrtbase". 

[1]: https://docs.microsoft.com/en-us/uwp/win32-and-com/win32-extension-apis

Thanks for the reminder Eryk Sun.  This means the test needs to be run yet one more time :)

import ctypes, locale, struct
crt_time = ctypes.CDLL('api-ms-win-crt-time-l1-1-0', use_errno=True)
locale.setlocale(locale.LC_ALL, 'de_DE')
buf = ctypes.create_string_buffer(1024)
tm = struct.pack('9i', 2019, 5, 6, 9, 50, 4, 0, 126, 1)
print('count:', crt_time.strftime(buf, 1024, b'%Z', tm))
print('value:', buf.value)
wbuf = ctypes.create_unicode_buffer(1024)
print('count:', crt_time.wcsftime(wbuf, 1024, '%Z', tm))
print('value:', wbuf.value)
crt_convert = ctypes.CDLL('api-ms-win-crt-convert-l1-1-0', use_errno=True)
print('count:', crt_convert.mbstowcs(wbuf, buf, 1024))
print('value:', wbuf.value)

The code crashes on this line:
print('count:', crt_time.strftime(buf, 1024, b'%Z', tm))

Thanks for your patience with this Charlie, but please try another run this time without the strftime() and mbstowcs() calls.  Honest, we are getting closer!

If the process crashes at the first print statement, I'm not sure how I can run the tests. Or should I try them separately?

You can safely execute each line individually (omitting the aforementioned count/value pairs) or depending on how the copy/paste is being done, just paste the script into a text editor (notepad) and comment out those lines.  Then copy-paste that modified script.

This time I'm really attempting to see if the internal encoding/decoding done in the CRT strftime() is failing somehow (it calls wcsftime).

print('count:', crt_time.wcsftime(wbuf, 1024, '%Z', tm)) also fails
but
crt_convert = ctypes.CDLL('api-ms-win-crt-convert-l1-1-0', use_errno=True)
print('count:', crt_convert.mbstowcs(wbuf, buf, 1024))
seems to work okay.

Here is another test, this time removing Python from the equation (mostly :)

import ctypes, struct
crt_locale = ctypes.CDLL('api-ms-win-crt-locale-l1-1-0', use_errno=True)
crt_time = ctypes.CDLL('api-ms-win-crt-time-l1-1-0', use_errno=True)
crt_locale._wsetlocale.restype = ctypes.c_wchar_p
print('old locale:', crt_locale._wsetlocale(0, 0))
tm = struct.pack('9i', 2019, 5, 6, 9, 50, 4, 0, 126, 1)
wbuf = ctypes.create_unicode_buffer(1024)
print('count:', crt_time.wcsftime(wbuf, 1024, '%Z', tm))
print('value:', wbuf.value)
print('new locale:', crt_locale._wsetlocale(0, 'de_DE'))
print('count:', crt_time.wcsftime(wbuf, 1024, '%Z', tm))
print('value:', wbuf.value)

And this is the result.

old locale: C
count: 28
value: Mitteleuropäische Sommerzeit
new locale: de_DE
count: -1
value:
Windows fatal exception: code 0xc0000374

Looks like

print('new locale:', crt_locale._wsetlocale(0, 'de_DE'))
print('count:', crt_time.wcsftime(wbuf, 1024, '%Z', tm))

is the culprit.

Final test, this time, no Python what so ever.  I've added a zip containing a simple C program (source and .exe) that performs the same test.

The output should be similar to:

--------
The current locale is now: C
The time zone is: 'Mountain Daylight Time' (22 characters)
--------
The updated locale is now: de_DE
The time zone is: 'Mountain Daylight Time' (22 characters)
--------

If this crashes, the problem is then within the CRT itself :(

This is the result
\issue36792>test.exe
--------
The current locale is now: C
The time zone is: 'Mitteleuropõische Sommerzeit' (28 characters)
--------
The updated locale is now: de_DE
The time zone is: '' (-1 characters)
--------
NB something is wrong with that string. Should be Mitteleuropäsiche Sommerzeit but that could just be an encoding thing.

Thanks again!  I will have some more tests for you to try tomorrow as I am out of time for today.

I'm currently of the belief that there is something Python is going to have to do to work around an issue within the CRT, but more testing will prove that theory.

I've added another test executable (issue36792-2.zip) which should bring some insight into where things are going wrong.  Please run and post the results.

I have managed to setup a VM that can reproduce the error.  Unfortunately, the error (heap corruption) is coming from within the UCRT.  Attempting to work around that, I came across another error in the UCRT.

Due to these errors in all available UCRT versions, the only available solution is to re-implement %Z handling in Python's implementation of strftime().

@steve.dower: If you could forward the following information to the interested parties for faster response, it would be great.

Everyone else, what follows contains UCRT internal details which is of no interest to Python.

In wcsftime() [ucrt\time\wcsftime.cpp, line 1018], the call to _mbstowcs_s_l() doesn't handle the error case of EILSEQ.  This error happens when the tzname (originally set by __tzset() from the Unicode name in the registry encoded to the ACP) contains an undecodable character.  In the 'de_DE' case, \u00E4.  _mbstowcs_s_l returns without changing `wnum` (which is 0).  The 'Z' case then blindly subtracts 1 (wnum=-1) and advances the string pointer, now pointing *before* the start when the format is just L"%Z" (like Python has).  Now when wcsftime() is returning it stores a '\0' at that current position thus causing a heap corruption.

Ok, no deal breaker, we can just update the _tzname array when we call setlocale().  Alas, another error :(  If _tzset() is called a second time, it fails when encoding the Unicode name due to an uninitialized variable.  In tzset_from_system_nolock() [ucrt\time\tzset.cpp, line 158], the function __acrt_WideCharToMultiByte() [appcrt\locale\widechartomultibyte.cpp] does not set the `lpUsedDefaultChar` flag when FALSE.  This is only an issue, it seems, on subsequent calls as `used_default_char` can be non-zero prior to the call.  A non-zero value there, causes the tzname to be set to the empty string even if the conversion succeeded.

I've received a detailed response from the UCRT team, and there are a few pieces here.

* the fact that tzname is cached in ACP is known and will be fixed
* the decoding bug is real, but it's due to the experimental UTF-8 support
* the experimental UTF-8 support was enabled because "de_DE" is not a known Windows locale name - try with "de-DE"

Perhaps it would be easy to do the replacement of underscores with hyphens on Windows in this function? I think that's safe enough, yes?

I can confirm that using "de-DE" does indeed avoid the crash.

> * the experimental UTF-8 support was enabled because "de_DE" is not a
> known Windows locale name - try with "de-DE"
>
> Perhaps it would be easy to do the replacement of underscores with hyphens
> on Windows in this function? I think that's safe enough, yes?
>

Even some well known locale names still use the utf-8 code page.  Most seem
to uncommon, but at least es-BR (Brazil) does and would still fall victim
to these UCRT bugs.

>

> Perhaps it would be easy to do the replacement of underscores with 
> hyphens on Windows in this function? I think that's safe enough, yes?

Recent releases of ucrt implement this translation from underscore to hyphen for us, so this suggestion is no longer always necessary. However, I don't know what the PEP-11 stance is regarding the modern lifecycle policy of Windows 10. Will Python support a release for as long as it's supported by the enterprise version? For example, support for 1709 enterprise ends on 2020-04-14, so if we followed that, then Python 3.9 would require Windows 10 1803 or higher. That seems wrong while we're still supporting Windows 8.1, but what will the stance be when Python supports only Windows versions that use the modern lifecycle?

> Even some well known locale names still use the utf-8 code page.  Most 
> seem to uncommon, but at least es-BR (Brazil) does and would still 
> fall victim to these UCRT bugs.

es-BR is a custom locale for the Spanish language in Brazil, as opposed to the common Portuguese locale (pt-BR). It's a Unicode-only locale, which means its ANSI codepage is 0. Since 0 is CP_ACP, its effective ANSI codepage is the system or process ANSI codepage. 

For example:

    >>> kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)
    >>> buf = (ctypes.c_wchar * 10)()

Portuguese in Brazil uses codepage 1252 as its ANSI codepage:

    >>> n = kernel32.GetLocaleInfoEx('pt-BR', 0x1004, buf, 10)
    >>> buf.value
    '1252'

Spanish in Brazil uses CP_ACP:

    >>> n = kernel32.GetLocaleInfoEx('es-BR', 0x1004, buf, 10)
    >>> buf.value
    '0'

hi-IN (Hindi, India) is a common Unicode-only locale:

    >>> n = kernel32.GetLocaleInfoEx('hi-IN', 0x1004, buf, 10)
    >>> buf.value
    '0'

ucrt has switched to using UTF-8 for Unicode-only locales:

    >>> locale.setlocale(locale.LC_CTYPE, 'hi_IN')
    'hi_IN'
    >>> ucrt = ctypes.CDLL('ucrtbase', use_errno=True)
    >>> ucrt.___lc_codepage_func()
    65001

Note that ucrt uses UTF-8 for Unicode-only locales only when using an explicitly named locale such as "hi_IN", "Hindi_India" or even just "Hindi". On the other hand, if a Unicode-only locale is used implicitly, ucrt instead uses the system ANSI codepage:

    >>> locale.setlocale(locale.LC_CTYPE, '')
    'Hindi_India.1252'
    >>> ucrt.___lc_codepage_func()
    1252

I suppose this is for backwards compatibility. Windows 10 at least supports setting the system ANSI codepage to UTF-8, or overriding the process ANSI codepage to UTF-8 via the application manifest "actveCodePage" setting. For the latter, I modified the manifest in a "python_utf8.exe" copy of the normal "python.exe" binary, which is simpler than having to reboot to change the system locale:

    C:\>python_utf8 -q

    >>> import locale
    >>> locale.setlocale(locale.LC_CTYPE, '')
    'Hindi_India.utf8'

> Will Python support a release for as long as it's supported by the enterprise version? For example, support for 1709 enterprise ends on 2020-04-14, so if we followed that, then Python 3.9 would require Windows 10 1803 or higher. That seems wrong while we're still supporting Windows 8.1, but what will the stance be when Python supports only Windows versions that use the modern lifecycle?

This needs a separate discussion on python-dev, but for issues like this I think it's fine to blame the underlying platform and recommend getting their updates. We don't have to work around every Windows bug for all time.

I'll read the rest of what's been happening on this issue when I'm less tired, but not quite up to it right now ;)

ucrt in Windows 10 v2004 uses an internal wide-character version of the time-zone name, which gets updated by _tzset() and kept in sync with the _tzname encoded strings. Also, note that the current implementation uses the locale's ANSI code page for both "de-DE" and "de_DE". UTF-8 has to be requested explicitly when using a BCP-47 locale name, such as "de_DE.UTF-8" or "de-DE.UTF-8", and UTF-8 is the only explicit codeset that's allowed in this case.