Title: Native Windows Python builds running on Europe/Moscow TZ report wrong time from when there is TZ environment variable also set to Europe/Moscow
Type: behavior Stage: resolved
Components: Windows Versions: Python 3.8
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: Nosy List: belopolsky, eryksun, mikekaganski, p-ganssle, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2021-06-08 19:05 by mikekaganski, last changed 2021-06-11 13:19 by mikekaganski. This issue is now closed.

Messages (5)
msg395355 - (view) Author: Mike Kaganski (mikekaganski) Date: 2021-06-08 19:05
On a Windows 10 system, which TZ is set to Moscow (UTC+3), I use a native Windows Python build (as opposed to e.g. one from Cygwin). Specifically, I tested both custom Python build created by LibreOffice project, as well as the Python by Python Software Foundation available from MS Store [1].

1. Open command prompt on Windows (cmd.exe).
2. Execute 'set TZ=Europe/Moscow'
3. Execute 'python'
4. In the Python prompt, execute 'import datetime'
5. Execute ''

The result of this is a time that is two hours off, e.g.

> datetime.datetime(2021, 6, 8, 19, 59, 21, 925240)

instead of proper

> datetime.datetime(2021, 6, 8, 21, 59, 21, 925240)

which appears when step #2 is skipped.
Possibly Python takes both system time zone information *and* the environment variable into account when calculating the time. I suppose it should only consider one or the other, not both.

Note that the problem does not happen with Cygwin's Python, which works fine with the same TZ environment variable value.

For a real-life problem that results from this, see case at [2], where unit test is failing only from 00:00 till 02:00 on my local system, obviously telling me that I should sleep at that time, not try to run unit tests :-)

msg395503 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-06-10 04:31
> 2. Execute 'set TZ=Europe/Moscow'

The Windows C runtime supports a simple format for the TZ environment variable, which is detailed in the documentation of _tzset() [1]. For example, it's "MSK-3" for Moscow Standard Time. It's -3 because the offset is from local time to UTC. 

The CRT does not verify that the TZ value is valid. "Europe/Moscow" is invalid, but it's blindly parsed anyway. There's no UTC offset, so it defaults to UTC (0 offset). It's parsed as supporting daylight saving time, according to U.S. rules. Currently this corresponds to UTC + 1. That's why there's a -2 hour delta from Moscow Standard Time, which does not observe daylight saving time.

I recommend that you unset the TZ environment variable before running Windows Python. Use the system timezone.

msg395605 - (view) Author: Mike Kaganski (mikekaganski) Date: 2021-06-11 05:44
Thank you Eryk! This is a good workaround for me. I have implemented it:

It looks like it's not a Python's problem; I suppose this may be closed. Thanks again!
msg395631 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-06-11 11:10
Note that this explanation in your commit is wrong and unhelpful: "likely because in the native Windows Python takes into account both system timezone data and the TZ environment variable". When TZ is set, localtime() is based only on the TZ value, and daylight saving time uses only U.S. rules (e.g. beginning 2021-03-14 and ending 2021-11-07). The value must be of the form "tzn [+|-]hh[:mm[:ss] ][dzn]", but there is no validation. So "Europe/Moscow" is invalid and gets parsed as UTC with U.S. DST. I recommend clearing the TZ variable because the value format is non-standard, and its DST support is U.S.-centric nonsense.
msg395633 - (view) Author: Mike Kaganski (mikekaganski) Date: 2021-06-11 13:19
@Eryk Sun: yes, of course you are right - but please see the date of the commit; I didn't know what you kindly explained me in your reply yesterday :-) Thank you again.
Date User Action Args
2021-06-11 13:19:47mikekaganskisetmessages: + msg395633
2021-06-11 11:10:39eryksunsetmessages: + msg395631
2021-06-11 05:44:58mikekaganskisetstatus: open -> closed
resolution: third party
messages: + msg395605

stage: resolved
2021-06-10 04:31:36eryksunsetnosy: + eryksun
messages: + msg395503
2021-06-10 02:05:16zach.waresetnosy: + belopolsky, p-ganssle
2021-06-08 19:05:21mikekaganskicreate