classification
Title: Native Windows Python builds running on Europe/Moscow TZ report wrong time from datetime.datetime.now when there is TZ environment variable also set to Europe/Moscow
Type: behavior Stage: resolved
Components: Windows Versions: Python 3.8
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: Nosy List: belopolsky, eryksun, mikekaganski, p-ganssle, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2021-06-08 19:05 by mikekaganski, last changed 2021-06-11 13:19 by mikekaganski. This issue is now closed.

Messages (5)
msg395355 - (view) Author: Mike Kaganski (mikekaganski) Date: 2021-06-08 19:05
On a Windows 10 system, which TZ is set to Moscow (UTC+3), I use a native Windows Python build (as opposed to e.g. one from Cygwin). Specifically, I tested both custom Python build created by LibreOffice project, as well as the Python by Python Software Foundation available from MS Store [1].

1. Open command prompt on Windows (cmd.exe).
2. Execute 'set TZ=Europe/Moscow'
3. Execute 'python'
4. In the Python prompt, execute 'import datetime'
5. Execute 'datetime.datetime.now()'

The result of this is a time that is two hours off, e.g.

> datetime.datetime(2021, 6, 8, 19, 59, 21, 925240)

instead of proper

> datetime.datetime(2021, 6, 8, 21, 59, 21, 925240)

which appears when step #2 is skipped.
Possibly Python takes both system time zone information *and* the environment variable into account when calculating the time. I suppose it should only consider one or the other, not both.

Note that the problem does not happen with Cygwin's Python, which works fine with the same TZ environment variable value.

For a real-life problem that results from this, see case at [2], where unit test is failing only from 00:00 till 02:00 on my local system, obviously telling me that I should sleep at that time, not try to run unit tests :-)

[1] https://www.microsoft.com/en-us/p/python-38/9mssztt1n39l
[2] https://gerrit.libreoffice.org/c/core/+/92217/2#message-f55091795e7cde9d75adc00ddb69451121b644f6
msg395503 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-06-10 04:31
> 2. Execute 'set TZ=Europe/Moscow'

The Windows C runtime supports a simple format for the TZ environment variable, which is detailed in the documentation of _tzset() [1]. For example, it's "MSK-3" for Moscow Standard Time. It's -3 because the offset is from local time to UTC. 

The CRT does not verify that the TZ value is valid. "Europe/Moscow" is invalid, but it's blindly parsed anyway. There's no UTC offset, so it defaults to UTC (0 offset). It's parsed as supporting daylight saving time, according to U.S. rules. Currently this corresponds to UTC + 1. That's why there's a -2 hour delta from Moscow Standard Time, which does not observe daylight saving time.

I recommend that you unset the TZ environment variable before running Windows Python. Use the system timezone.

---
[1] https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/tzset?view=msvc-160
msg395605 - (view) Author: Mike Kaganski (mikekaganski) Date: 2021-06-11 05:44
Thank you Eryk! This is a good workaround for me. I have implemented it: https://git.libreoffice.org/core/+/3bcaa4ba79477a21251ddaa06e0ea159196a7ffb

It looks like it's not a Python's problem; I suppose this may be closed. Thanks again!
msg395631 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2021-06-11 11:10
Note that this explanation in your commit is wrong and unhelpful: "likely because datetime.datetime.now in the native Windows Python takes into account both system timezone data and the TZ environment variable". When TZ is set, localtime() is based only on the TZ value, and daylight saving time uses only U.S. rules (e.g. beginning 2021-03-14 and ending 2021-11-07). The value must be of the form "tzn [+|-]hh[:mm[:ss] ][dzn]", but there is no validation. So "Europe/Moscow" is invalid and gets parsed as UTC with U.S. DST. I recommend clearing the TZ variable because the value format is non-standard, and its DST support is U.S.-centric nonsense.
msg395633 - (view) Author: Mike Kaganski (mikekaganski) Date: 2021-06-11 13:19
@Eryk Sun: yes, of course you are right - but please see the date of the commit; I didn't know what you kindly explained me in your reply yesterday :-) Thank you again.
History
Date User Action Args
2021-06-11 13:19:47mikekaganskisetmessages: + msg395633
2021-06-11 11:10:39eryksunsetmessages: + msg395631
2021-06-11 05:44:58mikekaganskisetstatus: open -> closed
resolution: third party
messages: + msg395605

stage: resolved
2021-06-10 04:31:36eryksunsetnosy: + eryksun
messages: + msg395503
2021-06-10 02:05:16zach.waresetnosy: + belopolsky, p-ganssle
2021-06-08 19:05:21mikekaganskicreate