classification
Title: datetime in Python 3.6+ no longer respects 'TZ' environment variable
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: adamwill, ammar2, belopolsky, jwilk
Priority: normal Keywords:

Created on 2017-04-12 21:43 by adamwill, last changed 2017-06-25 05:22 by ammar2. This issue is now closed.

Messages (4)
msg291574 - (view) Author: Adam Williamson (adamwill) Date: 2017-04-12 21:43
I can't figure out yet why this is, but it's very easy to demonstrate:

[adamw@adam anaconda (time-log %)]$ python35 
Python 3.5.2 (default, Feb 11 2017, 18:09:24) 
[GCC 7.0.1 20170209 (Red Hat 7.0.1-0.7)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import datetime
>>> os.environ['TZ'] = 'America/Winnipeg'
>>> datetime.datetime.fromtimestamp(0)
datetime.datetime(1969, 12, 31, 18, 0)
>>> os.environ['TZ'] = 'Europe/London'
>>> datetime.datetime.fromtimestamp(0)
datetime.datetime(1970, 1, 1, 1, 0)
>>> 

[adamw@adam anaconda (time-log %)]$ python3
Python 3.6.0 (default, Mar 21 2017, 17:30:34) 
[GCC 7.0.1 20170225 (Red Hat 7.0.1-0.10)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import datetime
>>> os.environ['TZ'] = 'America/Winnipeg'
>>> datetime.datetime.fromtimestamp(0)
datetime.datetime(1969, 12, 31, 16, 0)
>>> os.environ['TZ'] = 'Europe/London'
>>> datetime.datetime.fromtimestamp(0)
datetime.datetime(1969, 12, 31, 16, 0)
>>> 

That is, when deciding what timezone to use for operations that involve one, if the 'TZ' environment variable was set, Python 3.5 would use the timezone it was set to. Python 3.6 does not, it ignores it.

As you can see, if I twiddle the 'TZ' setting and call `datetime.datetime.fromtimestamp(0)` repeatedly under Python 3.5, I get different results - each one is the wall clock time at the epoch (timestamp 0) in the timezone specified as 'TZ'. If I do the same on Python 3.6, the 'TZ' setting is ignored and I always get the same result (the wall clock time of 'the epoch' in Vancouver, which is my real timezone, and which I guess is being picked up from /etc/localtime or whatever).

This wound up causing a problem in the Fedora / Red Hat installer, anaconda:

https://bugzilla.redhat.com/show_bug.cgi?id=1433560

The 'current time zone' can be changed in anaconda. Shortly after it starts up, it automatically tries to guess the correct time zone via geolocation, and the user can also explicitly choose a timezone in the installer interface (or set one in a kickstart). Whenever the timezone is set in this way, an underlying library (libtimezonemap - https://launchpad.net/timezonemap) sets 'TZ' to the chosen timezone. It turns out other code in anaconda relies on Python respecting that setting, which Python 3.6 does not do. As a consequence, anaconda with Python 3.6 winds up setting the system time incorrectly. Also, the timestamps on all its log files are different now, and there may well be other consequences I didn't figure out yet.

The same applies to, e.g., `datetime.datetime.now()`: you can perform the same experiment with Python 3.5 and 3.6. If you change the 'TZ' env var while calling `datetime.datetime.now()` after each change, on Python 3.5, the naive datetime object it returns is the current time *in that timezone*. On Python 3.6, regardless of what 'TZ' is set to, it always gives you the same time.

Is this an intended and/or desired change that we should adjust to somehow? Is there another way a running Python process can change what "the current" timezone is, for the purposes of datetime calculations like this?
msg291576 - (view) Author: Adam Williamson (adamwill) Date: 2017-04-12 22:37
Hmm, after a bit more poking I found this:

https://docs.python.org/3/library/time.html#time.tzset

"Note

Although in many cases, changing the TZ environment variable may affect the output of functions like localtime() without calling tzset(), this behavior should not be relied on."

It seems like that's kinda what we're dealing with here. If I extend my tests to change TZ, call the test function, then call `time.tzset()` and call the test function again, the *second* call to the test function gives the different result, i.e. the `time.tzset()` call does what it claims and changes Python's conception of the 'current' timezone.

So while that note has been there all along, it seems like the behaviour actually changed between 3.5 and 3.6, and a change to 'TZ' is now less likely to be respected without a `tzset()` call. But given the doc note, perhaps that can't be considered a bug.

anaconda doesn't call `time.tzset()` anywhere at present. It's also multi-threaded, so making sure all the threads call `time.tzset()` after any thread has changed what the 'current' timezone is will be lots of fun to implement, I guess :/
msg295642 - (view) Author: Jakub Wilk (jwilk) Date: 2017-06-10 14:40
This is a side-effect of fixing issue #28067.
msg296803 - (view) Author: Ammar Askar (ammar2) * (Python triager) Date: 2017-06-25 05:22
Closing this since as you pointed out this behavior is already documented.

As an aside, seeing as you guys already have pytz as a dependency. Instead of fixing this with tzset you can also just explicitly do datetime.datetime.fromtimestamp(time, pytz.timezone(os.environ['TZ']))
History
Date User Action Args
2017-06-25 05:22:50ammar2setstatus: open -> closed

nosy: + ammar2
messages: + msg296803

resolution: not a bug
stage: resolved
2017-06-10 14:40:40jwilksetnosy: + jwilk
messages: + msg295642
2017-04-12 22:37:09adamwillsetmessages: + msg291576
2017-04-12 21:45:41berker.peksagsetnosy: + belopolsky
2017-04-12 21:43:42adamwillcreate