This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: tizinfo.fromutc changed for tzinfo wih StdOffset=0, DstOffset=1
Type: behavior Stage: resolved
Components: Extension Modules Versions: Python 3.6, Python 3.4, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: belopolsky Nosy List: PeterJCLaw, alynn, belopolsky, peterjclaw, python-dev, serhiy.storchaka, tim.peters
Priority: normal Keywords: patch

Created on 2015-03-07 00:43 by peterjclaw, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
time_issues.py peterjclaw, 2015-03-07 11:33 Shorter demo of the issue
issue23600.diff belopolsky, 2015-09-27 20:53 review
Messages (15)
msg237407 - (view) Author: Peter J C Law (peterjclaw) Date: 2015-03-07 00:43
There's a difference in behaviour between the ``fromutc`` method on a tzinfo between Python 2 and Python 3, though only under the specific case of Summer Time in regions whose usual offset is 0.

From what I can tell, it's the Python 3 one which is wrong, based on it no longer matching the sample implementation provided in the docs and on it appearing to report the wrong times for ``datetime.now(tz)`` when on a machine configured for BST during June 2015.
Similar results can also be achieved using a manually constructed ``datetime`` for dates during summer 2014.

I've put the python script (and sample outputs) I used to investigate in a gist at https://gist.github.com/PeterJCLaw/d8bcc168d68acf066811#file-time_issues-py. The outputs there are for pythons 2.7.9 and 3.4.0.
msg237414 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2015-03-07 03:48
Peter,

Can you attach your demo script to the issue?  Better yet, is it possible to explain the issue without referring to 100 lines of code?
msg237440 - (view) Author: Peter J C Law (peterjclaw) Date: 2015-03-07 11:33
Hi,

Sorry for the overkill demo. I've attached a much shorter version, the key portion of which seems to be that, for the case of UK summer time the timezone, the tzinfo's `dst()` and `utcoffset()` methods return the same value.

This results in the delta between the two (which my understanding suggests equates to the non-dst offset of the timezone) is zero (which is right).

The python implementations (both in datetime.py and in the docs) cope with this by checking the DST difference and applying this after the timezone adjustments have happened.

From a look through the CPython implementation, it looks to me like it's checking the wrong value before applying the DST difference. Specifically, on line 3033 in Modules/_datetimemodule.c, it checks `delta` before applying the DST.

For the majority of timezones this happens to work since the standard offset is not 0 and it ends up doing the addition anyway (and no functionality is lost if the DST value is 0).

Having tested changing the checked value to being dst rather than delta this does appear to fix this and all the existing tests still pass.

Peter
msg251672 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2015-09-26 23:52
I am afraid you misunderstand how fromutc() method works.  Note that you rarely need to call it directly: use astimezone() method to convert between timezones.
msg251675 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2015-09-27 03:04
I expect Peter is correct:  the C fromutc() doesn't match the logic of the Python fromutc(), and there are no comments explaining why the C version changed the logic.

The last 4 lines of his `time_issues.py` show the difference.  The simplified UKSummerTime tzinfo always says the total UTC offset and the DST offset are +1:00:00.  The Python .fromutc() adds that hour to the datetime passed in, but the C .fromutc() does not.  That's because they implement different functions, not because Peter is using .fromutc() ;-)
msg251717 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2015-09-27 19:01
It looks like I introduced this bug ~ 5 years ago when I made a switch from integer minutes offset to an arbitrary timedelta.  It's rather amazing that it took so long to discover it.
msg251725 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2015-09-27 20:53
Attaching a patch that should fix the issue.
msg251734 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2015-09-28 00:10
Patch looks good to me!  Thanks :-)
msg251735 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2015-09-28 00:17
Thanks, Tim.  Now I need to figure out how to commit to multiple branches.  This goes to 3.4 through 3.6, right? or just to 3.5 and 3.6.
msg251737 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2015-09-28 00:41
Afraid that's a question for python-dev - I lost track of the active branches over year ago :-(
msg251741 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2015-09-28 02:24
I give up. Somehow my changes conflict with 

parent:      98335:0d3b64bbc82c
user:        Serhiy Storchaka <storchaka@gmail.com>
date:        Sun Sep 27 22:38:33 2015 +0300
summary:     Issue #25203: Failed readline.set_completer_delims() no longer left the

and my knowledge of hg is not enough to get out.  When are we switching to git?
msg251743 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2015-09-28 02:35
New changeset ff68705c56a8 by Alexander Belopolsky in branch '3.4':
Closes issue #23600: Wrong results from tzinfo.fromutc().
https://hg.python.org/cpython/rev/ff68705c56a8

New changeset c706c062f545 by Alexander Belopolsky in branch '3.5':
Closes issue #23600: Wrong results from tzinfo.fromutc().
https://hg.python.org/cpython/rev/c706c062f545

New changeset 4879c10ce982 by Alexander Belopolsky in branch 'default':
Closes issue #23600: Wrong results from tzinfo.fromutc().
https://hg.python.org/cpython/rev/4879c10ce982

New changeset cbcf82f92c25 by Alexander Belopolsky in branch '3.4':
Closes issue #23600: Wrong results from tzinfo.fromutc().
https://hg.python.org/cpython/rev/cbcf82f92c25

New changeset 848d665bc312 by Alexander Belopolsky in branch '3.5':
Closes issue #23600: Wrong results from tzinfo.fromutc().
https://hg.python.org/cpython/rev/848d665bc312
msg251744 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2015-09-28 02:39
OK, I have no idea how I managed to create two commits in 3.4 and 3.5 and loose the NEWS entry in the end.
msg251785 - (view) Author: Peter Law (PeterJCLaw) * Date: 2015-09-28 18:50
Awesome, thanks for fixing this.
msg251793 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2015-09-28 19:30
Thank you for your persistence and patience, Peter!  It shouldn't have been this hard for you :-(
History
Date User Action Args
2022-04-11 14:58:13adminsetgithub: 67788
2015-09-28 19:30:39tim.peterssetmessages: + msg251793
2015-09-28 18:50:44PeterJCLawsetnosy: + PeterJCLaw
messages: + msg251785
2015-09-28 17:48:47belopolskysetstatus: open -> closed
resolution: fixed
stage: commit review -> resolved
2015-09-28 02:39:30belopolskysetmessages: + msg251744
2015-09-28 02:35:34python-devsetnosy: + python-dev
messages: + msg251743
2015-09-28 02:24:20belopolskysetnosy: + serhiy.storchaka
messages: + msg251741
2015-09-28 00:41:20tim.peterssetmessages: + msg251737
2015-09-28 00:17:01belopolskysetmessages: + msg251735
components: + Extension Modules, - Library (Lib)
2015-09-28 00:10:32tim.peterssetmessages: + msg251734
2015-09-27 20:53:21belopolskysetfiles: + issue23600.diff
versions: + Python 3.4, Python 3.5
messages: + msg251725

keywords: + patch
stage: needs patch -> commit review
2015-09-27 19:01:21belopolskysetmessages: + msg251717
2015-09-27 18:49:31belopolskysetassignee: belopolsky
stage: needs patch
2015-09-27 03:04:08tim.peterssetmessages: + msg251675
2015-09-26 23:52:58belopolskysetnosy: + tim.peters
messages: + msg251672
2015-09-26 23:46:16belopolskysetversions: + Python 3.6, - Python 3.4
2015-03-07 15:02:41alynnsetnosy: + alynn
2015-03-07 11:33:17peterjclawsetfiles: + time_issues.py

messages: + msg237440
2015-03-07 03:48:03belopolskysetmessages: + msg237414
2015-03-07 03:10:47ned.deilysetnosy: + belopolsky
2015-03-07 00:43:03peterjclawcreate