classification
Title: tzinfo objects with sub-minute offsets are not supported (e.g. UTC+05:53:28)
Type: enhancement Stage: test needed
Components: Extension Modules Versions: Python 3.2
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: belopolsky Nosy List: Fergus.Noble, belopolsky, benjamin.peterson, haypo, jamesh, mark.dickinson
Priority: low Keywords: easy

Created on 2009-02-17 04:33 by jamesh, last changed 2012-11-04 07:11 by Fergus.Noble.

Files
File name Uploaded Description Edit
issue5288-proto.diff belopolsky, 2010-07-02 23:26
issue5288.diff belopolsky, 2010-07-05 04:43 patch for the C code
old-datetimemodule.c.gcov belopolsky, 2010-07-05 18:43 Test coverage before change
new-datetimemodule.c.gcov belopolsky, 2010-07-05 18:48 Test coverage after change
issue5288a.diff belopolsky, 2010-07-07 23:38
Messages (13)
msg82299 - (view) Author: James Henstridge (jamesh) Date: 2009-02-17 04:33
The datetime module does not support time zones whose offset from UTC is
not an integer number of minutes.

The Olson time zone database (used by most UNIX systems and Mac OS X)
has a number of time zones with historic offsets that use second
resolution (from before those locations switched to a rounded offset
from GMT).

If you are working purely with the Python date time module, you can
round these offsets to whole numbers of minutes and get consistent
results (this is what the pytz module does), there are problems if you
want to interact with other programs using the Olson database.

As an example, PostgreSQL can return dates with sub-minute UTC offsets
and it would be nice to be able to represent them exactly.

The documentation doesn't explain why offsets need to be a whole number
of minutes, and the interface (returning timedeltas) doesn't look like
it'd need changing to support second offsets.  I wouldn't expect any
more complexity in the code to support them either.
msg106374 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-05-24 16:46
As far as I can tell, the TZ offset code can be simplified by eliminating conversion from timedelta to int and back in utcoffset() and dst() methods of time and datetime objects.

The only reason for the restriction that I can think of is that some text representation of datetime only provide 4 digits for timezone. The behavior of formatting functions, can be preserved while allowing fractional minute offsets elsewhere.

On the other hand, given that this issue did not see a single response in more than a year, it seems that there is little interest in fixing it.
msg108606 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-06-25 15:24
This proved to require a lot of changes to C implementation because most of timezone arithmetics is done using integer operations with offset in minutes. It is easy, however to do this in pure python implementation which can be found at http://svn.python.org/view/*checkout*/sandbox/branches/py3k-datetime/datetime.py .

Marking this "easy" to attract volunteers who may want to do Python prototype.
msg109151 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-07-02 23:26
I am attaching a patch against sandbox version of datetime.py.  With this patch, there is a single place where subminute offset is rejected, _check_utc_offset() function.  I have also added "whole minute" asserts in places where sub-minute part of the offset is discarded or assumed 0.
msg109262 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-07-04 20:30
I am attaching a rough patch which removes timedelta -> int minutes -> timedelta round trips from utcoffset handling code.  I think the result is an improvement, but needs more polishing.

Mark,

Do you think this is worth pursuing?  I am not intending to add support for sub-minute offsets yet, just pass offsets around as timedeltas internally without unnecessary conversions to int and back.
msg109347 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-07-05 18:42
I am attaching a test coverage file for the patched datetimemodule.c.
msg109348 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-07-05 18:48
It looks like I attached coverage for the original file.  See new-datetimemodule.c.gcov for coverage after the change.
msg109504 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-07-07 21:35
Uploaded issue5288.diff to Rietveld:

  http://codereview.appspot.com/1698050
msg109508 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-07-07 23:38
issue5288a.diff addressed comments from Rietveld.
msg109510 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2010-07-08 00:06
C code changes eliminating round-trips between timdelta and int offsets committed in r82642 and Python code changes committed to sandbox in r82641.

Now the requested behavior change is easy and I am about +0.5 on relaxing the offset checks to allow seconds in the offsets and about +0.1 on removing the checks altogether and allowing arbitrary timedeltas even over 1 day.

The only substantive question in my mind is how to handle printing in formats like %z which only provide 4 digits for UTC offset.  Two options seem reasonable: a) truncate sub-minute offsets; and b) follow RFC 3339 recommendation and translate the time into the nearest whole-minute timezone.

I am slightly is favor of (a), but this whole problem may be a reason to reject this RFE.
msg174494 - (view) Author: Fergus Noble (Fergus.Noble) Date: 2012-11-02 07:56
Digging up an old issue but I am also interested in seeing this enhancement. Specifically to represent GPS time which is (currently) 16 seconds ahead of UTC.
msg174742 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2012-11-04 02:41
Is it practical to implement GPS time as datetime given that we don't have support for leap seconds?
msg174760 - (view) Author: Fergus Noble (Fergus.Noble) Date: 2012-11-04 07:11
GPS time doesn't include leap seconds so I think datetime is a good representation. If datetime doesn't know about leap seconds then there would still be some issues with finding the timedelta between a GPS time and a UTC time straddling a leap second but I guess a similar issue also exists with two UTC times.

For my application all the times I am dealing with are in a short period and will have the same UTC offset so its a little easier, I can probably avoid most of these issues.

However, wouldn't it be possible to implement the general case with a non-constant utcoffset function (and new fromutc implementation) in the tzinfo class? Of course there is no way to properly handle UTC times more than 6 months or so in the future...
History
Date User Action Args
2012-11-04 07:11:28Fergus.Noblesetmessages: + msg174760
2012-11-04 02:41:24belopolskysetmessages: + msg174742
2012-11-02 07:56:13Fergus.Noblesetnosy: + Fergus.Noble
messages: + msg174494
2010-07-08 00:06:07belopolskysetpriority: normal -> low
keywords: + easy, - patch
messages: + msg109510

stage: commit review -> test needed
2010-07-07 23:38:25belopolskysetfiles: + issue5288a.diff

messages: + msg109508
stage: test needed -> commit review
2010-07-07 21:35:38belopolskysetnosy: + benjamin.peterson
messages: + msg109504
2010-07-05 18:48:54belopolskysetfiles: + new-datetimemodule.c.gcov

messages: + msg109348
2010-07-05 18:43:00belopolskysetfiles: + old-datetimemodule.c.gcov

messages: + msg109347
2010-07-05 04:43:20belopolskysetfiles: - issue5288.diff
2010-07-05 04:43:10belopolskysetfiles: + issue5288.diff
2010-07-05 04:00:56belopolskysetfiles: - issue5288.diff
2010-07-05 04:00:50belopolskysetfiles: + issue5288.diff
keywords: + patch
2010-07-04 20:30:18belopolskysetfiles: + issue5288.diff

nosy: + mark.dickinson
messages: + msg109262

keywords: - patch, easy
2010-07-02 23:26:09belopolskysetfiles: + issue5288-proto.diff
keywords: + patch
messages: + msg109151
2010-06-25 15:24:06belopolskysetkeywords: + easy
nosy: + haypo
messages: + msg108606

2010-05-24 16:46:34belopolskysetmessages: + msg106374
2010-05-21 16:25:13belopolskysetversions: + Python 3.2, - Python 3.1, Python 2.7
nosy: + belopolsky

assignee: belopolsky
type: enhancement
stage: test needed
2009-02-17 04:33:24jameshcreate