classification
Title: datetime.datetime should have a timestamp() method
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: duplicate
Dependencies: Superseder: datetime needs an "epoch" method
View: 2736
Assigned To: belopolsky Nosy List: Alexandre.Zani, Jay.Taylor, Neil Muller, amaury.forgeotdarc, andersjm, barry, belopolsky, catlee, davidfraser, djc, erik.stephens, guettli, hodgestar, jribbens, lemburg, mark.dickinson, ping, pitrou, python-dev, r.david.murray, steve.roberts, tim.peters, tomster, vivanov, vstinner, werneck
Priority: normal Keywords: patch

Created on 2012-05-25 08:53 by djc, last changed 2012-06-08 16:38 by belopolsky. This issue is now closed.

Files
File name Uploaded Description Edit
issue14908.diff belopolsky, 2012-06-08 15:36 review
Messages (16)
msg161558 - (view) Author: Dirkjan Ochtman (djc) * (Python committer) Date: 2012-05-25 08:53
There's datetime.fromtimestamp() and datetime.timetuple(), but no datetime.timestamp(). It should be possible to round-trip a UNIX timestamp through a datetime.datetime.
msg161568 - (view) Author: Alexander Belopolsky (Alexander.Belopolsky) Date: 2012-05-25 11:39
See issue2736.
msg162263 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2012-06-04 15:15
I completely agree.  As easy (but obscure) as it is, it seems quite silly to have to go through time.mktime(dt.timetuple())
msg162267 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2012-06-04 15:45
Barry,

Did you review the long discussion under issue 2736?  Specifically, please  note the part about mxDateTime, ticks() and gmticks().  I am -0 on adding ticks() and gmticks() and as far as I can tell no one has implemented these methods for datetime without introducing some subtle bugs.

I also suspect that those who ask for a timestamp() method expect that it will do more than

EPOCH = datetime(1970, 1, 1)
def timestamp(t):
      return (t - EPOCH).total_seconds()

For example, account for leap seconds or magically guess the DST offset.  If we introduce a method that implements the code above, we have to prepare to explain why it does not do those things.

If instead we introduce a shortcut for time.mktime(t.timetuple()), we will have to deal with the fact that mktime() behaves differently on different systems.
msg162268 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2012-06-04 15:57
On Jun 04, 2012, at 03:45 PM, Alexander Belopolsky wrote:

>Did you review the long discussion under issue 2736?  Specifically, please
>note the part about mxDateTime, ticks() and gmticks().  I am -0 on adding
>ticks() and gmticks() and as far as I can tell no one has implemented these
>methods for datetime without introducing some subtle bugs.

I skimmed it, but I don't have time to read the whole bug.  I'll note that
I've recently had to convert a bunch of code from using mxDateTime to
datetime+time, for the only reason that we had to support Python 3 and mxDT
isn't yet ported.

>I also suspect that those who ask for a timestamp() method expect that it
>will do more than
>
>EPOCH = datetime(1970, 1, 1)
>def timestamp(t):
>      return (t - EPOCH).total_seconds()

Not really.

>For example, account for leap seconds or magically guess the DST offset.  If
>we introduce a method that implements the code above, we have to prepare to
>explain why it does not do those things.

That's fine, explain away!

>If instead we introduce a shortcut for time.mktime(t.timetuple()), we will
>have to deal with the fact that mktime() behaves differently on different
>systems.

Not really.  If developers even care, they will know that the epoch is
different for different systems.  We don't have to deal with it, we just have
to mention it.  Let them use more verbose "correct" code if they care.

Seriously, we should not over-engineer this.  It's purely a more discoverable
convenience and I think implementing the moral equivalent to
time.mktime(t.timetuple()) is exactly the right thing to do.
msg162269 - (view) Author: Alexander Belopolsky (Alexander.Belopolsky) Date: 2012-06-04 16:03
On Mon, Jun 4, 2012 at 11:57 AM, Barry A. Warsaw <report@bugs.python.org> wrote:
> I've recently had to convert a bunch of code from using mxDateTime to
> datetime+time,

That's a valuable experience.  How big of a deal was the lack of
.ticks() and .gmticks()?  How did you work around it?
msg162270 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2012-06-04 16:12
On Jun 04, 2012, at 04:03 PM, Alexander Belopolsky wrote:

>That's a valuable experience.  How big of a deal was the lack of
>.ticks() and .gmticks()?  How did you work around it?

It was *much* less of an issue than all the magic date format parsing that
mxDT supports.  That's actually something that I think is a more serious
deficiency, since .strptime() is pretty limited (e.g. how would you parse ISO
8601 dates both with and without the 'T'?).  A Python 3 compatible time format
parser would make for a very nice separate library (PyPI) and/or addition to
the stdlib (eventually).

FWIW, here's the changeset.

http://bazaar.launchpad.net/~gwibber-committers/gwibber/trunk/revision/1354
msg162278 - (view) Author: Alexandre Zani (Alexandre.Zani) Date: 2012-06-04 17:33
I think the easiest and most intuitive approach is to simply define timestamp() as being the reverse of fromtimestamp(). Don't worry about leap seconds and all that stuff.

If non-1970 epochs are a concern, this could be renamed to posix_timestamp or some such with perhaps a generic timestamp function that takes both a time and epoch.

But let's not let such a useful function not happen just because it won't solve everyone's problem.
msg162280 - (view) Author: Alexander Belopolsky (Alexander.Belopolsky) Date: 2012-06-04 17:45
On Mon, Jun 4, 2012 at 1:33 PM, Alexandre Zani <report@bugs.python.org> wrote:
> I think the easiest and most intuitive approach is to simply define timestamp()
> as being the reverse of fromtimestamp().

I would like to invite everyone to review the discussion leading to
closing of issue 2736.   We cannot implement "the reverse of
fromtimestamp()" because fromtimestamp() is not reversible in presence
of DST and because float cannot represent all values that datetime
supports.  Both issues can be resolved, but I did not see any solution
that I would call intuitive.

Is anyone motivated enough to port mxDT's ticks() method to datetime?
I don't think we need to reinvent the wheel.
msg162282 - (view) Author: Alexandre Zani (Alexandre.Zani) Date: 2012-06-04 19:21
I'm still reading through the issue you mentioned. (It's a painful read I have to admit) One major obstacle seems to be that during the DST switch over, an hour gets repeated and so the datetime object is ambiguous. (are you on the first or second hour?) I would argue that it isn't a problem that this function needs to solve. The ambiguity isn't tied to the conversion. It's tied to the datetime object itself.

Let's add an optional parameter to specify the DST status, doc the pitfall and not worry overmuch about it.

Side note: Let me know if I misunderstood Alexander, but if I didn't this should be documented with the datetime object. Based upon my understanding, the datetime object is a bad choice if you care about that ambiguity. That's not really clear.
msg162288 - (view) Author: Alexander Belopolsky (Alexander.Belopolsky) Date: 2012-06-04 20:20
On Mon, Jun 4, 2012 at 3:21 PM, Alexandre Zani <report@bugs.python.org> wrote:
> Let me know if I misunderstood Alexander, but if I didn't this should be documented with
> the datetime object. Based upon my understanding, the datetime object is a bad choice
> if you care about that ambiguity. That's not really clear.

The datetime object is neither good nor bad if you want to deal with
the DST ambiguity.  If you want to store times as local time, the best
practice is to include the UTC offset.  The datetime module now
provides the facilities to do so.  If you neglect the timezone and
deal with naive datetime objects instead, you should probably avoid
mixing it with POSIX timestamps in the same application.

Here is an example where naive datetime is quite appropriate: a
timekeeping application for a small business with a single location.
If you need to generate opening hours for a given week,
[datetime(y,m,d, 9, 0) + datetime(i) for i in range(5)] is an adequate
solution, but if you compare it to [datetime.fromtimestamp(x +
24*3600*i) for i in range(5)], you may be off by one hour if your week
spans the DST change.

In most applications, however, keeping local time without UTC offset
is a bad choice.  POSIX's timestamp only advantage is that it does no
allow keeping local time at all.
msg162316 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2012-06-05 01:28
Barry,

I looked at your changeset at http://bazaar.launchpad.net/~gwibber-committers/gwibber/trunk/revision/1354 and at the specification at http://developers.facebook.com/docs/reference/api/event/.

Am I correct that you are parsing "string containing an ISO-8601 formatted date/time or a UNIX timestamp; if it contains a time zone (not recommended), it will be converted to Pacific time before being stored and displayed"?

If so, it looks like converting input to POSIX timestamps is the wrong thing to do because users who specify naive time expect it to be displayed without conversion.

The way I read the specs, a better implementation would store input in naive datetime objects, converting UNIX timestamps or TZ-aware strings to Pacific timezone.
msg162452 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2012-06-07 03:29
I think the attached patch implements what was agreed to on python-dev.  We need more tests, particularly around DST change.  I tested the patch on OSX, but I am interested to hear feedback from users of other OSes.
msg162483 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2012-06-07 17:29
Updated patch adds a few more tests and improves error handling in C implementation.
msg162531 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2012-06-08 15:36
Added ReST documentation.  Will commit soon.  Merging nosy list from issue 2736.
msg162534 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2012-06-08 16:38
I associated my commit with issue 2736 by mistake, but it is probably a good thing because that issue contains a longer history.

Closing this as a duplicate.
History
Date User Action Args
2012-06-08 16:38:21belopolskysetstatus: open -> closed
superseder: datetime needs an "epoch" method
messages: + msg162534

resolution: duplicate
stage: commit review -> resolved
2012-06-08 15:37:16belopolskysetfiles: - issue14908.diff
2012-06-08 15:37:01belopolskysetfiles: + issue14908.diff

nosy: + tim.peters, ping, jribbens, guettli, amaury.forgeotdarc, mark.dickinson, davidfraser, pitrou, andersjm, catlee, tomster, werneck, hodgestar, Neil Muller, erik.stephens, steve.roberts, vivanov, python-dev, Jay.Taylor
messages: + msg162531

stage: patch review -> commit review
2012-06-07 17:29:51belopolskysetfiles: - issue14908.diff
2012-06-07 17:29:38belopolskysetfiles: + issue14908.diff
nosy: + vstinner
messages: + msg162483

2012-06-07 03:29:04belopolskysetfiles: + issue14908.diff
keywords: + patch
messages: + msg162452

stage: needs patch -> patch review
2012-06-05 19:34:03belopolskysetassignee: belopolsky

nosy: - Alexander.Belopolsky
stage: needs patch
2012-06-05 01:28:12belopolskysetmessages: + msg162316
2012-06-04 20:20:45Alexander.Belopolskysetmessages: + msg162288
2012-06-04 19:21:23Alexandre.Zanisetmessages: + msg162282
2012-06-04 17:45:51Alexander.Belopolskysetmessages: + msg162280
2012-06-04 17:33:08Alexandre.Zanisetnosy: + Alexandre.Zani
messages: + msg162278
2012-06-04 16:24:22r.david.murraysetnosy: + r.david.murray
2012-06-04 16:12:43barrysetmessages: + msg162270
2012-06-04 16:03:33Alexander.Belopolskysetmessages: + msg162269
2012-06-04 15:57:41barrysetmessages: + msg162268
2012-06-04 15:45:39belopolskysetmessages: + msg162267
2012-06-04 15:15:04barrysetmessages: + msg162263
2012-06-04 15:11:32barrysetnosy: + barry
2012-05-25 11:39:50Alexander.Belopolskysetnosy: + Alexander.Belopolsky
messages: + msg161568
2012-05-25 10:02:48pitrousetnosy: + lemburg, belopolsky

type: enhancement
components: + Library (Lib)
versions: + Python 3.3, - Python 2.7, Python 3.2
2012-05-25 08:53:19djccreate