datetime needs an "epoch" method #46988
Comments
|
If you try to convert datetime objects to seconds since epoch and back >>> dt = datetime(2008, 5, 1, 13, 35, 41, 567777)
>>> seconds = mktime(dt.timetuple())
>>> datetime.fromtimestamp(seconds) == dt
False
Current fix is to do
>>> seconds += (dt.microsecond / 1000000.0)
>>> datetime.fromtimestamp(seconds) == dt
True |
tebeka
mannequin
added
type-bug
|
That's expected as mktime is just a thin wrapper over libc mktime() and |
|
Attached a patch which adds a .totimetuple(...) method to The intention is that the dt.totimetuple(...) method is equivalent to: |
|
Patch adding documentation for datetime.totimestamp(...). |
|
I think the name is not good, should be "toepoch" or something like that. |
|
datetime has fromtimestamp already, so using totimestamp keeps naming |
|
See also bpo-1673409 |
|
I like the method, but I have some comments about the new method:
I wrote a similar patch before reading |
|
Here is a merged patch of the three patches. Except the C |
|
I would like to voice my opposition the totimestamp method. Representing time as a float is a really bad idea (originated at In my opinion both time.time() returning float and I would not mind integer based to/from timestamp methods taking and |
|
Le Saturday 15 November 2008 02:15:30 Alexander Belopolsky, vous avez écrit :
It's too late to break the API (Python3 is in RC stage ;-)), but we can create |
|
On Fri, Nov 14, 2008 at 8:37 PM, STINNER Victor <report@bugs.python.org> wrote:
While 1970 is the most popular epoch, I've seen 1900, 2000 and even
I would much rather have divmod implemented as you suggested in def toepoch(d):
x, y = divmod(d, timedellta(0, 1))
return x, y.microseconds |
|
Le Saturday 15 November 2008 04:17:50 Alexander Belopolsky, vous avez écrit :
I tried yesterday and it doesn't work! datetime.datetime(1970, 1, 1, 1, 0)
>>> t1 = epoch + timedelta(seconds=-1660000000)
>>> t2 = datetime.fromtimestamp(-1660000000)
>>> t2
datetime.datetime(1917, 5, 26, 1, 53, 20)
>>> t1 - t2
datetime.timedelta(0)
>>> t2 = datetime.fromtimestamp(-1670000000)
>>> t2
datetime.datetime(1917, 1, 30, 7, 6, 40)
>>> t1 = epoch + timedelta(seconds=-1670000000)
>>> t1 - t2
datetime.timedelta(0, 3600)We lost an hour durint the 1st World War :-) Whereas my implementation using mktime() works: -1670000000.0 |
|
Any thoughts to time zone/DST handling for naive datetime objects? E.g. For aware datetime objects, I think the time.mktime(dt.timetuple()) |
|
----- "Alexander Belopolsky" <report@bugs.python.org> wrote:
The point for me is that having to interact with Microsoft systems that require times means that the conversions have to be done. Is it better to have everybody re-implement this, with their own bugs, or to have a standard implementation? I think it's clearly better to have it as a method on the object. Of course, we should put docs in describing the pitfalls of this approach... |
|
On Mon, Nov 24, 2008 at 9:04 AM, David Fraser <report@bugs.python.org> wrote:
I did not see the "epoch" proposal as an interoperability with
As far as I know, interoperability with Microsoft systems requires
Yes, having a well documented high resolution "time since epoch" to I believe the correct approach would be to extend fromtimestamp (and In addition exposing the system gettimeofday in the time module to |
|
About the timestamp, there are many formats: (a) UNIX: 32 bits signed integer, number of seconds since the 1st
(b) UNIX64: 64 bits signed integer, number of seconds since the 1st
(c) UNIX: 32 bits unsigned integer, number of seconds since the 1st
(d) UUID60: 60 bits unsigned integer, number of 1/10 microseconds
(e) Win64: 64 bits unsigned integer, number of 1/10 microseconds since
(f) MSDOS DateTime or TimeDate: bitfield with 16 bits for the date and
|
|
Timedelta formats: (a) Win64: 64 bits unsigned integer, number of 1/10 microsecond
(b) 64 bits float, number of seconds
Other file formats use multiple numbers to store a duration: [AVI video]
[WAV audio]
[Ogg Vorbis]
|
|
That's an impressive summary, but what is your conclusion? I don't On Mon, Nov 24, 2008 at 11:00 AM, STINNER Victor <report@bugs.python.org> wrote:
|
|
Ooops, timestamp (c) is the *Mac* timestamp: seconds since the 1st
Hum, it's maybe not possible to choose between integer and float. Why
Attached file (timestamp.py) is a module to import/export timestamp in >>> import timestamp
>>> from datetime import datetime
>>> now = datetime.now()
>>> now
datetime.datetime(2008, 11, 24, 18, 7, 50, 216762)
>>> timestamp.exportUnix(now)
1227550070
>>> timestamp.exportUnix(now, True)
1227550070.2167621
>>> timestamp.exportMac(now)
3310394870L
>>> timestamp.exportWin64(now)
128720236702167620L
>>> timestamp.exportUUID(now)
134468428702167620L
>>> timestamp.importMac(3310394870)
datetime.datetime(2008, 11, 24, 18, 7, 50)
>>> timestamp.importUnix(1227550070)
datetime.datetime(2008, 11, 24, 18, 7, 50)
>>> timestamp.importUnix(1227550070.2167621)
datetime.datetime(2008, 11, 24, 18, 7, 50, 216762)It supports int and float types for import and export. |
|
On Mon, Nov 24, 2008 at 12:13 PM, STINNER Victor <report@bugs.python.org> wrote:
I would still prefer totimestamp()->(int, int) returning (sec, usec) |
Right, I prefer your solution ;-) |
|
On Mon, Nov 24, 2008 at 12:34 PM, STINNER Victor <report@bugs.python.org> wrote:
Great! What do you think about extending fromtimestamp(timestamp[, Also, are you motivated enough to bring this up on python-dev to get a |
|
----- "STINNER Victor" <report@bugs.python.org> wrote:
There are also the PyWinTime objects returned by PythonWin COM calls which are basically FILETIMEs |
|
belopolsky will be happy to see this new version of my patch:
|
|
About mktime() -> -1: see the bpo-1726687 (I found the fix in this Next job will be to patch datetime.(utc)fromtimestamp() to support |
|
Victor, As you explain in your own documentation, the proposed method is equivalent to On the patch itself:
|
AlexanderBelopolsky
mannequin
changed the title
|
On Fri, Dec 17, 2010 at 12:17 PM, Antoine Pitrou <report@bugs.python.org> wrote:
A better question is why datetime.utcfromtimestamp(s) exists given
It does, but it does not help much those who want a float - they would
Well, no one has come up with the code that does work so far. Note
Because in certain timezones fromtimestamp() can return the same |
??? EPOCH is not even a constant in the datetime module. And regardless, the point is *not* the number of characters typed, but
Yes, but a very obvious one at least.
Do you think many users even think of calling divmod() timedelta You apparently hold the opinion that the datetime module should be |
|
On Fri, Dec 17, 2010 at 1:17 PM, Antoine Pitrou <report@bugs.python.org> wrote:
No, and it does not belong there. A higher level library that uses
I don't see anything obvious about the choice between datetime(1970, 1, 1) + timedelta(seconds=s) is obvious, self-contained, short and does not require any knowledge
Let's see: def floattimestamp(t):
s, us = t.totimestamp()
return s + us * 1e-6and def floattimestamp(t):
s, us = t.totimestamp()
return s + us / 1000000which one is *obviously* correct? Are they *obviously* equivalent? Note that when timedelta.total_seconds() was first committed, it
Sure, but if the goal is to implement json serialization of datetime In any case, you ignore the hard question about totimestamp(): |
And so what was your point exactly?
I don't think the "time" module can be named "higher level", and it
Sigh.
I don't see anything suggesting it is a monstrosity. The name is
Both are obviously correct for whatever the non-perverted user has in
So? What is your point?
IMO, the solution would have the datetime object carry the offset from |
|
On Fri, Dec 17, 2010 at 2:35 PM, Antoine Pitrou <report@bugs.python.org> wrote:
Remember, most of the code is written once, but read and edited many
Yes, UTC not being a proper acronym in any human language is one That's correct, but most users expect their timestamps to be the same
I thought the point was obvious: conversion between time values and
Ditto. This is exactly what bpo-9527 is attempting to achieve. |
Ok. Too bad you don't live on the same planet than most of us. I bail |
|
On Fri, Dec 17, 2010 at 3:26 PM, Antoine Pitrou <report@bugs.python.org> wrote:
Sorry that my attempt at humor has proven to be too subtle. I was """ |
|
It looks like it's not possible to choose between float and (int, int) output type for datetime.totimestamp(). One is more practical (and enough for people who doesn't need an exact result), and one is needed to keep the same resolution than the datetime object. I think that we can add two methods:
I choosed the shortest name for float because I suppose that most users prefer float than a tuple, and so the API is symmetrical:
My patch have to be updated to use the timezone (and the DST thing?) and also to update the Python implementation. |
|
I am extremely disappointed by what has happened here. We are talking about a very simple method that everybody needs, and that has been reimplemented over and over again. I have been frustrated countless times by the lack of a utctotimestamp() method. I have watched beginners and experienced programmers alike suffer over and over again for the lack of this method, and spend hours trying to figure out why Python doesn't have it and how it should be spelled in Python. The discussion here has been stuck on assumptions that the method must meet all of the following ideals:
These ideals cannot all be met simultaneously and perfectly. The correct thing to do as an engineer is to choose a practical compromise and document the decision. The compromise that almost everyone chooses (because it is useful, convenient, has microsecond precision at least until the year 2100, and millisecond precision is frequently sufficient) is to use a floating-point number with an epoch of 1970-01-01. Floating-point seconds can be easily subtracted, added, serialized, and deserialized, and are a primitive data type in nearly every language and database. They are unmatched in ease of use. So everyone wastes time searching for the answer and figuring out how to write: import calendar
calendar.timegm(dt.utctimetuple()) + dt.microsecond * 1e-6We should use this as the definition of datetime.utctotimestamp(), document its limitations, and be done with it. Instead, this essential and useful method has now been held up for almost three YEARS by an inability to accept a simple engineering decision. Unbelievable. |
|
On Thu, Mar 31, 2011 at 2:52 PM, Ka-Ping Yee <report@bugs.python.org> wrote:
What exactly are you disappointed about? As far as I can tell, the
This is not what this issue has been about so far. It was about local EPOCH = datetime(1970, 1, 1)
def utctotimestamp(dt) :
return (dt - EPOCH).total_seconds()
These "beginners and experienced programmers" may want to reconsider
No it was actually stuck because of the inability to reliably obtain
Those who need to do arithmetics on time values more often deal with
Compared to what? I find integers much more suitable for representing
And this is the wrong answer. Someone else using (dt - I think the best we can do is to expand datetime.utcfromtimestamp() def utcfromtimestamp(s):
return EPOCH + timedelta(seconds=s)and either leave it as an exercise to the reader to solve def utctotimestamp(dt) :
return (dt - EPOCH) / timedelta(seconds=1) |
Plenty have proposed a satisfactory solution. No one has come up with a solution that is satisfactory to *you*, because you have overconstrained the problem. The reason we still have no utctotimestamp() after all these years is that you, and you alone as far as I know, refuse to accept a method that inverts utcfromtimestamp() with microsecond precision over its working range. Such a method is a perfectly reasonable and acceptable solution and would add a lot of value to Python as a language. I suspect you don't realize just how much pain you have unintentionally caused the world of Python users by singlehandedly blocking progress on this issue. I've seen them: students, friends, coworkers -- even very smart and capable people are stymied by it. No one thinks of looking in the calendar module. Maybe if you watched some of them struggle with this, you would understand.
To take this perspective is to miss the point of Python. |
|
I couldn't agree more with ping's position on this. It is against the spirit of what Python has set out to be, and the blocking needs to stop. Any chance we could get a .epoch() function into python 2.7 as well? |
|
On Mon, Apr 4, 2011 at 5:42 PM, Jay Taylor <report@bugs.python.org> wrote:
Adding votes to a tracker issue without a working patch will not move
I don't think any committer has a power to *block* a patch. I
No. |
|
Just to add another data point to this discussion: mxDateTime, which in large parts inspired the Python datetime module, The methods naturally return the Unix ticks value as float, Victor's patch could easily be updated to return floats as well, There's only one catch that Victor's patch doesn't include: mktime() |
|
Marc, could you maybe write a new patching taking care of the DST and |
|
STINNER Victor wrote:
Sorry, but no. I'm not really a fan of the datetime module and Note that dealing with DST in timezones other than the local time When dealing with date/time values, it is usually best to stay with Consequently, I'd suggest to only allow UTC and local timezone |
|
On Tue, Apr 5, 2011 at 4:33 AM, Marc-Andre Lemburg
Yes, mxDateTime's gmticks()/ticks() pair of functions present a much Google code search, http://www.google.com/codesearch?hl=en&sa=N&q=gmticks+lang:python returns only 13 hits for "gmticks". In several instances, the I would not use Google Code search as an ultimate arbiter on the
Well, search for gmticks does not return too many hits outside of """ Follow the link for some more colorful language describing developer's Note that it is likely that the bug MySQLdb developer complained about
Which in turn is a mistake IMO. Note that POSIX does not use float
This is not the goal that I would support. I would rather see code
Victor reported implementing two methods, one to return a float and
No, it is not about "only one catch". Victor's patch is simply wrong. |
|
MAL> Since most of the datetime module was inspired by mxDateTime, """ http://bytes.com/topic/python/answers/522572-datetime-timestamp I will also add that fromtimestamp() is not invertible in the presence of DST. That's why mxDatetime.ticks() takes a DST flag making it effectively a multi-valued function. Note that naive users, who just want to pass datetime values to an OS function expecting a float, most likely will not have means of properly obtaining DST flag. |
|
Alexander Belopolsky wrote:
mxDateTime needs those two methods, since it doesn't natively The .gmticks() method is always used when storing UTC values Note that implementing .gmticks() is fairly easy on POSIX
Note that he was referring to the .ticks() method, not the .gmticks()
The time module is our reference in this case and this tries hard Note that sub-second accuracy relies on a number of factors, On many systems, you only get 1/100s accuracy, on others, the timer OTOH, there's a new set of APIs for nano-second accuracy available BTW: The integer format was chose in order to keep the memory
No chance :-) In practice, the time module gets used a lot
I had a look at the last patch on this ticket.
True, so make that two issues ;-) |
|
Alexander Belopolsky wrote:
IMHO, the whole concept of DST is broken, but that's not our fault :-) Ditching the concept just because it is known to fail for Otherwise, we'd have to ditch the date support in the datetime module And I'm not even starting to talk about ditching the concept In reality, all these things hardly ever matter and if they do, BTW: A "timestamp" usually refers to the combination of date and Date/time is fun, isn't it ? |
|
Let me state my position on this issue once again. Converting datetime values to float is easy. If your dt is a naive instance representing UTC time: timestamp = (dt - datetime(1970, 1, 1)) / timedelta(seconds=1)If your dt is an aware instance: timestamp = (dt - datetime(1970, 1, 1, tzinfo=timezone.utc)) / timedelta(seconds=1)These recipes are easy to adjust for your application needs. One application may want millisecond or microsecond ticks, another might want to carry subsecond presision in a separate integer, third may want to avoid timestamps before 1970 or after 2038 or ignore microseconds altogether. No matter what a hypothetical datetime.epoch() will provide, most of applications will need to add a line or two to its code to serve their needs. Applications that will use dt.epoch() naively without thinking what dt represents (say local or UTC) will be buggy. The only related feature that I think is missing from datetime module is the ability to obtain local time as an aware datetime instance and to convert a naive datetime instance assumed to represent local time to an aware one. This is the subject of bpo-9527, but there is a resistance to adding that feature. |
|
On Tue, Apr 5, 2011 at 1:45 PM, Marc-Andre Lemburg
In Unix context, the term "timestamp" is usually associated with the See, for example: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/touch.html |
|
On 04/05/2011 18:22, Alexander Belopolsky wrote:
Refusing to cooperate with the rest of the world is not sane by my books. On 04/05/2011 21:06, Alexander Belopolsky wrote:
Please add these lines to the datetime module's documentation. In some OTOH a sinceepoch(epoch=datetime(1970,1,1)) method of the datetime class |
|
On Thu, Apr 7, 2011 at 6:20 AM, Velko Ivanov <report@bugs.python.org> wrote:
This is precisely what I suggested at the end of msg132697 above. See |
|
New changeset b55eac85e39c by Alexander Belopolsky in branch 'default': |
abalkin
added
docs
|
New changeset 6671c5039e15 by Alexander Belopolsky in branch 'default': |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: