The datetime module is missing some important methods for interacting with timestamps (i.e. seconds since 1970-01-01T00:00:00).

There are methods to convert from a timestamp, i.e. date.fromtimestamp and datetime.fromtimestamp, but there are no methods to convert back.

In addition, timedelta has no method for returning the number of seconds it represents (i.e. days*86400+seconds+microseconds/1000000).

Yes, that's true.

There is no datetime.totimestamp because the range
of time represented by a datetime object far
exceeds the range of a normal int-based Unix
timestamp (roughly 1970-2038).  Datetime objects
before the start of the Unix epoch would be
represented by negative numbers.  As far as I
know, the common Unix library functions which
accept epoch times wouldn't know what to do
with a negative number.

That said, you stated these missing methods
were important.  Can you offer some use
cases which would support that contention?
I personally don't think a argument for
symmetry would be a convincing use case and
that's the only one I can think of.

Almost everything you just said about time_t is wrong. time_t is signed,
and always has been (otherwise the 'end of time' for 32-bit time_t would
be 2106, not 2038). Also, time_t does not end at 2038 because nothing
says it must be 32 bits. Also, Python has 'long integers' which do not
overflow.

Also, I don't understand what you mean about use cases. The "use case"
is "dealing with anything which expects standard Unix time_t, for
example the Python standard library". The use case I have personally is
the program I was working on when I encountered the problem described in
this bug report. Also I think symmetry is a darn good argument. Why does
fromtimestamp exist if, as you claim, nobody uses time_t?

Jon> Almost everything you just said about time_t is wrong. time_t is
    Jon> signed, and always has been (otherwise the 'end of time' for 32-bit
    Jon> time_t would be 2106, not 2038). Also, time_t does not end at 2038
    Jon> because nothing says it must be 32 bits. Also, Python has 'long
    Jon> integers' which do not overflow.

My apologies about goofing up on the signedness of time_t.  What are you
going to do with a long integer that you can't do with a datetime object?
You clearly can't pass it directly to any Unix library functions which
expect time_t.  Converting it can overflow.

    Jon> Also, I don't understand what you mean about use cases. The "use
    Jon> case" is "dealing with anything which expects standard Unix time_t,
    Jon> for example the Python standard library". The use case I have
    Jon> personally is the program I was working on when I encountered the
    Jon> problem described in this bug report. Also I think symmetry is a
    Jon> darn good argument. Why does fromtimestamp exist if, as you claim,
    Jon> nobody uses time_t?

What should datetime.datetime(9999, 1, 1).totimestamp() return?  How would
you pass it to something which accepts a time_t?  The fromtimestamp
functions work simply because the range of time_t is a proper subset of the
range of Python's datetime objects.  Symmetry gets you little.  In
situations where you need Unix timestamps and you know your datetime objects
are within the bounds representable by time_t, you can define a convenience
function:

    def totimestamp(dt):
        return time.mktime(dt.timetuple()) + dt.microsecond/1e6

This will, of course, fail if the year is too big or too small (and will
fail in platform-dependent ways if the underlying platform's range of
representable dates has different bounds than Unix does).  Doing it without
resorting to calling time.mktime is also nontrivial.  Under the covers the
datetime module currently uses platform functions to get time information
anyway.  It doesn't do a lot of low-level time arithmethic itself.
Implementing fromtimestamp would require a fair amount of effort unless you
were willing to punt and just raise OverflowError for dates outside the
system's range.

Skip

unless I'm missing something important this will do the trick quite 
nicely:

    >>> from datetime import datetime
    >>> datetime(2007, 12, 24, 20, 0).strftime("%s")
    '1198522800'

For most imaginable use cases, where an epoch style time stamp is 
required this should be enough.

HTH,

Tom

Tom> unless I'm missing something important this will do the trick quite
    Tom> nicely:

    >>>> from datetime import datetime
    >>>> datetime(2007, 12, 24, 20, 0).strftime("%s")
    Tom>     '1198522800'

    Tom> For most imaginable use cases, where an epoch style time stamp is
    Tom> required this should be enough.

No fractions of a second...

Skip

> No fractions of a second...

If we're expecting floating-point, then everything you said earlier
about the limitations of ints was a bit redundant ;-)

>> No fractions of a second...

    Jon> If we're expecting floating-point, then everything you said earlier
    Jon> about the limitations of ints was a bit redundant ;-)

Yes, sorry.  I responded to the mail without going back and reviewing the
entire thread.  (It's been a couple months.)  Note however that my example
to_timestamp() function converts the microseconds field to seconds and
include them in the result.

So, are we concluding that nothing needs to be added to the datetime module?

Skip

One other thing worth noting is that %s is not universally
available.  Solaris, for example, lacks it in its strftime()
implementation.  (It has a bazillion other non-standard
format strings but not %s.)

Skip

Well, I still think that a convert-to-time_t function is essential, and
I don't buy any of the counter-arguments so far. The only argument I can
see is "should it return float or integer?" - floats are inaccurate and
integers can't represent partial seconds.

This is not going to go anywhere without someone offering a patch to
implement this.  Plus I suspect this has a much greater use in the C API
than in the Python API for datetime.

Skip has already provided what amounts to a patch. It just needs to be
decided whether to (a) not include it, (b) include it with the floating
point part, or (c) include it without the floating point part.

I couldn't comment as to how many people need it. I can say that I need
it, and anyone else who's used to manipulating dates and times either
"in a unixy sort of way" or with libraries that are expecting time_t's
will need it.

I keep needing to know the number of seconds that a timedelta represents,
so I implemented the following patch.

This returns only the integer portion, but could be modified to return a
floating point value.

Chris> I keep needing to know the number of seconds that a timedelta
    Chris> represents, so I implemented the following patch.

I can sympathize, but if we accept this patch, for symmetry reasons
shouldn't we also add .todays, .tomicroseconds and maybe even .toweeks?

Skip

timedelta.todays() could be useful in general I suppose.  I think
timedelta.toweeks() could be omitted since it's simple division by an
easy to recognize constant...also the timedelta docs say that it stores
data in terms of microseconds, seconds, and days.

OTOH tohours(), tominutes(), etc. (all the units that the constructor
uses) could be useful in some cases.

My feeling is that adding a method for each time unit bloats the API too
much.  Personally I've only ever wanted to know the number of seconds a
timedelta represents. It seems like seconds are a good base unit to
support since it allows easier interoperability with other python types
and external data.

I'm not sure this is still being considered, so I just wanted to add my
opinion.  This seems like such a simple request being made overly
complicated.  We just want seconds since epoch since that is used in
many other functions and libraries (i.e. the time module,
os.stat().st_mtime).  Why is there toordinal()?!?  If there is going to
be any toxxx() methods, there should definitely be a
tosecs()/totimestamp() method or an 'epoch/timestamp' attribute.  For
boundary cases, I would look to the std time module for guidance.

I would like to add a use case, generating random dates. Especially
generating random dates/times according to different probability
distributions and within ranges. It would be nicest if operations could
be done directly with datetime, date, time and timedelta objects, but
easy conversion to and from timestamps as requested here would also make
for reasonably convenient code (i.e. get timestamp conversions from
objects to represent ranges, generate random numbers and convert back)

Unassigning.  Haven't heard from Tim in quite awhile and he's made no 
input on this issue.  Also bump to 2.7.

You can have all this by using the time module's functions.

It is true that these don't work for all dates, but they are still
useful to have.

FWIW: mxDateTime has always had methods to convert the date/time values
to ticks and back again:

    http://www.egenix.com/products/python/mxBase/mxDateTime/

Since most of the datetime module was inspired by mxDateTime, I wonder
why these were left out.

I wrote this function is my program:

def timedelta2seconds(delta):
    """
    Convert a datetime.timedelta() objet to a number of second
    (floatting point number).

    >>> timedelta2seconds(timedelta(seconds=2, microseconds=40000))
    2.04
    >>> timedelta2seconds(timedelta(minutes=1, milliseconds=250))
    60.25
    """
    return delta.microseconds / 1000000.0 \
        + delta.seconds + delta.days * 60*60*24

About the use cases: I use it to compute the compression rate of an 
audio song (bytes / seconds), to compute the bit rate of a video 
(bytes / seconds). I'm using datetime.timedelta() to store the 
audio/video duration.

It's not related to time_t: see issue #2736 for 
datetime.datetime.totimestamp(). And about time_t: I don't about 31 
bits signed integer. It's not beacuse other programs have arbitrary 
limits than Python should also be limited.

About the patch: I don't like the name "tosecs", it's not consistent 
with the constructor: timedelta(seconds=...).tosec[ond]s(). And why 
dropping the microseconds? For short duration, microseconds are 
useful.

New patch based on catlee's patch: add toseconds() method to timedelta 
but use the float type to keep the microseconds. The error between the 
exact value and IEEE754 value is small: the error is smaller than one 
second even with 999999999 days:

 - with less than 16 days, the microsecond error is smaller than 
0,01%.
 - with less than 1000 days, the microsecond error is smaller than 1%
 - with 99500 days or more, the microsecond error is bigger than 90%

To compute the error in microseconds:
>>> d=100; m=1
>>> e=abs((timedelta(days=d, microseconds=m).toseconds() - 
d*86400)*1e6 - m)
>>> e*100/m
0.024044513702392578

For the seconds, this is no error even with 999999999 days. Error in 
seconds:

>>> d=999999999; s=1; timedelta(days=d, seconds=m).toseconds() - 
d*86400, s
(1.0, 1)

(no error, both values are the same)

See also issue2736

Chris> I keep needing to know the number of seconds that a timedelta
Chris> represents.

I propose an alternative approach that I believe will neatly solve 
fractional vs. whole seconds and multitude of conceivable toxxx methods:

Let's extend timedelta's div and floordiv methods to allow 

>>> (t - epoch) // timedelta(seconds=1)
--> whole seconds

and  

>>> (t - epoch) / timedelta(seconds=1)
--> fractional seconds

>>> (t - epoch) // timedelta(seconds=1)

I don't like this syntax, because I can't guess the result unit:
  datetime - datetime -> timedelta
but:
  timedelta / timedelta -> seconds? days? nanoseconds?

If you example, you used timedelta(seconds=1), but what is the result 
unit if you use timedelta(hours=1)? or timedelta(days=1, 
microseconds=1)?

The problem is that timedelta has no unit (or has multiple units), 
whereas timedelta.toseconds() are seconds. So about your example:

>>> (t - epoch).toseconds()
--> fractional seconds
>>> int((t - epoch).toseconds())
--> whole seconds

> timedelta / timedelta -> seconds? days? nanoseconds?

The quotient of two timedelta is a dimensionless number with no unit:
    timedelta(hours=1) / timedelta(minutes=5) == 12.0
This seems well defined, where is the ambiguity?

I was going to say the same as Amaury: timedelta / timedelta is
dimensionless (time units cancel each other in division) and the
advantage of this notation is that you get a way to express
timedelta.toxxx for all units accepted in constructor and even toweeks
becomes simple d / timedelta(7).

I've started flashing out a patch and then recalled that I've seen one
at issue2706 .  So instead of attaching a new patch here, I am going
to review the one in issue2706 now.

There was also some related discussion at issue4291 .   Apparently it
has been suggested that timedelta to integer and float conversions
would be a better alternative to timedelta / timedelta division.   I
disagree.  Integer conversion is ambiguous - should it be to seconds,
to microseconds or to timedelta.resolution (whatever that will become
in the future)?  Floating point conversion may loose precision as Tim
pointed out in msg26266 .  That would lead users to scratching their
heads over what to use float(d1)/float(d2) or float(d1)/int(d2) or
even int(d1)/int(d2) with true division on.

This said, let's move this discussion to issue2706 now.

On Fri, Nov 14, 2008 at 7:17 AM, Amaury Forgeot d'Arc
<report@bugs.python.org> wrote:
>
> Amaury Forgeot d'Arc <amauryfa@gmail.com> added the comment:
>
>> timedelta / timedelta -> seconds? days? nanoseconds?
>
> The quotient of two timedelta is a dimensionless number with no unit:
>    timedelta(hours=1) / timedelta(minutes=5) == 12.0
> This seems well defined, where is the ambiguity?
>
> ----------
> nosy: +amaury.forgeotdarc
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue1673409>
> _______________________________________
>

A timedelta.toseconds method (or equivalent) makes no sense. The number
of seconds in a day is not fixed (due to leap seconds) and relying on
such a method would introduce subtle bugs. The only way to find out the
number of seconds in a range of dates is if you have a concrete date range.

> The number of seconds in a day is not fixed (due to leap seconds)

POSIX timestamp doesn't count leap seconds. It looks like the datetime 
module is not aware of the leap seconds:

>>> print datetime.datetime(2006, 1, 1) - datetime.datetime(2005, 12, 
31)
1 day, 0:00:00

About my method: I finally prefer datetime/datetime or 
datetime//datetime instead of a toseconds() method. And to convert a 
timestamp to a timestamp: see my patch attached to issue #2736.

I removed my ".toseconds() method" patch because I prefer division. 
See issue #2706 for divison, divmod, etc.

I also think totimestamp() on datetime objects would be useful, I've
missed it myself a couple of times. The return value should be similar
to that of time.time(), i.e. a float.

Leap second handling is usually configurable. Default on Debian Linux
(but similar on RHEL and SuSE):

>>> int(date(1994,1,1).strftime("%s")) - int(date(1993,1,1).strftime("%s"))
31536000

After doing "cp /usr/share/zoneinfo/right/Europe/Berlin /etc/localtime":

>>> int(date(1994,1,1).strftime("%s")) - int(date(1993,1,1).strftime("%s"))
31536001

Also, NTP servers usually get this right. I don't think, Python should
promote a wrong date handling by default, even if it's convenient.

> (...) totimestamp() (...) return value should be similar
> to that of time.time(), i.e. a float

float is a source of many problems (rounding problems), especially for 
huge values: float is unable to store correctly microseconds for big 
values: see msg75426. A simple tuple (int, int) is simple and there is 
no rounding/float limit.

What you say is "with 99500 days or more, the microsecond error is
bigger than 90%". It means that with epoch starting at 1970, you can
still return timestamps with a 1-2 microsecond accuracy for the year 2242.

Additional precision would be overkill.

> A timedelta.toseconds method (or equivalent) makes no sense.
> The number of seconds in a day is not fixed (due to leap seconds) and
> relying on such a method would introduce subtle bugs.

You are misunderstanding what timedelta is. It is a fixed-length period
of time. It is stored as a number of (24x3600-second) days, seconds and
microseconds. There is no "start date" or "end date", so the concept of
leap seconds just does not apply.

With issue2706 accepted, I think this issue can now be rejected because proposed td.tosecs() can now be spelled simply as td/timedelta(seconds=1).

The other RFE for a totimestamp() method is a duplicate of issue2736.

Closing this as a duplicate of issue 2736, as suggested.  I'll combine the nosy lists.

(BTW, as well as the newly introduced division methods, td.tosecs already exists, except that it's spelt td.total_seconds.)

On Wed, Apr 21, 2010 at 2:03 PM, Mark Dickinson <report@bugs.python.org> wrote:
..
> (BTW, as well as the newly introduced division methods, td.tosecs already exists, except that it's spelt td.total_seconds.)

Aggrrr!  How did that slip in? :-)

86399999913600.0

Shouldn't td.total_seconds() return Decimal?  Maybe in py4k ...

> Aggrrr!  How did that slip in? :-)

Blame Antoine.  :)  (See issue 5788 and revision 76529.)

> Shouldn't td.total_seconds() return Decimal?  Maybe in py4k ...

Yes, that would have been nice.

I'm not sure that the Decimal type is well-established enough yet that it's okay to return Decimal instances from random unrelated modules, though.  Maybe someday.