classification
Title: datetime.strftime('%s') should respect tzinfo
Type: enhancement Stage: needs patch
Components: Extension Modules, Library (Lib) Versions: Python 3.3
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Daniel.O'Connor, belopolsky, bignose, mumino, r.david.murray, santa4nt
Priority: normal Keywords: easy, patch

Created on 2011-08-15 02:31 by Daniel.O'Connor, last changed 2012-09-20 12:23 by mumino.

Files
File name Uploaded Description Edit
strftime.patch mumino, 2012-09-20 12:23 patch for strftime("%s") review
Messages (11)
msg142095 - (view) Author: Daniel O'Connor (Daniel.O'Connor) Date: 2011-08-15 02:31
It isn't possible to add a timezone to a naive datetime object which means that if you are getting them from some place you can't directly control there is no way to set the TZ.

eg pywws' DataStore returns naive datetime's which are in UTC. There is no way to set this and hence strftime seems to think they are in local time.

I can sort of see why you would disallow changing a TZ once set but it doesn't make sense to prevent this for naive DTs.

Also, utcnow() returns a naive DT whereas it would seem to be more sensible to return it with a UTC TZ.
msg142129 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-08-15 14:09
In what way does 'replace' not satisfy your need to set the tzinfo?

As for utcnow, we can't change what it returns for backward compatibility reasons, but you can get a non-naive utc datatime by doing datetime.now(timezone.utc).  (I must admit, however, that at least this morning I can't wrap my head around how that works based on the docs :(.
msg142130 - (view) Author: Daniel O'Connor (Daniel.O'Connor) Date: 2011-08-15 14:23
On 15/08/2011, at 23:39, R. David Murray wrote:
> R. David Murray <rdmurray@bitdance.com> added the comment:
> 
> In what way does 'replace' not satisfy your need to set the tzinfo?

Ahh that would work, although it is pretty clumsy since you have to specify everything else as well.

In the end I used calendar.timegm (which I only found out about after this).

> As for utcnow, we can't change what it returns for backward compatibility reasons, but you can get a non-naive utc datatime by doing ´

That is a pity :(

> datetime.now(timezone.utc).  (I must admit, however, that at least this morning I can't wrap my head around how that works based on the docs :(.

OK.. I am only using 2.7 so I can't try that :)

> 
> ----------
> nosy: +r.david.murray
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue12750>
> _______________________________________
>
msg142131 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-08-15 15:36
Ah.  Well, pre-3.2 datetime itself did not generate *any* non-naive datetimes.

Nor do you need to specify everything for replace.  dt.replace(tzinfo=tz) should work just fine.
msg142150 - (view) Author: Daniel O'Connor (Daniel.O'Connor) Date: 2011-08-15 22:57
On 16/08/2011, at 1:06, R. David Murray wrote:
> R. David Murray <rdmurray@bitdance.com> added the comment:
> 
> Ah.  Well, pre-3.2 datetime itself did not generate *any* non-naive datetimes.
> 
> Nor do you need to specify everything for replace.  dt.replace(tzinfo=tz) should work just fine.

OK.

I did try this and it seems broken though..
In [19]: now = datetime.datetime.utcnow()

In [21]: now.replace(tzinfo = pytz.utc)
Out[21]: datetime.datetime(2011, 8, 15, 22, 54, 13, 173110, tzinfo=<UTC>)

In [22]: datetime.datetime.strftime(now, "%s")
Out[22]: '1313414653'

In [23]: now
Out[23]: datetime.datetime(2011, 8, 15, 22, 54, 13, 173110)

[ur 8:22] ~ >date -ujr 1313414653
Mon 15 Aug 2011 13:24:13 UTC

i.e. it appears that replace() applies the TZ offset to a naive datetime object effectively assuming it is local time rather than un-timezoned (which is what the docs imply to me)

> ----------
> resolution:  -> invalid
> stage:  -> committed/rejected
> status: open -> closed
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue12750>
> _______________________________________
>
msg142190 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2011-08-16 12:30
OK. At a minimum there is a doc issue here, so I'm reopening.
msg142244 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-08-17 01:00
> i.e. it appears that replace() applies the TZ offset to a naive datetime
> object effectively assuming it is local time rather than un-timezoned
> (which is what the docs imply to me)

I don't understand your issue.  The replace method does not assume anything, it just replaces whatever fields you specify with new values.  You can replace tzinfo just like any other field, year, month, day, etc while preserving the other fields.  I think this is fairly well documented. I think what you are looking for is the astimezone() method which, however may not work well on naive datetime instances simply because a naive instance may be ambiguous in presence of DST.  However, if you start with an aware UTC datetime, you should be able to use astimezone() to convert to any local TZ.
msg142245 - (view) Author: Daniel O'Connor (Daniel.O'Connor) Date: 2011-08-17 01:55
On 17/08/2011, at 10:30, Alexander Belopolsky wrote:
> Alexander Belopolsky <alexander.belopolsky@gmail.com> added the comment:
> 
>> i.e. it appears that replace() applies the TZ offset to a naive datetime
>> object effectively assuming it is local time rather than un-timezoned
>> (which is what the docs imply to me)
> 
> I don't understand your issue.  The replace method does not assume anything, it just replaces whatever fields you specify with new values.  You can replace tzinfo just like any other field, year, month, day, etc while preserving the other fields.  I think this is fairly well documented. I think what you are looking for is the astimezone() method which, however may not work well on naive datetime instances simply because a naive instance may be ambiguous in presence of DST.  However, if you start with an aware UTC datetime, you should be able to use astimezone() to convert to any local TZ.

Hmm I see, it would appear the problem lies with strftime().

[ur 10:34] ~ >ipython-2.7 
Python 2.7.2 (default, Aug  6 2011, 23:46:16) 
Type "copyright", "credits" or "license" for more information.
IPython 0.10.2 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [48]: now = datetime.datetime.utcnow()
In [49]: nowtz = now.replace(tzinfo = pytz.utc)
In [50]: nowadl = now.replace(tzinfo = pytz.timezone('Australia/Adelaide'))
In [51]: now
Out[51]: datetime.datetime(2011, 8, 17, 1, 53, 51, 451118)
In [52]: nowtz
Out[52]: datetime.datetime(2011, 8, 17, 1, 53, 51, 451118, tzinfo=<UTC>)
In [53]: nowadl
Out[53]: datetime.datetime(2011, 8, 17, 1, 53, 51, 451118, tzinfo=<DstTzInfo 'Australia/Adelaide' CST+9:30:00 STD>)
In [54]: now.strftime("%F %r %s")
Out[54]: '2011-08-17 01:53:51 AM 1313511831'
In [55]: nowtz.strftime("%F %r %s")
Out[55]: '2011-08-17 01:53:51 AM 1313511831'
In [56]: nowadl.strftime("%F %r %s")
Out[56]: '2011-08-17 01:53:51 AM 1313511831'

Wed 17 Aug 2011 01:54:52 UTC
[ur 11:24] ~ >date +%s
1313546093
[ur 11:24] ~ >date -ujr `date +%s`
Wed 17 Aug 2011 01:54:59 UTC
[ur 11:24] ~ >date -ujr 1313511831
Tue 16 Aug 2011 16:23:51 UTC

i.e. strftime disregards tzinfo and seems to treat the time as LT (I think).

It certainly doesn't behave the way I'd expect after using strftime(3) et al :)
msg142249 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2011-08-17 03:12
> it would appear the problem lies with strftime()

Yes, strftime('%s') ignores tzinfo at the moment.  This is not a bug. Support for '%s' format code is incidental and not documented in Python.

Nevertheless I think this is a good feature request.  I am changing the title to make it more explicit.
msg142250 - (view) Author: Daniel O'Connor (Daniel.O'Connor) Date: 2011-08-17 03:16
On 17/08/2011, at 12:42, Alexander Belopolsky wrote:
> Alexander Belopolsky <alexander.belopolsky@gmail.com> added the comment:
> 
>> it would appear the problem lies with strftime()
> 
> Yes, strftime('%s') ignores tzinfo at the moment.  This is not a bug. Support for '%s' format code is incidental and not documented in Python.
> 
> Nevertheless I think this is a good feature request.  I am changing the title to make it more explicit.

OK thanks!
msg170808 - (view) Author: Mümin Öztürk (mumino) Date: 2012-09-20 12:23
I made a patch for datetime.strftime('%s'). it takes tzinfo into consideration.


>>> datetime.datetime(1970,1,1).strftime("%s")   
'-7200'

>>> datetime.datetime(1970,1,1, tzinfo=datetime.timezone.utc).strftime("%s")
'0'

datetime.date still behave as naive datetime.datetime
>>> datetime.date(1970,1,1).strftime("%s")
'-7200'
History
Date User Action Args
2012-09-20 12:23:51muminosetfiles: + strftime.patch

nosy: + mumino
messages: + msg170808

keywords: + patch
2011-08-17 07:02:29santa4ntsetnosy: + santa4nt
2011-08-17 03:16:59Daniel.O'Connorsetmessages: + msg142250
2011-08-17 03:12:25belopolskysetversions: - Python 2.7, Python 3.2
title: datetime.datetime how to correctly attach a timezone to an existing naive datetime -> datetime.strftime('%s') should respect tzinfo
messages: + msg142249

components: + Extension Modules
keywords: + easy
stage: resolved -> needs patch
2011-08-17 01:55:54Daniel.O'Connorsetmessages: + msg142245
2011-08-17 01:00:55belopolskysetmessages: + msg142244
2011-08-16 12:30:50r.david.murraysetstatus: closed -> open


title: datetime.datetime timezone problems -> datetime.datetime how to correctly attach a timezone to an existing naive datetime
nosy: + belopolsky
versions: + Python 3.2, Python 3.3
messages: + msg142190
resolution: not a bug ->
2011-08-16 06:39:46bignosesetnosy: + bignose
2011-08-15 22:57:03Daniel.O'Connorsetmessages: + msg142150
2011-08-15 15:36:17r.david.murraysetstatus: open -> closed
resolution: not a bug
messages: + msg142131

stage: resolved
2011-08-15 14:23:44Daniel.O'Connorsetmessages: + msg142130
2011-08-15 14:09:45r.david.murraysetnosy: + r.david.murray
messages: + msg142129
2011-08-15 02:31:12Daniel.O'Connorcreate