Title: No way to generate or parse timezone as produced by datetime.isoformat()
Type: enhancement Stage: commit review
Components: Extension Modules Versions: Python 3.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: belopolsky Nosy List: Anders.Hovmöller, Arfrever, Eric.Hanchrow, Roman.Evstifeev, SilentGhost, aymeric.augustin, barry, belopolsky, berker.peksag, cvrebert, davydov, deronnax, flying sheep, gvanrossum, haypo, jcea, jstasiak, jwilk, karlcow, kirpit, martin.panter, mcepl, merwok, mihaic, nagle, pbryan, perey, piotr.dobrogost, r.david.murray, roysmith, shanmbic, sirex, sonots, tim.peters, ztane
Priority: normal Keywords: easy, patch

Created on 2015-08-28 22:34 by gvanrossum, last changed 2017-07-27 19:39 by belopolsky.

File name Uploaded Description Edit
datetime.patch shanmbic, 2015-10-04 19:58 review
datetime2.patch shanmbic, 2015-10-05 22:44 review
Repositories containing patches
Messages (20)
msg249305 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2015-08-28 22:34
The datetime isoformat() function for an aware datetime appends the timezone in the form +HH:MM or -HH:MM. But the %z format produces (strftime) or parses (strptime) either +HHMM or -HHMM. I looked it up on Wikipedia, and the ISO 8601 standard indeed uses the colon. It would be nice if there was a new format character that could produce or parse the colon...
msg249815 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2015-09-04 19:30
This request is similar to (if not a duplicate of) issue 15873.

Gnu date uses the following extensions to %z instruction of strftime/strptime:

       %z     +hhmm numeric timezone (e.g., -0400)

       %:z    +hh:mm numeric timezone (e.g., -04:00)

       %::z   +hh:mm:ss numeric time zone (e.g., -04:00:00)

       %:::z  numeric time zone with : to necessary precision (e.g., -04, +05:30)

I think it is reasonable to add those to python.
msg252284 - (view) Author: (shanmbic) * Date: 2015-10-04 19:58
I modified the "format_utcoffset" function in "_datetimemodule.c", to accept one more parameter "secondsrequired" . It is a boolean variable (PyObject) , which when set to true, the function will return the offset formatted as "+HH:MM:SS" or "-HH:MM:SS". I also modified the "wrap_strftime" function to also accept the strings "%:z" and "%::z" as format specifiers. This is my first contribution to python , so my approach of modifying a function's default parameters might not be correct. Any help is appreciated 
msg252294 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-10-05 00:39
I added some comments on Rietveld. I guess the documentation should get notices and What’s New entries saying it is new in 3.6. Test cases would be good, including negative ones to check error handling is sensible if the Z is missing. Also shanmbic, perhaps look at signing the contributor agreement <> if you haven’t already.

Adding to the list of format codes means that the statement at the top of the list about them all being required by C89 will need fixing.

For parsing, perhaps the existing %z code could be extended to accept colons, without needing to specify a new %:z code. Although if %:z is added for formatting, it should also be supported for parsing.

I am not convinced that it is worth adding %::z or %:::z to Python. The documentation of tzinfo.utcoffset() says it returns the offset “in minutes”, hinting that sub-minute offsets are not supported. RFC 3339 acknowledges that sub-minute offsets exist in history, but AFAIK neither RFC 3339 nor ISO 8601 support them. So I think it is too specialized to build %::z into Python. And I don’t imagine the %:::z necessary precision version would be used much either.
msg252364 - (view) Author: (shanmbic) * Date: 2015-10-05 22:44
Yeah , I agree that including '%:::z' might not be necessary at all . Including '%:z' and '%::z' format specifiers will give an upper edge to Python. Although , if not going with '%::z' or seconds offset, for now only '%:z' can be a valuable addition . 
I have rectified my code, corrected the indentations and have added cases , where it will also handle any strings of form '%:z' and '%::z' i.e. any character other than 'z' also raising exceptions for cases "%:" or "%::", where no character is specified (Invalid string).   
In case , if we only go with '%:z' format specifier , I will remove the code for '%::z'. 
P.S. I have filled the contributor agreement form. Thanks for the awesome help here. :)
msg252366 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2015-10-05 23:16
I think it is very likely that issue 5288 will result in at least seconds being allowed in UTC offsets in Python 3.6.  I am assuming that much in my PEP 495 work in issue 24773.
msg252449 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-10-07 02:59
Okay I am happy to be wrong about Python allowing seconds resolution.

I notice that Issue 12006 was committed, which adds a second table of Python-only codes for ISO 8601 values. The proposed entries here could be incorporated into that table, and the text above it modified to suit. That would solve my C89 library problem.
msg270530 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-07-16 01:56
Since this is closely related to issue 15873, I am merging the nosy lists.

As far as I can tell, the patch just needs tests and a final decision on issue 5288. I don't think it is at all controversial, but we need to relax the offset restrictions before ::z code can go in.

shanmbic, you can use my PEP 495 branch to test.  See <issue24773-s3">>.
msg270531 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-07-16 01:59
Sorry, the tracker messed up the URL: go to

and select branch:issue24773-s3
msg270543 - (view) Author: Antti Haapala (ztane) * Date: 2016-07-16 06:49
"Be conservative in what you do, be liberal in what you accept from others" they say. Also Z as a timezone designator is also widely used in iso 8601 timestamps. I believe the effort should be made to *parse* *any/all* of the ISO 8601 supported time-zone codes with one conversion, the list is not that long, just 'Z', HH, HH:MM, HHMM, longest match. Literal 'Z' really does not need to be supported for *output* at all, but for input, please.

Otherwise this will still go down the road of iso8601 library, which just tries to support all the YYmmddTHHMMSS.FFFFFFzzzz variants. It uses regular expressions to parse the dates as it is faster than trying N different formats with `strptime`
msg270573 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-07-16 17:09
"Be conservative in what you do, be liberal in what you accept from others"

I would agree to this approach with respect to a proposed fromisoformat() method (see issue 15873), but setptime seems to be about specifying an exact format.  For example, "%Y%m%d" will not accept "2016-07-16".
msg270575 - (view) Author: Antti Haapala (ztane) * Date: 2016-07-16 17:26
Alexander: that is true, because they are *separate* conversion flags. 

However even the POSIX standard strptime has some leniency: '%m` and `%d` accept the numbers *without* leading zeroes. This actually also means that one cannot use `%Y%m%d` to detect an invalid ISO timestamp:

    >>> datetime.datetime.strptime('111122', '%Y%m%d')
    datetime.datetime(1111, 2, 2, 0, 0)

The `arrow` library depends on the supposed "strict" behaviour of strptime that has never been guaranteed, which often results in very buggy behaviour under some conditions.


(Also, it must be noted that GNU date program doesn't use these formats to *parse* dates, and POSIX strptime in *C* library outright ignores any timezone information)
msg270579 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2016-07-16 18:21
Antti, while I see some convenience in making %z parsing promiscuous, there is clear utility in adding %:z to strftime.  If we do that, not allowing the same for parsing will be odd.  Let's start with that.  A case for a promiscuous %z can be made later.

On a separate issue, note that datetime.isoformat() has recently grown a timespec option.  See issue 19475.  I wonder if it would make sense to add a tzspec option to control the way timezone is formatted.
msg270826 - (view) Author: Anders Hovmöller (Anders.Hovmöller) Date: 2016-07-19 13:56
> The `arrow` library depends on the supposed "strict" behaviour of strptime that has never been guaranteed, which often results in very buggy behaviour under some conditions.

Well… the arrow library accepts all sorts of broken input and gives you a date back. I think they even think that’s a feature and not a bug so no use in trying to help people who are adamant that they don’t want to be helped :/
msg299213 - (view) Author: Naotoshi Seo (sonots) Date: 2017-07-26 07:05
Any progress here? I want `%:z` strftime.

By the way, ruby strptime works as:

irb(main):008:0* Time.strptime('2015-01-01T01:01:01+09:00', '%Y-%m-%dT%H:%M:%S%z')
=> 2015-01-01 01:01:01 +0900
irb(main):009:0> Time.strptime('2015-01-01T01:01:01+0900', '%Y-%m-%dT%H:%M:%S%z')
=> 2015-01-01 01:01:01 +0900
irb(main):010:0> Time.strptime('2015-01-01T01:01:01UTC', '%Y-%m-%dT%H:%M:%S%z')
=> 2015-01-01 01:01:01 UTC
irb(main):011:0> Time.strptime('2015-01-01T01:01:01Z', '%Y-%m-%dT%H:%M:%S%z')
=> 2015-01-01 01:01:01 UTC
irb(main):020:0> Time.strptime('2015-01-01T01:01:01Asia/Tokyo', '%Y-%m-%dT%H:%M:%S%z')
=> 2015-01-01 01:01:01 +0900

irb(main):015:0* Time.strptime('2015-01-01T01:01:01+09:00', '%Y-%m-%dT%H:%M:%S%Z')
=> 2015-01-01 01:01:01 +0900
irb(main):016:0> Time.strptime('2015-01-01T01:01:01+0900', '%Y-%m-%dT%H:%M:%S%Z')
=> 2015-01-01 01:01:01 +0900
irb(main):017:0> Time.strptime('2015-01-01T01:01:01UTC', '%Y-%m-%dT%H:%M:%S%Z')
=> 2015-01-01 01:01:01 UTC
irb(main):018:0> Time.strptime('2015-01-01T01:01:01Z', '%Y-%m-%dT%H:%M:%S%Z')
=> 2015-01-01 01:01:01 UTC
irb(main):019:0> Time.strptime('2015-01-01T01:01:01Asia/Tokyo', '%Y-%m-%dT%H:%M:%S%Z')
=> 2015-01-01 01:01:01 +0900
msg299237 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2017-07-26 14:52
Alexander, can you summarize the status of this issue? Maybe we can move forward for 3.7?
msg299239 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2017-07-26 15:28
This issue is waiting for the final decision on #5288.  If sub-minute offsets support is accepted, I still don't think we need %::z because %:z can simply add non-zero :seconds as needed.  There are also some review comments on the latest patch that have not been addressed.  Overall this is an easy issue and it would be quite doable to get it resolved for 3.7.
msg299241 - (view) Author: Guido van Rossum (gvanrossum) * (Python committer) Date: 2017-07-26 15:49
Please do! I have no opinion on %::z but maybe you can find inspiration in the Zen of Python. :-)
msg299333 - (view) Author: John Nagle (nagle) Date: 2017-07-27 18:31
As the original author of the predecessor bug report (issue 15873) in 2012, I would suggest that there's too much bikeshedding here. I filed this bug because there was no usable ISO8601 date parser available.  PyPi contained four slightly different buggy ones, and three more versions were found later.  

I suggested following RFC3339, "Date and Time on the Internet: Timestamps", section 5.6, which specifies a clear subset of ISO8601.  Five years later, I suggest just going with that. Fancier variations belong in non-standard libraries.

Date parsing should not be platform-dependent.  Using an available C library was convenient, but not portable. 

Let's get this done.
msg299339 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2017-07-27 19:39

An RFC3339 parser is beyond the scope of this issue which is limited to adding str[fp]time code(s) to produce and consume RFC3339-formatted timezones.

We can still have fromisoformat() constructor implemented in 3.7, but someone needs to address the issues raised at #15873.
Date User Action Args
2017-07-27 19:39:34belopolskysetmessages: + msg299339
2017-07-27 18:31:45naglesetmessages: + msg299333
2017-07-26 15:49:33gvanrossumsetmessages: + msg299241
2017-07-26 15:28:51belopolskysetmessages: + msg299239
versions: + Python 3.7, - Python 3.6
2017-07-26 14:52:02gvanrossumsetmessages: + msg299237
2017-07-26 07:05:44sonotssetnosy: + sonots
messages: + msg299213
2016-12-13 11:58:44sirexsetnosy: + sirex
2016-07-19 13:56:48Anders.Hovmöllersetmessages: + msg270826
2016-07-16 18:21:37belopolskysetmessages: + msg270579
2016-07-16 17:26:26ztanesetmessages: + msg270575
2016-07-16 17:09:22belopolskysetmessages: + msg270573
2016-07-16 06:49:01ztanesetnosy: + ztane
messages: + msg270543
2016-07-16 01:59:24belopolskysetmessages: + msg270531
2016-07-16 01:56:41belopolskysetnosy: + barry, jcea, roysmith, nagle, haypo, jwilk, mcepl, merwok, Arfrever, r.david.murray, davydov, cvrebert, karlcow, SilentGhost, perey, flying sheep, mihaic, aymeric.augustin, Roman.Evstifeev, berker.peksag, piotr.dobrogost, kirpit, Anders.Hovmöller, jstasiak, Eric.Hanchrow, deronnax, pbryan

messages: + msg270530
stage: patch review -> commit review
2015-10-07 02:59:56martin.pantersetmessages: + msg252449
2015-10-05 23:16:36belopolskysetnosy: + tim.peters
messages: + msg252366
2015-10-05 22:44:11shanmbicsetfiles: + datetime2.patch

messages: + msg252364
2015-10-05 00:39:47martin.pantersetcomponents: + Extension Modules, - ctypes
2015-10-05 00:39:01martin.pantersetnosy: + martin.panter

messages: + msg252294
stage: needs patch -> patch review
2015-10-04 19:58:24shanmbicsetfiles: + datetime.patch

components: + ctypes

keywords: + patch
nosy: + shanmbic
hgrepos: + hgrepo319
messages: + msg252284
2015-09-29 16:30:56belopolskysetkeywords: + easy
assignee: belopolsky
2015-09-04 19:32:23belopolskysettype: enhancement
stage: needs patch
2015-09-04 19:30:08belopolskysetnosy: + belopolsky
messages: + msg249815
2015-08-28 22:34:41gvanrossumcreate