classification
Title: document that strptime() does not support the Feb 29 if the format does not contain the year
Type: behavior Stage: patch review
Components: Documentation, Library (Lib) Versions: Python 3.8, Python 3.7, Python 3.6, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Arfrever, Martin.Morrison, Matthew.Earl, belopolsky, bradengroom, brett.cannon, docs@python, fbidu, hynek, p-ganssle, pconnell, pitrou, swalker, taleinat, vstinner
Priority: normal Keywords: easy, patch

Created on 2013-10-24 10:02 by Matthew.Earl, last changed 2018-10-31 13:29 by taleinat.

Pull Requests
URL Status Linked Edit
PR 10243 open toanant, 2018-10-30 15:20
Messages (13)
msg201108 - (view) Author: Matthew Earl (Matthew.Earl) Date: 2013-10-24 10:02
datetime.datetime.strptime() without a year fails on Feb 29 with:

>>> datetime.datetime.strptime("Feb 29", "%b %d")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/auto/ensoft-sjc/thirdparty/lib/python3.3/_strptime.py", line 511, in _strptime_datetime
    return cls(*args)
ValueError: day is out of range for month

This is because without a year specified the year is assumed to be 1900, which is not a leap year. The underlying _strptime._strptime() function has some munging such that it doesn't itself fail (see #14157):

>>> _strptime._strptime("Feb 29", "%b %d")
((1900, 2, 29, 0, 0, 0, 0, 60, -1, None, None), 0)

...however datetime.datetime.__init__() is called with this tuple as *args, causing the validation failure.
msg201114 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-10-24 11:21
I don't think that the issue can be called a bug. If we pick another year (ex: 1904), you cannot compare two datetimes anymore:
"This solution has some very undesirable properties - namely that Mar 1st is now less than Feb 29th!"
http://bugs.python.org/issue14157#msg160637

If you want to handle "Feb 29", add an explicit year.

This issue is maybe a documentation issue: datetime.datetime.strptime() should warn users that calling datetime.datetime.strptime() without year may fail for Feb 29.
msg201128 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2013-10-24 13:30
I agree with Victor: we should document that proper Feb 29/leap year support requires a specified year, else constantly accepting Feb 29 as valid would lead to more errors than fix.
msg201137 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-10-24 14:05
Note: the actual explanation is that Feb 29th doesn't exist in *1900* which is the default year in strptime(). The same error happens if you ask for "Feb 30" or "Apr 31", it has nothing to do with leap years specifically.

In other words, the documentation looks sufficient to me as-is, and adding special wording for this would only make it longer than it should be.
msg201139 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-10-24 14:09
> In other words, the documentation looks sufficient to me as-is, and adding special wording for this would only make it longer than it should be.

Adding a note would not hurt.
msg201151 - (view) Author: Matthew Earl (Matthew.Earl) Date: 2013-10-24 16:13
Out of interest, what's the reason for accepting the time.strptime() version as a bug, but not datetime.datetime.strptime()? Is it that time.strptime() is meant to be a simple parsing from string to tuple (with minimal checks), whereas datetime.datetime.strptime() should represent an actual point in time, therefore extra validation is expected to occur?

If so I'm happy to either close or add a small note to the docs (I don't mind which.)
msg201158 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2013-10-24 17:08
> what's the reason for accepting the time.strptime()
> version as a bug, but not datetime.datetime.strptime()?

In case of time.strptime(), we have an option of returning (1900, 2, 29, ..) which while not being a valid date, is a valid (time)tuple:

>>> time.mktime((1900, 2, 29, 0, 0, 0, 0, 0, 0))
-2203873200.0

The time module treats 1900-02-29 as 1900-03-01:

>>> time.mktime((1900, 3, 1, 0, 0, 0, 0, 0, 0))
-2203873200.0


Datetime is stricter than that:

>>> datetime(1900, 2, 29)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: day is out of range for month


There is no valid datetime value that can reasonably be returned from datetime.strptime('Feb 29', '%b %d').
msg328780 - (view) Author: Braden Groom (bradengroom) * Date: 2018-10-29 00:56
> In other words, the documentation looks sufficient to me as-is, and adding special wording for this would only make it longer than it should be.

I agree with Victor here. It seems like this can be closed. There haven't been any comments from other people hitting this issue for 5 years.
msg328805 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-29 11:43
Paul Ganssle: Do you want to work on a PR? Or should we close the issue?
msg328807 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2018-10-29 11:52
This is a great, easy, doc-only issue for a new contributor to handle.
msg328907 - (view) Author: Felipe Rodrigues (fbidu) * Date: 2018-10-30 12:21
What about adding some more information in the exception message in addition to the docs? I mean, if we're raising an issue because someone inserted Feb 29, we add a note about leap year in the exception message
msg328925 - (view) Author: Paul Ganssle (p-ganssle) * Date: 2018-10-30 14:39
@Victor: You mean a PR to fix the *issue* or a PR to add this to the docs?

The current behavior is pretty counter-intuitive, particularly because it also fails because of the (relatively) little-known fact that 1900 happens to not be a leap year because it is evenly divisible by 100 but not by 400.

I think it's pretty simple for end-users to work around this:

def strptime_smarter(dtstr, fmt):
    try:
        return datetime.strptime(dtstr, fmt)
    except ValueError:
        tt = time.strptime(dtstr, fmt)
        if tt[0:3] == (1900, 2, 29):
            return datetime(1904, *tt[1:6])
        raise


But this is largely a problem that arises because we don't have any concept of a "partial datetime", see this dateutil issue: https://github.com/dateutil/dateutil/issues/449

What users want when they do `datetime.strptime("Feb 29", "%b %d")` is something like `(None, 2, 29)`, but we're both specifying an arbitrary default year *and* enforcing that the final date be legal. I think the best solution would be to change the default year to 2000 for *all* dates, but for historical reasons that is just not feasible. :(

Another option is that we could allow specifying a "default date" from which missing values would be drawn. We have done this in dateutil.parser.parse: https://dateutil.readthedocs.io/en/stable/parser.html#dateutil.parser.parse

The biggest problem in dateutil is that the default value for "default date" is the *current date*, which causes many problems with reproducibility. For `datetime.strptime`, the default value would be `datetime(1900, 1, 1)`, which has none of those same problems.

Still, adding such a parameter to `datetime.strptime` seems like a lot of effort to go through to just to make it *easier* for people to work around this bug in `strptime`, particularly since in this case you can *kinda* do the same thing with:

    strptime('1904 ' + dtstr, '%Y %b %d')

Long-winded carping on about datetime issues aside, I think my final vote is for leaving the behavior as-is and documenting it. Looking at the documentation, the only documentation I see for what happens when you don't have %Y, %m or %d is:

    For time objects, the format codes for year, month, and day should not be used,
    as time objects have no such values. If they’re used anyway, 1900 is substituted
    for the year, and 1 for the month and day.

This only makes sense in the context of `strftime`. I think for starters we should document the behavior of strptime when no year, month or day are specified. As part of that documentation, we can add a footnote about Feb 29th.

I can make a PR for this, but as Tal mentions, I think this is a good issue for a first-time contributor, so I'd like to give someone else an opportunity to take a crack at this.
msg328995 - (view) Author: Tal Einat (taleinat) * (Python committer) Date: 2018-10-31 13:29
> I think for starters we should document the behavior of strptime when no year, month or day are specified. As part of that documentation, we can add a footnote about Feb 29th.

This sounds good to me.

We could also improve the exception raised in the specific case of Feb 29th; IMO this should be a separate PR.
History
Date User Action Args
2018-10-31 13:29:59taleinatsetmessages: + msg328995
2018-10-30 15:20:51toanantsetkeywords: + patch
stage: patch review
pull_requests: + pull_request9556
2018-10-30 14:39:55p-gansslesetmessages: + msg328925
2018-10-30 14:03:47p-gansslesetnosy: + p-ganssle
2018-10-30 12:21:55fbidusetnosy: + fbidu
messages: + msg328907
2018-10-29 11:52:56taleinatsetversions: + Python 3.6, Python 3.7, Python 3.8, - Python 3.2, Python 3.3
nosy: + taleinat

messages: + msg328807

keywords: + easy
2018-10-29 11:43:42vstinnersetmessages: + msg328805
2018-10-29 00:56:33bradengroomsetnosy: + bradengroom
messages: + msg328780
2013-10-24 17:08:23belopolskysetmessages: + msg201158
2013-10-24 16:13:21Matthew.Earlsetmessages: + msg201151
2013-10-24 14:09:23vstinnersetstatus: pending -> open

messages: + msg201139
2013-10-24 14:05:01pitrousetstatus: open -> pending

messages: + msg201137
2013-10-24 13:52:14vstinnersetassignee: docs@python

nosy: + docs@python
components: + Documentation
title: datetime.datetime.strptime without a year fails on Feb 29 -> document that strptime() does not support the Feb 29 if the format does not contain the year
2013-10-24 13:30:51brett.cannonsetnosy: + brett.cannon
messages: + msg201128
2013-10-24 11:21:49vstinnersetmessages: + msg201114
2013-10-24 10:02:17Matthew.Earlcreate