classification
Title: crash in datetime.strftime
Type: crash Stage: resolved
Components: Library (Lib), Windows Versions: Python 3.4, Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: tim.golden Nosy List: Ramchandra Apte, belopolsky, flox, gladman, haypo, patrick.vrijlandt, python-dev, tim.golden
Priority: normal Keywords: patch

Created on 2011-12-29 09:04 by patrick.vrijlandt, last changed 2013-11-12 13:33 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
issue13674.diff tim.golden, 2013-11-06 15:02 review
Messages (21)
msg150323 - (view) Author: patrick vrijlandt (patrick.vrijlandt) Date: 2011-12-29 09:04
This causes a crash in python 3.2.2 and 3.2, but not in 2.7.2

C:\Python32>python
Python 3.2 (r32:88445, Feb 20 2011, 21:29:02) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime
>>> datetime.datetime(1899,12,31).strftime("%y")

The crash happens with %y but not with %Y.
The crash happens with any year < 1900.
On 2.7.2 a ValueError is raised because strftime requires year >= 1900. This is what IMHO should happen (and would have saved me a lot of time)
msg150325 - (view) Author: Ramchandra Apte (Ramchandra Apte) * Date: 2011-12-29 09:15
This bug can not be reproduced in Python 3.2.2 on Ubuntu.
Since Python 2.7.2 on your system raises a ValueError for dates below 1900 ,your system's strftime probably does not allow dates below 1900 (unlike Ubuntu).
Python 3.2.2's datetime.strftime should handle this error from strftime().
msg150335 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2011-12-29 16:59
This is happening on Windows x86 against the current tip. The MS C runtime can handle older dates; it's just that we're taking 1900 off the year at some point. (At least, I think that's what's happening). FWIW you only need time.strftime to reproduce the error:

  import time
  time.strftime("%y", (1899, 1, 1, 0, 0, 0, 0, 0, 0))

If no-one gets there first I'll dig into the timemodule strftime wrapper.
msg150341 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2011-12-29 18:20
... and that's because the tm struct defines the tm_year field as an offset from 1900. Sorry for the false start. I'll look at the MS runtime stuff instead
msg150343 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-12-29 18:44
timemodule.c has the following check:

#if defined(_MSC_VER) || defined(sun)
    if (buf.tm_year + 1900 < 1 || 9999 < buf.tm_year + 1900) {
        PyErr_SetString(PyExc_ValueError,
                        "strftime() requires year in [1; 9999]");
        return NULL;
    }
#endif
msg150345 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2011-12-29 19:28
Yes, but the MS crt strftime *for %y* requires a year >= 1900 (ie 
tm_year >= 0). Looks like we need a special check.
msg150353 - (view) Author: patrick vrijlandt (patrick.vrijlandt) Date: 2011-12-29 22:38
Is it relevant that 2.7.2 _does_ throw a correct exception?
msg150368 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2011-12-30 10:24
Well, the code in 2.x is quite different from that in 3.x.
Specifically, the 2.x code assumes that, for Windows, no
year before 1900 is valid for any of the formats. So a
simple check throws a ValueError for tm_year < 0. For 3.x
the assumption was that Windows can handle any year as far
back as 0; in fact that's not true for the %y format.

I'll propose a patch to timemodule.c but I'll have to take
it to python-dev as I'm not 100% sure of the best place for
it.
msg150369 - (view) Author: patrick vrijlandt (patrick.vrijlandt) Date: 2011-12-30 10:42
Somewhere in the code is also/still a seperate check concerning strftime:

PythonWin 3.2 (r32:88445, Feb 20 2011, 21:29:02) [MSC v.1500 32 bit (Intel)] on win32.
Portions Copyright 1994-2008 Mark Hammond - see 'Help/About PythonWin' for further copyright information.
>>> import datetime
>>> datetime.datetime(-1, 1, 1)
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
ValueError: year is out of range
>>> datetime.datetime(0, 1, 1)
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
ValueError: year is out of range
>>> datetime.datetime(1, 1, 1)
datetime.datetime(1, 1, 1, 0, 0)
>>> datetime.datetime(1, 1, 1).strftime("Y")
Traceback (most recent call last):
  File "<interactive input>", line 1, in <module>
ValueError: year=1 is before 1000; the datetime strftime() methods require year >= 1000
>>>
msg150372 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2011-12-30 11:03
Yes, sorry. I wasn't clear enough. There *are* still checks
in the 3.x code (for the kind of thing you're showing). But
the checks assume 1000 <= year <= maxint is ok for all format
parameters on Windows. In fact, for %y, only 1900 <= year is ok.
msg198688 - (view) Author: (gladman) Date: 2013-09-30 10:32
On IDLE this:

Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:06:53) [MSC v.1600 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> from datetime import datetime
>>> datetime(1878, 12, 31).strftime('%d %b %y')

causes a crash on Windows.  I am surprised that this bug still exists as it is not far off two years old now.
msg198692 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-09-30 11:39
> I am surprised that this bug still exists as it is not far off two years old now.

You should report the bug to Microsoft who distributes a buggy C runtime library.
msg198693 - (view) Author: (gladman) Date: 2013-09-30 12:08
On 30/09/2013 12:39, STINNER Victor wrote:
> 
> STINNER Victor added the comment:
> 
>> I am surprised that this bug still exists as it is not far off two years old now.
> 
> You should report the bug to Microsoft who distributes a buggy C runtime library.
> 
> ----------
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue13674>
> _______________________________________
> 

Thank you for your suggestion.

But, given that this bug has been present for some two years, I would
hope that Python developers would have already done what you suggest and
this leads me to doubt that any approach that I made to Microsoft would
achieve any practical benefit.

In practice it is pretty well always necessary in building applications
on top of widely used compilers and libraries to have to workaround the
many bugs that they contain.

Since there is evidently no workaround for this issue in the Python
3.3.2 code, I have to assume that one is not considered to be
necessary, presumably because Microsoft has fixed (or intends to fix)
this issue.
msg198694 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2013-09-30 12:14
In reality (as I'm sure you can guess) it's just that no-one's got to
the point of fixing it. I did start off, but it's not a trivial fix and
clearly it got sidelined (with no-one shouting). Sometimes that's just
the way it is.

I'll see if I can dig out whatever code I had managed to change. I'm
certainly happy to review and apply a patch if someone wants to supply one.
msg198700 - (view) Author: (gladman) Date: 2013-09-30 14:24
On 30/09/2013 13:14, Tim Golden wrote:
> 
> Tim Golden added the comment:
> 
> In reality (as I'm sure you can guess) it's just that no-one's got to
> the point of fixing it. I did start off, but it's not a trivial fix and
> clearly it got sidelined (with no-one shouting). Sometimes that's just
> the way it is.
> 
> I'll see if I can dig out whatever code I had managed to change. I'm
> certainly happy to review and apply a patch if someone wants to supply one.

Thank you for your comment Tim.

To be honest, Python is just so reliable that I was genuinely very
surprised to find something that actually crashes it :-)

It's hardly a priority but I thought it might be worth a comment in case
it was a bug that had simply been forgotten about.
msg202268 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2013-11-06 15:02
Attached is a patch with tests
msg202269 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-11-06 15:23
+        if (strchr("y", outbuf[1]) && buf.tm_year < 0)

hum... why not simply outbuf[1] == 'y' ? It would be more explicit and less surprising.

For the unit test, it would be nice to test also asctime(), even if time.asctime() doesn't use asctime() of the C library. And it's better to run tests on all platforms. Only test_y_before_1900() should behave differently on other platforms, but it would be nice to run test_y_before_1900() on platforms supporting "%y" with year < 1900. In my experience, other operating systems have also their own issues. For example, time.strftime() has a specific test to Windows, but also Solaris and AIX.
msg202271 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2013-11-06 16:19
On 06/11/2013 15:23, STINNER Victor wrote:
> +        if (strchr("y", outbuf[1]) && buf.tm_year < 0)
>
> hum... why not simply outbuf[1] == 'y' ? It would be more explicit
> and less surprising.

Ummm. I have no idea what I was thinking about there. I think it was 
somehow connected with the strchr check a few lines earlier. Anyhow, 
fixed now, thanks.

> For the unit test, it would be nice to test also asctime(), even if
> time.asctime() doesn't use asctime() of the C library. And it's
> better to run tests on all platforms. Only test_y_before_1900()
> should behave differently on other platforms, but it would be nice to
> run test_y_before_1900() on platforms supporting "%y" with year <
> 1900. In my experience, other operating systems have also their own
> issues. For example, time.strftime() has a specific test to Windows,
> but also Solaris and AIX.

I'm not sure where time.asctime comes into it. The implementation 
doesn't use time.strftime, but even if it did, I don't see the need to 
add a test under this issue: the unit test for strftime should be enough 
to cover any direct or indirect use of the function.

I'm happy to open up the other tests.

TJG
msg202683 - (view) Author: Roundup Robot (python-dev) Date: 2013-11-12 12:58
New changeset 1537f14cc690 by Tim Golden in branch '3.3':
Issue13674 Correct crash with strftime %y format under Windows
http://hg.python.org/cpython/rev/1537f14cc690

New changeset 41a4c55db7f2 by Tim Golden in branch 'default':
Issue13674 Correct crash with strftime %y format under Windows
http://hg.python.org/cpython/rev/41a4c55db7f2
msg202684 - (view) Author: Tim Golden (tim.golden) * (Python committer) Date: 2013-11-12 13:00
I've committed the changes with a variant of the pre-1900 test running on all platforms. I think there's scope for more testing of the boundary conditions of strftime but that would be for another issue. I want to get this one in now as it's a crasher on Windows.
msg202688 - (view) Author: Roundup Robot (python-dev) Date: 2013-11-12 13:33
New changeset c61147d66843 by Tim Golden in branch 'default':
Issue #13674 Updated NEWS
http://hg.python.org/cpython/rev/c61147d66843

New changeset 49db4851c63b by Tim Golden in branch '3.3':
Issue #13674 Updated NEWS
http://hg.python.org/cpython/rev/49db4851c63b

New changeset 3ff7602ee543 by Tim Golden in branch 'default':
Issue #13674 Null merge with 3.3
http://hg.python.org/cpython/rev/3ff7602ee543
History
Date User Action Args
2013-11-12 13:33:53python-devsetmessages: + msg202688
2013-11-12 13:00:20tim.goldensetstatus: open -> closed
resolution: fixed
messages: + msg202684

stage: resolved
2013-11-12 12:58:18python-devsetnosy: + python-dev
messages: + msg202683
2013-11-06 16:19:42tim.goldensetmessages: + msg202271
2013-11-06 15:23:14hayposetmessages: + msg202269
2013-11-06 15:02:54tim.goldensetkeywords: + patch
files: + issue13674.diff
messages: + msg202268
2013-11-06 12:46:47tim.goldensetcomponents: + Windows
versions: + Python 3.4, - Python 3.2
2013-11-06 12:46:21tim.goldensetassignee: tim.golden
2013-09-30 14:24:00gladmansetmessages: + msg198700
2013-09-30 12:14:16tim.goldensetmessages: + msg198694
2013-09-30 12:08:07gladmansetmessages: + msg198693
2013-09-30 11:39:31hayposetmessages: + msg198692
2013-09-30 10:32:16gladmansetnosy: + gladman
messages: + msg198688
2011-12-30 11:03:59tim.goldensetmessages: + msg150372
2011-12-30 10:42:41patrick.vrijlandtsetmessages: + msg150369
2011-12-30 10:24:59tim.goldensetmessages: + msg150368
2011-12-29 22:38:25patrick.vrijlandtsetmessages: + msg150353
2011-12-29 19:28:08tim.goldensetmessages: + msg150345
2011-12-29 18:44:39hayposetmessages: + msg150343
2011-12-29 18:20:34tim.goldensetmessages: + msg150341
2011-12-29 16:59:28tim.goldensetnosy: + tim.golden
messages: + msg150335
2011-12-29 12:12:32floxsetnosy: + belopolsky, haypo, flox

versions: + Python 3.3
2011-12-29 09:15:09Ramchandra Aptesetnosy: + Ramchandra Apte
messages: + msg150325
2011-12-29 09:04:17patrick.vrijlandtcreate