classification
Title: str(datetime_obj) doesn't include microseconds if their value is 0
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.1, Python 3.2, Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ezio.melotti Nosy List: doerwalter, ezio.melotti, georg.brandl, pitrou, r.david.murray, tim.peters
Priority: low Keywords: patch

Created on 2009-11-17 22:02 by ezio.melotti, last changed 2009-12-18 15:41 by ezio.melotti. This issue is now closed.

Files
File name Uploaded Description Edit
issue7342.patch ezio.melotti, 2009-12-07 22:22 Fixed the test to use a datetime obj with a specific number of microseconds.
Messages (9)
msg95404 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2009-11-17 22:02
Last night, test_strptime failed on one of the buildbots [1] with the
following error:
test test_strptime failed -- Traceback (most recent call last):
  File
"C:\buildslave\3.x.moore-windows\build\lib\test\test_strptime.py", line
279, in test_fraction
    tup, frac = _strptime._strptime(str(now), format="%Y-%m-%d %H:%M:%S.%f")
  File "C:\buildslave\3.x.moore-windows\build\lib\_strptime.py", line
332, in _strptime
    (data_string, format))
ValueError: time data '2009-11-16 17:17:14' does not match format
'%Y-%m-%d %H:%M:%S.%f'

The reason is that the __str__ method of datetime objects
(datetime.now() in the test) adds the trailing .%f only if the
microseconds are != 0. Since the possible values of microseconds are
between 0  and 999999 (both included), there is 1 possibility out of
1000000 that the microseconds of datetime.now() are equal to 0 (and
depending on how the value is incremented it might not include 0 among
the possible values at all). Apparently that was the cause of the
failure in the test.

This can be reproduced easily doing:
>>> from datetime import datetime
>>> str(datetime.now())
'2009-11-17 22:11:23.522951'
>>> str(datetime(2012, 12, 21)) # no microseconds
'2012-12-21 00:00:00'

This behavior is defined in Modules/datetimemodule.c, in the C function
datetime_isoformat (line 4145 on py3k, introduced by tim_one in r30151)
and in isoformat_time (line 1278 on trunk, called by datetime_isoformat,
introduced by walter.doerwald in r55714).

Is there a valid reason to omit the microseconds in case they are equal
to 0 instead of just adding the trailing '.000000'?

If the behavior doesn't change the test can be probably fixed checking
the value of the microseconds before the call to str(). The
documentation and other tests to check this should also be added.

(Thanks to R. David Murray for pointing me in the right direction while
I was investigating the problem.)

[1]:
http://www.python.org/dev/buildbot/all/builders/x86%20XP-5%203.x/builds/52/steps/test/logs/stdio
msg95405 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-11-17 22:08
Yes, there is a valid reason. You certainly don't want spurious
microseconds to be displayed when a datetime object was constructed from
a second-precise source (e.g. parsing e-mail headers).
msg95406 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2009-11-17 22:13
This behavior is intentional and is documented in the
datetime.isoformat() docs:

"""
Return a string representing the date and time in ISO 8601 format,
YYYY-MM-DDTHH:MM:SS.mmmmmm or, if microsecond is 0, YYYY-MM-DDTHH:MM:SS 
...
"""

It was Guido's idea ;-)  The point is that __str__ is supposed to
produce "nice" output, and ".0000000" was thought to be more annoying
than useful, since the common case is that datetime objects don't use
microseconds.
msg95532 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2009-11-20 07:08
If __str__ is supposed to produce "nice" output, the microsecond shouldn't 
be visible at all imho (special cases are not special enough to break the 
rules).
If the date/time object is read by a human he probably doesn't care of 
the microseconds anyway, if it's parsed by a machine the '0 microseconds' 
situation must be special-cased to avoid failures like the one mentioned in 
the first message.
The fact that the documentation of datetime.isotime() mentions it is not 
enough if the user doesn't know that it's used by str(), so a note should be 
added to the doc.
I don't know if/how the situation can be fixed though.
msg95536 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2009-11-20 11:57
Ezio, it was Guido's design decision, it was intentional, and it's been
documented from the start(*).  So you can disagree with it, but you
won't get anywhere claiming it's "a bug":  intentional, documented
behaviors are never "bugs".  At best you can make a case for that
they're design errors.

But that won't fly either.  It's no more inconsistent than that, e.g.,
most floating-point numbers in practice aren't usually displayed with an
exponent, but if an exponent is needed, one is added to the output:

>>> .25
0.25
>>> .25 * 10**100
2.5e+99

Overly literal readings don't help your case either.  Yes, __str__ aims
at producing nice output, but it's pedantic to argue as if that is, or
should be, __str__'s /only/ goal.  Python's design just isn't that
simple-minded ;-)  If someone uses non-zero microseconds, presumably
they want to see them.

Utterly trivial mechanical parsing of str() output is a non-goal in Python.

Of course the failing test should be repaired, and that was a good
catch.  But the chance of changing the language to make the test work
as-is is approximately 0%.


(*) The datetime docs already say:

"""
__str__( ) 

For a datetime instance d, str(d) is equivalent to d.isoformat(' '). 
"""
msg95537 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2009-11-20 12:23
Yes, I wrote the previous message from a cellphone and I wasn't able to
check the doc. It is indeed already documented in datetime.__str__ --
sorry for the noise.
I also noticed that the microseconds are not the only thing that can
change in datetime.isoformat(), there are also the separator and the UTC
offset, so if someone is parsing the output of str(d)/d.isoformat(), he
should already include other checks anyway.

I agree that only the test should be fixed (and possibly others tests
should be added to check that str() outputs the right thing when there
is a UTC offset and/or a different separator).

I'll work on a patch for that.
msg96079 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2009-12-07 22:22
Simple patch that fixes the test using a datetime object with a specific
number of microseconds instead of using datetime.now().

The test only checks that _strptime._strptime returns the correct value
for the microseconds, in test_datetime there are already other tests
(e.g. TestTime.test_str) that check that the microseconds are present
only if the value is different from 0.
msg96344 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2009-12-13 19:02
Committed in r76804. Leaving it open until I port this to 2.6 and 3.x.
msg96568 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2009-12-18 15:41
Done in r76874 (release26-maint), r76875 (py3k), r76876 (release31-maint).
History
Date User Action Args
2009-12-18 15:41:36ezio.melottisetstatus: open -> closed
resolution: fixed
messages: + msg96568

stage: commit review -> resolved
2009-12-13 19:02:35ezio.melottisetmessages: + msg96344
stage: patch review -> commit review
2009-12-07 22:22:12ezio.melottisetkeywords: + patch
files: + issue7342.patch
messages: + msg96079

stage: test needed -> patch review
2009-11-20 12:23:41ezio.melottisetassignee: georg.brandl -> ezio.melotti
messages: + msg95537
components: - Documentation, Extension Modules
nosy: tim.peters, doerwalter, georg.brandl, pitrou, ezio.melotti, r.david.murray
2009-11-20 11:57:51tim.peterssetmessages: + msg95536
2009-11-20 07:08:51ezio.melottisetmessages: + msg95532
2009-11-17 22:13:27tim.peterssetmessages: + msg95406
2009-11-17 22:08:53pitrousetnosy: + pitrou
messages: + msg95405
2009-11-17 22:02:49ezio.melotticreate