classification
Title: Irregular behavior of datetime.__str__()
Type: enhancement Stage:
Components: Library (Lib) Versions:
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, skip.montanaro, tim.peters, tungwaiyip, zooko
Priority: normal Keywords:

Created on 2004-11-28 05:18 by tungwaiyip, last changed 2007-10-15 15:20 by zooko. This issue is now closed.

Messages (12)
msg54311 - (view) Author: Wai Yip Tung (tungwaiyip) Date: 2004-11-28 05:18
From documentation of datetime.isoformat() and 
__str__() :

Return a string representing the date and time in ISO 
8601 format, YYYY-MM-DDTHH:MM:SS.mmmmmm or, if 
microsecond is 0, YYYY-MM-DDTHH:MM:SS


This behavior assume if microsecond is 0, it means the 
user don't need microsecond precision. This is a poor 
assumption because obviously the user may want 
microsecond precision but its value just happen to be 0. 
Now the output is irregular the user can't even use string 
slicing without checking the length of the output first.

Similar behavior found in 

timedelta.__str__()

time.isoformat()

msg54312 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2004-11-28 19:58
Logged In: YES 
user_id=357491

But there is a reason for this behavior; strftime() and strptime() do not 
support microsecond values.  If you are working with those two functions 
those values will cause you issues when specifying the directives.  And 
the chances of actually having a 0 microsecond if you are using them is 
rather small to say the least.
msg54313 - (view) Author: Wai Yip Tung (tungwaiyip) Date: 2004-11-29 05:30
Logged In: YES 
user_id=561546

I don't understand the issue with strftime() and strptime(). If 
datetime supports microsecond and time doesn't, the user just 
have to trim the microsecond off like:
  strptime( str(datetime.now())[:-7], format)

The problem is the above won't work. And that's why I filed 
this bug. It fails if datetime.now() just happen to have 
microsecond value of 0.

How often this happen is not the issue. The issue is it should 
be deterministic. Actually an issue that happens 1/1000th of 
time is a lot more problematic than an issue that happens 
consistently.

A preferable design is to have datetime to take an extra flag 
to indicate if microsecond is wanted. Or a datetime class that  
supports second precision and a subclass that supports 
microsecond. The user should make a choice on how much 
precision should be used, not leaving it up to chance.
msg54314 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2004-12-01 22:07
Logged In: YES 
user_id=357491

OK, look at it in terms of math; trailing zeros are almost always 
truncated from numbers because they are superfluous.  The same is 
happening here; the superfluous zeros after the decimal are just being 
truncated.

And you can also just as easily detect if there are no trailing zeros and 
tack them on if you prefer::

iso = datetime.datetime.now().isoformat()
if '.' not in iso: iso = '%s.000000' % iso

Because the docs clearly state this behavior I am changing this to a 
feature request for Python 2.5 .

But I do understand the desire to make parsing easier.  If Tim Peters is 
okays the API change I will patch it to take an optional argument which 
will force a microsecond output.  But personally I am -0 on the option.

You listening by any chance, Tim?
msg54315 - (view) Author: Tim Peters (tim.peters) * (Python committer) Date: 2004-12-02 02:50
Logged In: YES 
user_id=31435

Sorry, I really don't see a point to this.  The claim that slicing 
doesn't work is unbelievable on two counts:

1. If someone has a datetime object D, and wants, e.g.,
   the year, they don't do str(D), try to slice out the year
   digits, then convert them to an int(!) -- they just do
   D.year.  Same for the other members (including
   D.microsecond).

2. Slicing works anyway.  For example, str(D)[:4] gets
   the year digits regardless of D.microsecond.  str(D)[17:]
   gets the seconds likewise, and float(str(D)[17:]) gets
   the number of seconds as a float regardless of whether
   D.microsecond is 0.

If you want D without microseconds, the intended way to do 
it is one of:

a) Don't set microseconds if you don't want microseconds;

or, if you're stuck with unwanted microseconds,

b) Use D.replace(microsecond=0) to get a copy of D
   but with microsecond forced to 0.

The current behavior is by design, and documented, so won't 
be changed regardless.

I'm somewhere between -1 and -0 on complicating signatures 
to give a shortcut for something that's rarely needed and 
easy to get anyway.
msg54316 - (view) Author: Wai Yip Tung (tungwaiyip) Date: 2004-12-02 19:17
Logged In: YES 
user_id=561546

You didn't get the problem case. The microsecond is wanted 
but it happens to be 0. Now I can't get a timestamp aligned 
without some doing some extra checking.

I also hate complicating signatures. So my recommendation 
really is to remove the one line of if statement and have it 
always output the microsecond as a fixed size string. Python 
will have one less test case. The user will get a fix sized string 
they can easily slice to get any precision they want.

The current logic really only complicate things. If you take it 
one step further and skip minute and second if they are all 0, 
it will probably be more of an annoyonce than useful. Nor is 
the trimming trailing zeros rationale is actually taking place 
here. On Windows str(now()) always have the last 3 digits 0, 
except in the 1/1000 of time when the microsecond happens 
to be 0.

Nothing is fatal flawed here. Right now I just avoid str() and 
use the strptime() instead. The problem is just the extra logic 
in str() actually make it less useful.
msg55574 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2007-09-02 03:37
I'm going to offer one more argument here, then close the ticket.
(Tim already told you the behavior wasn't going to change.)
str() is a convenience function intended to give conveniently
human-readable output.  It's not intended to be a one-size-fits-
all routine.  Humans are used to not seeing fractions of a second
in times when there are none.  In those situations where you
unambiguously need microseconds displayed, use something like
this:

    >>> str(dt.replace(microsecond=0)) + ".%06d" % dt.microsecond
    '2007-09-01 22:30:36.000032'
    >>> dt.strftime("%H:%M:%S") + ".%06d" % dt.microsecond
    '22:30:36.000032'
msg56434 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2007-10-15 04:15
Here is a note for the next person who comes to this ticket wondering
why isoformat() exhibits this slightly un-Pythonic behavior.  If you
want to use isoformat() to produce, for example, timestamps for your
logfiles, you'll need to do something like the following.  (I do hope
you noticed the documentation and didn't use isoformat() in the naive
way, or your log files will very rarely have a different format than you
expected.)

    d = datetime.datetime.utcfromtimestamp(when)
    if d.microsecond:
        return d.isoformat(" ")[:-3]+"Z"
    else:
        return d.isoformat(" ") + ".000Z"

http://allmydata.org/trac/tahoe/browser/src/allmydata/node.py#L21
msg56436 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2007-10-15 12:50
Zooko> Here is a note for the next person who comes to this ticket
    Zooko> wondering why isoformat() exhibits this slightly un-Pythonic
    Zooko> behavior.

What are you referring to, that it doesn't display any microseconds when the
microsecond field happens to be 0 or that it doesn't truncate the fractions
of a second to milliseconds?

Skip
msg56438 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2007-10-15 13:11
I meant that it special-cases .microseconds == 0.  If I want to produce
a custom output format using Python Standard Library, I expect to have
to slice, add my own fields and so forth, but I don't expect to need an
"if" to handle a special-case that is there for improving the appearance
to human readers.  That's something I had to do a lot more often when I
worked in Perl.

Even if the cost of changing the definition of isoformat() is too high
at this point, I still wanted to post my code from http://allmydata.org
as an example to other users so that they can use isoformat() safely.
msg56441 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2007-10-15 13:47
Zooko> I meant that it special-cases .microseconds == 0.

Tim indicated in his comment that the behavior is both by design and
documented and isn't going to change.  In an earlier comment I showed how to
achieve the result you ased for in one line.  Here's another example using
your desire for millisecond display resolution:

    >>> dt.replace(microsecond=0).strftime("%Y-%m-%dT%H:%M:%S") + ".%03dZ" % (dt.microsecond//1000)
    '2007-10-15T08:24:02.509Z'

Also, I have a patch for py3k which adds a %f format specifier to strftime.
I still have to make some other additions, but you're more than welcome to
review what's there now:

    http://bugs.python.org/issue1158

Skip
msg56443 - (view) Author: Zooko O'Whielacronx (zooko) Date: 2007-10-15 15:20
Thank you for the one-liner.  I was about to use it in the allmydata.org
project, but I remembered that my programming partner would probably
prefer the larger but more explicit if:else: over the clever one-liner.
 Perhaps it will be useful to someone else.

I'll have a look at issue1158.
History
Date User Action Args
2007-10-15 15:20:32zookosetmessages: + msg56443
2007-10-15 13:47:39skip.montanarosetmessages: + msg56441
2007-10-15 13:11:30zookosetmessages: + msg56438
2007-10-15 12:50:06skip.montanarosetmessages: + msg56436
2007-10-15 04:15:40zookosetnosy: + zooko
messages: + msg56434
2007-09-02 03:37:50skip.montanarosetstatus: open -> closed
nosy: + skip.montanaro
resolution: rejected
messages: + msg55574
2004-11-28 05:18:10tungwaiyipcreate