classification
Title: dateTime.now() return format missing decimal seconds.
Type: behavior Stage: resolved
Components: Windows Versions: Python 3.2, Python 3.3, Python 3.4, Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Dave, mark.dickinson, r.david.murray, vstinner
Priority: normal Keywords:

Created on 2013-02-05 21:43 by Dave, last changed 2013-02-07 01:06 by Dave. This issue is now closed.

Messages (14)
msg181484 - (view) Author: Dave (Dave) Date: 2013-02-05 21:43
Calling datetime.datetime.now() will return only the Date and time to the second w/o the decimal portion when the second increments when also running firefox w/shockwave flash enabled on a windows 7 machine.

  
Example output: 
counter1 is: 23360 time is: 2013-02-05 16:32:24.999000      
counter1 is: 23361 time is: 2013-02-05 16:32:25
counter1 is: 23362 time is: 2013-02-05 16:32:25.002000
Notice the missing decimal value on the middle one.

To reproduce:
Code:
    counter = 0
    while(True):
        counter +=1
            time = datetime.datetime.now()
            numericDateTime = 
            print("counter1 is: " + str(counter) + " time is: " + time)

I can get this to occur every time that firefox is running with Shockwave flash enabled.
datetime.now() works fine w/o firefox running w/shockwave flash enabled.

I believe the datetime.now() should always return the same format.
msg181485 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2013-02-05 21:46
> I believe the datetime.now() should always return the same format.

It's now how it was implemented:
http://docs.python.org/dev/library/datetime.html#datetime.datetime.__str__
and
http://docs.python.org/dev/library/datetime.html#datetime.datetime.isoformat
msg181486 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2013-02-05 21:48
See also issue #1074462
msg181487 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2013-02-05 21:57
This is a feature request rather than a bug report;  changing fields accordingly.

Also, given the discussion leading to the rejection of issue 1074462, and the fact that you can now use %f in strftime as an easy way to get consistent output (issue 1158), I'm closing this as rejected.

>>> d = datetime.datetime(2013, 10, 11, 12, 34, 56, 0)
>>> d.strftime("%Y-%m-%d %H:%M:%S.%f")
'2013-10-11 12:34:56.000000'
msg181490 - (view) Author: Dave (Dave) Date: 2013-02-05 23:01
I appreciate you guys looking into this so quickly, but let's dig a little deeper.

1. STINNER Victor, you claim this is already fixed in 3.4 by the link, however this doesn't really help since I'm not even up to 3.3 yet (though I'm considering it, I meant to select 3.2 above).  No one has yet provided any insight into why this condition only occurs when firefox is up running shockwave flash...(to me this is an even bigger question as I have a 7 other rather under tasked cores at my disposal).

2.  Mark Dickinson, let's break this down.  I've already worked around it as I’m sure that EVERYONE ELSE WHO USES IT has already done as well...  We know as coders it's more difficult to work with a varying format (no advantage there).  Which begs the question why not fix it so everyone doesn't have to?  By the issue you referenced it's been known and unfixed since at least since 2004.  Also, no curiosity why this only occurs when running firefox w/shockwave flash?

Again I do appreciate you guys looking into this so quickly, but I don't know if this is actually going to get fixed, why it wasn't fixed in 2004, or is the Python motto "it's not broke if there exists a workaround".  Odd that more time was spent talking about this (and Mark and myself independently implemented actual fixes) than the fix would have taken to apply to the baseline(I used to see this much other places I've worked).  But I'm happy to say, I never saw responses back so quickly, which is very hopeful.

Sincerely,
Dave
msg181491 - (view) Author: Dave (Dave) Date: 2013-02-05 23:10
Point was/is that I'd be willing to fix this so that others don't have to.  It's for OTHERS SAKE that I submitted this issue as my system is already bullet proof from this defect/lack of feature situation.  This is also my first attempt to get involved with the freeware community.

Thanks,
Dave
msg181492 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-02-05 23:12
I think Victor meant "not" instead of "now".

It doesn't only occur when your run particular programs, it occurs whenever the microseconds are zero.  It is possible that your particular combination of programs produces a timing pattern that means you see microseconds zero more often in your tests.

Keep in mind that the str is a *convenience* representation.  As such it is *much* more convenient to not print the zero microseconds in the very common case of constructed datetimes that have zero microseconds.

If you want to strictly control the representation you want to use specific formatting, which you can do.  

Thus the rejection.
msg181493 - (view) Author: Dave (Dave) Date: 2013-02-06 00:28
Thanks David Murry for clearing up STINNER Victor comments.  I already feel like I work here;)  So "it's not broke if there exists a workaround".  

In that case it's time to update the documents (which often takes longer than the code to update) to reflect this inconsistency so others won't run into this like I did (it took time to trouble shoot this as it was occurring prior to my logging being set up).  It then took time to trap until I started web browsing:)

State clearly in the API that the output format is either:
"2013-02-05 16:32:24.999000" or      
"2013-02-05 16:32:25"
depending on the time returned.


I would also suggest deprecating the now(), __str__ and any method that exhibits this inconsistent multi-format output behavior since everyone after reading the updated comments will switch to another method immediately for simpler/more robust code. 

David Murry, said "Keep in mind that the str is a *convenience* representation.  As such it is *much* more convenient to not print the zero microseconds in the very common case of constructed datetimes that have zero microseconds."  Interesting since currently datetime does print the zeros microseconds in my example: "2013-02-05 16:32:24.999000" (notice the last 3 zeros are microseconds returned/printed by datetime.now()).  Apparently this is inconvenient by your standards was well.  

I'm moving the status back to open for the documentation update and we'll see how anyone feels about the deprecation.

All these little efforts have added up to something great "Python" and I hope these discussion will help further that cause.  I appreciate everyone who has participated.

Thanks,
Dave
msg181494 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-02-06 01:53
No, it is not "it's not broke because there's a workaround", it is not broken because it is *working as designed*.

The str of any object in python is intended to be a convenient representation of that object.  Note that this applies to *all* objects in Python, not just datetime.  (Thus, talking about deprecating __str__ doesn't make any sense.)

The most convenient representation of a datetime has been deemed to be yyyy-mm-dd hh:mm:ss if there microseconds is equal to zero, and yyy-mm-dd hh:mm:ss.MMMMMM if microseconds is not equal to zero, and in practice this has, in general, worked out very well.  It is too bad that Python can't read the programmer's mind and know whether or not zero microseconds are important in a given str call, but sadly it can't, so the above algorithm is the most convenient for the typical uses of str on a datetime.

Now, if you are writing an application and you are doing output that needs to be in a particular format, you should *format your output* to be the way you want it.  This applies to any data, not just datetimes.  It means using a format specification, to which the str of an object might or might not be an input.  In the case of datetime, you don't want to use its str as the input to the format, because that can loose information, as you have discovered.  Instead you want to specify an strftime format string that causes the data to be displayed *exactly as you want it to be*:

    >>> counter1 = 23360
    >>> x = datetime.datetime(2013, 2, 5, 20, 45)
    >>> x
    datetime.datetime(2013, 2, 5, 20, 45)
    >>> str(x)
    '2013-02-05 20:45:00'
    >>> "counter1 is: {:6d} time is: {:%Y-%m-%d %H:%M:%S.%f}".format(counter1, x)
    'counter 1 is:  23360 time is: 2013-02-05 20:45:00.000000'

str is a generic function that is a convenience.  A format specification is, well, *specific* to a particular application.  Note how it is necessary to also use a format for the counter in order to obtain output records with consistent spacing.

Now, as far as documentation goes, __str__ references isoformat, and isoformat says:

    datetime.isoformat(sep='T')

        Return a string representing the date and time in ISO 8601 format,
        YYYY-MM-DDTHH:MM:SS.mmmmmm or, if microsecond is 0,
        YYYY-MM-DDTHH:MM:SS

It is hard to see how the documentation can get clearer than that.  

(As an aside, I *can* see the argument, as was made in the other issue, that the isoformat method should provide a way to control the micorseconds display.  That would be a feature request, but given the history of this issue you should probably raise it on the python-ideas mailing list first in order to get buy-in before opening a new request for enhancement for it.  Not optimal, I know, but sometimes you have to deal with history when suggesting/requesting a change in a project.

Also note that datetime.now() does not have *any* format.  It is a datetime object.  Only when turning it in to a string by some method does formatting come in to play.

I want to thank you for your enthusiasm for making things better.  I hope that my more detailed explanation will help you understand the reasoning behind the design decisions that have been made here.
msg181534 - (view) Author: Dave (Dave) Date: 2013-02-06 15:48
Thanks for the reply, STINNER Victor reply makes more sense in hindsight.  Legacy often rules and we can work with/around things knowing it's full behavior.  

Since this is not documented for datetime.now()(where this issue began), can we add comments something like what was done for __str__, so the two possible output formats are documented? 

Thanks,
Dave
msg181558 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-02-06 18:55
No.  As I said, datetime.now() returns a *datetime object*.  Formatting only becomes involved when you format an object, and that applies to *any* datetime object, and is correctly documented in __str__ + isoformat.

Please do not reopen the issue again unless you get our agreement.
msg181566 - (view) Author: Dave (Dave) Date: 2013-02-06 19:47
Ok, as a c++ guy, it looked like it's returning a string.  The documentation says "Return the current local date and time", but it's actually returning a datetime object (likely an object pointer)  initialized to the current time.  I think this is where every class inherits from a common base class which must include the __str__ method or something to that effect.  Then printing the ptr/ref to the datetime.now() object actually just call's it's __str__ method.  I get this now.

I need to study the Python inner workings to get a better sense of this, but this has helped much (my current books are more functional, but too primitive).  

Thanks again David Murray for taking the time to fully address this issue and to everyone who participated.  Issue closed, action to me to learn more python.  If anyone knows a good deeper book on it, please pass on the title.

Sincerely,
Dave
msg181567 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2013-02-06 19:56
You are correct.  Effectively every class has an __str__, and that is what gets called when you print something without specifying any other formatting.  (I say effectively, because if there is no __str__ the __repr__ gets used, which every class *does* have via inheritance from the base object 'object'.)

For what it is worth, I just gave David Beasly's "Python Essential Reference" to someone who is a relatively new Python programmer but an experienced programmer, and he loved it.
msg181577 - (view) Author: Dave (Dave) Date: 2013-02-07 01:06
I'll order it. 

Thanks again,
Dave
History
Date User Action Args
2013-02-07 01:06:12Davesetmessages: + msg181577
2013-02-06 19:56:26r.david.murraysetmessages: + msg181567
2013-02-06 19:47:28Davesetmessages: + msg181566
2013-02-06 18:55:57r.david.murraysetstatus: open -> closed
resolution: not a bug
messages: + msg181558

versions: + Python 2.7, Python 3.3, Python 3.4
2013-02-06 15:48:05Davesetstatus: closed -> open
resolution: not a bug -> (no value)
messages: + msg181534
2013-02-06 01:53:32r.david.murraysetstatus: open -> closed
resolution: not a bug
messages: + msg181494

stage: resolved
2013-02-06 00:28:03Davesetstatus: closed -> open
type: enhancement -> behavior
resolution: rejected -> (no value)
messages: + msg181493
2013-02-05 23:12:01r.david.murraysetnosy: + r.david.murray
messages: + msg181492
2013-02-05 23:10:05Davesetmessages: + msg181491
2013-02-05 23:01:30Davesetmessages: + msg181490
versions: + Python 3.2, - Python 3.4
2013-02-05 21:57:51mark.dickinsonsetstatus: open -> closed
versions: + Python 3.4, - Python 3.2
type: behavior -> enhancement
messages: + msg181487

resolution: rejected
2013-02-05 21:48:54mark.dickinsonsetnosy: + mark.dickinson
messages: + msg181486
2013-02-05 21:46:07vstinnersetnosy: + vstinner
messages: + msg181485
2013-02-05 21:43:47Davecreate