classification
Title: logging decimal point should come from locale
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: pitrou, rhettinger, skip.montanaro, vinay.sajip, xiang.zhang
Priority: normal Keywords:

Created on 2017-03-31 17:18 by skip.montanaro, last changed 2017-04-06 18:47 by vinay.sajip. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 931 closed python-dev, 2017-03-31 17:40
Messages (11)
msg290927 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2017-03-31 17:18
The logging module hard codes the decimal point for timestamps to be ",". It should use locale.localeconv()["decimal_point"] instead.
msg290933 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2017-03-31 17:58
It's not exactly a decimal point, more a "decimal mark" as per ISO 8601. From the Wikipedia article for the standard at  "https://en.wikipedia.org/wiki/ISO_8601#Times -

"However, a fraction may only be added to the lowest order time element in the representation. A decimal mark, either a comma or a dot (without any preference as stated in resolution 10 of the 22nd General Conference CGPM in 2003, but with a preference for a comma according to ISO 8601:2004) is used as a separator between the time element and its fraction."

and the citation is

"ISO 8601:2004(E), ISO, 2004-12-01, 4.2.2.4 ... the decimal fraction shall be divided from the integer part by the decimal sign specified in ISO 31-0, i.e. the comma [,] or full stop [.]. Of these, the comma is the preferred sign."

Nothing about picking a decimal point based on the current locale.
msg290942 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2017-03-31 21:37
It's Vinay's code, so what he wants should carry the most weight. I did this as much as an exercise in figuring out the whole pull request/bug report process as anything.
msg290944 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2017-03-31 21:48
One example demonstrating that the datetime module at least prefers a decimal point:

>>> import dateutil.parser
>>> t = '1993-04-21 08:03:00,123'
>>> dateutil.parser.parse(t)
datetime.datetime(1993, 4, 21, 8, 3, 0, 123000)
>>> dateutil.parser.parse(t).isoformat()
'1993-04-21T08:03:00.123000'

Looking at datetime.py, it appears the dot is hard-coded there. Maybe there would be value in the right hand (logging) and the left hand (datetime) doing things the same way?
msg290954 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2017-04-01 04:00
+1 for changing this to a period.  Every time I see the comma, it seems odd to me.  It also is harder to parse back into a regular float.  In addition, it is hazard when people use commas and delimiters between the fields in a log file.
msg290958 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2017-04-01 04:59
Although I prefer period but such a change could break backwards compatibility so I don't know it's worth or not. Codes like `datetime.strptime('1993-04-21 08:03:00,123', '%Y-%m-%d %H:%M:%S,%f')` (usually for such a simple parse goal I like to use the builtin datetime library) would break with this change and IIUC, the datetime format could be customized for logging.
msg290971 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2017-04-01 08:51
I would prefer to keep things as they are - which is conforming to the preferences expressed in ISO 8601, and preserving backwards compatibility. I agree that a period has some advantages, but I went with what the standard said as closely as I could. I'm not sure it has ever been a problem in practice - I don't remember any other issue raising it, and this functionality has been the same since around 2003 IIRC.
msg291164 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2017-04-05 07:22
Since the date format itself isn't localized (fortunately), there is no reason to localize the decimal point either.  People wanting a localized logging format can easily override the default configuration.  And this proposal would break compatibility with existing log parsing tools.

We could probably bikeshed the default logging configuration for a long time, but there is value in not changing the defaults from one release to another.
msg291227 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2017-04-06 13:49
I would like to close this issue now, without making changes. Will do in one day, unless someone pipes up.
msg291235 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2017-04-06 16:04
Vinay> I would like to close this issue now...

Go for it.

As I indicated in a previous comment, the exercise was as much to try and
come to grips with the process as to actually make the change. There
certainly appear to be good reasons to leave well enough alone. My primary
(though minor) concerns at this point are:

* the datetime module hard-coded it one way (period) while the logging
package hard-coded it the other way.

* other logging packages I've used/inherited in other languages
(admittedly, pretty much Americo-centric) all seem to have used periods.

This only became an issue for me because I recently started using Flask,
which sets up the logging environment and provides no straightforward API
for me to reconfigure its logger. (Peter Otten demonstrated a way to do
this using functools.partial, which, while doable, certainly doesn't strike
me as straightforward.) In cases where I'm in complete control, configuring
my own logging environment makes sense. (In reality, when I'm in complete
control, I tend to roll my own 20-line Logger class and not use the logging
module at all, but that's a historical artifact of me discovering
performance issues several years ago in applications which logged heavily.
Those issues may well not exist today.)

Skip
msg291245 - (view) Author: Vinay Sajip (vinay.sajip) * (Python committer) Date: 2017-04-06 18:47
> and provides no straightforward API for me to reconfigure its logger.

As far as I know, you just need to do something which configures a Flask logger, then call logging.config.dictConfig() with a suitable configuration which doesn't disable existing loggers and configures the Flask logger how you want. (The dictConfig call should replace any existing configuration with what you've passed to it). See this:

https://stackoverflow.com/questions/11816236/how-to-config-flask-app-logger-from-a-configure-file
History
Date User Action Args
2017-04-06 18:47:28vinay.sajipsetstatus: open -> closed

messages: + msg291245
stage: resolved
2017-04-06 16:04:10skip.montanarosetstatus: pending -> open

messages: + msg291235
2017-04-06 13:49:26vinay.sajipsetstatus: open -> pending
resolution: not a bug
messages: + msg291227
2017-04-05 07:22:59pitrousetnosy: + pitrou
messages: + msg291164
2017-04-01 08:51:28vinay.sajipsetmessages: + msg290971
2017-04-01 04:59:45xiang.zhangsetnosy: + xiang.zhang
messages: + msg290958
2017-04-01 04:00:20rhettingersetnosy: + rhettinger
messages: + msg290954
2017-03-31 21:48:14skip.montanarosetmessages: + msg290944
2017-03-31 21:37:03skip.montanarosetmessages: + msg290942
2017-03-31 17:58:57vinay.sajipsetnosy: + vinay.sajip
messages: + msg290933
2017-03-31 17:40:47python-devsetpull_requests: + pull_request1116
2017-03-31 17:18:07skip.montanarocreate