Title: Misleading note in Statistics module documentation
Type: enhancement Stage: resolved
Components: Documentation Versions: Python 3.4
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: steven.daprano Nosy List: Journeyman08, docs@python, iritkatriel, rhettinger, steven.daprano
Priority: normal Keywords: patch

Created on 2015-02-25 18:40 by Journeyman08, last changed 2020-12-19 10:21 by steven.daprano. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 23842 closed rhettinger, 2020-12-18 17:57
Messages (6)
msg236612 - (view) Author: Jake (Journeyman08) Date: 2015-02-25 18:40
In the statistics module documentation, there is a note that states that 

"The mean is strongly affected by outliers and is not a robust estimator for central location: the mean is not necessarily a typical example of the data points. For more robust, although less efficient, measures of central location, see median() and mode()"

While I appreciate the intention, this is quite misleading.  The implication is that the mean, median and mode are different ways to estimate one "central location", however, in reality they are very different things (albeit which refer to a similar notion).

The sample mean is an unbiased estimator of the true mean but it need not be unbiased as an estimator of the true median or modes and vice versa for the median and mode.

To make this clearer I would rephrase to 

"The mean is strongly affected by outliers and is not necessarily representative of the central tendency of the data. For cases with large outliers or very low sample size, see median() and mode()"

Apologies if this is seen as frivolous, but statistics can be hard enough to remain very clear about even when the words are used precisely.
msg383228 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2020-12-17 11:03
I agree with Jake's comment, but I think the solution is to remove that Note altogether. This document is a software manual, not a statistics textbook, and as such it should just state clearly what the statistics module does. If someone doesn't know whether they need the mean or the median, they really need to read a more fundamental text before writing their code.
msg383335 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2020-12-19 00:12
I strongly oppose this change, and I dispute the characterisation of 
this as a misleading note. It is not misleading, and I argue that every 
word of it is factually correct. Jake, if you disagree, then please 
provide some citations.

Irit: it is ridiculous to describe a two paragraph (nine line) note as 
"a statistics textbook". That is an exaggerated position that doesn't 
help the discussion. It's not a textbook, it is a short note that helps 
users whose knowledge of statistics is naive to understand which 
statistic is better for them.

"If someone doesn't know whether they need the mean or the median, they 
really need to read a more fundamental text before writing their code."

I totally disagree. This module is not intended only for statisticians 
and experts, and the user who isn't sure which average to use shouldn't 
have to read a textbook on the fundamentals of statistics.
msg383361 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-12-19 05:42
Steven, you are the module maintainer.  So if you're sure about the current wording, go ahead and close this.
msg383362 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2020-12-19 06:32
I'm willing to give Irit and Jake opportunity to make their case. 
Particularly if they can demonstrate that I got my facts wrong.

I'm going to close the ticket, but if anyone feels strongly enough to 
respond with a good argument, or better still citations demonstrating 
that the comment is factually wrong, I am open to revising or removing 
the wording.
msg383368 - (view) Author: Steven D'Aprano (steven.daprano) * (Python committer) Date: 2020-12-19 10:21
Sorry Raymond, I missed this before closing the task.

> FWIW, Allen Downey also had concerns about this wording.

I don't recognise the name, who is Allen Downey and what concerns does 
he have?
Date User Action Args
2020-12-19 10:21:10steven.dapranosetmessages: + msg383368
2020-12-19 07:24:04steven.dapranosetstatus: open -> closed
resolution: rejected
stage: patch review -> resolved
2020-12-19 06:32:31steven.dapranosetmessages: + msg383362
2020-12-19 05:42:47rhettingersetmessages: + msg383361
2020-12-19 00:58:59rhettingersetmessages: - msg383343
2020-12-19 00:57:51rhettingersetassignee: docs@python -> steven.daprano
2020-12-19 00:57:36rhettingersetmessages: + msg383343
2020-12-19 00:12:12steven.dapranosetmessages: + msg383335
2020-12-18 17:57:06rhettingersetkeywords: + patch
nosy: + rhettinger

pull_requests: + pull_request22703
stage: patch review
2020-12-17 11:03:49iritkatrielsetnosy: + iritkatriel
messages: + msg383228
2015-02-25 19:14:14SilentGhostsetnosy: + steven.daprano
2015-02-25 18:40:46Journeyman08create