This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Rewrite math.erf() and math.erfc() from scratch
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.6, Python 3.5, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: brett.cannon Nosy List: benjamin.peterson, brett.cannon, eric.smith, georg.brandl, larry, lemburg, mark.dickinson, ned.deily, python-dev, serhiy.storchaka, stutzbach, teoliphant, terry.reedy, vstinner
Priority: release blocker Keywords:

Created on 2016-01-14 20:35 by brett.cannon, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (10)
msg258218 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-01-14 20:35
If you look Modules/mathmodule.c you will notice there is a comment that goes with erf() and erfc() stating that the algorithms were taken from a book entitled 'Numerical Recipes'. Unfortunately that book has a license dictating that any information from the book is only allowed for non-commercial use; commercial use requires negotiating a license (http://numerical.recipes/aboutNR3license.html). That's bad for anyone who has a commercial distribution of Python as that's a special requirement they have to follow.

It would be best to do a clean room implementation of both math.erf() and math.erfc() that does not use information from 'Numerical Recipes' in order to not be violating that license. That way Python can be sold commercially without having to negotiate a separate license just for those two functions.

Unfortunately this code exists since at least Python 2.7, so I have flagged all Python releases as needing the eventual clean room implementation applied to it (although Python 3.2 goes out of security maintenance next month so I don't know how critical it is to fix that far back).
msg258279 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2016-01-15 08:47
The comment is unfortunate. The code in Modules/mathmodule.c *was* written from scratch (by me). All I took from Numerical Recipes was the idea of using continued fractions from one part of the domain and a power-series expansion for another part. If you compare the code with the NR code, there's really no similarity beyond that part.

Perhaps just deleting the NR reference is the way to go.
msg258280 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2016-01-15 09:05
... and the way I read it, the NR licence applies specifically to their code, not to the basic numerical ideas set out in the text (which is all we're using). We're not using the actual code from NR at all.

IOW, IANAL but I really don't think there's an issue here.
msg258281 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-01-15 09:06
erf() is a part of C99. May be move hand-writen implementation to Modules/_math.c and use libc erf() if available?
msg258283 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2016-01-15 09:20
On 15.01.2016 10:05, Mark Dickinson wrote:
> 
> Mark Dickinson added the comment:
> 
> ... and the way I read it, the NR licence applies specifically to their code, not to the basic numerical ideas set out in the text (which is all we're using). We're not using the actual code from NR at all.
> 
> IOW, IANAL but I really don't think there's an issue here.

The license is a copyright license, so it only applies to the
actual code from the book. The ideas would have to be patented
to be protected. Copyright in some code or text is not enough
to (potentially) prevent someone else from reusing the ideas.

If someone is aware of a patent on the algorithm, we may have
an issue. Otherwise, there's no issue if we're using Mark's
implementation.
msg258284 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2016-01-15 09:20
@Serhiy: Sure, that would work (the same way that we do for log1p). I *think* I tried this at the time, but it turns out that some libm implementations of erf and erfc are pretty bad, so our tests failed. (But I may be misremembering.)

In any case, the proposal to use the libm erf/erfc should be a separate issue, I think.
msg258305 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-01-15 16:28
Thanks for the historical information, Mark! I'll either update the comment or flat-out delete it so it doesn't trip anyone else up.

I'll also scale back the scope of the update since it's just a cleanup and not an IP issue.
msg258311 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-01-15 17:39
New changeset 76eb752e5447 by Brett Cannon in branch '3.5':
Issue #26114: Remove a reference to 'Numerical Recipes'.
https://hg.python.org/cpython/rev/76eb752e5447

New changeset 8ad701463cd7 by Brett Cannon in branch 'default':
Merge for issue #26114
https://hg.python.org/cpython/rev/8ad701463cd7
msg258312 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2016-01-15 17:41
New changeset faac8f09020d by Brett Cannon in branch '2.7':
Issue #26114: Remove mention of 'Numerical Recipes'.
https://hg.python.org/cpython/rev/faac8f09020d
msg258384 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2016-01-16 11:46
Some additional history: the original 1986 book has separate copyright notices for text (Cambridge U Press), and computer programs (Numerical Recipes Software).  It also has an offer to sell diskettes (Fortran or Pascal), 15 pounds each.  There is no additional license statements that I can see, even for the disks.  I vaguely remember that there was some upset or controversy when a 'license' was added to the 2nd edition.

The copyrighted but license-free erf and erfc functions on p. 164) use the incomplete gamma functions, which use continued fractions and series for different subdomains.  Applied Statistics algorithm AS 66 (about 1973), for normal integrals, appears to directly use series and continued fractions for different subdomains.  It is based on a similar algorithm from 1969.  Both series and continued fraction expansions for erf/normal-integral were known well before that.
History
Date User Action Args
2022-04-11 14:58:26adminsetnosy: + ned.deily
github: 70302
2016-01-16 11:46:27terry.reedysetnosy: + terry.reedy
messages: + msg258384
2016-01-15 17:42:13brett.cannonsetstatus: open -> closed
resolution: fixed
stage: needs patch -> resolved
2016-01-15 17:41:59python-devsetmessages: + msg258312
2016-01-15 17:39:16python-devsetnosy: + python-dev
messages: + msg258311
2016-01-15 16:28:13brett.cannonsetassignee: brett.cannon
messages: + msg258305
versions: - Python 3.2, Python 3.3, Python 3.4
2016-01-15 09:20:46mark.dickinsonsetmessages: + msg258284
2016-01-15 09:20:01lemburgsetmessages: + msg258283
2016-01-15 09:06:34serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg258281
2016-01-15 09:05:44mark.dickinsonsetmessages: + msg258280
2016-01-15 08:47:59mark.dickinsonsetmessages: + msg258279
2016-01-14 20:57:33vstinnersetnosy: + vstinner
2016-01-14 20:38:20brett.cannonsetnosy: - peter
2016-01-14 20:38:01brett.cannonsetnosy: + teoliphant, peter
2016-01-14 20:35:41brett.cannoncreate