The documentation for distutils claims that sys.exec_prefix is used in certain cases to install data_files, but this is simply not true (maybe it was true in the past or this sentence was copy/pasted from somewhere else?)

If you’re not sure about the reason for that sentence, I think you should not remove it from the docs but investigate more (look at the history, test the conditions (package with extension module), etc).

> If you’re not sure about the reason for that sentence, I think you should not remove it from the docs

If the docs are wrong, their history doesn't matter that much: the docs should be fixed regardless.

> test the conditions (package with extension module)

I did that before posting this bug report. I also looked at the distutils sources. I couldn't find any evidence that data_files are ever installed in sys.exec_prefix.

Just for fun, let's look at the history. That piece of documentation goes back to

commit 632bda3aa06879396561dde5ed3d93ee8fb8900c
Author: Fred Drake <fdrake@acm.org>
Date:   Fri Mar 8 22:02:06 2002 +0000

    Add more explanation of how data_files is used (esp. where the files end up
    in the installation and how that location is determined).

I haven't tried to compile that Python version from 2002 but the distutils sources from that time look quite close to what they are today (scary, no?). So my gut feeling is that those docs were wrong even at the time that they were written.

There is also

commit fa2f4b6d8e297eda09d8ee52dc4a3600b7d458e7
Author: Greg Ward <gward@python.net>
Date:   Sat Jun 24 17:22:39 2000 +0000

    Changed the default installation directory for data files (used by
    the "install_data" command to the installation base, which is usually just
    sys.prefix.  (Any setup scripts out there that specify data files will have
    to set the installation directory, relative to the base, explicitly.)

with commit message stating that data_files are installed in sys.prefix.

> If the docs are wrong, their history doesn't matter that much

What I was saying is that I am not sure that the docs are wrong.

Distutils is more touchy that the rest of the stdlib and I err on the side of caution.

Well, I did try it on a minimal Python project. I also read the distutils sources and understood why it installs data_files in sys.prefix by default. So what more do you need to be convinced?

Did you try with a minimal project containing a C extension?

Did you install in a system where sys.prefix != sys.exec_prefix?

> Did you try with a minimal project containing a C extension?
> Did you install in a system where sys.prefix != sys.exec_prefix?

Yes to both questions.

Though it's difficult to say for certain (distutils' option selection logic is obscure and difficult to follow), it seems that Jeroen's analysis is right.

> it seems that Jeroen's analysis is right.

So would you be willing to merge the PR then?

New changeset 598e15d4feaee3849a91d92c9ca51f17baafe19c by Antoine Pitrou (jdemeyer) in branch 'master':
bpo-25592: Improve documentation of distutils data_files (GH-9767)
https://github.com/python/cpython/commit/598e15d4feaee3849a91d92c9ca51f17baafe19c

Thanks for the analysis and PR!  Sorry that the slow process here was frustrating.

New changeset ebae1ce9c40e62ce52dc968f86ed11578d2fcdfd by Antoine Pitrou (Miss Islington (bot)) in branch '3.7':
bpo-25592: Improve documentation of distutils data_files (GH-9767) (GH-11701)
https://github.com/python/cpython/commit/ebae1ce9c40e62ce52dc968f86ed11578d2fcdfd

Fixing this on 2.7 would require additional investigation (distutils might have diverged), so I'm closing now.

> Fixing this on 2.7 would require additional investigation (distutils might have diverged)

Let's be honest, we are talking about distutils here. So it's way more likely that it didn't diverge and that the behavior is exactly the same on 2.7 and 3.8. So I would suggest to backport it to 2.7 also.

Jeroen you're right.  Could you still give it a quick check?  Then I'll mark the PR for a 2.7 backport.

> Could you still give it a quick check?

I did just that. For reference, these are the steps:
- Checkout the "2.7" branch of the cpython git repo
- ./configure --prefix=/tmp/prefix --exec-prefix=/tmp/eprefix && make && make install
- Install pip (https://pip.pypa.io/en/stable/installing/#installing-with-get-pip-py)
- /tmp/prefix/pip install --no-deps --verbose pari_jupyter  # This requires the PARI/GP library to be installed
- Note that all Jupyter stuff is installed in /tmp/prefix/share/jupyter while the directory /tmp/eprefix/share does not exist

(note typo in the above: /tmp/prefix/pip should be /tmp/prefix/bin/pip)

New changeset 40a101df8c64a1e55cca2780248d7a92bdcccd57 by Antoine Pitrou (Miss Islington (bot)) in branch '2.7':
bpo-25592: Improve documentation of distutils data_files (GH-9767) (GH-11734)
https://github.com/python/cpython/commit/40a101df8c64a1e55cca2780248d7a92bdcccd57

Ok, this is now pushed to 2.7 as well. Thank you Jeroen!