classification
Title: distutils: build should use a version-specific build directory
Type: behavior Stage: resolved
Components: Distutils Versions: Python 3.5
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, alexis, benjamin.peterson, darabos, eric.araujo, georg.brandl, ned.deily, r.david.murray, ronaldoussoren, tarek, techtonik
Priority: normal Keywords:

Created on 2012-07-22 08:32 by techtonik, last changed 2020-10-22 21:35 by eric.araujo. This issue is now closed.

Messages (13)
msg166109 - (view) Author: anatoly techtonik (techtonik) Date: 2012-07-22 08:32
This scenario fails:
  python setup.py install
  python3 setup.py install

Because it seems like code once built for Python 2 is not rebuild when installing for Python 3.
msg166134 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-07-22 12:59
Could you please be more specific about what you are trying to do and what error you are seeing?
msg170113 - (view) Author: Dániel Darabos (darabos) Date: 2012-09-09 16:09
I've had the same issue. I was installing Pyglet. I first installed it under Python 2, then:

$ sudo python3 setup.py install
running install
running build
running build_py
running install_lib
[...]
creating /Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pyglet/window/xlib
copying build/lib/pyglet/window/xlib/__init__.py -> /Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pyglet/window/xlib
byte-compiling /Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pyglet/__init__.py to __init__.cpython-32.pyc
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/pyglet/__init__.py", line 276
    print '[%d] %s%s %s' % (thread, indent, name, location)
                       ^
SyntaxError: invalid syntax


Deleting the 'build' directory from the Python 2 run fixed the problem and I could install the package under Python 3.
msg170135 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2012-09-09 20:57
It seems to me this is just a special case of the more general issue of building multiple versions of the same distribution, say with multiple versions of Python 2.  In general, anytime you change your build environment, you should start with no existing build directory. That's just good practice.  Distutils doesn't keep track of every change you might make in a build environment between invocations of setup.py.  I suggest that this be closed as "won't fix".
msg170196 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-09-10 15:08
> It seems to me this is just a special case of the more general issue of building multiple versions of the
> same distribution, say with multiple versions of Python 2.  In general, anytime you change your build
> environment, you should start with no existing build directory.

Maybe Distutils could do that for you.

> Distutils doesn't keep track of every change you might make in a build environment between invocations
> of setup.py.

Could you detail what you mean with “change in a build environment”?  Timestamps are tracked, C headers dependencies too, changes to setup.py (and maybe setup.cfg too), so I think it wouldn’t be a stretch to also track the Python version used to run setup.py.  On the other hand, even with the same version there can be differences in the command-line options, the available modules (using Pythons with the same version from different virtual environments), etc.  Anyway, using one build directory per version (X.Y.Z) could easily solve a part of these issues.

What do people think?
msg170245 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2012-09-10 22:54
Currently users can run e.g.:
$ python2.7 setup.py build -b build-2.7 install
$ python3.2 setup.py build -b build-3.2 install

If default name of build directory had to be changed, then it would be useful to also include name of Python implementation (e.g. "build-cpython-2.7").
msg170485 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2012-09-14 16:32
I'd change the name of the subdirectories of build instead of the name of the build directory itself. That way distutils will still create one build directory regardless of how many python versions you use, and the name of that directory is easily predicatble (which is nice when using basic tools to clean up a directory).

The distribute/setuptools projects do this as well, and implementing this should only require a fairly trival patch to distutils.commands.build (the defaults for the various build_ directories need to be updated).

BTW. How likely is the acceptance of such a patch? IIRC distutils is more or less closed for development w.r.t. new features. Would that also apply to this feature request?
msg170486 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-09-14 16:40
> I'd change the name of the subdirectories of build instead of the name of the build directory itself.
Yep that’s what I was thinking about (or if I wasn’t, I should have been :)

> IIRC distutils is more or less closed for development w.r.t. new features.
Not more or less: it is frozen.  The only exceptions are changes required by other changes like the pycache or ABI tag PEPs.

> Would that also apply to this feature request?
If we agree this is a bug, we can fix it.  What do RMs think?
msg170532 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2012-09-15 21:48
Changing of name of subdirectory would break e.g. shell scripts, which set PYTHONPATH=build/lib when calling Python scripts, so this change rather should not be made in micro releases.
msg170588 - (view) Author: Ronald Oussoren (ronaldoussoren) * (Python committer) Date: 2012-09-17 06:28
I don't think this is a bugfix, but a new feature and as such the behavior should only be changed for 3.4.

The easiest workaround for this issue is to use setuptools or distribute when installing, as those tools do set the various build directories to version-specific values.
msg170607 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-09-17 13:53
Okay, then it’s distutils2 material.
msg170608 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-09-17 13:54
A doc note for distutils in stable versions could also be added.
msg379359 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2020-10-22 21:35
setuptools is recommended over distutils, and doesn’t have the issue (per previous messages).
History
Date User Action Args
2020-10-22 21:35:20eric.araujosetstatus: open -> closed
messages: + msg379359

assignee: eric.araujo ->
resolution: wont fix
stage: needs patch -> resolved
2014-03-13 21:07:18eric.araujosettitle: distutils2: build should use a version-specific build directory -> distutils: build should use a version-specific build directory
stage: needs patch
components: + Distutils, - Distutils2
versions: + Python 3.5, - 3rd party, Python 3.4
2012-09-17 13:54:00eric.araujosetmessages: + msg170608
2012-09-17 13:53:14eric.araujosettitle: distutils: build should use a version-specific build directory -> distutils2: build should use a version-specific build directory
messages: + msg170607
components: - Distutils
versions: + 3rd party, Python 3.4, - Python 2.7, Python 3.2, Python 3.3
2012-09-17 06:28:14ronaldoussorensettype: behavior
messages: + msg170588
2012-09-15 21:48:25Arfreversetmessages: + msg170532
2012-09-14 16:40:51eric.araujosetnosy: + georg.brandl, benjamin.peterson
messages: + msg170486
2012-09-14 16:32:15ronaldoussorensetnosy: + ronaldoussoren
messages: + msg170485
2012-09-10 22:54:48Arfreversetmessages: + msg170245
2012-09-10 22:43:52Arfreversetnosy: + Arfrever
2012-09-10 15:08:39eric.araujosettitle: distutils: build_py_2to3 should use a separate build directory -> distutils: build should use a version-specific build directory
messages: + msg170196
versions: + Python 2.7
2012-09-09 20:57:04ned.deilysetnosy: + ned.deily

messages: + msg170135
versions: - Python 3.1
2012-09-09 16:09:57darabossetnosy: + darabos
messages: + msg170113
2012-07-22 12:59:24r.david.murraysetnosy: + r.david.murray
messages: + msg166134
2012-07-22 08:32:55techtonikcreate