Title: distutils and python -B
Type: behavior Stage: resolved
Components: Distutils, Tests Versions: Python 3.3
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: eric.araujo Nosy List: Arfrever, alexis, eric.araujo, georg.brandl, pitrou, ssbarnea, tarek
Priority: normal Keywords:

Created on 2011-05-19 16:20 by pitrou, last changed 2013-03-28 20:14 by eric.araujo. This issue is now closed.

Messages (12)
msg136301 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-05-19 16:20
I get the following failure under a fresh checkout:

FAIL: test_package_data (distutils.tests.test_build_py.BuildPyTestCase)
Traceback (most recent call last):
  File "/home/antoine/t/cpython/Lib/distutils/tests/", line 61, in test_package_data
    self.assertTrue("__init__.pyc" in files)
AssertionError: False is not true
msg136321 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-05-19 17:31
This is due to PYTHONDONTWRITEBYTECODE being set. Not sure this is worth fixing.
msg136344 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-05-20 00:11
Duplicate of #12117.
msg146943 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-11-03 16:31
I have changed packaging in dad02a080bbc to work even under -O or -B.

packaging gives control over the creation of pyc and/or pyo files to the user with its own explicit options (--compile/--no-compile and --optimize 0/1/2), and the behavior should not change if the calling Python happens to run with -B or -O for whatever reason.

Other libraries make different choices: For example, compileall gives no option to generate pyo instead of pyc, you have to call it with python -O.  I argue that distutils and packaging are different.  Calling “python -B build” means that you don’t want pyc files to be created for the modules that are imported for this invocation, but it should not mean that distutils needs to behave as if --no-compile was given.

Fixing this is a small change, and in my opinion a bug fix.  Of course, I would not remove the DistutilsByteCompileError to preserve compatibility, but I would remove the warnings/exceptions when running under -B.  Can I do it?
msg147488 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-11-12 14:28
A 1999 comment in build_py from Greg Ward agrees with me:

    # XXX hey! we can't control whether we optimize or not; that's up
    # to the invocation of the current Python interpreter (at least
    # according to the py_compile docs).  That sucks.

If Tarek doesn’t object, I will make the change I proposed in my earlier message in distutils.
msg148500 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * (Python triager) Date: 2011-11-28 16:24
Please don't backport this change to distutils without backporting --no-compile options to "build" command in distutils (issue #13400). Otherwise users would lose possibility of disabling byte-compilation during running this command.

distutils tests could use locally modified environment with PYTHONDONTWRITEBYTECODE variable removed.
msg148502 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-11-28 16:26
> Please don't backport this change to distutils without backporting --no-compile options to
> "build" command in distutils (issue #13400).
The feature freeze on distutils rules this out.

> Otherwise users would lose possibility of disabling byte-compilation during running this command.
python build build_py --no-compile --optimize=0
msg154202 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-02-25 08:02
I thought about this and am not sure about my position.

I am 100% convinced that what packaging does is the right thing.  Modules like py_compile, compileall (in 3.2+) and packaging are able to compile with or without optimization depending on options that are given to them, independently from the optimization level of the calling Python.

distutils however depends on the calling Python’s -B and -O options, and I just don’t think there is much value in fixing this bug and causing obscure bugs for people who’ve been using -B to control distutils for the past months.

Antoine, you’re a pragmatist and a fixer of bugs.  Arfrever, you’re using this misfeature.  I’d like to know your opinion.
msg185420 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2013-03-28 10:18
Can this be closed?
msg185451 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2013-03-28 14:37
In the absence of support for doing the right thing (which would break compatibility), I won’t change distutils here.
msg185468 - (view) Author: Sorin Sbarnea (ssbarnea) * Date: 2013-03-28 20:02
Can we have a fix for this? ... one that would not require me to drop using PYTHONDONTWRITEBYTECODE.

I do have PYTHONDONTWRITEBYTECODE=1 on my user profile and installing packages became a nightmare.
msg185469 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2013-03-28 20:14
See #12117.
Date User Action Args
2013-03-28 20:14:39eric.araujosetmessages: + msg185469
2013-03-28 20:02:05ssbarneasetnosy: + ssbarnea
messages: + msg185468
2013-03-28 14:37:45eric.araujosetstatus: open -> closed

messages: + msg185451
stage: commit review -> resolved
2013-03-28 10:18:38georg.brandlsetstatus: pending -> open
nosy: + georg.brandl
messages: + msg185420

2012-02-25 08:02:03eric.araujosetstatus: open -> pending
resolution: fixed
messages: + msg154202
2011-11-28 19:27:10vstinnersetnosy: - vstinner
2011-11-28 16:26:12eric.araujosetmessages: + msg148502
2011-11-28 16:24:47Arfreversetnosy: + Arfrever
messages: + msg148500
2011-11-12 14:28:48eric.araujosetmessages: + msg147488
title: test_distutils failure -> distutils and python -B
2011-11-03 16:31:34eric.araujosetstatus: closed -> open
messages: + msg146943

assignee: tarek -> eric.araujo
resolution: duplicate -> (no value)
stage: commit review
2011-05-20 00:11:39vstinnersetstatus: open -> closed

nosy: + vstinner
messages: + msg136344

resolution: duplicate
2011-05-19 17:31:12pitrousetmessages: + msg136321
2011-05-19 16:20:59pitrousetnosy: + alexis
2011-05-19 16:20:53pitroucreate