classification
Title: __debug__ in compile(optimize=1)
Type: behavior Stage: resolved
Components: Interpreter Core Versions: Python 3.7, Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: SilentGhost, arigo, eryksun, josh.r, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2014-07-27 11:32 by arigo, last changed 2017-12-15 11:28 by serhiy.storchaka. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 4880 merged serhiy.storchaka, 2017-12-15 09:53
PR 4882 merged serhiy.storchaka, 2017-12-15 11:02
Messages (9)
msg224119 - (view) Author: Armin Rigo (arigo) * (Python committer) Date: 2014-07-27 11:32
The documentation of the built-in compile() function is not 100% clear but I think it says that giving the "optimize=1" argument turns "__debug__" to false in the compiled code ( https://docs.python.org/3.5/library/functions.html?highlight=compile#compile ).  It doesn't work this way in practice, though:

    python3.5
    >>> exec(compile("print(__debug__)", "exec", "exec", optimize=1))
    True

I'd recommend to fix either the documentation or the source code.
msg224120 - (view) Author: SilentGhost (SilentGhost) * (Python triager) Date: 2014-07-27 11:43
I can reproduce your example on 3.4, but for the comparison:

>>> exec(compile("if __debug__: print(42)", "exec", "exec", optimize=1))

>>> exec(compile("if __debug__: print(42)", "exec", "exec", optimize=0))
42

So, it's not as straightforward as one might imagine.
msg224126 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2014-07-27 12:56
If __debug__ were referenced from the code object's co_consts instead of checking locals, globals and builtins (LOAD_NAME), then optimize=1 would work consistently for a given code object. 

Currently in 3.4.1:

    >>> dis.dis(compile("if __debug__: x = 1", "", "exec", optimize=0))
      1           0 LOAD_CONST               0 (1)
                  3 STORE_NAME               0 (x)
                  6 LOAD_CONST               1 (None)
                  9 RETURN_VALUE

    >>> dis.dis(compile("if __debug__: x = 1", "", "exec", optimize=1))
      1           0 LOAD_CONST               0 (None)
                  3 RETURN_VALUE

    >>> dis.dis(compile("x = __debug__", "", "exec", optimize=0))
      1           0 LOAD_NAME                0 (__debug__)
                  3 STORE_NAME               1 (x)
                  6 LOAD_CONST               0 (None)
                  9 RETURN_VALUE

    >>> dis.dis(compile("x = __debug__", "", "exec", optimize=1))
      1           0 LOAD_NAME                0 (__debug__)
                  3 STORE_NAME               1 (x)
                  6 LOAD_CONST               0 (None)
                  9 RETURN_VALUE

With the current design, an exec can override builtins.__debug__ in globals or locals:

    >>> exec("print(__debug__)", globals(), {"__debug__": "spam"})
    spam
msg266770 - (view) Author: Josh Rosenberg (josh.r) * (Python triager) Date: 2016-05-31 18:54
I just opened #27169: "__debug__ is not optimized out at compile time for anything but `if:` and `while:` blocks" as a more general case of this bug.

This bug covers inconsistent behavior, but ultimately, it's caused by incomplete optimization: `__debug__` doesn't act as a compile time constant outside of `if:` and `while:` blocks, so the behavior is inconsistent for all other uses in the `compile` case where `optimize` doesn't match the main interpreter session, and inefficient for all other uses whether or not `compile` is involved.

If #27169 is resolved as I suggested (or in some other similar way), it would also resolve this bug.
msg308377 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-12-15 10:35
New changeset 3325a6780c81f1ea51190370b5454879c4862a37 by Serhiy Storchaka in branch 'master':
bpo-27169: The __debug__ constant is now optimized out at compile time. (#4880)
https://github.com/python/cpython/commit/3325a6780c81f1ea51190370b5454879c4862a37
msg308379 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-12-15 10:46
Is it worth to backport this change?
msg308380 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017-12-15 10:54
> Is it worth to backport this change?

It's an optimization, usually the answer is no.

The change is short and can be seen as a bugfix, so it's up to you, it's ok to backport to 3.6, and maybe even 2.7, if it's not too hard to write the backport.
msg308382 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-12-15 11:26
New changeset b82da9ebb20053637e731fd40589e1e52f9f8f6e by Serhiy Storchaka in branch '3.6':
[3.6] bpo-27169: The __debug__ constant is now optimized out at compile time. (GH-4880) (#4882)
https://github.com/python/cpython/commit/b82da9ebb20053637e731fd40589e1e52f9f8f6e
msg308384 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-12-15 11:28
The backport to 3.6 is straightforward, but backporting to 2.7 would require larger changes.
History
Date User Action Args
2017-12-15 11:28:34serhiy.storchakasetstatus: open -> closed
versions: + Python 3.6, Python 3.7, - Python 3.4, Python 3.5
type: behavior
messages: + msg308384

resolution: fixed
stage: patch review -> resolved
2017-12-15 11:26:28serhiy.storchakasetmessages: + msg308382
2017-12-15 11:02:27serhiy.storchakasetpull_requests: + pull_request4779
2017-12-15 10:54:23vstinnersetnosy: + vstinner
messages: + msg308380
2017-12-15 10:46:03serhiy.storchakasetmessages: + msg308379
2017-12-15 10:35:50serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg308377
2017-12-15 09:53:13serhiy.storchakasetkeywords: + patch
stage: patch review
pull_requests: + pull_request4775
2016-05-31 18:54:36josh.rsetnosy: + josh.r
messages: + msg266770
2014-07-27 12:56:27eryksunsetnosy: + eryksun
messages: + msg224126
2014-07-27 11:43:17SilentGhostsetversions: + Python 3.4, Python 3.5
nosy: + SilentGhost

messages: + msg224120

components: + Interpreter Core
2014-07-27 11:32:00arigocreate