Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

__debug__ in compile(optimize=1) #66289

Closed
arigo mannequin opened this issue Jul 27, 2014 · 9 comments
Closed

__debug__ in compile(optimize=1) #66289

arigo mannequin opened this issue Jul 27, 2014 · 9 comments
Labels
3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@arigo
Copy link
Mannequin

arigo mannequin commented Jul 27, 2014

BPO 22091
Nosy @arigo, @vstinner, @serhiy-storchaka, @eryksun, @MojoVampire
PRs
  • bpo-27169: The __debug__ constant is now optimized out at compile time. #4880
  • [3.6] bpo-27169: The __debug__ constant is now optimized out at compile time. (GH-4880) #4882
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2017-12-15.11:28:34.891>
    created_at = <Date 2014-07-27.11:32:00.282>
    labels = ['interpreter-core', 'type-bug', '3.7']
    title = '__debug__ in compile(optimize=1)'
    updated_at = <Date 2017-12-15.11:28:34.888>
    user = 'https://github.com/arigo'

    bugs.python.org fields:

    activity = <Date 2017-12-15.11:28:34.888>
    actor = 'serhiy.storchaka'
    assignee = 'none'
    closed = True
    closed_date = <Date 2017-12-15.11:28:34.891>
    closer = 'serhiy.storchaka'
    components = ['Interpreter Core']
    creation = <Date 2014-07-27.11:32:00.282>
    creator = 'arigo'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 22091
    keywords = ['patch']
    message_count = 9.0
    messages = ['224119', '224120', '224126', '266770', '308377', '308379', '308380', '308382', '308384']
    nosy_count = 6.0
    nosy_names = ['arigo', 'vstinner', 'SilentGhost', 'serhiy.storchaka', 'eryksun', 'josh.r']
    pr_nums = ['4880', '4882']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue22091'
    versions = ['Python 3.6', 'Python 3.7']

    @arigo
    Copy link
    Mannequin Author

    arigo mannequin commented Jul 27, 2014

    The documentation of the built-in compile() function is not 100% clear but I think it says that giving the "optimize=1" argument turns "__debug__" to false in the compiled code ( https://docs.python.org/3.5/library/functions.html?highlight=compile#compile ). It doesn't work this way in practice, though:

        python3.5
        >>> exec(compile("print(__debug__)", "exec", "exec", optimize=1))
        True

    I'd recommend to fix either the documentation or the source code.

    @SilentGhost
    Copy link
    Mannequin

    SilentGhost mannequin commented Jul 27, 2014

    I can reproduce your example on 3.4, but for the comparison:

    >> exec(compile("if __debug__: print(42)", "exec", "exec", optimize=1))

    >>> exec(compile("if __debug__: print(42)", "exec", "exec", optimize=0))
    42

    So, it's not as straightforward as one might imagine.

    @SilentGhost SilentGhost mannequin added the interpreter-core (Objects, Python, Grammar, and Parser dirs) label Jul 27, 2014
    @eryksun
    Copy link
    Contributor

    eryksun commented Jul 27, 2014

    If __debug__ were referenced from the code object's co_consts instead of checking locals, globals and builtins (LOAD_NAME), then optimize=1 would work consistently for a given code object.

    Currently in 3.4.1:

        >>> dis.dis(compile("if __debug__: x = 1", "", "exec", optimize=0))
          1           0 LOAD_CONST               0 (1)
                      3 STORE_NAME               0 (x)
                      6 LOAD_CONST               1 (None)
                      9 RETURN_VALUE
    
        >>> dis.dis(compile("if __debug__: x = 1", "", "exec", optimize=1))
          1           0 LOAD_CONST               0 (None)
                      3 RETURN_VALUE
    
        >>> dis.dis(compile("x = __debug__", "", "exec", optimize=0))
          1           0 LOAD_NAME                0 (__debug__)
                      3 STORE_NAME               1 (x)
                      6 LOAD_CONST               0 (None)
                      9 RETURN_VALUE
    
        >>> dis.dis(compile("x = __debug__", "", "exec", optimize=1))
          1           0 LOAD_NAME                0 (__debug__)
                      3 STORE_NAME               1 (x)
                      6 LOAD_CONST               0 (None)
                      9 RETURN_VALUE

    With the current design, an exec can override builtins.__debug__ in globals or locals:

        >>> exec("print(__debug__)", globals(), {"__debug__": "spam"})
        spam

    @MojoVampire
    Copy link
    Mannequin

    MojoVampire mannequin commented May 31, 2016

    I just opened bpo-27169: "__debug__ is not optimized out at compile time for anything but `if:` and `while:` blocks" as a more general case of this bug.

    This bug covers inconsistent behavior, but ultimately, it's caused by incomplete optimization: `__debug__` doesn't act as a compile time constant outside of `if:` and `while:` blocks, so the behavior is inconsistent for all other uses in the `compile` case where `optimize` doesn't match the main interpreter session, and inefficient for all other uses whether or not `compile` is involved.

    If bpo-27169 is resolved as I suggested (or in some other similar way), it would also resolve this bug.

    @serhiy-storchaka
    Copy link
    Member

    New changeset 3325a67 by Serhiy Storchaka in branch 'master':
    bpo-27169: The __debug__ constant is now optimized out at compile time. (bpo-4880)
    3325a67

    @serhiy-storchaka
    Copy link
    Member

    Is it worth to backport this change?

    @vstinner
    Copy link
    Member

    Is it worth to backport this change?

    It's an optimization, usually the answer is no.

    The change is short and can be seen as a bugfix, so it's up to you, it's ok to backport to 3.6, and maybe even 2.7, if it's not too hard to write the backport.

    @serhiy-storchaka
    Copy link
    Member

    New changeset b82da9e by Serhiy Storchaka in branch '3.6':
    [3.6] bpo-27169: The __debug__ constant is now optimized out at compile time. (GH-4880) (bpo-4882)
    b82da9e

    @serhiy-storchaka
    Copy link
    Member

    The backport to 3.6 is straightforward, but backporting to 2.7 would require larger changes.

    @serhiy-storchaka serhiy-storchaka added the 3.7 (EOL) end of life label Dec 15, 2017
    @serhiy-storchaka serhiy-storchaka added the type-bug An unexpected behavior, bug, or error label Dec 15, 2017
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    3 participants