Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PYTHONDONTWRITEBYTECODE and PYTHONOPTIMIZE have no effect #76026

Closed
Perlence mannequin opened this issue Oct 23, 2017 · 8 comments
Closed

PYTHONDONTWRITEBYTECODE and PYTHONOPTIMIZE have no effect #76026

Perlence mannequin opened this issue Oct 23, 2017 · 8 comments
Assignees
Labels
3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@Perlence
Copy link
Mannequin

Perlence mannequin commented Oct 23, 2017

BPO 31845
Nosy @ncoghlan, @ericsnowcurrently, @Perlence
PRs
  • bpo-31845: Fix reading flags from environment #4105
  • bpo-22257: Mention startup refactoring in What's New #4286
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/ncoghlan'
    closed_at = <Date 2017-10-25.02:14:21.810>
    created_at = <Date 2017-10-23.11:25:30.491>
    labels = ['interpreter-core', 'type-bug', '3.7']
    title = 'PYTHONDONTWRITEBYTECODE and PYTHONOPTIMIZE have no effect'
    updated_at = <Date 2017-11-05.04:58:48.549>
    user = 'https://github.com/Perlence'

    bugs.python.org fields:

    activity = <Date 2017-11-05.04:58:48.549>
    actor = 'ncoghlan'
    assignee = 'ncoghlan'
    closed = True
    closed_date = <Date 2017-10-25.02:14:21.810>
    closer = 'ncoghlan'
    components = ['Interpreter Core']
    creation = <Date 2017-10-23.11:25:30.491>
    creator = 'Perlence'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 31845
    keywords = ['patch']
    message_count = 8.0
    messages = ['304794', '304796', '304798', '304899', '304955', '304956', '305582', '305584']
    nosy_count = 3.0
    nosy_names = ['ncoghlan', 'eric.snow', 'Perlence']
    pr_nums = ['4105', '4286']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue31845'
    versions = ['Python 3.7']

    @Perlence
    Copy link
    Mannequin Author

    Perlence mannequin commented Oct 23, 2017

    Setting PYTHONDONTWRITEBYTECODE doesn't seem to have an effect in Python 3.7.0a2:

        $ python -V
        Python 3.7.0a2
        $ env PYTHONDONTWRITEBYTECODE=1 python -c 'import sys; print(sys.dont_write_bytecode)'
        False

    @Perlence Perlence mannequin added 3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) labels Oct 23, 2017
    @Perlence
    Copy link
    Mannequin Author

    Perlence mannequin commented Oct 23, 2017

    Looks like PYTHONOPTIMIZE has no effect either:

        $ python -V
        Python 3.7.0a2
        $ env PYTHONOPTIMIZE=1 python -c 'import sys; print(sys.flags.optimize)'
        0

    @Perlence Perlence mannequin changed the title Envvar PYTHONDONTWRITEBYTECODE is not respected PYTHONDONTWRITEBYTECODE and PYTHONOPTIMIZE have no effect Oct 23, 2017
    @Perlence
    Copy link
    Mannequin Author

    Perlence mannequin commented Oct 23, 2017

    It seems 1abcf67[1] is the first bad commit.

    [1]1abcf67

    @ncoghlan
    Copy link
    Contributor

    Huh, it appears the tests for these features are relying solely on the command line options, and not checking that the environment variables are also setting the flags properly. We should be able to account for that omission with a single new test in test_cmd_line that sets every relevant environment variable and checks for the expected sys.flags contents.

    As far as what's actually going wrong goes, it's this sequence of events in Py_Main:

    _Py_InitializeCore(&core_config);
    ... other code ...
    apply_command_line_and_environment(&cmdline);
    

    _Py_InitializeCore is setting the internal flags appropriately based on the runtime environment, but then Py_Main is stomping over those environmental settings with the settings from the command line.

    Historically, these operations happened the other way around, so it was solely up to the code reading the environment variables to ensure they played nice with each other. Now the command line processing logic needs to be updated to also ensure that it only ever increases these values and never reduces them.

    @ncoghlan ncoghlan self-assigned this Oct 24, 2017
    @ncoghlan
    Copy link
    Contributor

    New changeset d7ac061 by Nick Coghlan in branch 'master':
    bpo-31845: Fix reading flags from environment (GH-4105)
    d7ac061

    @ncoghlan
    Copy link
    Contributor

    We could still use some more comprehensive test cases for the env var handling and the way that interacts with the command line settings, but the merged PR includes at least a rudimentary check for the four that directly affect sys.flags without any tricky side effects (PYTHONDEBUG, PYTHONVERBOSE, PYTHONOPTIMIZE, PYTHONDONTWRITEBYTECODE).

    @ncoghlan ncoghlan added the type-bug An unexpected behavior, bug, or error label Oct 25, 2017
    @ncoghlan
    Copy link
    Contributor

    ncoghlan commented Nov 5, 2017

    Cross-linking to the work-in-progress RFE that introduced the error: https://bugs.python.org/issue22257

    @ncoghlan
    Copy link
    Contributor

    ncoghlan commented Nov 5, 2017

    New changeset 1b46131 by Nick Coghlan in branch 'master':
    bpo-22257: Mention startup refactoring in What's New (GH-4286)
    1b46131

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    1 participant