Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_compileall fails to build in the installed location #65463

Closed
doko42 opened this issue Apr 16, 2014 · 8 comments
Closed

test_compileall fails to build in the installed location #65463

doko42 opened this issue Apr 16, 2014 · 8 comments
Assignees
Labels
tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error

Comments

@doko42
Copy link
Member

doko42 commented Apr 16, 2014

BPO 21264
Nosy @brettcannon, @terryjreedy, @doko42, @ezio-melotti, @bitdancer, @voidspace
Superseder
  • bpo-25099: test_compileall fails when run by unprivileged user on installed Python
  • Files
  • issue21264.diff
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/brettcannon'
    closed_at = <Date 2015-10-09.22:11:37.698>
    created_at = <Date 2014-04-16.18:59:40.013>
    labels = ['type-bug', 'tests']
    title = 'test_compileall fails to build in the installed location'
    updated_at = <Date 2015-10-09.22:11:37.695>
    user = 'https://github.com/doko42'

    bugs.python.org fields:

    activity = <Date 2015-10-09.22:11:37.695>
    actor = 'brett.cannon'
    assignee = 'brett.cannon'
    closed = True
    closed_date = <Date 2015-10-09.22:11:37.698>
    closer = 'brett.cannon'
    components = ['Tests']
    creation = <Date 2014-04-16.18:59:40.013>
    creator = 'doko'
    dependencies = []
    files = ['35162']
    hgrepos = []
    issue_num = 21264
    keywords = ['patch']
    message_count = 8.0
    messages = ['216553', '217979', '218085', '251031', '251039', '251099', '251610', '252670']
    nosy_count = 6.0
    nosy_names = ['brett.cannon', 'terry.reedy', 'doko', 'ezio.melotti', 'r.david.murray', 'michael.foord']
    pr_nums = []
    priority = 'normal'
    resolution = 'duplicate'
    stage = 'resolved'
    status = 'closed'
    superseder = '25099'
    type = 'behavior'
    url = 'https://bugs.python.org/issue21264'
    versions = ['Python 3.5', 'Python 3.6']

    @doko42
    Copy link
    Member Author

    doko42 commented Apr 16, 2014

    the installation directory is non-writable, and the byte code files don't exist.

    [1/1] test_compileall
    test test_compileall failed -- Traceback (most recent call last):
      File "/usr/lib/python3.4/test/test_compileall.py", line 194, in test_no_args_respects_force_flag
        self.assertRunOK('-f', PYTHONPATH=self.directory)
      File "/usr/lib/python3.4/test/test_compileall.py", line 144, in assertRunOK
        *self._get_run_args(args), **env_vars)
      File "/usr/lib/python3.4/test/script_helper.py", line 69, in assert_python_ok
        return _assert_python(True, *args, **env_vars)
      File "/usr/lib/python3.4/test/script_helper.py", line 55, in _assert_python
        "stderr follows:\n%s" % (rc, err.decode('ascii', 'ignore')))
    AssertionError: Process return code is 1, stderr follows:

    1 test failed:
    test_compileall
    Re-running failed tests in verbose mode
    Re-running test 'test_compileall' in verbose mode

    ======================================================================
    FAIL: test_no_args_respects_force_flag (test.test_compileall.CommandLineTests)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/usr/lib/python3.4/test/test_compileall.py", line 194, in test_no_args_respects_force_flag
        self.assertRunOK('-f', PYTHONPATH=self.directory)
      File "/usr/lib/python3.4/test/test_compileall.py", line 144, in assertRunOK
        *self._get_run_args(args), **env_vars)
      File "/usr/lib/python3.4/test/script_helper.py", line 69, in assert_python_ok
        return _assert_python(True, *args, **env_vars)
      File "/usr/lib/python3.4/test/script_helper.py", line 55, in _assert_python
        "stderr follows:\n%s" % (rc, err.decode('ascii', 'ignore')))
    AssertionError: Process return code is 1, stderr follows:

    @doko42
    Copy link
    Member Author

    doko42 commented May 6, 2014

    what happens here:

    PYTHONPATH=$(pwd) python3.4 -X faulthandler -S -m compileall
    Skipping current directory
    Listing '/home/packages/python/3.4/x'...
    Compiling '/home/packages/python/3.4/x/foo.py'...
    Listing '/usr/lib/python3.4/'...
    Listing '/usr/lib/python3.4/plat-x86_64-linux-gnu'...
    Listing '/usr/lib/python3.4/lib-dynload'...

    so even when overwriting PYTHONPATH, the standard locations are searched and tried to compile, which fails when these are read only.

    Is this behaviour expected?

    The tests itself can be fixed by passing the the test directory as an argument, as done in other tests too.

    @doko42 doko42 added the tests Tests in the Lib/test dir label May 6, 2014
    @doko42
    Copy link
    Member Author

    doko42 commented May 7, 2014

    adding unittest developers

    @terryjreedy
    Copy link
    Member

    Verified failure on installed 3.4.3, Win 7, with 3 rather than 1 error (all similar).
    test_no_args_compiles_path (test.test_compileall.CommandLineTests) ... FAIL
    test_no_args_respects_force_flag (test.test_compileall.CommandLineTests) ... FAIL
    test_no_args_respects_quiet_flag (test.test_compileall.CommandLineTests) ... FAIL
    test_pep3147_paths_doubleoptimize (test.test_compileall.CommandLineTests) ... ok

    First traceback ends with
    AssertionError: Process return code is 1, command line was: ['C:\\Programs\\Python34\\python.exe', '-X', 'faulthandler', '-S', '-m', 'compil
    eall'], stderr follows:

    followed by another ending with
    File "C:\Programs\Python34\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
    UnicodeEncodeError: 'charmap' codec can't encode character '\U00011111' in position 38: character maps to <undefined>

    Why is an astral char in the error message?

    Will try patch.

    @terryjreedy terryjreedy added the type-bug An unexpected behavior, bug, or error label Sep 18, 2015
    @terryjreedy
    Copy link
    Member

    David, I am nosying you because this issue proposes to modify two test functions you added to test_compileall. See my prior message.

    For me, the test failed at every first occurrence of 'PYTHONPATH=' in the 3 functions. With one occurrence removed, the next in the same function failed. With all 6 removed, I get 1 error:

    FAIL: test_no_args_compiles_path (test.test_compileall.CommandLineTests)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "C:\Programs\Python34\lib\test\test_compileall.py", line 179, in test_no_args_compiles_path
        self.assertNotCompiled(self.initfn)
      File "C:\Programs\Python34\lib\test\test_compileall.py", line 159, in assertNotCompiled
        self.assertFalse(os.path.exists(path))
    AssertionError: True is not false

    The tests all pass in current uninstalled 3.4.3+ repository. If I remove all 6, I get the error above. Changing how the directory is passed apparently changes what get compiled. From the title of the test ad the comment "# Note that -l is implied for the no args case.", this is expected and is the point of the test. So deleting 'PYTHONPATH=' must be wrong. I believe we should detect that modifying the environment will not work before calling assertRunOK and skip the latter if it cannot work.

    The other two functions were added in bpo-19532 by RDM. I think we need to know why they were written as they are before changing them. Perhaps they pass with 'PYTHONPATH=' removed because they are incomplete.

    @bitdancer
    Copy link
    Member

    Yes the point is to test compilation of the path. If I run python3 -m compilelall on my installed python, it works fine (zero return code). I can't really investigate this further right now, but it isn't obvious to me what the answer is from the information in the issue so far, except that it doesn't *look* like it should be necessary to skip the tests unless the installation directories are read only *and* the .pyc files in them are non-existent or out of date.

    @brettcannon
    Copy link
    Member

    Issue bpo-25099 seems to also be related to a similar issue. A proposed fix can be found over there.

    @brettcannon brettcannon self-assigned this Sep 25, 2015
    @brettcannon
    Copy link
    Member

    I consider this fixed by way of issue bpo-25099. If that's wrong then feel free to open the issue again.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    4 participants