Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tests altered the execution environment in isolated mode #72841

Closed
serhiy-storchaka opened this issue Nov 10, 2016 · 12 comments
Closed

Tests altered the execution environment in isolated mode #72841

serhiy-storchaka opened this issue Nov 10, 2016 · 12 comments
Labels
3.7 (EOL) end of life tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error

Comments

@serhiy-storchaka
Copy link
Member

BPO 28655
Nosy @vstinner, @serhiy-storchaka, @tirkarthi
PRs
  • [3.6] bpo-28655: Fix test_import.test_missing_source_legacy() #9589
  • [3.6] bpo-28655: Fix test bdb for isolate mode #10220
    		•
    Dependencies
  • bpo-31837: ParseError in test_all_project_files()
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2018-09-29.07:14:57.314>
created_at = <Date 2016-11-10.08:46:58.206>
labels = ['3.7', 'type-bug', 'tests']
title = 'Tests altered the execution environment in isolated mode'
updated_at = <Date 2018-10-29.20:32:25.890>
user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2018-10-29.20:32:25.890>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-09-29.07:14:57.314>
closer = 'xtreak'
components = ['Tests']
creation = <Date 2016-11-10.08:46:58.206>
creator = 'serhiy.storchaka'
dependencies = ['31837']
files = []
hgrepos = []
issue_num = 28655
keywords = []
message_count = 12.0
messages = ['280497', '281274', '326459', '326466', '326468', '326474', '326475', '326477', '326479', '326480', '326670', '328866']
nosy_count = 3.0
nosy_names = ['vstinner', 'serhiy.storchaka', 'xtreak']
pr_nums = ['9589', '10220']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue28655'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']
    @serhiy-storchaka
    Copy link
    Member Author

    Following tests altered the execution environment in isolated mode:

    test_asyncio
    test_ctypes
    test_email
    test_idle
    test_import
    test_importlib
    test_json
    test_lib2to3

    $ ./python -I -S -m test.regrtest -vv test_asyncio test_ctypes test_email test_idle test_import test_importlib test_json test_lib2to3 >/dev/null
Warning -- sys.path was modified by test_asyncio
  Before: (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])
  After:  (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/home/serhiy/py/cpython3.6-debug/Lib', '/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug']) 
Warning -- sys.path was modified by test_ctypes
  Before: (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])
  After:  (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/home/serhiy/py/cpython3.6-debug/Lib', '/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug']) 
Warning -- sys.path was modified by test_email
  Before: (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])
  After:  (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/home/serhiy/py/cpython3.6-debug/Lib', '/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug']) 
Warning -- sys.path was modified by test_idle
  Before: (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])
  After:  (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/home/serhiy/py/cpython3.6-debug/Lib', '/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug']) 
Warning -- files was modified by test_import
  Before: []
  After:  ['@test_30748_tmp.pyc'] 
Warning -- sys.path was modified by test_importlib
  Before: (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])
  After:  (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/home/serhiy/py/cpython3.6-debug/Lib', '/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug']) 
Warning -- sys.path was modified by test_json
  Before: (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])
  After:  (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/home/serhiy/py/cpython3.6-debug/Lib', '/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug']) 
/home/serhiy/py/cpython3.6-debug/Lib/lib2to3/tests/test_parser.py:393: UserWarning: ParseError on file /home/serhiy/py/cpython3.6-debug/Lib/lib2to3/main.py (bad input: type=22, value='=', context=('', (130, 38)))
  warnings.warn('ParseError on file %s (%s)' % (filepath, err))
/home/serhiy/py/cpython3.6-debug/Lib/lib2to3/tests/test_parser.py:393: UserWarning: ParseError on file /home/serhiy/py/cpython3.6-debug/Lib/lib2to3/tests/pytree_idempotency.py (bad input: type=22, value='=', context=('', (49, 33)))
  warnings.warn('ParseError on file %s (%s)' % (filepath, err))
Warning -- sys.path was modified by test_lib2to3
  Before: (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])
  After:  (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/home/serhiy/py/cpython3.6-debug/Lib', '/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])

    @serhiy-storchaka serhiy-storchaka added 3.7 (EOL) end of life tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error labels Nov 10, 2016
    @serhiy-storchaka
    Copy link
    Member Author

    Most warnings are fixed in bpo-19398. The only warnings are left in test_import and test_lib2to3.

    $ ./python -I -S -m test.regrtest -vv test_import test_lib2to3 >/dev/null
Warning -- files was modified by test_import
  Before: []
  After:  ['@test_15631_tmp.pyc'] 
/home/serhiy/py/cpython3.6/Lib/lib2to3/tests/test_parser.py:393: UserWarning: ParseError on file /home/serhiy/py/cpython3.6/Lib/lib2to3/main.py (bad input: type=22, value='=', context=('', (130, 38)))
  warnings.warn('ParseError on file %s (%s)' % (filepath, err))
/home/serhiy/py/cpython3.6/Lib/lib2to3/tests/test_parser.py:393: UserWarning: ParseError on file /home/serhiy/py/cpython3.6/Lib/lib2to3/tests/pytree_idempotency.py (bad input: type=22, value='=', context=('', (49, 33)))
  warnings.warn('ParseError on file %s (%s)' % (filepath, err))

    @tirkarthi
    Copy link
    Member

    The dependency bpo-31837 dealing with test_lib2to3 failure was resolved by Victor in bpo-30117 and hence I marked it as a duplicate. The other issue regarding test_import is not reproducible on master as of 2aaf98c. test_import warning was reproducible with 14e976e and it seems to have been fixed later. The same issue with test_import was reported by Victor in bpo-30834 which was closed as outdated since it was not reproducible. I will try to bisect to the commit that fixed test_import. I propose closing this as outdated as part of triaging.

    Thanks much Serhiy and Victor for the reports and fixes.

    @vstinner
    Copy link
    Member

    I confirm that "./python -I -S -m test.regrtest -j0 --fail-env-changed test_asyncio test_ctypes test_email test_idle test_import test_importlib test_json test_lib2to3" now pass on master. I close the issue.

    Thanks Karthikeyan ;-)

    @serhiy-storchaka
    Copy link
    Member Author

    test_import still fails on 3.6.

    @vstinner
    Copy link
    Member

    test_import still fails on 3.6.

    Oh, I only tested master. I wrote PR 9589 to fix test_import on 3.6.

    @vstinner vstinner reopened this Sep 26, 2018
    @tirkarthi
    Copy link
    Member

    Thanks Serhiy. Thanks Victor for the fix. I can confirm this on 3.6. Interestingly --fail-env-changed flag didn't catch this but running in verbose mode failed.

    --fail-env-changed passes

    ➜ cpython git:(3707bcf) ./python.exe -I -S -m test.regrtest -j0 --fail-env-changed test_asyncio test_ctypes test_email test_idle test_import test_importlib test_json test_lib2to3
    Run tests in parallel using 6 child processes
    0:00:04 load avg: 3.51 [1/8] test_import passed
    0:00:07 load avg: 3.51 [2/8] test_ctypes passed
    0:00:09 load avg: 3.87 [3/8] test_idle passed
    0:00:19 load avg: 6.23 [4/8] test_importlib passed
    0:00:27 load avg: 6.05 [5/8] test_json passed
    0:00:29 load avg: 5.72 [6/8] test_email passed
    running: test_asyncio (59 sec 815 ms), test_lib2to3 (51 sec 972 ms)
    0:01:06 load avg: 4.33 [7/8] test_asyncio passed (1 min 4 sec) -- running: test_lib2to3 (58 sec 276 ms)
    0:01:09 load avg: 4.33 [8/8] test_lib2to3 passed (1 min 244 ms)

    == Tests result: SUCCESS ==

    All 8 tests OK.

    # warnings are present

    ➜ cpython git:(3707bcf) ./python.exe -I -S -m test.regrtest -vv test_import test_lib2to3 >/dev/null
    Warning -- files was modified by test_import
    Before: []
    After: ['@test_10093_tmp.pyc']

    Total duration: 1 min 9 sec
    Tests result: SUCCESS

    @tirkarthi
    Copy link
    Member

    Verified the fix in the PR and there are no warnings

    ➜ cpython git:(pr_9589) ./python.exe
    Python 3.6.6+ (heads/pr_9589:844abda318, Sep 26 2018, 20:59:26)
    [GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    ➜ cpython git:(pr_9589) ./python.exe -I -S -m test.regrtest -vv test_import test_lib2to3 >/dev/null
    ➜ cpython git:(pr_9589) ./python.exe -I -S -m test.regrtest -j0 --fail-env-changed test_asyncio test_ctypes test_email test_idle test_import test_importlib test_json test_lib2to3
    Run tests in parallel using 6 child processes
    0:00:06 load avg: 3.19 [1/8] test_import passed
    0:00:10 load avg: 3.73 [2/8] test_ctypes passed
    0:00:12 load avg: 3.73 [3/8] test_idle passed
    0:00:20 load avg: 4.08 [4/8] test_importlib passed
    0:00:27 load avg: 4.07 [5/8] test_json passed
    0:00:29 load avg: 4.15 [6/8] test_email passed
    running: test_asyncio (59 sec 700 ms), test_lib2to3 (49 sec 60 ms)
    0:01:04 load avg: 3.97 [7/8] test_asyncio passed (1 min 2 sec) -- running: test_lib2to3 (54 sec 18 ms)
    0:01:07 load avg: 3.97 [8/8] test_lib2to3 passed (55 sec 380 ms)

    == Tests result: SUCCESS ==

    All 8 tests OK.

    Total duration: 1 min 7 sec
    Tests result: SUCCESS

    Thanks

    @vstinner
    Copy link
    Member

    I can confirm this on 3.6. Interestingly --fail-env-changed flag didn't catch this but running in verbose mode failed.

    Oh... When using -j0, regrtest doesn't spawn worker processes using -I: the flag is simply omitted... As a side effect, forget() in test_import doesn't remove pyc_file from the current directory.

    I created bpo-34812 to track this issue.

    @vstinner
    Copy link
    Member

    New changeset 43500a5 by Victor Stinner in branch '3.6':
    bpo-28655: Fix test_import.test_missing_source_legacy() (GH-9589)
    43500a5

    @tirkarthi
    Copy link
    Member

    I guess this can be safely closed as fixed since 3.6 changes were merged and I verified the PR changes locally at msg326477. The issue regarding using -I along with -j0 is tracked at bpo-34812 .

    Thanks Serhiy and Victor for the fixes :)

    @vstinner
    Copy link
    Member

    New changeset 4687bc9 by Victor Stinner (Maite Giménez) in branch '3.6':
    bpo-28655: Fix test bdb for isolate mode (GH-10220)
    4687bc9

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.7 (EOL) end of life tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @vstinner @serhiy-storchaka @tirkarthi