classification
Title: Tests altered the execution environment in isolated mode
Type: behavior Stage: resolved
Components: Tests Versions: Python 3.7, Python 3.6, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: 31837 Superseder:
Assigned To: Nosy List: serhiy.storchaka, vstinner, xtreak
Priority: normal Keywords:

Created on 2016-11-10 08:46 by serhiy.storchaka, last changed 2018-10-29 20:32 by vstinner. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 9589 merged vstinner, 2018-09-26 15:05
PR 10220 merged python-dev, 2018-10-29 17:25
Messages (12)
msg280497 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-10 08:46
Following tests altered the execution environment in isolated mode:

test_asyncio
test_ctypes
test_email
test_idle
test_import
test_importlib
test_json
test_lib2to3

$ ./python -I -S -m test.regrtest -vv test_asyncio test_ctypes test_email test_idle test_import test_importlib test_json test_lib2to3 >/dev/null
Warning -- sys.path was modified by test_asyncio
  Before: (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])
  After:  (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/home/serhiy/py/cpython3.6-debug/Lib', '/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug']) 
Warning -- sys.path was modified by test_ctypes
  Before: (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])
  After:  (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/home/serhiy/py/cpython3.6-debug/Lib', '/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug']) 
Warning -- sys.path was modified by test_email
  Before: (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])
  After:  (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/home/serhiy/py/cpython3.6-debug/Lib', '/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug']) 
Warning -- sys.path was modified by test_idle
  Before: (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])
  After:  (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/home/serhiy/py/cpython3.6-debug/Lib', '/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug']) 
Warning -- files was modified by test_import
  Before: []
  After:  ['@test_30748_tmp.pyc'] 
Warning -- sys.path was modified by test_importlib
  Before: (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])
  After:  (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/home/serhiy/py/cpython3.6-debug/Lib', '/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug']) 
Warning -- sys.path was modified by test_json
  Before: (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])
  After:  (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/home/serhiy/py/cpython3.6-debug/Lib', '/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug']) 
/home/serhiy/py/cpython3.6-debug/Lib/lib2to3/tests/test_parser.py:393: UserWarning: ParseError on file /home/serhiy/py/cpython3.6-debug/Lib/lib2to3/main.py (bad input: type=22, value='=', context=('', (130, 38)))
  warnings.warn('ParseError on file %s (%s)' % (filepath, err))
/home/serhiy/py/cpython3.6-debug/Lib/lib2to3/tests/test_parser.py:393: UserWarning: ParseError on file /home/serhiy/py/cpython3.6-debug/Lib/lib2to3/tests/pytree_idempotency.py (bad input: type=22, value='=', context=('', (49, 33)))
  warnings.warn('ParseError on file %s (%s)' % (filepath, err))
Warning -- sys.path was modified by test_lib2to3
  Before: (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])
  After:  (3071075980, ['/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'], ['/home/serhiy/py/cpython3.6-debug/Lib', '/usr/local/lib/python36.zip', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/Lib/', '/home/serhiy/py/cpython3.6-debug/build/lib.linux-i686-3.6-pydebug'])
msg281274 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-11-20 15:12
Most warnings are fixed in issue19398. The only warnings are left in test_import and test_lib2to3.

$ ./python -I -S -m test.regrtest -vv test_import test_lib2to3 >/dev/null
Warning -- files was modified by test_import
  Before: []
  After:  ['@test_15631_tmp.pyc'] 
/home/serhiy/py/cpython3.6/Lib/lib2to3/tests/test_parser.py:393: UserWarning: ParseError on file /home/serhiy/py/cpython3.6/Lib/lib2to3/main.py (bad input: type=22, value='=', context=('', (130, 38)))
  warnings.warn('ParseError on file %s (%s)' % (filepath, err))
/home/serhiy/py/cpython3.6/Lib/lib2to3/tests/test_parser.py:393: UserWarning: ParseError on file /home/serhiy/py/cpython3.6/Lib/lib2to3/tests/pytree_idempotency.py (bad input: type=22, value='=', context=('', (49, 33)))
  warnings.warn('ParseError on file %s (%s)' % (filepath, err))
msg326459 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-09-26 12:39
The dependency issue31837 dealing with test_lib2to3 failure was resolved by Victor in issue30117 and hence I marked it as a duplicate. The other issue regarding test_import is not reproducible on master as of  2aaf98c16ae3070378de523a173e29644037d8bd. test_import warning was reproducible with 14e976e00e65bf343ba0fca016c3c9132a843daf and it seems to have been fixed later. The same issue with test_import was reported by Victor in issue30834 which was closed as outdated since it was not reproducible. I will try to bisect to the commit that fixed test_import. I propose closing this as outdated as part of triaging.

Thanks much Serhiy and Victor for the reports and fixes.
msg326466 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-09-26 13:53
I confirm that "./python -I -S -m test.regrtest -j0 --fail-env-changed test_asyncio test_ctypes test_email test_idle test_import test_importlib test_json test_lib2to3" now pass on master. I close the issue.

Thanks Karthikeyan ;-)
msg326468 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-09-26 14:34
test_import still fails on 3.6.
msg326474 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-09-26 15:07
> test_import still fails on 3.6.

Oh, I only tested master. I wrote PR 9589 to fix test_import on 3.6.
msg326475 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-09-26 15:13
Thanks Serhiy. Thanks Victor for the fix. I can confirm this on 3.6. Interestingly `--fail-env-changed` flag didn't catch this but running in verbose mode failed.

`--fail-env-changed` passes

➜  cpython git:(3707bcf02b) ./python.exe -I -S -m test.regrtest -j0 --fail-env-changed test_asyncio test_ctypes test_email test_idle test_import test_importlib test_json test_lib2to3
Run tests in parallel using 6 child processes
0:00:04 load avg: 3.51 [1/8] test_import passed
0:00:07 load avg: 3.51 [2/8] test_ctypes passed
0:00:09 load avg: 3.87 [3/8] test_idle passed
0:00:19 load avg: 6.23 [4/8] test_importlib passed
0:00:27 load avg: 6.05 [5/8] test_json passed
0:00:29 load avg: 5.72 [6/8] test_email passed
running: test_asyncio (59 sec 815 ms), test_lib2to3 (51 sec 972 ms)
0:01:06 load avg: 4.33 [7/8] test_asyncio passed (1 min 4 sec) -- running: test_lib2to3 (58 sec 276 ms)
0:01:09 load avg: 4.33 [8/8] test_lib2to3 passed (1 min 244 ms)

== Tests result: SUCCESS ==

All 8 tests OK.

# warnings are present

➜  cpython git:(3707bcf02b) ./python.exe -I -S -m test.regrtest -vv test_import test_lib2to3 >/dev/null
Warning -- files was modified by test_import
  Before: []
  After:  ['@test_10093_tmp.pyc']

Total duration: 1 min 9 sec
Tests result: SUCCESS
msg326477 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-09-26 15:33
Verified the fix in the PR and there are no warnings

➜  cpython git:(pr_9589) ./python.exe
Python 3.6.6+ (heads/pr_9589:844abda318, Sep 26 2018, 20:59:26)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
➜  cpython git:(pr_9589) ./python.exe -I -S -m test.regrtest -vv test_import test_lib2to3 >/dev/null
➜  cpython git:(pr_9589) ./python.exe -I -S -m test.regrtest -j0 --fail-env-changed test_asyncio test_ctypes test_email test_idle test_import test_importlib test_json test_lib2to3
Run tests in parallel using 6 child processes
0:00:06 load avg: 3.19 [1/8] test_import passed
0:00:10 load avg: 3.73 [2/8] test_ctypes passed
0:00:12 load avg: 3.73 [3/8] test_idle passed
0:00:20 load avg: 4.08 [4/8] test_importlib passed
0:00:27 load avg: 4.07 [5/8] test_json passed
0:00:29 load avg: 4.15 [6/8] test_email passed
running: test_asyncio (59 sec 700 ms), test_lib2to3 (49 sec 60 ms)
0:01:04 load avg: 3.97 [7/8] test_asyncio passed (1 min 2 sec) -- running: test_lib2to3 (54 sec 18 ms)
0:01:07 load avg: 3.97 [8/8] test_lib2to3 passed (55 sec 380 ms)

== Tests result: SUCCESS ==

All 8 tests OK.

Total duration: 1 min 7 sec
Tests result: SUCCESS


Thanks
msg326479 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-09-26 16:07
> I can confirm this on 3.6. Interestingly `--fail-env-changed` flag didn't catch this but running in verbose mode failed.

Oh... When using -j0, regrtest doesn't spawn worker processes using -I: the flag is simply omitted... As a side effect, forget() in test_import doesn't remove pyc_file from the current directory.

I created bpo-34812 to track this issue.
msg326480 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-09-26 16:09
New changeset 43500a5907eb9ae2e470dcbffe73012cd456f5a1 by Victor Stinner in branch '3.6':
bpo-28655: Fix test_import.test_missing_source_legacy() (GH-9589)
https://github.com/python/cpython/commit/43500a5907eb9ae2e470dcbffe73012cd456f5a1
msg326670 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-09-29 07:14
I guess this can be safely closed as fixed since 3.6 changes were merged and I verified the PR changes locally at msg326477. The issue regarding using -I along with -j0 is tracked at issue34812 .

Thanks Serhiy and Victor for the fixes :)
msg328866 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-10-29 20:32
New changeset 4687bc993a275eaeb27a8b2068b128ce1f464818 by Victor Stinner (Maite Giménez) in branch '3.6':
bpo-28655: Fix test bdb for isolate mode (GH-10220)
https://github.com/python/cpython/commit/4687bc993a275eaeb27a8b2068b128ce1f464818
History
Date User Action Args
2018-10-29 20:32:25vstinnersetmessages: + msg328866
2018-10-29 17:25:52python-devsetpull_requests: + pull_request9536
2018-09-29 07:14:57xtreaksetstatus: open -> closed
resolution: fixed
messages: + msg326670
2018-09-26 16:09:42vstinnersetmessages: + msg326480
2018-09-26 16:07:24vstinnersetmessages: + msg326479
2018-09-26 15:33:24xtreaksetmessages: + msg326477
2018-09-26 15:13:34xtreaksetmessages: + msg326475
2018-09-26 15:07:24vstinnersetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg326474
2018-09-26 15:05:53vstinnersetpull_requests: + pull_request8986
2018-09-26 14:34:43serhiy.storchakasetmessages: + msg326468
2018-09-26 13:53:54vstinnersetstatus: open -> closed
resolution: fixed
messages: + msg326466

stage: resolved
2018-09-26 12:39:17xtreaksetnosy: + xtreak
messages: + msg326459
2017-10-21 17:22:48serhiy.storchakasetdependencies: + ParseError in test_all_project_files()
2016-11-20 15:12:54serhiy.storchakasetmessages: + msg281274
2016-11-10 08:46:58serhiy.storchakacreate