classification
Title: distutils tests fail with recent 3.7 branch
Type: Stage: resolved
Components: Distutils Versions: Python 3.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: doko, dstufft, eric.araujo, ncoghlan, ned.deily, serhiy.storchaka, steve.dower, vstinner, xtreak
Priority: normal Keywords: 3.7regression

Created on 2018-09-26 08:06 by doko, last changed 2019-06-17 07:37 by ned.deily. This issue is now closed.

Messages (12)
msg326428 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2018-09-26 08:06
two distutils tests fail when running the tests from an installed location. 3.7 branch 20180925, succeeded with a 20180911 snapshot. Is there any locale setting required?

======================================================================
ERROR: test_non_ascii (distutils.tests.test_log.TestLog) (errors='surrogateescape')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.7/distutils/tests/test_log.py", line 26, in test_non_ascii
    log.debug('Dεbug\tMėssãge')
  File "/usr/lib/python3.7/distutils/log.py", line 41, in debug
    self._log(DEBUG, msg, args)
  File "/usr/lib/python3.7/distutils/log.py", line 34, in _log
    stream.write('%s\n' % msg)
  File "/usr/lib/python3.7/encodings/cp437.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_map)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u0117' in position 7: character maps to <undefined>

======================================================================
ERROR: test_find_executable (distutils.tests.test_spawn.SpawnTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.7/distutils/tests/test_spawn.py", line 95, in test_find_executable
    rv = find_executable(program)
  File "/usr/lib/python3.7/distutils/spawn.py", line 176, in find_executable
    path = os.environ['PATH']
  File "/usr/lib/python3.7/os.py", line 678, in __getitem__
    raise KeyError(key) from None
KeyError: 'PATH'

----------------------------------------------------------------------
msg326433 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-09-26 08:46
The test in distutils/tests/test_log.py matches the current code (with changes added in issue34421), but the code in distutils/log.py doesn't match the current code. Looks like the stdlib and tests are unsynchronized.
msg326434 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-09-26 08:57
I have reproduced this error by checking out the revision before issue34421 changes.

git checkout 77b92b15a5e5c84b91d3fd9d02f63db432fa8903~ Lib/distutils/log.py
./python -m test -v -m test_log test_distutils
msg326438 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-09-26 10:13
Test case failure regarding test_find_executable was fixed with issue34530 on master and backported.

Thanks
msg326440 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2018-09-26 10:57
issue34530 was already fixed on 20180904, so there's something else...
msg326442 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-09-26 11:18
I don't understand this bug report.

"./python -m test test_distutils -v -m test_non_ascii" pass in 3.6, 3.7 and master branches.

> two distutils tests fail when running the tests from an installed location. 3.7 branch 20180925, succeeded with a 20180911 snapshot.

I don't know what are these branches.

I tested the v3.7.0 test: "./python -m test test_distutils -v -m test_non_ascii" pass as well. It seems like you tested something between 3.7.0 and the current 3.7 branch.

I suggest you to wait for the 3.7.1 release which should be release next weeks.

--

distutils/log.py evolved a lot. The code now uses:

            try:
                stream.write('%s\n' % msg)
            except UnicodeEncodeError:
                # emulate backslashreplace error handler
                encoding = stream.encoding
                msg = msg.encode(encoding, "backslashreplace").decode(encoding)
                stream.write('%s\n' % msg)

Your traceback shows a stream.write() call on line 34, which isn't the case on the current 3.7 branch. So you are testing an outdated 3.7 commit.
msg326443 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-09-26 11:19
Hum, to be clear, I suggest to close the issue (FIXED), since both failures have been fixed in development branches.
msg326450 - (view) Author: Matthias Klose (doko) * (Python committer) Date: 2018-09-26 11:58
well, it was top of the branch yesterday
msg326462 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-09-26 12:54
> well, it was top of the branch yesterday

Hum, I don't understand because we don't have the same line number:

  File "/usr/lib/python3.7/distutils/log.py", line 34, in _log
    stream.write('%s\n' % msg)

$ git show bbdf8723324e31675f298dd273733cc13e1518df:Lib/distutils/log.py|gvim -

(...)
            try:
                stream.write('%s\n' % msg)
            except UnicodeEncodeError:
                # emulate backslashreplace error handler
                encoding = stream.encoding   # <~~~ line 34
                msg = msg.encode(encoding, "backslashreplace").decode(encoding)
                stream.write('%s\n' % msg)
msg326521 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-09-27 05:41
@doko As an update 3.7.1rc1 is available . The fixes should be there. I think they are worth checking out to see if the cases still fail or if you can add the links for the branches 20180925 and 20180911 snapshot. I can do a clean rebuild of these branches to test them.

3.7.rc1 announcement : https://mail.python.org/pipermail/python-dev/2018-September/155329.html

Thanks
msg327263 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2018-10-07 04:28
@doko, Is this still reproducible with either 3.7.1rc1 or current 3.7 head? Otherwise, let's consider closing it.
msg345806 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2019-06-17 07:37
Since there has been no followup on this, I assume it is no longer a problem for anyone and am closing.
History
Date User Action Args
2019-06-17 07:37:32ned.deilysetstatus: open -> closed
resolution: out of date
messages: + msg345806

stage: resolved
2018-10-07 04:28:22ned.deilysetnosy: + ned.deily
messages: + msg327263
2018-09-27 05:41:41xtreaksetmessages: + msg326521
2018-09-26 12:54:05vstinnersetmessages: + msg326462
2018-09-26 11:58:00dokosetmessages: + msg326450
2018-09-26 11:19:55vstinnersetmessages: + msg326443
2018-09-26 11:18:54vstinnersetmessages: + msg326442
2018-09-26 10:57:11dokosetmessages: + msg326440
2018-09-26 10:13:24xtreaksetmessages: + msg326438
2018-09-26 10:06:34xtreaksetnosy: + xtreak
2018-09-26 08:57:26serhiy.storchakasetmessages: + msg326434
2018-09-26 08:46:08serhiy.storchakasetmessages: + msg326433
2018-09-26 08:10:46dokosetnosy: + serhiy.storchaka, steve.dower
2018-09-26 08:06:20dokocreate