classification
Title: Got deprecation warning when running test_shutil.py on Windows NT 6
Type: behavior Stage: resolved
Components: Tests, Windows Versions: Python 3.4, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: serhiy.storchaka Nosy List: BreamoreBoy, ezio.melotti, python-dev, serhiy.storchaka, vajrasky, zach.ware
Priority: normal Keywords: easy, patch

Created on 2013-12-23 14:52 by vajrasky, last changed 2014-08-07 16:41 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
silent_warning_shutil_rmtree.patch vajrasky, 2014-08-06 14:33 review
Messages (11)
msg206861 - (view) Author: Vajrasky Kok (vajrasky) * Date: 2013-12-23 14:52
You don't have to be an administrator get this deprecation warning. I am not sure whether showing the deprecation warning is intended behaviour or not.

C:\Users\vajrasky\Code\cpython>PCbuild\python.exe Lib\test\test_shutil.py
..s..........s..s....ss...s..ss.ss..ssssssss....s.ss.ss..........ss.C:\Users\vaj
rasky\Code\cpython\lib\ntpath.py:309: DeprecationWarning: The Windows bytes API
has been deprecated, use Unicode filenames instead
  st = os.lstat(path)
C:\Users\vajrasky\Code\cpython\lib\shutil.py:357: DeprecationWarning: The Window
s bytes API has been deprecated, use Unicode filenames instead
  names = os.listdir(path)
C:\Users\vajrasky\Code\cpython\lib\shutil.py:363: DeprecationWarning: The Window
s bytes API has been deprecated, use Unicode filenames instead
  mode = os.lstat(fullname).st_mode
C:\Users\vajrasky\Code\cpython\lib\shutil.py:370: DeprecationWarning: The Window
s bytes API has been deprecated, use Unicode filenames instead
  os.unlink(fullname)
C:\Users\vajrasky\Code\cpython\lib\shutil.py:374: DeprecationWarning: The Window
s bytes API has been deprecated, use Unicode filenames instead
  os.rmdir(path)
.sss.............
----------------------------------------------------------------------
Ran 85 tests in 0.741s

OK (skipped=28)
msg224459 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-08-01 00:23
The deprecation warnings are coming from the various os calls so are nothing to do with test_shutil.  I think this can be closed as "not a bug".
msg224783 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2014-08-04 23:16
Even if the problem is not in test_shutil, those warnings shouldn't be there.
Vajrasky, can you check if you can still reproduce this issue, and see if you can fix it (either by converting the filenames to unicode, or by silencing the warnings)?
msg224837 - (view) Author: Vajrasky Kok (vajrasky) * Date: 2014-08-05 14:46
Okay, here is the problem. In Windows, you get DeprecationWarning if you pass bytes object to method such as os.lstat.

foo.py
======
import os
os.lstat(b"C:\\Users\\vajrasky\\Code\\cpython\\python.bat")

C:\Users\vajrasky\Code\cpython>python.bat -Wdefault foo.py
foo.py:2: DeprecationWarning: The Windows bytes API has been deprecated, use Unicode filenames instead
  os.lstat(b"C:\\Users\\vajrasky\\Code\\cpython\\python.bat")

The test that throws this error is this one:

    def test_rmtree_works_on_bytes(self):
        tmp = self.mkdtemp()
        victim = os.path.join(tmp, 'killme')
        os.mkdir(victim)
        write_file(os.path.join(victim, 'somefile'), 'foo')
        victim = os.fsencode(victim)
        self.assertIsInstance(victim, bytes)
        shutil.rmtree(victim)  => This one calls os.lstat

So I am not sure how to fix this problem. Disable the test for Windows? If we convert the path to Unicode then the test losses the meaning.
msg224838 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-08-05 14:50
Perhaps you could change the test to catch the warning and confirm its
presence on Windows?
msg224861 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-08-05 17:03
with self.assertWarns(DeprecationWarning) if windows else contextlib.ExitStack():
    shutil.rmtree(victim)
msg224935 - (view) Author: Vajrasky Kok (vajrasky) * Date: 2014-08-06 14:33
Okay, here is the patch based on Serhiy's suggestion.
msg224950 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-08-06 16:22
Is this the only place in tests which emits deprecation warnings on Windows?
msg225014 - (view) Author: Vajrasky Kok (vajrasky) * Date: 2014-08-07 14:36
In test_shutil.py, yes, this is the only place. But in other tests, we got different type of warnings, such as:

test_mailbox.py:46: RuntimeWarning: tests may fail, delete still pending for @test_4456_tmp

test_decimal:5608: UserWarning: C tests skipped: no module named _decimal.
msg225025 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014-08-07 16:40
New changeset e306cf07046a by Serhiy Storchaka in branch '3.4':
Issue #20056: Fixed deprecation warning about bytes path in test_shutil on
http://hg.python.org/cpython/rev/e306cf07046a

New changeset 8480179d2a7f by Serhiy Storchaka in branch 'default':
Issue #20056: Fixed deprecation warning about bytes path in test_shutil on
http://hg.python.org/cpython/rev/8480179d2a7f
msg225027 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-08-07 16:41
Thank you Vajrasky.
History
Date User Action Args
2014-08-07 16:41:46serhiy.storchakasetstatus: open -> closed
resolution: fixed
messages: + msg225027

stage: patch review -> resolved
2014-08-07 16:40:01python-devsetnosy: + python-dev
messages: + msg225025
2014-08-07 14:36:36vajraskysetmessages: + msg225014
2014-08-06 16:22:45serhiy.storchakasetassignee: serhiy.storchaka
messages: + msg224950
stage: patch review
2014-08-06 14:33:11vajraskysetfiles: + silent_warning_shutil_rmtree.patch
keywords: + patch
messages: + msg224935
2014-08-05 17:03:12serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg224861
2014-08-05 14:50:04zach.waresetmessages: + msg224838
2014-08-05 14:46:47vajraskysetmessages: + msg224837
versions: + Python 3.5, - Python 3.3
2014-08-04 23:16:09ezio.melottisetkeywords: + easy

messages: + msg224783
components: + Windows
2014-08-01 00:23:33BreamoreBoysetnosy: + BreamoreBoy
messages: + msg224459
2014-02-15 15:12:12ezio.melottisetnosy: + ezio.melotti, zach.ware
2013-12-23 14:52:44vajraskycreate