Title: os.readlink does not accept pathlib.Path on Windows
Type: behavior Stage: resolved
Components: Library (Lib), Windows Versions: Python 3.8, Python 3.7, Python 3.6
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: berker.peksag, eryksun, girtsf, gregory.p.smith, paul.moore, serhiy.storchaka, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: patch

Created on 2018-08-12 03:41 by girtsf, last changed 2019-10-23 20:57 by steve.dower. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8740 merged berker.peksag, 2018-08-12 05:36
Messages (9)
msg323427 - (view) Author: Girts Folkmanis (girtsf) * Date: 2018-08-12 03:41
Documentation for os.readlink says "Changed in version 3.6: Accepts a path-like object.". This works fine on macOS, but blows up on Windows:

Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:06:47) [MSC v.1914 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import pathlib
>>> os.readlink(pathlib.Path('foo'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: readlink() argument 1 must be str, not WindowsPath
msg323460 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2018-08-13 05:10
Steve and/or Eryk, I was adding some tests for os.readlink() in PR 8740 and I noticed that os.readlink() always returns str on Windows. However, the documentation for os.readlink() says:

    If the path is a bytes object (direct or indirectly), the result will
    be a bytes object.

The question is, should I add support for bytes as part of PR 8740? And what is the best way to implement it? (e.g. use CreateFileA() if PyUnicode_Check(path.object) returns false?)
msg323463 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2018-08-13 05:38
In msg235615 (Issue 9949), Zachary said using bytes paths on Windows is deprecated, but I can't see the actual conversation because Rietveld seems to be down: I think the os.readlink() documentation needs to be updated if we decide to keep the status quo.
msg323465 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-08-13 07:14
Currently the behavior doesn't match the documentation. Initially I thought that this can be solved by adding the support of path-like objects and backporting this up to 3.6. But os.readlink() on Windows doesn't not support also bytes paths, and never supported. This looks to me now more like a new feature. In 3.6 and 3.7 we can resolve this issue by just updating the documentation.

Bytes paths on Windows were deprecated before 3.6. Since implementing PEP 529, their should be supported on Windows if they are supported on other platforms. We shouldn't use an 8-bit API like CreateFileA, but instead decode bytes paths with UTF-8 and use a Unicode API (this should be implemented in path_converter()).
msg323486 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2018-08-13 16:32
Serhiy is exactly right, but to emphasise, the best way to do paths now is to use argument clinic with its path_t type. On Windows, .narrow is a flag that indicates whether you should PyUnicode_FSEncode() any results before returning them, and .wide is always initialized correctly.

Do not use any *A APIs - only *W from now on :)
msg323488 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2018-08-13 18:08
Thanks for the suggestions! I've updated PR 8740. I will submit separate PRs to fix os.readlink() documentation in 3.6 and 3.7 branches.
msg323560 - (view) Author: Berker Peksag (berker.peksag) * (Python committer) Date: 2018-08-15 10:03
New changeset e0b5b2096ead4cc094a129ce7602ac5c0e998086 by Berker Peksag in branch 'master':
bpo-34384: Fix os.readlink() on Windows (GH-8740)
msg355263 - (view) Author: Gregory P. Smith (gregory.p.smith) * (Python committer) Date: 2019-10-23 20:51
can this be closed?
msg355264 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2019-10-23 20:57
Date User Action Args
2019-10-23 20:57:52steve.dowersetstatus: open -> closed
resolution: fixed
messages: + msg355264

stage: patch review -> resolved
2019-10-23 20:51:16gregory.p.smithsetstatus: pending -> open
nosy: + gregory.p.smith
messages: + msg355263

2019-01-14 10:02:56serhiy.storchakasetstatus: open -> pending
2018-08-15 10:03:52berker.peksagsetmessages: + msg323560
2018-08-13 18:08:17berker.peksagsetmessages: + msg323488
2018-08-13 16:32:30steve.dowersetmessages: + msg323486
2018-08-13 07:14:03serhiy.storchakasetmessages: + msg323465
2018-08-13 05:38:52berker.peksagsetmessages: + msg323463
2018-08-13 05:10:20berker.peksagsetnosy: + eryksun
messages: + msg323460
2018-08-12 06:52:05serhiy.storchakasetnosy: + paul.moore, tim.golden, berker.peksag, serhiy.storchaka, zach.ware, steve.dower
components: + Windows
2018-08-12 06:36:18serhiy.storchakasetversions: + Python 3.6, Python 3.8
2018-08-12 05:36:51berker.peksagsetkeywords: + patch
stage: patch review
pull_requests: + pull_request8225
2018-08-12 03:41:11girtsfcreate