classification
Title: add os.fspath()
Type: enhancement Stage: resolved
Components: Versions: Python 3.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: brett.cannon Nosy List: Jelle Zijlstra, brett.cannon, buchuki, ethan.furman, martin.panter, python-dev, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2016-06-02 18:22 by ethan.furman, last changed 2016-07-04 17:27 by brett.cannon. This issue is now closed.

Files
File name Uploaded Description Edit
issue27186-fsencode.buchuki.patch buchuki, 2016-06-02 23:05 review
issue27186.patch Jelle Zijlstra, 2016-06-02 23:17 patch to add os.fspath and PyOS_FsPath review
issue27186-DirEntry-fspath.patch Jelle Zijlstra, 2016-06-02 23:59 adds fspath support to os.DirEntry review
issue27186-pathlib.buchuki.patch buchuki, 2016-06-03 00:23
issue27186-glossary.buchuki.patch buchuki, 2016-06-03 18:32 path-like glossary entry
issue27186-DirEntry-fspath.patch Jelle Zijlstra, 2016-06-04 19:40 patch to add fspath support to os.DirEntry review
issue27186-os_path_t.patch Jelle Zijlstra, 2016-06-04 23:45 patch for the path_t converter in os review
Messages (42)
msg266971 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-02 22:06
New changeset 59a52a9dd0dc by Ethan Furman in branch 'default':
issue27186 -- initial docs, tests, and python version of os.fspath
https://hg.python.org/cpython/rev/59a52a9dd0dc
msg266979 - (view) Author: Dusty Phillips (buchuki) * Date: 2016-06-02 22:50
Make os.fsencode and os.fsdecode able to accept a PathLike by calling into fspath. Additionally adds test for PathLike objects.
msg266982 - (view) Author: Dusty Phillips (buchuki) * Date: 2016-06-02 23:05
Test __fspath__ returning bytes as well.
msg266984 - (view) Author: Jelle Zijlstra (Jelle Zijlstra) * Date: 2016-06-02 23:17
This patch adds the C implementation (copied from the PEP).

Some notes:
- I added a new .h file in Include/ because there didn't seem to be an obvious existing place to put it.
- There was some uncertainty about whether we should Py_INCREF the string or bytes returned when a string/bytes is passed in. Running test -l and -L works with the Py_INCREF in and we're adding a new reference to the module, so I think this is correct.
msg267004 - (view) Author: Dusty Phillips (buchuki) * Date: 2016-06-03 00:23
This patch adds fspath protocol and constructor initialization to pathlib.Path.
msg267105 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-06-03 15:01
Note:  My schedule changed -- I won't be at the sprints today, but tomorrow.  I'll finish reviewing these patches and commit this afternoon if no one beats me to it.
msg267127 - (view) Author: Dusty Phillips (buchuki) * Date: 2016-06-03 18:26
Adding a glossary entry for path-like. The references are all correct except:

:class:`os.PathLike` doesn't link to anything because PathLike hasn't been added to the os module yet. Similarly, :meth:`__fspath__` does not link to anything, although I think this is acceptable.
msg267285 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-06-04 17:33
Dusty:  The pathlib tests looks like it's only testing the PurePath implementation; while this /should/ be sufficient, please add tests for the other classes as well.
msg267293 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-04 19:06
New changeset 780cbe18082e by Ethan Furman in branch 'default':
issue27186:  add C version of os.fspath(); patch by Jelle Zijlstra
https://hg.python.org/cpython/rev/780cbe18082e
msg267295 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-06-04 19:11
New changeset 00991aa5fdb5 by Ethan Furman in branch 'default':
issue27182: update fsencode and fsdecode for os.path(); patch by Dusty Phillips
https://hg.python.org/cpython/rev/00991aa5fdb5

(had wrong issue # in commit)
msg267299 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-06-04 19:23
Jelle:  the DirEntry patch looks good so far, but it needs a test for a bytes path.
msg267301 - (view) Author: Jelle Zijlstra (Jelle Zijlstra) * Date: 2016-06-04 19:40
Thanks for reviewing. This patch adds tests with a bytes DirEntry.
msg267306 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-04 19:50
New changeset e672cf63d08a by Ethan Furman in branch 'default':
issue27186: add PathLike ABC
https://hg.python.org/cpython/rev/e672cf63d08a
msg267316 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-04 20:48
New changeset 76b2ddaee6bd by Ethan Furman in branch 'default':
issue27186: fix fsencode/fsdecode and update tests; patch by Jelle Zijlstra
https://hg.python.org/cpython/rev/76b2ddaee6bd
msg267322 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-04 21:39
New changeset 254125a265d2 by Ethan Furman in branch 'default':
issue27186: add open/io.open; patch by Jelle Zijlstra
https://hg.python.org/cpython/rev/254125a265d2
msg267327 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-06-04 21:46
Jelle:  We still need os.open if you would like to work on that.  :)
msg267328 - (view) Author: Jelle Zijlstra (Jelle Zijlstra) * Date: 2016-06-04 21:53
Sure, I'll do that too. Now that os.PathLike and PyOS_FSPath exist, it should also be possible to add support to os.path.
msg267336 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-06-04 22:37
os.path is actually two different modules:  posixpath.py and ntpath.py

posixpath.py is being tracked in issue26027

ntpath.py is being tracked in issue27184
msg267348 - (view) Author: Jelle Zijlstra (Jelle Zijlstra) * Date: 2016-06-04 23:45
This patch adds support for fspath to a number of functions in the os module by augmenting the path_t argument converter.

The tests only cover a subset of the functions that use path_t, because some (e.g., unlink) have destructive side effects.
msg267429 - (view) Author: Jelle Zijlstra (Jelle Zijlstra) * Date: 2016-06-05 16:36
I moved my patch for the path_t converter to issue26027, which already covers posixmodule.c. I'm creating issue27231 to track adding os.fspath() support to posixpath.py. With that, I think this issue is done.
msg267465 - (view) Author: Dusty Phillips (buchuki) * Date: 2016-06-05 19:43
Ethan: Can you clarify what you mean by "testing the other classes"? PureWindowsPath and PurePosixPath are tested by extension of _BasePurePathTest. So I think you mean that _BasePathTest also needs testing, but I don't see anything in there that would mirror the constructor testing that is happening in PurePath.
msg268060 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-06-09 21:37
Re-opening as there are several patches that got put in here that have not been applied (e.g. glossary entry, os.DirEntry, etc.).
msg268062 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-06-09 22:01
I'm starting to catch up on everything you all did for PEP 519 and I wanted to say thanks! It looks like everything that needs to be done has been committed, has a patch, or just needs docs. The only thing I needed to do post-commit is tweak the docstrings to be PEP 8 compatible and reword some documention and docstrings.
msg268070 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-09 23:59
New changeset cec1f00c538d by Brett Cannon in branch 'default':
Issue #27186: Document PyOS_FSPath().
https://hg.python.org/cpython/rev/cec1f00c538d
msg268146 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-10 19:34
New changeset a5a013ca5687 by Brett Cannon in branch 'default':
Issue #27186: Add os.PathLike support to pathlib.
https://hg.python.org/cpython/rev/a5a013ca5687
msg268158 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-10 21:37
New changeset 5a62d682636e by Brett Cannon in branch 'default':
Issue #27186: Add os.PathLike support to DirEntry
https://hg.python.org/cpython/rev/5a62d682636e
msg268329 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-06-12 05:49
Test_fspath_protocol_bytes() (added in revision 5a62d682636e) fails on Windows:

http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%203.x/builds/7777/steps/test/logs/stdio
======================================================================
ERROR: test_fspath_protocol_bytes (test.test_os.TestScandir)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_os.py", line 2958, in test_fspath_protocol_bytes
    bytes_entry = self.create_file_entry(name=bytes_filename)
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_os.py", line 2932, in create_file_entry
    return self.get_entry(os.path.basename(filename))
  File "C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_os.py", line 2923, in get_entry
    entries = list(os.scandir(path))
TypeError: os.scandir() doesn't support bytes path on Windows, use Unicode instead
msg268393 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-12 18:11
New changeset 6a35aa1995ab by Brett Cannon in branch 'default':
Issue #27186: skip bytes path test for os.scandir() on Windows
https://hg.python.org/cpython/rev/6a35aa1995ab
msg268394 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-06-12 18:11
Thanks for the notice, Martin!
msg268399 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-12 20:21
New changeset 1ccd6196115a by Brett Cannon in branch 'default':
Issue #27186: add Include/osmodule.h to the proper build rules
https://hg.python.org/cpython/rev/1ccd6196115a
msg268414 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-06-13 03:06
Should the skip logic perhaps check for sys.platform == "win32" instead? The buildbots are still failing.
msg268415 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-13 03:38
New changeset f384c5c14488 by Martin Panter in branch 'default':
Issue #27186: Skip scandir(bytes) test with os.name == "nt"
https://hg.python.org/cpython/rev/f384c5c14488
msg268433 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2016-06-13 10:23
I went with os.name == "nt", which is what other scandir() tests use. I’m not sure there is a practical different. Anyway the buildbots seem happier now.
msg268457 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-06-13 17:39
Thanks for catching my screw-up, Martin; I misread the checks in the file by noticing the "nt" bit but not picking up it was comparing against sys.name instead of sys.platform.
msg268628 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-06-15 19:25
os.fspath() will be changed to ensure the output of calling obj.__fspath__() is a str or bytes object.

So the final behavior of calling os.fspath() will be to return a str or bytes or to raise an exception.

I'll update the code for this change, as well is the places in the stdlib that do/should be using os.fspath().
msg269204 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-24 19:03
New changeset ea7b6a7827a4 by Brett Cannon in branch 'default':
Issue #27186: Update os.fspath()/PyOS_FSPath() to check the return
https://hg.python.org/cpython/rev/ea7b6a7827a4
msg269205 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-06-24 19:06
And now that I have updated os.fspath() I realize that Ethan said he was going to implement the update. :( If you started on the work, Ethan, I'm really sorry for duplicating your (potential) work.

I'm going to commit Dusty's glossary term next and then go through and update the docs to use it. At that point that will leave updating os.path as the last explicit step in PEP 519 that needs to be short of updating "What's New". We're getting there. :)
msg269207 - (view) Author: Roundup Robot (python-dev) Date: 2016-06-24 19:21
New changeset 9c57178f13dc by Brett Cannon in branch 'default':
Issue #27186: Define what a "path-like object" is.
https://hg.python.org/cpython/rev/9c57178f13dc
msg269208 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-06-24 19:28
I think with the glossary change, this issue is done! Thanks everyone for the help with getting this far!

Now on to os.path...
msg269209 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2016-06-24 19:33
Brett, no worries.  My time has been extremely limited.

I'll get the other locations in the stdlib fixed sometime in the next two months if no one beats me to it.
msg269795 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-07-04 16:41
Maybe merge Include/osmodule.h and Modules/posixmodule.h?
msg269800 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2016-07-04 17:27
I'm fine with merging the two files if you want to do it. Build rules will also need updating if they do get merged.
History
Date User Action Args
2016-07-04 17:27:39brett.cannonsetmessages: + msg269800
2016-07-04 16:41:02serhiy.storchakasetmessages: + msg269795
2016-06-24 19:33:23ethan.furmansetmessages: + msg269209
2016-06-24 19:28:56brett.cannonsetstatus: open -> closed
resolution: fixed
messages: + msg269208

stage: commit review -> resolved
2016-06-24 19:21:55python-devsetmessages: + msg269207
2016-06-24 19:06:26brett.cannonsetmessages: + msg269205
2016-06-24 19:03:52python-devsetmessages: + msg269204
2016-06-15 19:25:56ethan.furmansetmessages: + msg268628
2016-06-13 17:39:05brett.cannonsetmessages: + msg268457
2016-06-13 10:23:17martin.pantersetmessages: + msg268433
2016-06-13 03:38:24python-devsetmessages: + msg268415
2016-06-13 03:06:44martin.pantersetmessages: + msg268414
2016-06-12 20:21:30python-devsetmessages: + msg268399
2016-06-12 18:11:49brett.cannonsetmessages: + msg268394
2016-06-12 18:11:27python-devsetmessages: + msg268393
2016-06-12 05:49:45martin.pantersetnosy: + martin.panter
messages: + msg268329
2016-06-10 21:40:49brett.cannonsetstage: resolved -> commit review
2016-06-10 21:40:43brett.cannonsetresolution: fixed -> (no value)
2016-06-10 21:37:29python-devsetmessages: + msg268158
2016-06-10 19:34:05python-devsetmessages: + msg268146
2016-06-09 23:59:26python-devsetmessages: + msg268070
2016-06-09 22:01:18brett.cannonsetmessages: + msg268062
2016-06-09 21:37:57brett.cannonsetstatus: closed -> open
assignee: brett.cannon
messages: + msg268060
2016-06-07 07:39:49berker.peksagsetstage: test needed -> resolved
2016-06-05 19:43:17buchukisetmessages: + msg267465
2016-06-05 16:36:55Jelle Zijlstrasetstatus: open -> closed
resolution: fixed
messages: + msg267429
2016-06-04 23:45:41Jelle Zijlstrasetfiles: + issue27186-os_path_t.patch

messages: + msg267348
2016-06-04 22:37:55ethan.furmansetmessages: + msg267336
2016-06-04 21:53:36Jelle Zijlstrasetmessages: + msg267328
2016-06-04 21:46:23ethan.furmansetmessages: + msg267327
2016-06-04 21:39:15python-devsetmessages: + msg267322
2016-06-04 20:48:17python-devsetmessages: + msg267316
2016-06-04 19:50:04python-devsetmessages: + msg267306
2016-06-04 19:40:32Jelle Zijlstrasetfiles: + issue27186-DirEntry-fspath.patch

messages: + msg267301
2016-06-04 19:23:44ethan.furmansetmessages: + msg267299
2016-06-04 19:11:15ethan.furmansetmessages: + msg267295
2016-06-04 19:06:53python-devsetmessages: + msg267293
2016-06-04 17:33:17ethan.furmansetmessages: + msg267285
2016-06-03 18:32:16buchukisetfiles: + issue27186-glossary.buchuki.patch
2016-06-03 18:31:52buchukisetfiles: - issue27186-glossary.buchuki.patch
2016-06-03 18:26:22buchukisetfiles: + issue27186-glossary.buchuki.patch

messages: + msg267127
2016-06-03 15:01:01ethan.furmansetmessages: + msg267105
2016-06-03 00:25:16buchukisetfiles: - issue27186.buchuki.patch
2016-06-03 00:23:41buchukisetfiles: + issue27186-pathlib.buchuki.patch

messages: + msg267004
2016-06-02 23:59:11Jelle Zijlstrasetfiles: + issue27186-DirEntry-fspath.patch
2016-06-02 23:17:22Jelle Zijlstrasetfiles: + issue27186.patch
nosy: + Jelle Zijlstra
messages: + msg266984

2016-06-02 23:05:06buchukisetfiles: + issue27186-fsencode.buchuki.patch

messages: + msg266982
2016-06-02 22:50:35buchukisetfiles: + issue27186.buchuki.patch

nosy: + buchuki
messages: + msg266979

keywords: + patch
2016-06-02 22:06:53python-devsetnosy: + python-dev
messages: + msg266971
2016-06-02 18:23:01ethan.furmanlinkissue27182 dependencies
2016-06-02 18:22:21ethan.furmancreate