Meta issue to track adding PEP-519 support in the various stdlib modules.

posix module: bpo-26027

importlib: bpo-26667

nt module: bpo-27184

os.fspath(): bpo-27186

Isn't the nt module just an alias of the posix module?

Nope.

There is a posixpath.py and an ntpath.py, and they are not the same.

FYI, I'm working on a patch for builtins.open to call PyOS_FSPath.

This patch makes the Python and C versions of open()/io.open() support the fspath protocol.

Sorry, Serhiy, I had my module names mixed up.

nt and posix are the same, but ntpath and posixpath are not (I may have those first two names wrong again, but hopefully you get the idea).

New changeset 00991aa5fdb5 by Ethan Furman in branch 'default':
bpo-27182: update fsencode and fsdecode for os.path(); patch by Dusty Phillips
https://hg.python.org/cpython/rev/00991aa5fdb5

In the patch that was just committed, the path_type variables are undefined (os.py lines 892 and 908). There is also no test for these error cases—we should test the cases where the path has no __fspath__ or does not return a bytes or str.

Attached patch fixes the undefined variable and adds additional tests.

Currently, os.fspath will raise an exception if the thing passed in is not str/bytes/PathLike, and that error message will proclaim that str or bytes or PathLike is required; however, this is not true in cases such as Path (which doesn't allow bytes), and incomplete in cases such as os.open (which also allows ints).

On the other hand, if the thing has a functional __fspath__ (meaning calling it doesn't raise an exception) then os.fspath will return whatever that method returns, which could be complete garbage.

So os.fspath is being too strict, too open, and too lax all at the same time.

Given Guido's reluctance to check the output of __fspath__(), plus the current difficulty of painless integration with existing functions, I think we should have os.fspath() only raise an exception if obj.__fspath__ exists and calling it raises an exception, otherwise we return the result of calling obj.__fspath__(), or obj if it doesn't have __fspath__.

In case that wasn't clear, attached is a unit test that passes when the above changes are implemented.

If we do that, then os.* functions that accept fds would also work on objects whose __fspath__ method returns an integer. I don't think that is desirable (I was just writing a test to ensure that fspath returning an integer throws an error).

Functions that only accept file descriptors should not be updated to work with __fspath__() as it will never return an int/fd.

As for Ethan's suggestion, are you saying you want to toss the str/bytes check from os.fspath()? If so then you will need to go to python-dev and bring that up as the PEP clearly specifies that str/bytes is checked for and specifically in the order of the Python code. The thinking behind the current design is that since __fspath__() has to be explicitly implemented that people will do so properly, versus accidentally passing in some type that isn't str/bytes like the pre-PEP 519 world (i.e. trust the __fspath__() implementors to do the right thing and only protect against someone passing in something wrong from complicated code flow).

There has been discussion about using the path.__fspath__() if hasattr(path, '__fspath__') else path idiom in os.path so that the pre-existing type-checks can do their thing instead of checking twice, although that's different from how os.fspath() works (then again, since this is all new code we could argue that going our own route in os.path is acceptable in the name of performance).

New changeset 6239673d5e1d by Brett Cannon in branch 'default':
Issue bpo-27182: Document os.PathLike.
https://hg.python.org/cpython/rev/6239673d5e1d

Just a quick update: between the patches for issue bpo-26027 and issue bpo-26667, the necessary code to make os.path work with path-like objects is done. At this point I'm just waiting for code reviews on those patches.

Here is an odd patch because I don't know where else to put it ATM that adds the remaining support in the os module/package not covered by other issues w/ patches (specifically os.walk() and os.fwalk()). I think everything else simply falls through thanks to os.path and path_converter.

The os and os.path modules are now done! The means PEP-519 is finished. At this point individual modules will need to be checked to see if they do (not) support os.PathLike.

New changeset 3417d324cbf9 by Brett Cannon in branch 'default':
Issue bpo-27182: Add support for path-like objects to PyUnicode_FSDecoder().
https://hg.python.org/cpython/rev/3417d324cbf9