pathlib is really nice, but currently it's rather inconvenient to use due to the lack of support in other parts of the stdlib for Path objects. For historical reasons, everything accepts string paths, but few places accept Paths. As an example: configparser.ConfigParser.read() but there are lots of others.

I'm opening this bug to start a conversation about better support for Path objects in the stdlib. Against all hope, I wish there was a simple way to extend the compatibility, but I don't like having to sprinkle str(some_path) calls everywhere (kind of defeats the purpose of having the nicer pathlib API IMHO). I suspect instead that it will be a matter of adding type tests or str() conversions to the relevant methods, but there may be other issues to discuss, like is it even a good idea to do this? ;)

Since we're unlikely to ever change all the places, I'd say it's better to be consistent. I'd rather write str(path) all over the place than having to look up in the docs each time if that specific API happens to support passing Paths directly.

However, Antoine has been positive towards more utility methods on Path objects. (Not that ConfigParser read() would fall in that category :)

On Oct 06, 2014, at 03:43 PM, Georg Brandl wrote:

I'd rather write str(path) all over the place than having to look up in the
docs each time if that specific API happens to support passing Paths
directly.

Have you tried to write a large-ish application using path objects?
str-infection is definitely a disincentive to using pathlib. :/

I was about to suggest deriving your own Path class from Path and str, but got a base class layout conflict because Path objects define lots of __slots__ :(

I was about to suggest deriving your own Path class from Path and str

That would be a rather horrible solution.
It has already been suggested to create a "path protocol". I would suggest Barry tries to float and investigate the idea on python-ideas:
https://mail.python.org/pipermail/python-ideas/2014-May/027869.html

That would be a rather horrible solution.

I know :)

As for adding convenience methods to Path objects, yes, I'm quite open to that. Best is to open an issue when you have a specific idea (and a patch :-)).

I think I'm missing something here because the idea of doing path = str(path) at the API boundary for an old function to support both Path and str objects for paths seems fairly minimal. Only when manipulating a path is wanting a Path object going to come up, and in that case can't you just do path = pathlib.Path(path) instead?

I agree with Brett. Making old functions support Path sounds trivial to me, and I don't think there are any backward compatibility issues. Having to do str(path) every time you call a function is ugly. So I think that all stdlib functions that currently take a string path should be able to take a Path object.

I'll add to that the functions in the zipfile module.

path = str(path) is minimal, but has the side effect of accepting almost any object, which is definitely not what you'd like ("where did that file named '<type object at ...>' come from?!")

Fine, so you add an if isinstance... in front of it and you're done.

Making old functions support Path sounds trivial to me

We're looking forward to trivial patches that enable Path handling consistently throughout the stdlib.

Fine, so you add an if isinstance... in front of it and you're done.

Easier said than done in C modules.

Georg: You're right, I forgot about C modules which make this not as trivial as I thought.

As for Python modules: If there's general agreement that this is a feature we want, I'll be happy to make a patch for the zipfile module to accept Path arguments.

Well, if you use an isinstance check you privilege the stdlib Path over any other pathlike implementation. Since it *is* in the stdlib, this isn't an automatic reason for rejection, but it does have a bit of a code smell to it. Why should everything that deals with path strings have to have intimate knowledge of Path objects?

I originally wrote here "Maybe we need a __path__ magic method" as a half-joke, but rereading the issue I see that this has in fact been proposed seriously, and referenced by Antoine (the pathlib author).

I'm -1 on just sprinkling support for Path throughout the stdlib. Do it in a universally applicable fashion or don't do it at all, IMO.

I'm -1 on just sprinkling support for Path throughout the stdlib.
Do it in a universally applicable fashion or don't do it at all, IMO.

How about doing it per module?

Please be conservative with adding methods to Path.

FWIW, my own experiences with pathlib have shown that utility methods have the disadvantage of duplicating well-established Python idioms. This reduces code readability in my opinion. While it is desirable that the rather inconvenient os.path may be superseded by pathlib in the long run, utility methods like Path.glob, Path.open, Path.mkdir, and Path.readtext have convenient stdlib counterparts with long tradition, and using the methods seems disruptive and confusing to me.

This is a further argument for having a path protocol instead IMO.

Random idea: what if pathlib.Path defined a .path attribute that was a plain string? Then you could write p.path instead of str(p), and "if hasattr(p, 'path'): p = p.path". This would be the new protocol. Advantage is also that DirEntry (returned by the new os.scandir()) already supports it. :-)

I thought about it some more, and personally I'd prefer each function to do str(path) internally rather than if hasattr(p, 'path'): p = p.path. Even if it means we'll have to deal with "where did that file named '<type object at ...>' come from?!" errors. I think it's clumsy that the path protocol is to access path.path. We already have a protocol for getting a string out of an object and it's str(object). I think we should use it even if it makes debugging harder in the case where someone mistakenly passes a non-path object to a function that wants a path. (And without using isinstance either.)

str(object) is not a protocol for getting a string out of an object. It's a protocol for getting a string for print(). str is defined for every object and therefore is useless for getting a string out of "string-like" object (as float for floats and bytes for bytes). Perhaps we need a new special method string that relates to str as index to int.

Here the aim is really to distinguish path-like objects from other objects, not to accept arbitrary strings.

A ".path" attribute sounds like the wrong name, though: a Path has a path? And the Path's path is not a Path? Ouch :-)

I think it's actually very reasonable for a Path to have a path attribute
that's a string. The DirEntry has two string attributes: name (the last
component) and path (the full path). The Path object already has the
former. Adding the latter makes sense to me. After all you've gotta give it
*some* name, and 'path' is used (unsurprisingly) in this meaning already in
many places.

The shortest idiom in libraries wanting to support this would be

    path = gettattr(arg, 'path', arg)

This extracts the path attribute from a DirEntry or Path object, and
assumes the argument is a string otherwise. I think this is relatively
reasonable to encode in C as well.

Would location work as an attribute name?

Only if we changed DirEntry to support that too. But it's a kind of
high-falootin' word that also has some other connotations (e.g.
geographical location, and the HTTP Location header). I've never heard it
use in relation to filenames -- those are invariably called some variant of
file, file name, path, full path. Really, the argument Antoine brings up
doesn't hold much weight.

Personally I thought the name path fit; just trying to see if some other option might work that Antoine would also like.

In any case I don't think "location" is any better ;-) If "path" fits other people then good.

OK, I'll add 'path' to unblock changes to the stdlib (but I won't close
this issue).

New changeset 7e9605697dfc by Guido van Rossum in branch '3.4':
Issue bpo-22570: Add 'path' attribute to pathlib.Path objects.
https://hg.python.org/cpython/rev/7e9605697dfc

New changeset 9c49c417a68a by Guido van Rossum in branch '3.5':
Issue bpo-22570: Add 'path' attribute to pathlib.Path objects. (Merge 3.4->3.5)
https://hg.python.org/cpython/rev/9c49c417a68a

New changeset d5f96a5da219 by Guido van Rossum in branch 'default':
Issue bpo-22570: Add 'path' attribute to pathlib.Path objects. (Merge 3.5->3.6)
https://hg.python.org/cpython/rev/d5f96a5da219

No docs? ;)

New changeset 2e3c31ab586a by Guido van Rossum in branch '3.4':
Docs for issue bpo-22570.
https://hg.python.org/cpython/rev/2e3c31ab586a

New changeset 408f8b255b56 by Guido van Rossum in branch '3.5':
Docs for issue bpo-22570. (Merge 3.4->3.5)
https://hg.python.org/cpython/rev/408f8b255b56

New changeset 759b2cecc289 by Guido van Rossum in branch '3.4':
Add versionadded (3.4.5) to docs for issue bpo-22570.
https://hg.python.org/cpython/rev/759b2cecc289

New changeset 1a6b485e717f by Guido van Rossum in branch '3.5':
Add versionadded (3.4.5) to docs for issue bpo-22570. (Merge 3.4->3.5)
https://hg.python.org/cpython/rev/1a6b485e717f

New changeset eab349b5c6d7 by Guido van Rossum in branch '3.5':
Cross-reference os.DirEntry and pathlib.Path for issue bpo-22570.
https://hg.python.org/cpython/rev/eab349b5c6d7

New changeset 97ab0ccac893 by Guido van Rossum in branch 'default':
Docs for issue bpo-22570. (Merge 3.5->3.6)
https://hg.python.org/cpython/rev/97ab0ccac893

So, I added docs, mentioning the getattr(arg, 'path', arg) idiom, and (for 3.5 and 3.6) also cross-referencing with DirEntry.

I'm not sure whether to now close this issue or whether to leave it open to remind people of adding patches using the new idiom to various stdlib modules. Opinions?

Also, since pathlib is provisional, I felt okay with adding this to 3.4.5 and 3.5.2.

We could leave this open as a meta issue and spin off individual issues for any specific module people choose to update to support Path objects, setting those new issues as dependencies here.

Opened bpo-26027 for adding support in the posix module.

Should the path attribute be only string, or other types are acceptable (bytes, file descriptor, and even None if the function accepts None)?

Let's say that the path attribute should be str or bytes -- this matches
the behavior of DirEntry. (But for pathlib.Path it is always a str.) It
cannot be None or FD. But note that the getattr(x, 'path', x) idiom returns
x unchanged if x is None or an FD (or a stream, for that matter).

Here's an alternate idea I thought of now. Maybe path.path should be a pointer to the same Path object, so every function will do this:

str(arg.path) if hasattr(arg, 'path') else arg

What I like about this is that it avoids having the path argument be a string. I don't like misnomers :)

I'd understand though if people won't like this suggestion since it introduces another step, of turning the path back into a string.

I appreciate the thought, Ram, but I'm not a fan.

Using 'path' for this field is not a misnomer (example: DictEntry uses 'path' for the same field).

See issue bpo-26027 for a follow-up idea.

str(arg.path) if hasattr(arg, 'path') else arg

DirEntry.path can be bytes.

You can't win 'em all... :-)

New changeset d0c8b2c1544e by Serhiy Storchaka in branch '3.5':
Issue bpo-22570: Renamed Py_SETREF to Py_XSETREF.
https://hg.python.org/cpython/rev/d0c8b2c1544e

New changeset 719c11b6b6ff by Serhiy Storchaka in branch 'default':
Issue bpo-22570: Renamed Py_SETREF to Py_XSETREF.
https://hg.python.org/cpython/rev/719c11b6b6ff

New changeset 7197809a7428 by Serhiy Storchaka in branch '2.7':
Issue bpo-22570: Renamed Py_SETREF to Py_XSETREF.
https://hg.python.org/cpython/rev/7197809a7428

Sorry, these changesets were related to bpo-26200.

Reopening as we need to rename the path attribute to __fspath__ once Brett's PEP is accepted (which will be soon). https://github.com/brettcannon/path-pep/blob/master/pep-0NNN.rst

The 3.4 and 3.5 versions of this should probably just be reversed before their releases (early June).

PEP-519 is accepted now. We need to revert the commits from http://bugs.python.org/issue22570#msg257634

And those from http://bugs.python.org/issue22570#msg257632 (these are the actual code -- the others were the docs).

Done. The revs are 90e58a77d386, 97198545e6c3, ade839421b8f.