This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Document various options for getting the absolute path from pathlib.Path objects
Type: enhancement Stage: needs patch
Components: Documentation Versions: Python 3.9
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: docs@python Nosy List: 4-launchpad-kalvdans-no-ip-org, ChrisBarker, John-Hennig, Zim, barneygale, brett.cannon, docs@python, eryksun, florisla, paul.moore, veky
Priority: normal Keywords:

Created on 2019-12-18 18:20 by brett.cannon, last changed 2022-04-11 14:59 by admin.

Messages (15)
msg358638 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2019-12-18 18:20
The question on how best to get an absolute path from a pathlib.Path object keeps coming up (see https://bugs.python.org/issue29688, https://discuss.python.org/t/add-absolute-name-to-pathlib-path/2882/, and https://discuss.python.org/t/pathlib-absolute-vs-resolve/2573 as examples).

As pointed out across those posts, getting the absolute path is surprisingly subtle and varied depending on your needs. As such we should probably add a section somewhere in the pathlib docs explaining the various ways and why you would choose one over the other.
msg358854 - (view) Author: Chris Barker (ChrisBarker) * Date: 2019-12-24 18:17
Yes Please!

I'd offer to help, but I really don't get the intricacies involved. I will offer to proofread and copy-edit though, if that's helpful.

And I note that coincidentally, just in the last week, I needed to make an absolute path from a Path, and it took me far too long to figure out that .resolve() would do it for me. Then I needed to do it again three days later, and it again took a while -- "resolve" is simply not mnemonic for me, and I'm guessing a lot of people have the same issue.

And I didn't find .absolute(), cause it's not documented. I see in issue #29688 that there are reasons for that, but I'll make a plea:

Please document .absolute(), even if those docs say something like "may not work in all circumstances, not well tested". Alternatively, if it's decided that folks should just use .resolve() in all cases anyway, then make .absolute() an alias for .resolve(). 

Or if that's not a good option, then at least put some prominent notes in resolve() so people will find it.

Also -- I needed to read the resolve() docs carefully (and then test) to see if it was what I wanted - which I know, is what this issue is about.

In short -- I understand that this is a complex issue, but making an absolute path is a pretty common use case, and we've had os.path.abspath() for decades, so there should be one obvious way to do it, and it should be easily discoverable.

NOTE: even if there is no one to do the work of properly testing .absolute() at this point, it would b nice to at least decide now what the long term goal is -- will there be an absolute() or is resolve() all we really need?
msg361698 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2020-02-10 14:32
> In short -- I understand that this is a complex issue, but making an absolute path is a pretty common use case, and we've had os.path.abspath() for decades, so there should be one obvious way to do it, and it should be easily discoverable.

+1 on this.

Given that (as far as I can tell from the various discussions) `resolve` works fine as long as the file exists, maybe the key distinction to make is whether you have an existing file or not.

(More subtle questions like UNC path vs drive letter, mentioned on the Discourse thread, are probably things that we can defer to a "more advanced cases" discussion in the docs).
msg361877 - (view) Author: Floris Lambrechts (florisla) Date: 2020-02-12 11:33
I've written an "Absolute paths" section based on the knowledge I found in the various threads.

Any review is appreciated.

https://github.com/florisla/cpython/tree/pathlib-chapter-absolute-paths

With some related documentation changes:

https://github.com/florisla/cpython/tree/absolute-path-related-improvements
msg361878 - (view) Author: Paul Moore (paul.moore) * (Python committer) Date: 2020-02-12 11:38
You've provided links to your branches, but not to the specific text you're proposing to add. Can you link to a diff or something that shows what you've added more precisely?
msg361885 - (view) Author: Floris Lambrechts (florisla) Date: 2020-02-12 12:18
This is the new chapter:

https://github.com/florisla/cpython/commit/c146ad3d086fe9e401284c12fc670ea4f9398f3b
msg362030 - (view) Author: Vedran Čačić (veky) * Date: 2020-02-15 20:51
If we want something mnemonic, I'm sure nothing beats __abs__. (Hey, we have __truediv__ already!;)
msg362578 - (view) Author: Floris Lambrechts (florisla) Date: 2020-02-24 09:54
@ChrisBarker,

Could you review the proposed addition to the documentation?

https://github.com/florisla/cpython/commit/c146ad3d086fe9e401284c12fc670ea4f9398f3b
msg362579 - (view) Author: Floris Lambrechts (florisla) Date: 2020-02-24 09:55
(sorry, didn't see the GitHub comments before... I'll process those first.)
msg362627 - (view) Author: Floris Lambrechts (florisla) Date: 2020-02-25 07:44
Based on the feedback received in GitHub here:
https://github.com/florisla/cpython/commit/c146ad3d086fe9e401284c12fc670ea4f9398f3b

I made a new revision of the 'Absolute paths' chapter here:
https://github.com/florisla/cpython/blob/pathlib-chapter-absolute-paths-2/Doc/library/pathlib.rst#absolute-paths

Further feedback is welcome.

Changes:

* Be more 'in your face' about Path.resolve() being the recommended
  approach.
* Add separate section on Windows considerations
* Explain difference between Path.resolve() and os.path.isabs() w.r.t.
  checking for drive.
* Refer to 'mapped share' instead of 'mapped network share'.
* Explain replacement of substitute drive with final path.
* Mention os.path.abspath's upcasing of drive letter in case of
  a path missing a root.
* Mention different handling of junctions versus symlinks w.r.t.
  relative parts.

For brevity, I've kept the wording on substitute drive and handling of
junctions very short.

For the same reason I did not not include eryksun's (interesting!) info
on why mapped and substitute drives are non-canonical.

Not mentioning Path.resolve()'s behavior w.r.t. non-existing files since
that's documented in resolve() itself.
msg387634 - (view) Author: John Hennig (John-Hennig) Date: 2021-02-24 17:40
@Floris:
> Not mentioning Path.resolve()'s behavior w.r.t. non-existing files since
that's documented in resolve() itself.

I don't see it mentioned in the documentation of `resolve()`, or anywhere else in the docs, that on Windows (but not on other platforms) `resolve()` does *not* resolve a relative path to an absolute path if the file does not exist. As opposed to `absolute()`, which works as expected on any platform.

Linux:
```
Python 3.6.9 (default, Oct  8 2020, 12:12:24)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pathlib import Path
>>> file = Path('new.txt')
>>> file.exists()
False
>>> file.resolve()
PosixPath('/home/user/new.txt')
```

Windows:
```
Python 3.9.2 (tags/v3.9.2:1a79785, Feb 19 2021, 13:44:55) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from pathlib import Path
>>> file = Path('new.txt')
>>> file.exists()
False
>>> file.resolve()
WindowsPath('new.txt')
>>> file.absolute()
WindowsPath('d:/home/new.txt')
>>> file.touch()
>>> file.resolve()
WindowsPath('D:/home/new.txt')
>>> file.unlink()
>>> file.resolve()
WindowsPath('new.txt')
>>>
```
msg416465 - (view) Author: EZ (Zim) Date: 2022-03-31 23:03
First, I hope we all agree:
'C:\Windows' and '/usr/bin' == absolute path
'Windows' and 'bin' == relative path
'C:\Program Files' and '/bin' == absolute path
'C:\Windows\..\Program Files' and '/usr/../bin' == relative path

It is very confusing between these two, but despite claims otherwise, absolute() does not work as expected. However, to sum up the findings below: 
* absolute() fails to resolve paths with relative steps (esp "..") but will always add the dir structure even if the file doesn't exist.
* resolve() will always give an absolute path.
** unless the file doesn't exist -- 
** unless unless the path includes a '..'! 
* There's also a related problem with is_absolute() being incorrect with relative paths (such as '..'), which usually results in it saying absolute().is_absolute() is True when it is obviously False.

Done on Windows 10, python 3.9.5

>>> ini
WindowsPath('desktop.ini/..')
>>> ini.resolve().is_absolute()
True
>>> ini.absolute()
WindowsPath('C:/Users/zim/Downloads/desktop.ini/..')
>>> ini.absolute().is_absolute()
True

This second should not be True, there is a trailing '..' not resolved by absolute()

Now let's create a truly messy path:
>>> ini.resolve()
WindowsPath('C:/Users/zim/Downloads')
>>> ini = ini / "ntuser.ini"
>>> ini.exists()
False
>>> ini.resolve()
WindowsPath('C:/Users/zim/Downloads/ntuser.ini')
>>> ini = ini / "../ntuser.ini"
>>> ini.exists()
False
>>> ini.resolve()
WindowsPath('C:/Users/zim/Downloads/ntuser.ini')
>>> ini = ini / "../../ntuser.ini"
>>> ini.resolve()
WindowsPath('C:/Users/zim/ntuser.ini')
>>> ini.exists()
True
>>> ini.absolute()
WindowsPath('C:/Users/zim/Downloads/desktop.ini/../ntuser.ini/../ntuser.ini/../../ntuser.ini')
>>> ini.absolute().is_absolute()
True

absolute() not only doesn't give an absolute path, but is_absolute() is somehow ok with that.

Now a file that doesn't exist:
>>> mike = Path("palin.jpg")
>>> mike.resolve()
WindowsPath('palin.jpg')
>>> mike.resolve().is_absolute()
False
>>> mike.absolute()
WindowsPath('C:/Users/zim/Downloads/palin.jpg')
>>> mike.absolute().is_absolute()
True

Finally, absolute() is right about the right thing, but resolve() is not terribly wrong. is_absolute() is correctly False here (for once). 

The problem is that the after a resolve() call, a Path object can still be used to create a file (good), but if resolve() is used before file creation, then the full path will not be there as should be expected (bad). This seems like a bug with resolve()

What if a file is non existent AND relative? Things get more confusing.

>>> badrel = Path('../circus.jpg')
>>> badrel
WindowsPath('../circus.jpg')
>>> badrel.absolute()
WindowsPath('C:/Users/zim/Downloads/../circus.jpg')
>>> badrel.resolve()
WindowsPath('C:/Users/zim/circus.jpg')
>>> badrel.exists()
False

So, absolute() still acts like the normal trash fire it is with relative paths, but what's this, resolve() actually gives an absolute path?!

I should note resolve() only behaves unpredictably on Windows. It correctly resolves non-existent files no matter what on macOS and Linux (caveat: my linux test was done with python 3.6). However, absolute() always fails to distill paths with relative steps regardless of OS.

So, it seems clear:
Bug 1: resolve() should work the same with non-existent files with incomplete paths on Windows as it does on *nix platforms, as it does on Windows when handling existent files and non-existent ones with parent path notation.
Bug 2: Obviously if absolute() is supposed to be in the lib, it should be documented, and it likely should be distinct from resolve(), but most of all: it should return actual absolute paths! If these cannot be fulfilled, it should be set to be deprecated (after resolve() is fixed, hopefully)
Bug 3: is_absolute() should actually detect absolute paths, instead it seems to report True if the path contains a root starting point, but ignores relative changes in between. (this issue exists on all three major OSs)
msg416470 - (view) Author: Vedran Čačić (veky) * Date: 2022-04-01 06:33
> First, I hope we all agree:
> 'C:\Windows\..\Program Files' and '/usr/../bin' == relative path

I don't agree. To me, absolute means regardless of a reference point. So, absolute path would be a path that refers to the same entity from whichever directory you reference it. And that is surely the case for these two.
msg416488 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2022-04-01 14:09
> Now a file that doesn't exist:
> >>> mike = Path("palin.jpg")
> >>> mike.resolve()
> WindowsPath('palin.jpg')

This is a bug in resolve(). It was fixed in 3.10+ by switching to ntpath.realpath(). I don't remember why a fix for 3.9 was never applied. Work on the PR may have stalled due to a minor disagreement.

> 'C:\Windows\..\Program Files' and '/usr/../bin' == relative path

No, a relative path depends on either the current working directory or, for a symlink target, the path of the directory that contains the symlink.

In Windows, a rooted path such as r"\spam" is a relative path because it depends on the drive of the current working directory. For example, if the current working directory is r"Z:\eggs", then r"\spam" resolves to r"Z:\spam". Also, a drive-relative paths such as "Z:spam" depends on the working directory of the given drive. Windows supports a separate working directory for each drive. For example, if the working directory of drive "Z:" is r"Z:\eggs", then "Z:spam" resolves to r"Z:\eggs\spam".
msg416668 - (view) Author: Barney Gale (barneygale) * Date: 2022-04-04 13:40
The docs for PurePath.is_absolute() say:

> A path is considered absolute if it has both a root and (if the flavour allows) a drive

This does not preclude it from having ".." segments.

PurePath.absolute() is documented as of bpo-29688 / 3.11, see: https://docs.python.org/3.11/library/pathlib.html#pathlib.Path.absolute

The documentation for the absolute() method is deliberately placed alongside resolve() for ease of comparison. Both methods make a path absolute, but resolve() also follows symlinks, and consequently is able to safely elide ".." segments.
History
Date User Action Args
2022-04-11 14:59:24adminsetgithub: 83271
2022-04-04 13:40:35barneygalesetmessages: + msg416668
2022-04-01 19:23:53brett.cannonsetnosy: + barneygale
2022-04-01 14:09:00eryksunsetnosy: + eryksun
messages: + msg416488
2022-04-01 06:33:14vekysetmessages: + msg416470
2022-03-31 23:03:08Zimsetnosy: + Zim, - 45757
messages: + msg416465
versions: + Python 3.9
2021-03-17 21:03:554-launchpad-kalvdans-no-ip-orgsetnosy: + 4-launchpad-kalvdans-no-ip-org
2021-02-24 17:40:07John-Hennigsetnosy: + John-Hennig
messages: + msg387634
2020-02-25 07:44:09florislasetmessages: + msg362627
2020-02-24 09:55:54florislasetmessages: + msg362579
2020-02-24 09:54:56florislasetmessages: + msg362578
2020-02-15 20:51:59vekysetnosy: + veky
messages: + msg362030
2020-02-12 12:18:46florislasetmessages: + msg361885
2020-02-12 11:38:54paul.mooresetmessages: + msg361878
2020-02-12 11:33:07florislasetnosy: + florisla
messages: + msg361877
2020-02-10 14:32:10paul.mooresetnosy: + paul.moore
messages: + msg361698
2019-12-28 02:09:2545757setnosy: + 45757
2019-12-24 18:17:28ChrisBarkersetnosy: + ChrisBarker
messages: + msg358854
2019-12-18 18:20:50brett.cannoncreate