classification
Title: Provide expanduser() on Path objects
Type: enhancement Stage: resolved
Components: Library (Lib) Versions: Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, Claudiu.Popa, David.Edelsohn, antoine.pietri, cvrebert, fletom, haypo, neologix, pitrou, python-dev, rominf, serhiy.storchaka, steve.dower, vajrasky, zach.ware
Priority: low Keywords: patch

Created on 2013-11-25 20:02 by pitrou, last changed 2015-01-10 08:53 by haypo. This issue is now closed.

Files
File name Uploaded Description Edit
pathlib.patch Claudiu.Popa, 2013-12-01 09:41 review
pathlib1.patch Claudiu.Popa, 2013-12-05 08:23 review
issue19776.patch Claudiu.Popa, 2014-05-13 06:03 review
issue19776_1.patch Claudiu.Popa, 2014-06-17 05:05 review
issue19776_2.patch Claudiu.Popa, 2014-07-15 14:23
issue19776_3.patch Claudiu.Popa, 2014-07-19 06:30 review
issue19776_4.patch Claudiu.Popa, 2014-07-26 07:57 review
pathlib_expanduser.patch serhiy.storchaka, 2014-07-26 09:49 review
pathlib_expanduser_2.patch serhiy.storchaka, 2014-09-20 20:32 review
test_pathlib_empty_homedir.patch haypo, 2015-01-06 11:16 review
Messages (32)
msg204387 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2013-11-25 20:02
Something like expanduser() may be useful on Path objects.
msg204640 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2013-11-28 01:57
I wanted to suggest to modify the resolve() method, but '~' is a valid filename... (I tested, and it's annoying because depending on command, sometimes ~ is expanded to /home/haypo sometimes not...)
msg204897 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2013-12-01 09:41
Hello. Here's a patch for `expanduser()`.
msg205272 - (view) Author: Vajrasky Kok (vajrasky) * Date: 2013-12-05 04:11
Documentation, please (Doc/library/pathlib.rst)!

This is the docstring of Path.cwd.

"""Return a new path pointing to the current working directory
(as returned by os.getcwd()).
"""

This is the docstring of Path.expanduser.
""" Return a new path with expanded ~ and ~user constructs.
"""

Perhaps we need to add "(as returned by os.path.expanduser)"?
msg205280 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2013-12-05 08:23
Thanks, Vajrasky! Here's the new version of the patch.
msg208494 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-01-19 19:20
Antoine, is this feature still wanted?
msg216798 - (view) Author: Fletcher Tomalty (fletom) Date: 2014-04-18 20:04
+1

It's very annoying to have to import expand user from os.path, when pathlib should be a full replacement for that module. Thanks to those working on this!
msg217209 - (view) Author: Roman Inflianskas (rominf) Date: 2014-04-26 19:55
I think that `absolute` method should call `expanduser` and `expandvars` (do you plan to include it?) automatically. This should be optional (via default arguments: `expanduser=True, expandvars=True`.
msg217339 - (view) Author: Antoine Pietri (antoine.pietri) * Date: 2014-04-28 03:42
> I think that `absolute` method should call `expanduser` and `expandvars` (do you plan to include it?) automatically. This should be optional (via default arguments: `expanduser=True, expandvars=True`.

I think it shouldn't. (Or shouldn't be set to True by default anyway).
absolute() method resolves symlinks, and it would make no sense to expand tildes and vars, which are purely a "shell syntax".

. and .. are real things in the filesystem, ~ is just a notation commonly used (since it's in the SCL spec), but it's not *part* of the path, that's why you can totally have a valid ~ file.

Making absolute() expand tildes would be illogic, unintuitive and unpythonic.

(+1 for the .expanduser() patch though, I went here after searching for this feature in the docs).
msg218407 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-05-13 06:03
Added a new version of the patch with improvements suggested by Berker Peksag. Thanks for the review!
msg220422 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-06-13 06:56
Antoine, how does my latest patch look?
msg220516 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-06-14 01:05
Claudiu, sorry for the delay. I'm going to take a look soon!
msg220791 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-06-17 05:05
Attached the new version of the patch with fixes according to Antoine's review.
msg223090 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-07-15 06:54
Since all the comments have been addressed, it would be nice if this gets committed.
msg223111 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-07-15 14:23
This new patch fixes some comments from Serhiy. Thanks for the review!
msg223137 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-07-15 18:53
Doc example is still looks confusing. Is your home directory /home?

There is a question. What should pathlib's expanduser() do in case when user directory can't be determined (or user does not exist)? Perhaps unlike to os.path.expanduser() it should raise an exception (as in many other pathlib's methods).
msg223461 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-07-19 06:30
No, my home directory is actually /root. The attached patch should be clearer now (I hope). Regarding your question, wouldn't checking for this duplicate what os.path.expanduser already does? (Unless checking for the exact same string before returning it.)
msg223770 - (view) Author: Charles-Fran├žois Natali (neologix) * (Python committer) Date: 2014-07-23 19:44
> There is a question. What should pathlib's expanduser() do in case
> when user directory can't be determined (or user does not exist)?
> Perhaps unlike to os.path.expanduser() it should raise an exception
> (as in many other pathlib's methods).

Let's see what POSIX says:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_01

"""
If the system does not recognize the login name, the results are undefined.
"""

Helpful, isn't it ;-)

Behaving like os.path.expanduser() would have the advantage of consistency (which is in turn consistent with Unix shells), OTOH, it seems wrong to just return the unexpanded form: for example, if someone calls expanduser('~apache') and there's apache user, returning '~apache' could be dangerous if there was an '~apache' folder in the CWD.

So I think it would be better to raise an exception.
msg224027 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-07-26 07:57
Here's a version of the patch which raises ValueError when the path can't be expanded. Hopefully, the used approach is good enough.
msg224037 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-07-26 09:49
Here is alternative patch which doesn't use os.path.expanduser() and reimplement it's logic. Differences:

* expanduser() is part of concrete path API. This method access environment.
* RuntimeError is raised when user home can't be determined.
* Currently ntpath.expanduser() uses heuristic to expand path with specified username. This works with default homedirs but can return wrong result when homedirs was moved to different locations. WindowsPath.expanduser() also uses heuristic, but more robust. Of course it would be better to get other users homedirs from Windows API, and perhaps we should defer this issue until implementing pwd or like on Windows.
* Expanded tests.

Interesting, common idiom to escape tilda in relative path (adding "./" prefix) doesn't work with pathlib, because "." components are ignored.
msg224039 - (view) Author: Claudiu Popa (Claudiu.Popa) * Date: 2014-07-26 10:03
Looks good.
msg227167 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-09-20 18:57
Serhiy, would you like to update your patch?
msg227174 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2014-09-20 20:32
Sorry. Here is updated patch.
msg233153 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-12-28 19:03
Steve, would you like to give an opinion on the Windows aspects of this patch? Otherwise I will simply commit it soon.
msg233154 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2014-12-28 20:07
I thought that USERPROFILE was the preferred environment variable and should be checked before HOME, but I could be wrong. Consistency with the existing expanduser function is more important probably.

There's almost certainly an API to find an arbitrary user directory, but it may only be available to administrators, so the guess is probably necessary anyway. The directory name doesn't necessarily match the user name either, especially since a lot of Windows users these days authenticate with their email address.
msg233220 - (view) Author: Roundup Robot (python-dev) Date: 2014-12-30 21:09
New changeset bee697b0fd18 by Antoine Pitrou in branch 'default':
Issue #19776: Add a expanduser() method on Path objects.
https://hg.python.org/cpython/rev/bee697b0fd18
msg233513 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2015-01-06 10:41
The test fails on the buildbot "PPC64 AIX 3.x". It looks like the home directory of the buildbot slave user is /home/shager/.

http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/2994/steps/test/logs/stdio

======================================================================
FAIL: test_expanduser (test.test_pathlib.PosixPathTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shager/cpython-buildarea/3.x.edelsohn-aix-ppc64/build/Lib/test/test_pathlib.py", line 2002, in test_expanduser
    self.assertEqual(p3.expanduser(), P(otherhome) / 'Documents')
AssertionError: PosixPath('/Documents') != PosixPath('Documents')
msg233516 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-01-06 11:08
I don't really know how to investigate that failure. Perhaps David wants to look into it?
msg233518 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2015-01-06 11:16
> I don't really know how to investigate that failure.

I don't think that it's specific to AIX, it just depends on the list of users on your system. On my Fedora 21, I have many users which have an empty home directory:

- nobody
- dbus
- polkitd
- usbmuxd
- nm-openconnect
- tcpdump
- qemu
- radvd
- systemd-journal-gateway
- systemd-timesync
- systemd-network
- systemd-resolve
- systemd-bus-proxy

If the test pick one of this user instead of a user with a non-empty home directory, the test fails with the same error. I propose a simple patch: test_pathlib_empty_homedir.patch
msg233790 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2015-01-10 00:08
Victor, your patch sounds ok to me.
msg233803 - (view) Author: Roundup Robot (python-dev) Date: 2015-01-10 08:00
New changeset 63dac5212552 by Victor Stinner in branch 'default':
Issue #19776: Fix test_pathlib.test_expanduser()
https://hg.python.org/cpython/rev/63dac5212552
msg233805 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2015-01-10 08:53
Ok, test_pathlib now pass on the buildbot "PPC64 AIX 3.x". I close the issue. Nice enhancement of the Path object.
History
Date User Action Args
2015-01-10 08:53:47hayposetstatus: open -> closed
resolution: fixed
messages: + msg233805
2015-01-10 08:00:32python-devsetmessages: + msg233803
2015-01-10 00:08:29pitrousetmessages: + msg233790
2015-01-06 11:16:51hayposetfiles: + test_pathlib_empty_homedir.patch

messages: + msg233518
2015-01-06 11:08:37pitrousetnosy: + David.Edelsohn
messages: + msg233516
2015-01-06 10:41:38hayposetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg233513
2014-12-30 21:10:15pitrousetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2014-12-30 21:09:58python-devsetnosy: + python-dev
messages: + msg233220
2014-12-28 20:07:41steve.dowersetmessages: + msg233154
2014-12-28 19:03:22pitrousetmessages: + msg233153
2014-12-28 19:01:19pitrousetnosy: + zach.ware, steve.dower
2014-12-24 16:55:53cvrebertsetnosy: + cvrebert
2014-09-20 20:32:24serhiy.storchakasetfiles: + pathlib_expanduser_2.patch

messages: + msg227174
2014-09-20 18:57:12pitrousetmessages: + msg227167
2014-07-26 10:03:10Claudiu.Popasetmessages: + msg224039
2014-07-26 09:49:48serhiy.storchakasetfiles: + pathlib_expanduser.patch

messages: + msg224037
2014-07-26 07:57:47Claudiu.Popasetfiles: + issue19776_4.patch

messages: + msg224027
2014-07-23 19:44:20neologixsetnosy: + neologix
messages: + msg223770
2014-07-19 06:30:24Claudiu.Popasetfiles: + issue19776_3.patch

messages: + msg223461
2014-07-15 18:53:07serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg223137
2014-07-15 14:23:51Claudiu.Popasetfiles: + issue19776_2.patch

messages: + msg223111
stage: commit review -> patch review
2014-07-15 06:54:24Claudiu.Popasetmessages: + msg223090
stage: patch review -> commit review
2014-06-17 05:05:27Claudiu.Popasetfiles: + issue19776_1.patch

messages: + msg220791
2014-06-14 01:05:46pitrousetmessages: + msg220516
2014-06-13 06:56:30Claudiu.Popasetmessages: + msg220422
stage: patch review
2014-05-13 06:03:02Claudiu.Popasetfiles: + issue19776.patch

messages: + msg218407
2014-04-28 03:42:48antoine.pietrisetnosy: + antoine.pietri
messages: + msg217339
2014-04-26 19:55:20rominfsetnosy: + rominf
messages: + msg217209
2014-04-18 20:04:24fletomsetnosy: + fletom
messages: + msg216798
2014-01-19 19:20:05Claudiu.Popasetmessages: + msg208494
2013-12-05 08:23:09Claudiu.Popasetfiles: + pathlib1.patch

messages: + msg205280
2013-12-05 04:11:35vajraskysetnosy: + vajrasky
messages: + msg205272
2013-12-01 09:41:52Claudiu.Popasetfiles: + pathlib.patch

nosy: + Claudiu.Popa
messages: + msg204897

keywords: + patch
2013-11-28 01:57:43hayposetnosy: + haypo
messages: + msg204640
2013-11-26 02:29:21Arfreversetnosy: + Arfrever
2013-11-25 20:02:58pitroucreate