Title: os.path.realpath on Windows does not follow symbolic links
Type: behavior Stage: patch review
Components: Library (Lib), Windows Versions: Python 3.6
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Christian Åkerström, Ethan Smith, eric.smith, haypo, ishimoto, jason.coombs, living180, ncdave4life, paul.moore, pitrou, steve.dower, stutzbach, takluyver, tim.golden, zach.ware
Priority: normal Keywords: needs review

Created on 2010-09-25 15:03 by stutzbach, last changed 2017-07-05 05:30 by Ethan Smith.

File name Uploaded Description Edit ncdave4life, 2012-03-09 23:12 Python 3.2 fix for issue 9949
issue9949.tar.bz2 living180, 2012-04-15 18:52 Python 3.3 fix for issue #9949
issue9949-v2.tar.bz2 living180, 2012-04-16 01:11 Python 3.3 fix for issue #9949, version 2
issue9949-v3.patch living180, 2012-06-03 20:04 Python 3.3 fix for issue #9949, version 3 review
issue9949-v4.patch living180, 2015-02-09 15:38 Python 3.5 fix for issue #9949, version 4 review
Messages (15)
msg117374 - (view) Author: Daniel Stutzbach (stutzbach) (Python committer) Date: 2010-09-25 15:03
In Lib/

# realpath is a no-op on systems without islink support
realpath = abspath

However, Windows Vista and newer support symbolic links and other Python methods support them.

(noticed this through source code inspection; haven't actually tested it)
msg142585 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-08-20 23:56
The issue #12799 has been marked as a duplicate of this issue:
"From a look at the code, realpath() does not seem to resolve symlinks under Windows, even though we have the _getfinalpathname function to do that. Please indulge with me if I'm wrong :)"
msg155270 - (view) Author: Dave Burton (ncdave4life) Date: 2012-03-09 23:12
This is a patch for the os.path.realpath() bug under Windows,  "os.path.realpath on Windows does not follow symbolic links"

ntpath.diff fixes the realpath() function to resolve symbolic links to their targets (tested under Windows 7).

Note: I tried to make the equivalent patch for Python 2.7, but it didn't work because the "from nt import _getfinalpathname" fails.
msg155371 - (view) Author: Dave Burton (ncdave4life) Date: 2012-03-11 03:15
It seems that the nt module is implemented in the posixmodule.c source file, and the Python 3 version contains the posix__getfinalpathname entry point, but the Python 2 version does not.

I presume that PyWin32 could also be used to work around this.  Too bad it isn't automatically included with Python:
msg155372 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2012-03-11 03:22
file, and the Python 3 version contains the posix__getfinalpathname entry
point, but the Python 2 version does not.
> I presume that PyWin32 could also be used to work around this.  Too bad
it isn't automatically included with Python:

We can't use external packages in the standard library. If you're looking
for a Python which includes pywin32, ActiveState provides this.

I think we can just implement the function we need.
msg155376 - (view) Author: Dave Burton (ncdave4life) Date: 2012-03-11 04:03

The change is nearly identical in Python 2.7 to the change for Python 3.2.  The only difference is that instead of:

+        elif isinstance(path, bytes):
+            path = os.getcwdb()

It is:

+        elif isinstance(path, unicode):
+            path = os.getcwdu()
msg158353 - (view) Author: Daniel Harding (living180) * Date: 2012-04-15 18:52
I have attached a series of patches with (hopefully) provide more robust fix for this issue, against the Python 3.3 branch.  It handles both bytes and str objects, paths that do not actually exist on the filesystem, and removal of the '\\?\' prefix returned by _getfinalpathname, unless the supplied path had that prefix already.  A couple of the patches contain some separate infrastructure that could hopefully be used to simplify some of the other Windows code in posixmodule.c (One immediate possibility would be to combine the code provided in these patches with the symbolic-link resolution code in the Windows stat functions - there is quite a bit of duplication there that could be eliminated.)

One thing these patches do not address is resolving a broken symbolic link.  The Windows API function GetFinalPathNameByHandle does not handle this case (because CreateFile cannot be used to get a handle if the symbolic link is broken).  This functionality could be implemented by manually following the reparse points, but that would basically require reimplementing GetFinalPathNameByHandle.

Finally, this patch could be fairly easily backported to Python 3.2, but that shouldn't be done without careful consideration.  It changes the return value from os.path.realpath on Windows even when there are no symbolic links in the path (the returned value will have the actual casing as stored on the filesystem, instead of the casing supplied by the user).  I don't think it should be backported to Python 2.7, because that version, like all Python versions before Python 3.2 are unaware of symbolic links on Windows (e.g. lexists is the same function as exists).

The patches were generated using git (I use git-hg) - if that format is a problem, let me know and I can regenerate them.
msg158390 - (view) Author: Daniel Harding (living180) * Date: 2012-04-16 01:11
Uploading a new series of patches - they are all the same as the first set, except for 0006-Make-realpath-follow-symbolic-links-on-Windows.patch.  I realized that I could use os.readlink to handle broken symbolic links, so I changed the logic of os.path.realpath slightly to do that (including recursive symlink detection).
msg162231 - (view) Author: Daniel Harding (living180) * Date: 2012-06-03 20:04
The previous version of this patch did not handle bytes arguments correctly and could fail in conjunction with a non-ASCII compatible encoding.  Also, if the result was a UNC path, it was not being handled correctly (the returned value would have started with a single backslash, not two).  Version 3 of the patch fixes these issues.  I am also attaching a single, condensed patch as requested by the Lifecycle of a Patch devguide.  I can also provide a patch series as before if desired.
msg166197 - (view) Author: Atsuo Ishimoto (ishimoto) * Date: 2012-07-23 02:02
Yet another patch to support symlink on Windows.
msg223299 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-16 23:46
@Zach can you do a patch review on this as it's holding up #13837 ?
msg223801 - (view) Author: Atsuo Ishimoto (ishimoto) * Date: 2014-07-24 01:57
I have unlinked my patch since it doesn't looks correct now. Sorry for disturbing.
msg229059 - (view) Author: Zachary Ware (zach.ware) * (Python committer) Date: 2014-10-11 05:34
Daniel: It's taken two years, but I've reviewed your patch :).  There are a few things that need to be addressed, but the basic change looks pretty good.  If you're still interested in seeing this fixed, I look forward to reviewing an updated patch; otherwise, I'll try to pick this up myself soon.
msg235615 - (view) Author: Daniel Harding (living180) * Date: 2015-02-09 15:38
@Zach - thanks for the review.  Sorry that it has taken a few months to pick this issue back up.  Anyway, here is an updated patch.  It is pretty different than the previous patch, to the point that I would consider it a completely new patch.

Anyway, here's a basic rundown of what the patch contains:

Two functions in Modules/posixmodule.c were modified to handle bytes objects as well as str objects:  win_readlink and _getfinalpathname.  This was necessary to allow os.path.realpath() to continue returning bytes when supplied with bytes.  I know you said in your review that you didn't care about this because using bytes paths on Windows is deprecated, but I think it still important to maintain backwards compatibility.

A new implementation of realpath() was added to Lib\, used when _getfinalpathname is available (when _getfinalpathname is not available, realpath remains a synonym for abspath, as it was previously).  The logic here has been completely reworked and is roughly modeled on the realpath implementation from Lib\

A number of tests were added to Lib\test\ for realpath() functionality.  Three of the tests were based on realpath() tests from, but one is completely new:  test_realpath_broken_symlinks.

A note about improved Windows support was added to the documentation for realpath(), but no other doc changes have been made.

Anyway, feedback is welcome - I'd love to get this code in sufficient shape to see a working realpath() implementation for Windows land in Python.
msg275287 - (view) Author: Christian Åkerström (Christian Åkerström) Date: 2016-09-09 08:16
Any update on this? Would be great with a fix for python symlinks on Windows.
Date User Action Args
2017-07-05 05:30:39Ethan Smithsetnosy: + Ethan Smith
2016-09-09 08:22:42BreamoreBoysetnosy: - BreamoreBoy
2016-09-09 08:16:00Christian Åkerströmsetnosy: + Christian Åkerström
messages: + msg275287
2015-12-12 16:39:21SilentGhostsetkeywords: + needs review, - patch
nosy: + paul.moore, steve.dower

components: + Windows
versions: + Python 3.6, - Python 3.5
2015-02-09 15:38:09living180setfiles: + issue9949-v4.patch

messages: + msg235615
2014-10-11 05:34:38zach.waresetmessages: + msg229059
versions: + Python 3.5, - Python 2.7, Python 3.3, Python 3.4
2014-10-01 17:09:44takluyversetnosy: + takluyver
2014-07-24 01:57:33ishimotosetmessages: + msg223801
2014-07-24 01:55:36ishimotosetfiles: - issue9949.patch
2014-07-17 13:58:08brian.curtinsetnosy: - brian.curtin
2014-07-16 23:46:44BreamoreBoysetnosy: + BreamoreBoy
messages: + msg223299
2013-11-05 21:13:29zach.waresetnosy: + zach.ware
stage: needs patch -> patch review

versions: + Python 3.4, - Python 3.2
2012-10-07 11:09:22hyneklinkissue13837 dependencies
2012-07-23 02:02:15ishimotosetfiles: + issue9949.patch
nosy: + ishimoto
messages: + msg166197

2012-06-03 20:04:49living180setfiles: + issue9949-v3.patch
keywords: + patch
messages: + msg162231
2012-04-16 01:11:39living180setfiles: + issue9949-v2.tar.bz2

messages: + msg158390
2012-04-15 18:52:37living180setfiles: + issue9949.tar.bz2
nosy: + living180
messages: + msg158353

2012-03-11 04:03:25ncdave4lifesetmessages: + msg155376
2012-03-11 03:22:49brian.curtinsetmessages: + msg155372
2012-03-11 03:15:13ncdave4lifesetmessages: + msg155371
2012-03-09 23:12:06ncdave4lifesetfiles: +
nosy: + ncdave4life
messages: + msg155270

2011-08-20 23:56:12hayposetnosy: + pitrou, tim.golden, haypo

messages: + msg142585
versions: + Python 2.7, Python 3.3
2010-09-25 15:03:13stutzbachcreate