classification
Title: os.path.realpath() may produce incorrect results
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.1, Python 3.2, Python 3.3, Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: giampaolo.rodola, martin.panter, mkovtun, mocksoul, santoso.wijaya, serhiy.storchaka, stromsund, terry.reedy, vstinner, wm
Priority: normal Keywords: patch

Created on 2011-03-04 17:17 by mkovtun, last changed 2013-02-14 19:09 by serhiy.storchaka. This issue is now closed.

Files
File name Uploaded Description Edit
poc.py wm, 2011-03-05 21:03
test_issue11397.patch santoso.wijaya, 2011-03-20 20:01 unittest for this issue (generated against 2.7 branch) review
issue11397.patch santoso.wijaya, 2011-03-21 10:37 Patch and unittest (against 2.7 branch) review
issue11397_py31.patch santoso.wijaya, 2011-03-28 17:54 Patch against 3.1 review
issue11397_py32.patch santoso.wijaya, 2011-03-28 17:55 Patch against 3.2 review
issue11397_py32_2.patch santoso.wijaya, 2011-04-28 14:39 Patch against 3.2 review
issue11397_py27_2.patch santoso.wijaya, 2011-04-28 15:13 Patch against 2.7 review
issue11397_py31_2.patch santoso.wijaya, 2011-04-28 15:13 Patch against 3.1 review
realpath_link_stack.py martin.panter, 2011-08-20 06:03
Messages (13)
msg130060 - (view) Author: Mikhail Kovtun (mkovtun) Date: 2011-03-04 17:17
This bug appears in Python 2.4, 2.5, 2.6; not tested in Python 2.7.

How to reproduce on Linux:
{{{
$ mkdir ~/testsymlinks
$ cd ~/testsymlinks
$ mkdir adir
$ ln -s ../adir adir/blink
$ mkdir -p  adir/cdir/ddir
$ ln -s adir/cdir/ddir/.. xlink
$ ln -s xlink/../blink zlink
}}}
Then:
{{{
$ readlink zlink -f
/home/user/testsymlinks/adir
}}}
but:
{{{
$ python
Python 2.4.3 (#1, Sep  3 2009, 15:37:37) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-46)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os.path
>>> os.path.realpath('zlink')
'/home/user/testsymlinks/blink'
>>>
}}}
msg130132 - (view) Author: Wojciech Muła (wm) Date: 2011-03-05 21:03
Confirmed in python3.2.

Lib/posixpath.py/_resolve_link: path returned by readlink
is normalized by normpath. In this case readlink('zlink')
returns 'xlink/../blink', then normpath is called, and
returns 'blink'. This ends processing because 'blink'
isn't a link.

I've attached procedure that does [IMHO] correct symlink
resolution (this is just proof-of-concept, it is slow and
doesn't deal with cyclic links).
msg131386 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-03-19 04:50
2.5 and 2.6 are only open for security issues.
I am guessing that 2.7, 3.1, and 3.3 are affected.
I cannot test as I only have Windows currently.
msg131531 - (view) Author: Santoso Wijaya (santoso.wijaya) * Date: 2011-03-20 20:01
Attaching a unittest that will manifest this bug.
msg131628 - (view) Author: Santoso Wijaya (santoso.wijaya) * Date: 2011-03-21 10:37
I'm attaching a patch that fixes this bug. I'm using a different algorithm altogether than the currently existing one, to allow a stack-based approach that lends itself to fixing this bug (as inspired by Wojciech). Hopefully it won't expose new, subtler bugs...
msg132409 - (view) Author: Santoso Wijaya (santoso.wijaya) * Date: 2011-03-28 17:54
Attaching patches against 3.1 and 3.2 (due to unicode and slightly different unittest layout).
msg133749 - (view) Author: Santoso Wijaya (santoso.wijaya) * Date: 2011-04-14 16:56
ping?
msg133750 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2011-04-14 17:02
Sorry, patches sometimes sit awhile before a developer who can do something does do something.
msg133758 - (view) Author: Santoso Wijaya (santoso.wijaya) * Date: 2011-04-14 18:13
Oh, I understand. I'm just wondering especially for this one because it has no assigned dev.
msg133777 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2011-04-14 23:20
I tested issue11397_py32.patch:
 - you should use os.fsencode(sep) instead of sep.encode()
 - os.path.realpath('../'*10) raises IndexError('pop from empty list') instead of giving '/'
msg134653 - (view) Author: Santoso Wijaya (santoso.wijaya) * Date: 2011-04-28 06:51
(For 3.2) Patch v2. Added some more corner case tests.
msg142514 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2011-08-20 06:03
Another infinite loop that isn't caught in Python 3.2.1: With the symbolic link
    link => link/inside
a readlink("link") call will keep looping.

Anyhow, the proposed solution in issue11397_py32_2.patch does not account for paths with multiple independent references to the same link. Example link:
    here => .
Calling readlink("here/here") for me should return "/media/disk/home/vadmium" (my current directory, containing the "here" link), but the proposed version returns an "/media/disk/home/vadmium/here/here" (incompletely resolved).

I suggest something similar to "realpath_link_stack.py" I am attaching. I think the main difference is it pops each link off the cycle-detection stack after it has been resolved.
msg182116 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2013-02-14 19:09
Already fixed in issue6975.
History
Date User Action Args
2013-02-14 19:09:19serhiy.storchakasetstatus: open -> closed

nosy: + serhiy.storchaka
messages: + msg182116

resolution: out of date
stage: needs patch -> resolved
2012-01-12 16:33:06stromsundsetnosy: + stromsund
2012-01-12 16:29:11mocksoulsetnosy: + mocksoul
2011-08-20 06:03:26martin.pantersetfiles: + realpath_link_stack.py
nosy: + martin.panter
messages: + msg142514

2011-04-28 15:13:15santoso.wijayasetfiles: + issue11397_py31_2.patch
2011-04-28 15:13:03santoso.wijayasetfiles: + issue11397_py27_2.patch
2011-04-28 14:39:31santoso.wijayasetfiles: + issue11397_py32_2.patch
2011-04-28 14:39:16santoso.wijayasetfiles: - issue11397_py32_2.patch
2011-04-28 06:51:24santoso.wijayasetfiles: + issue11397_py32_2.patch

messages: + msg134653
2011-04-14 23:20:18vstinnersetmessages: + msg133777
2011-04-14 18:13:58santoso.wijayasetmessages: + msg133758
2011-04-14 17:02:39terry.reedysetmessages: + msg133750
2011-04-14 16:56:53santoso.wijayasetmessages: + msg133749
2011-03-28 17:55:11santoso.wijayasetfiles: + issue11397_py32.patch
2011-03-28 17:54:54santoso.wijayasetfiles: + issue11397_py31.patch

messages: + msg132409
2011-03-22 18:49:38santoso.wijayasetnosy: terry.reedy, vstinner, giampaolo.rodola, santoso.wijaya, wm, mkovtun
components: + Library (Lib), - Extension Modules
2011-03-21 10:40:44vstinnersetnosy: + vstinner
2011-03-21 10:37:10santoso.wijayasetfiles: + issue11397.patch
nosy: terry.reedy, giampaolo.rodola, santoso.wijaya, wm, mkovtun
messages: + msg131628
2011-03-20 20:08:42terry.reedysetnosy: terry.reedy, giampaolo.rodola, santoso.wijaya, wm, mkovtun
stage: test needed -> needs patch
2011-03-20 20:01:16santoso.wijayasetfiles: + test_issue11397.patch

messages: + msg131531
keywords: + patch
nosy: terry.reedy, giampaolo.rodola, santoso.wijaya, wm, mkovtun
2011-03-19 04:50:24terry.reedysetversions: + Python 3.1, Python 2.7, Python 3.3, - Python 2.6, Python 2.5
nosy: + terry.reedy

messages: + msg131386

stage: test needed
2011-03-05 23:58:11santoso.wijayasetnosy: + santoso.wijaya
2011-03-05 21:03:26wmsetfiles: + poc.py
versions: + Python 3.2
nosy: + wm

messages: + msg130132
2011-03-05 18:33:33giampaolo.rodolasetnosy: + giampaolo.rodola
2011-03-04 17:17:17mkovtuncreate