classification
Title: Implement os.path.samefile and os.path.sameopenfile on Windows
Type: enhancement Stage:
Components: Versions: Python 3.2, Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: asvetlov, ocean-city, sandberg
Priority: normal Keywords: patch

Created on 2009-05-10 10:02 by sandberg, last changed 2010-09-09 20:39 by ocean-city. This issue is now closed.

Files
File name Uploaded Description Edit
samefile.patch ocean-city, 2009-10-14 09:54
Messages (10)
msg87518 - (view) Author: Erik Carstensen (sandberg) Date: 2009-05-10 10:02
It would be nice if samefile / sameopenfile was present on Windows.
Right now I usually work around this by keeping a platform-specific hack
for Windows that approximates samefile by comparing normalized paths;
this is ugly and doesn't handle junctions correctly.

In one of my projects I have written a C implementation of samefile,
which I manually monkey-patched os.path with. It would probably be
rather easy to adapt it to become a native part of ntpath.

My code relies on GetFileInformationByHandle, which is only available in
Windows 2000 professional and newer
(http://msdn.microsoft.com/en-us/library/aa364952(VS.85).aspx); if I
understood it correctly this should not be a problem as Python 2.6 and
newer doesn't support older versions of Windows.

Unfortunately I don't use Windows myself, but I have rdesktop access to
an XP machine with cygwin and Visual Studio 2005 installed (which seems
insufficient to build python 2.6, at least), so it will be difficult for
me to test my code. But I'll post some code soon.
msg87840 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2009-05-15 20:50
At this point, 2.7/3.2 are the only targets for new features.
msg93970 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2009-10-14 09:54
Here is my experimental patch.
msg93973 - (view) Author: Erik Carstensen (sandberg) Date: 2009-10-14 11:22
An alternative solution which I would have considered, is to extend
stat/fstat on Windows to set st_dev and st_ino to sensible values (based
on dwVolumeSerialNumber and nFileIndexLow/High), and then use the POSIX
implementations of samefile/sameopenfile.

There may be one potential problem with this solution though: It would
require stat to perform a CreateFile in addition to GetFileAttributesEx,
and I don't know if there are situations where one is allowed to call
the latter but not the former on a file. There would be no such problems
with fstat, though.

In your patch, I think the dwShareMode parameter to CreateFile* should
be changed to FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE so
make the operation less intrusive.
msg93979 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2009-10-14 12:30
> extend stat/fstat on Windows to set st_dev and st_ino to sensible
> values (based on dwVolumeSerialNumber and nFileIndexLow/High)

Once I considered this approach, but problems was that
nFileIndexLow/High can change every time file handle is opened, so it's
not unique. See remarks in following page.
http://msdn.microsoft.com/en-us/library/aa363788%28VS.85%29.aspx

> I think the dwShareMode parameter to CreateFile* should
> be changed to FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE
> so make the operation less intrusive.

Probably you are right. I must admit I'm not familiar with this *shared*
flag, and I was careless about its usage.
msg93986 - (view) Author: Erik Carstensen (sandberg) Date: 2009-10-14 13:51
> Once I considered this approach, but problems was that
> nFileIndexLow/High can change every time file handle is opened, so
> it's not unique.

Ah, I see, then your approach makes sense.

There's another part of your patch that I don't understand:

+except ImportError: # not running on Windows - mock up something sensible
+    from posixpath import samefile # XXX

In what situations will this happen, and are we guaranteed in those
cases that samefile will be found in posixpath?
msg93993 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2009-10-14 14:48
I'm not sure about this neither. So, XXX is in comment. ;-)

On abspath() above, it also tries to import native method
_getfullpathname(), and when it fails alternative implementation runs.
(probably when someone calls ntpath.abspath from linux or somewhere.....
Does it really happen? I don't know)

I was not sure what kind of alternative implementation is appropriate,
so I borrowed posixmodule's one. Probably more appropriate
implementation may exist.
msg95931 - (view) Author: Erik Carstensen (sandberg) Date: 2009-12-03 13:31
> Once I considered this approach, but problems was that
> nFileIndexLow/High can change every time file handle is opened, so it's
> not unique. See remarks in following page.
> http://msdn.microsoft.com/en-us/library/aa363788%28VS.85%29.aspx

Actually, that page only says that file identities may vary over time,
and that it may happen during operations such as defragmentation.
However, I have been in contact with a customer who observed a file
system where the file index actually changed every time a file was
closed and re-opened, given that nobody else kept the file open
inbetween. This was on some kind of network mount, I wasn't told which
kind (and I couldn't reproduce it with samba).

This means that it's actually essential that you don't close the file
between the CreateFile calls (and yes, your patch does this correctly).
msg96016 - (view) Author: Andrew Svetlov (asvetlov) * (Python committer) Date: 2009-12-06 01:38
Please note: patch for http://bugs.python.org/issue1578269 is already has 
implementation for samefile. It's easer to add sameopenfile is same way 
than maintain two different approaches.

Unfortunately solution will work only starting from Windows Vista and 
Windows Server 2008.
msg115973 - (view) Author: Hirokazu Yamamoto (ocean-city) * (Python committer) Date: 2010-09-09 20:39
Both os.path.samefile and os.path.sameopenfile are now in py3k.
And release27-maint is in feature freeze, so I think this issue
should be closed.

# Implemented on
os.path.samefile: #1578269
os.path.sameopenfile: #7566
History
Date User Action Args
2010-09-09 20:39:36ocean-citysetstatus: open -> closed
resolution: out of date
messages: + msg115973
2009-12-06 01:38:55asvetlovsetnosy: + asvetlov
messages: + msg96016
2009-12-03 15:00:39terry.reedysetnosy: - terry.reedy
2009-12-03 13:31:30sandbergsetmessages: + msg95931
2009-10-14 14:48:09ocean-citysetmessages: + msg93993
2009-10-14 13:51:45sandbergsetmessages: + msg93986
2009-10-14 12:30:09ocean-citysetmessages: + msg93979
2009-10-14 11:22:50sandbergsetmessages: + msg93973
2009-10-14 09:54:41ocean-citysetfiles: + samefile.patch

nosy: + ocean-city
messages: + msg93970

keywords: + patch
2009-05-15 20:50:43terry.reedysetnosy: + terry.reedy

messages: + msg87840
versions: - Python 2.6, Python 2.5, Python 3.0, Python 3.1
2009-05-10 10:02:47sandbergcreate