classification
Title: New realpath breaks setuptools_scm
Type: Stage: resolved
Components: Windows Versions: Python 3.8
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: eryksun, jaraco, paul.moore, steve.dower, tim.golden, zach.ware
Priority: normal Keywords: 3.8regression

Created on 2020-05-22 18:31 by jaraco, last changed 2020-05-22 20:37 by jaraco. This issue is now closed.

Messages (5)
msg369629 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2020-05-22 18:31
I've encountered an apparent regression on Python 3.8 on Windows when the current working directory is in a symlink to another volume and one runs `setup.py develop` on a project using setuptools_scm (https://github.com/pypa/setuptools_scm/issues/436).

I suspect the issue stems from changes in issue9949 and is likely a manifestation of the issues articulated by Eryk in issue9949#msg350138.
msg369637 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2020-05-22 20:11
It seems the underlying reason the behavior fails is the (intended) difference resolving the empty path to a realpath when a symlink points to another volume. The failing routine invokes realpath early (https://github.com/pypa/setuptools_scm/blob/d7c122e14c9eaca96574dec0ea530ad7204965a9/src/setuptools_scm/file_finder.py#L19).

```
path master # pwd
C:\Users\jaraco\code\main\path
path master # py -3.7 -c "import os; print(os.path.realpath(''))"
C:\Users\jaraco\code\main\path
path master # py -3.8 -c "import os; print(os.path.realpath(''))"
\\vmware-host\Shared Folders\home\code\main\path
```

But that doesn't explain why this issue would not have been triggered on non-Windows systems.
msg369639 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2020-05-22 20:25
It's because on Unix:

```
>>> os.path.relpath('/Users/jaraco/code/main/path/.flake8', '')
'.flake8'
```

But on Windows, relpath raises an error for the comparable call:

```
>>> os.path.relpath('\\\\vmware-host\\shared folders\\home\\code\\main\\path\\.flake8', '')
ValueError: path is on mount '\\\\vmware-host\\shared folders', start on mount 'C:'
```

So it seems it may not be a bug in Python, but merely a consequence of Python 3.8 honoring symlinks in realpath, and exposing the unfortunate weakness of resolving relative real paths when root volumes aren't in a shared namespace.

Does Python have any advice for downstream users running into this or similar issues? If they want cross-platform compatibility, must they avoid use of realpath? Do you have any tips specific to what setuptools_scm is doing?
msg369641 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2020-05-22 20:30
To make matters more complicated, `git rev-parse --show-toplevel` also returns the realpath:

```
# git rev-parse --show-toplevel
//vmware-host/Shared Folders/home/code/main/path
```
msg369644 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2020-05-22 20:37
Good news is it looks like the issue with setuptools_scm can be address in a relatively straightforward manner (https://github.com/pypa/setuptools_scm/issues/436#issuecomment-632899446).

I think that means the answers to the above questions are:

- Downstream users do not need to avoid the use of realpath.
- When using realpath, make sure to do it consistently, especially before invoking relpath.
History
Date User Action Args
2020-05-22 20:37:50jaracosetstatus: open -> closed
resolution: not a bug
messages: + msg369644

stage: resolved
2020-05-22 20:30:09jaracosetmessages: + msg369641
2020-05-22 20:25:18jaracosetmessages: + msg369639
2020-05-22 20:11:53jaracosetmessages: + msg369637
2020-05-22 19:57:35jaracosetnosy: + eryksun
2020-05-22 18:31:28jaracocreate