classification
Title: distutils write into symlinks instead of replacing them
Type: behavior Stage:
Components: Distutils Versions: Python 3.1, Python 3.2, Python 3.3, Python 2.7, Python 2.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: eric.araujo Nosy List: eric.araujo, mgorny, tarek
Priority: normal Keywords:

Created on 2013-06-13 05:18 by mgorny, last changed 2013-08-04 10:43 by mgorny. This issue is now closed.

Messages (4)
msg191057 - (view) Author: Michał Górny (mgorny) * Date: 2013-06-13 05:18
We're doing heavy wrapping of Python scripts on Gentoo in order to efficiently support having multiple versions of Python installed. For that reason, every Python script installed using the package manager is renamed, and a symlink to a common wrapper binary is installed in its place.

For example, setuptools install looks like:

  /usr/bin/easy_install -> python-exec
  /usr/bin/easy_install-python2.7
  /usr/bin/easy_install-python3.2

Using the wrappers like this allow us to actively support user preference of Python version and map it to supported ones. Symlinking a common binary makes this more maintainable since there's just one place where wrapper needs to be updated.

However, it seems that setup.py is failing hard in replacing symlinks. Whenever user upgrades any installed package, either externally or due to some tool like easy_install or pip, the relevant setup.py writes into /usr/bin/easy_install rather than properly replacing it with a new file.

The effect, as you may imagine, is the wrapper being replaced with a random Python script and system-wide breakage.

The proper thing to do would be to write into temporary file in the destination directory, then use os.rename() to atomically replace the destination with the temporary file.

Since distutils is no longer maintained, I would be equally happy with simply os.remove() before writing the scripts. The problem may affect distutils2 as well.
msg191084 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2013-06-13 16:25
There are a handful of issues related to symlinks handling in distutils.  From the discussion on #15205 , the state of things is that basically distutils has no defined behaviour with respect to symlinks, and it’s not clear to me what the desired behaviour would be.
msg191085 - (view) Author: Michał Górny (mgorny) * Date: 2013-06-13 16:46
Well, I don't see much relevance between the two bugs, to be honest :). I think this bug is more of a symptom of a deeper issue with the way distutils is installing files.

But the issue is causing repeating issues for our users, and I don't really know what is the best way of at least helping our users avoid them.

If there were an agreement that this specific behavior of distutils is unintended, we can prepare a patch. I don't really want to diverge from the 'upstream' behavior in Gentoo just to fix our issue, and I don't really see another good way of fixing it.
msg194342 - (view) Author: Michał Górny (mgorny) * Date: 2013-08-04 10:43
Oh, sorry. I've looked throughout the code again and it seems that distutils is doing the right thing. It's setuptools/distribute that is broken.
History
Date User Action Args
2013-08-04 10:43:05mgornysetstatus: open -> closed
resolution: not a bug
messages: + msg194342
2013-06-13 16:46:02mgornysetmessages: + msg191085
2013-06-13 16:25:53eric.araujosetmessages: + msg191084
2013-06-13 05:19:06mgornysettype: behavior
2013-06-13 05:18:55mgornycreate