Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

distutils dereferences symlinks on Mac OS X but not on Linux #59410

Closed
olliewalsh mannequin opened this issue Jun 27, 2012 · 6 comments
Closed

distutils dereferences symlinks on Mac OS X but not on Linux #59410

olliewalsh mannequin opened this issue Jun 27, 2012 · 6 comments
Assignees
@merwok
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@olliewalsh
Copy link
Mannequin

olliewalsh mannequin commented Jun 27, 2012

BPO 15205
Nosy @ericvsmith, @tarekziade, @merwok, @zooba

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = 'https://github.com/merwok'
closed_at = <Date 2021-02-03.18:37:35.768>
created_at = <Date 2012-06-27.12:53:40.583>
labels = ['type-bug', 'library']
title = 'distutils dereferences symlinks on Mac OS X but not on Linux'
updated_at = <Date 2021-02-03.18:37:35.768>
user = 'https://bugs.python.org/olliewalsh'

bugs.python.org fields:

activity = <Date 2021-02-03.18:37:35.768>
actor = 'steve.dower'
assignee = 'eric.araujo'
closed = True
closed_date = <Date 2021-02-03.18:37:35.768>
closer = 'steve.dower'
components = ['Distutils']
creation = <Date 2012-06-27.12:53:40.583>
creator = 'olliewalsh'
dependencies = []
files = []
hgrepos = []
issue_num = 15205
keywords = []
message_count = 6.0
messages = ['164149', '164542', '164544', '164643', '164644', '386440']
nosy_count = 6.0
nosy_names = ['eric.smith', 'tarek', 'eric.araujo', 'Arfrever', 'olliewalsh', 'steve.dower']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue15205'
versions = ['Python 2.7']
@olliewalsh
Copy link
Mannequin Author

olliewalsh mannequin commented Jun 27, 2012

Hi,

This is related to bpo-12585.

Distutils sdist builds a package tree using hardlinks to the source if supported by the OS. This is then tarred/zipped/etc...

If the source contains symbolic links to external files:

On Linux (and apparently Solaris) they are not dereferenced and the resulting package is broken.

On OSX (and apparently all BSD kernels) a hardlink to a symlink will dereference the symlink first and the resulting tgz package is ok. However I would expect issue bpo-8876 to be more likely as the symlink could references a different filesystem which would case hardlinking to fail.

In bpo-12585 I assume that zip dereferences the symlinks which masks the issue.

distutils2 appears to always copy instead of hardlinking resolving all of these issues.

distutils can be monkey patched to do the same:
http://article.gmane.org/gmane.comp.python.distutils.devel/2078

@olliewalsh olliewalsh mannequin assigned merwok Jun 27, 2012
@olliewalsh olliewalsh mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Jun 27, 2012
@merwok
Copy link
Member

merwok commented Jul 2, 2012

As I said in the other bug, symlinks handling in not fully specified: some functions (like copy_file and copy_tree) have arguments to control copy/link/symlink, but the higher-level commands don’t expose that choice to the user. At present I am not sure if having symlinks in a distribution makes sense, and if the install command (or platform-specific systems for bdists) supports it. I am not sure what the correct behavior would be here.

(BTW the example you link to is not at all monkey-patching, but clean extension of distutils using the provided hooks. :)

@merwok
Copy link
Member

merwok commented Jul 2, 2012

(Hm OK “del os.link” is definitely a hack :)

@olliewalsh
Copy link
Mannequin Author

olliewalsh mannequin commented Jul 4, 2012

Hi,

As I said in the other bug, symlinks handling in not fully specified

Yes. I've mostly submitted a bug report in case anybody else encounters the problem.

At present I am not sure if having symlinks in a distribution makes sense

I agree.
In my case symlinks are being used to restructure a large legacy module into smaller packages. When creating sdists the symlinks are dereferenced on another developer's host (OSX) and all is well but they on my host (Linux) I get packages with broken symlinks.

BTW the example you link to is not at all monkey-patching

My mistake. When I worked around this issue I did monkey-patch sdist. I also just wrapped the source path is os.path.readlink() before hardlinking. I think the approach in that link is a much better solution so I have referred to it instead.

-Ollie

@olliewalsh
Copy link
Mannequin Author

olliewalsh mannequin commented Jul 4, 2012

PS that should be os.readlink()

@zooba
Copy link
Member

zooba commented Feb 3, 2021

Distutils is now deprecated (see PEP-632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils.

If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at https://github.com/pypa/setuptools

@zooba zooba closed this as completed Feb 3, 2021
@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@merwok merwok

Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@merwok @zooba