-
-
Notifications
You must be signed in to change notification settings - Fork 29.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
distutils sdist is too lazy w.r.t. recalculating MANIFEST #52934
Comments
The sdist command in distutils calculates the MANIFEST file from a template (by default MANIFEST.in). The code in sdist assumes that the contents of MANIFEST only depends on MANIFEST.in and setup.py, which can cause files to be missed when building an sdist. In particular: given a MANIFEST.in file that includes all documentation in a 'Doc' subdirectory, using"recursive-include Doc *.txt" in the template, the MANIFEST won't get regenerated when a new file is added to the documentation subdirectory and hence that new file is not included in sdist distributions. |
One way to fix this: is to always recreate MANIFEST when an explicit MANIFEST.in file exists, as in the attached patch. (The patch is not perfect: I'd rename "template_newer" before committing) |
The same problem occurs without a manifest template: you can have options or imports in setup.py that will change the MANIFEST file. For example : if setup() has: packages=['foo'] And if you add a subpackage "bar" in "foo" with some modules, with an existing MANIFEST, they won't be added. IOW, the MANIFEST file should be recalculated *everytime*. I am going to remove all the "don't rebuilt MANIFEST" code in Distutils *and* Distutils2. |
done in r81255, r81256 (2.6), r81258 (py3), r81260 (3.1) Thanks Ronald |
This change just wrecked Mercurial's release build process. We've been building Mercurial release tarballs with a Makefile target wrapped around sdist for most of five years, and we've never had or wanted a MANIFEST.in file. We generate an exact, complete, and correct MANIFEST ourselves with: hg manifest > MANIFEST This fix ignores that MANIFEST entirely and attempts to build a new one even though MANIFEST.in doesn't exist. So not only does this change years of documented behavior in a way that very nearly caused us to ship a tarball missing over a thousand files, it means doing what we want to do with sdist (ship precisely the files listed in our version control manifest) is now significantly harder. Unfortunately, this fix is now in a long-lived Python release which we support, which means we're stuck with it. |
this fix was done to avoid generating broken MANIFEST file with sdist. IIUC your problem is more about avoiding generating a MANIFEST file at *all* via sdist, so we should add a --no-manifest option to the sdist command. This can be added today in Mercurial setup.py file as well so you don't have the problem with 2.7. I can contribute this change if you want. |
Ok, we need a change that will work with Python 2.4 through 2.7. |
This issue has broken our build system too. A bug has been fixed, but another one is open now. If the MANIFEST file has been produced by distutils it should be The problem is how to make the distinction. Maybe an special line at |
I like the idea of a marker in a comment line in the MANIFEST, we can introduce this as a bug fix to avoid this regression |
Attaching a patch to implement the marker idea. No doc changes yet. It should work transparently: you don’t have to give an option to have automatic recalculation, you don’t have to give an option to have it leave your manual MANIFEST alone. (For the curious, history is at http://beta.intuxication.org/hg/merwok/python-patches. Yay for TDD and Mercurial.) |
New version of the patch to address Tarek’s comments and add NEWS and docs entries. |
This is a regression introduced in 2.6.6rc1. After discussion in irc with merwok, it is agreed that he will revert to pre-2.6.6rc1 behavior and not apply any fix. It's way too late to be changing behavior in 2.6.6. Folks using Python 2.6 will just have to live with the devil they know. |
2.6 regression reverted in r83992. Tarek is okay with my patch for the other branches. I noticed the first fix did not have a versionadded in the docs, so I’ll add that and commit. |
Patch committed in r83993 (py3k), r83994 (3.1) and r83996 (2.7). I got the version wrong in 3.x and fixed it in r83995 and r83998. |
Patch ported to distutils2 (http://bitbucket.org/tarek/distutils2/changeset/5abba4a77f5a). Closing. Please reopen with priority set to “release blocker” if 2.6 behaves badly before Monday. Thanks. |
There is a bug in the new code. See bpo-11104. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: