classification
Title: pysetup create should not convert package_data to extra_files
Type: behavior Stage: resolved
Components: Distutils2 Versions: Python 3.3, 3rd party
process
Status: closed Resolution: fixed
Dependencies: 11805 Superseder:
Assigned To: eric.araujo Nosy List: alexis, christian.heimes, eric.araujo, erik.bray, python-dev
Priority: normal Keywords: patch

Created on 2012-01-04 14:58 by christian.heimes, last changed 2012-02-05 11:30 by eric.araujo. This issue is now closed.

Files
File name Uploaded Description Edit
fix-pysetup-create-package_data.diff eric.araujo, 2012-01-05 03:45 review
Messages (6)
msg150610 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2012-01-04 14:58
As requested in http://bugs.python.org/issue13703#msg150609

./python Lib/test/regrtest.py test_packaging
[1/1] test_packaging
Warning -- threading._dangling was modified by test_packaging
Warning -- sysconfig._SCHEMES was modified by test_packaging
test test_packaging failed -- Traceback (most recent call last):
  File "/home/heimes/dev/python/randomhash/Lib/packaging/tests/test_create.py", line 168, in test_convert_setup_py_to_cfg
    """))
AssertionError: '[metadata]\nname = pyxfoil\nversion = 0.2\nsummary = Python bindings for the Xf [truncated]... != '[metadata]\nname = pyxfoil\nversion = 0.2\nsummary = Python bindings for the Xf [truncated]...
  [metadata]
  name = pyxfoil
  version = 0.2
  summary = Python bindings for the Xfoil engine
  download_url = UNKNOWN
  home_page = http://www.python-science.org/project/pyxfoil
  maintainer = André Espaze
  maintainer_email = andre.espaze@logilab.fr
  description = My super Death-scription
         |barbar is now on the public domain,
         |ho, baby !
  
  [files]
  packages = pyxfoil
      babar
      me
  modules = my_lib
      mymodule
  scripts = my_script
      bin/run
- extra_files = setup.py
+ extra_files = Martinique/Lamentin/dady
+     Martinique/Lamentin/mumy
+     Martinique/Lamentin/sys
+     Martinique/Lamentin/bro
+     setup.py
      README
-     pyxfoil/fengine.so
      Pom
      Flora
      Alexander
+     pyxfoil/fengine.so
-     Martinique/Lamentin/dady
-     Martinique/Lamentin/mumy
-     Martinique/Lamentin/sys
-     Martinique/Lamentin/bro
  
  resources =
      README.rst = {doc}
      pyxfoil.1 = {man}
  


1 test failed:
    test_packaging
msg150618 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-01-04 16:58
Thanks, I will check this.
msg150653 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-01-05 03:45
The bug is caused by code in packaging.create that iterates over a dict (package_data) to extend a list (extra_files).  Instead of just calling sorted to make output deterministic, I’d prefer to fix that more serious behavior bug (see also #13463, #11805 and #5302 for more !fun package_data bugs).  Problem is that the setup.cfg syntax does not define how to give more than one value.  If it’s judged acceptable to disallow paths with embedded spaces, we could do something like this:

[files]
package_data =
    spam = first second third

Otherwise we’d need to use multiple lines (requested in #5302):

[files]
package_data =
    spam = first
    spam = second
    spam = third

We probably don’t want that.  An intermediate idea:

[files]
package_data =
    spam = first
           second
           third

Not sure this would be the nicest thing for people to write, and for us (me) to extend the setup.cfg parser for.

Anyway, attached patch fixes the code so that package_data in setup.py becomes package_data in setup.cfg and adapts the tests to check that, disabling multi-value package_data for now.  I tested it with distutils2 and pypy, so it should fix the hash change in your clone.
msg150669 - (view) Author: Erik Bray (erik.bray) * Date: 2012-01-05 15:44
FWIW, I'm for the first option for specifying package_data:

[files]
package_data =
    spam = first second third

I'm pretty sure this is how I ended up implementing it in d2to1, since I needed this functionality.

Theoretically spaces could be supported with an escape sequence, but I don't think that's worth complicating things for if package_data is deprecated anyways.  I'm all for making it difficult for anyone trying to include filenames with spaces in their source code.
msg152666 - (view) Author: Roundup Robot (python-dev) Date: 2012-02-05 09:49
New changeset edb6f9fb54ac by Éric Araujo in branch 'default':
Stop converting package_data to extra_files in pysetup create (#13712).
http://hg.python.org/cpython/rev/edb6f9fb54ac
msg152677 - (view) Author: Roundup Robot (python-dev) Date: 2012-02-05 11:23
New changeset 730c2e4aaf9c by Éric Araujo in branch 'default':
Stop converting package_data to extra_files in pysetup create (#13712).
http://hg.python.org/distutils2/rev/730c2e4aaf9c

New changeset ea717d8e71d0 by Éric Araujo in branch 'python3':
Merge fixes for #13901, #11805, #13712 and other improvements
http://hg.python.org/distutils2/rev/ea717d8e71d0
History
Date User Action Args
2012-02-05 11:30:54eric.araujosetstatus: open -> closed
resolution: fixed
stage: resolved
2012-02-05 11:23:57python-devsetmessages: + msg152677
2012-02-05 09:49:21python-devsetnosy: + python-dev
messages: + msg152666
2012-02-04 07:05:15eric.araujolinkissue13910 superseder
2012-02-04 07:04:25eric.araujosetdependencies: + package_data only allows one glob per-package
2012-01-05 15:44:35erik.braysetmessages: + msg150669
2012-01-05 03:45:59eric.araujosetfiles: + fix-pysetup-create-package_data.diff
title: test_packaging depends on hash order -> pysetup create should not convert package_data to extra_files
nosy: + erik.bray

messages: + msg150653

keywords: + patch
2012-01-04 16:58:14eric.araujosetmessages: + msg150618
versions: + 3rd party
2012-01-04 14:58:04christian.heimescreate