classification
Title: distutils’ build_py fails when package string is unicode
Type: behavior Stage: needs patch
Components: Distutils Versions: Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: eric.araujo Nosy List: Boris.FELD, eric.araujo, patrick.andrew, tarek
Priority: normal Keywords: easy

Created on 2012-02-04 20:41 by patrick.andrew, last changed 2014-01-10 10:10 by Boris.FELD.

Files
File name Uploaded Description Edit
patch-Lib-distutils-command-build_py.py patrick.andrew, 2012-02-04 20:41 build_py package name support patch
Messages (6)
msg152643 - (view) Author: Patrick Andrew (patrick.andrew) Date: 2012-02-04 20:41
When a package list is built using Unicode strings, distutils fails to build the package with a TypeError.

This patch alternatively checks for 'unicode' as the instance type and also prints the type in the type error for easier debugging.
msg152671 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-02-05 10:11
Thanks for the report and patch.  I think distutils was not written with Unicode in mind, or maybe even before Python had a unicode type.  Technically, http://docs.python.org/distutils/setupscript#additional-meta-data says that unicode is not allowed for metadata fields (nothing is said about py_modules, packages and the like), but we’ve fixed a couple of bugs related to unicode, so I think this is a reasonable request.  Can you post (part of) your failing setup script?
msg152715 - (view) Author: Patrick Andrew (patrick.andrew) Date: 2012-02-06 00:56
From py-logilab-common 0.57.1 port for FreeBSD. No patches applied:


package init file './test/__init__.py' not found (or not a regular file)
Traceback (most recent call last):
  File "setup.py", line 170, in <module>
    install()
  File "setup.py", line 166, in install
    **kwargs
  File "/usr/local/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/local/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python2.7/distutils/command/build.py", line 127, in run
    self.run_command(cmd_name)
  File "/usr/local/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python2.7/distutils/command/build_py.py", line 93, in run
    self.build_packages()
  File "/usr/local/lib/python2.7/distutils/command/build_py.py", line 372, in build_packages
    self.build_module(module, module_file, package)
  File "/usr/local/lib/python2.7/distutils/command/build_py.py", line 333, in build_module
    "'package' must be a string (dot-separated), list, or tuple")
TypeError: 'package' must be a string (dot-separated), list, or tuple
*** Error code 1

Stop in /usr/src/ports/devel/py-logilab-common.


This package's setup.py is auto-generating the packages list with the current working directory.


def get_packages(directory, prefix):
    """return a list of subpackages for the given directory"""
    result = []
    for package in os.listdir(directory):
        absfile = join(directory, package)
        if isdir(absfile):
            if exists(join(absfile, '__init__.py')) or \
                   package in ('test', 'tests'):
                if prefix:
                    result.append('%s.%s' % (prefix, package))
                else:
                    result.append(package)
                result += get_packages(absfile, result[-1])
    return result

...

packages = [modname] + get_packages(os.getcwd(), modname)

kwargs['packages'] = packages

setup(...
    **kwargs)

where modname is imported from __pkginfo__.py:
distname = 'logilab-common'
modname = 'common'


What's interesting is there is no explicit unicode string definition within this package list gerneration, yet the final packages list looks like:

['logilab.common', u'logilab.common.test', u'logilab.common.test.data', u'logilab.common.test.data.find_test', u'logilab.common.ureports']
msg207230 - (view) Author: Boris FELD (Boris.FELD) * Date: 2014-01-03 15:27
I've the same problem today with package https://pypi.python.org/pypi/httpretty/0.7.1 but only when I try to install one of my project which requires httpretty, if I try to install it directly it works like a charm.

pip install httpretty -> works
pip install mypkg -> doesn't works

Looks like HTTPretty is using __file__ variable in setup.py (https://github.com/gabrielfalcao/HTTPretty/blob/master/setup.py#L35) and pip seems to pass the file as unicode:

http://0bin.net/paste/dQfsSAmguWNYyY7w#0O/gcrWA44wKicfTdsGT4KqRYhbZLyhN9BUXNQD1XZA=

At the last line: "__file__=u'/home/lothiraldan/.virtualenvs/test/build/httpretty/setup.py'"
msg207236 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2014-01-03 19:28
It’s strange that this would happen when installing as a dependency and not when installing directly.  Pip can change faster than stdlib is released, could you report the bug to them and see if it’s possible to pass __file__ as a byte string?
msg207840 - (view) Author: Boris FELD (Boris.FELD) * Date: 2014-01-10 10:10
An issue has been opened in pip repository: https://github.com/pypa/pip/issues/1441
History
Date User Action Args
2014-01-10 10:10:47Boris.FELDsetmessages: + msg207840
2014-01-03 19:28:13eric.araujosetmessages: + msg207236
2014-01-03 15:27:18Boris.FELDsetnosy: + Boris.FELD
messages: + msg207230
2012-07-02 20:56:49eric.araujosettype: behavior
stage: needs patch
2012-02-06 00:56:06patrick.andrewsetmessages: + msg152715
2012-02-05 10:11:17eric.araujosetkeywords: + easy
assignee: tarek -> eric.araujo
messages: + msg152671

title: Lib/distutils/command/build_py fails when package string is unicode -> distutils’ build_py fails when package string is unicode
2012-02-04 20:41:07patrick.andrewcreate