Title: distutils’ build_py fails when package string is unicode
Type: behavior Stage: needs patch
Components: Distutils Versions: Python 2.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: eric.araujo Nosy List: Boris.FELD, eric.araujo, patrick.andrew, tarek
Priority: normal Keywords: easy

Created on 2012-02-04 20:41 by patrick.andrew, last changed 2014-01-10 10:10 by Boris.FELD.

File name Uploaded Description Edit patrick.andrew, 2012-02-04 20:41 build_py package name support patch
Messages (6)
msg152643 - (view) Author: Patrick Andrew (patrick.andrew) Date: 2012-02-04 20:41
When a package list is built using Unicode strings, distutils fails to build the package with a TypeError.

This patch alternatively checks for 'unicode' as the instance type and also prints the type in the type error for easier debugging.
msg152671 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-02-05 10:11
Thanks for the report and patch.  I think distutils was not written with Unicode in mind, or maybe even before Python had a unicode type.  Technically, says that unicode is not allowed for metadata fields (nothing is said about py_modules, packages and the like), but we’ve fixed a couple of bugs related to unicode, so I think this is a reasonable request.  Can you post (part of) your failing setup script?
msg152715 - (view) Author: Patrick Andrew (patrick.andrew) Date: 2012-02-06 00:56
From py-logilab-common 0.57.1 port for FreeBSD. No patches applied:

package init file './test/' not found (or not a regular file)
Traceback (most recent call last):
  File "", line 170, in <module>
  File "", line 166, in install
  File "/usr/local/lib/python2.7/distutils/", line 152, in setup
  File "/usr/local/lib/python2.7/distutils/", line 953, in run_commands
  File "/usr/local/lib/python2.7/distutils/", line 972, in run_command
  File "/usr/local/lib/python2.7/distutils/command/", line 127, in run
  File "/usr/local/lib/python2.7/distutils/", line 326, in run_command
  File "/usr/local/lib/python2.7/distutils/", line 972, in run_command
  File "/usr/local/lib/python2.7/distutils/command/", line 93, in run
  File "/usr/local/lib/python2.7/distutils/command/", line 372, in build_packages
    self.build_module(module, module_file, package)
  File "/usr/local/lib/python2.7/distutils/command/", line 333, in build_module
    "'package' must be a string (dot-separated), list, or tuple")
TypeError: 'package' must be a string (dot-separated), list, or tuple
*** Error code 1

Stop in /usr/src/ports/devel/py-logilab-common.

This package's is auto-generating the packages list with the current working directory.

def get_packages(directory, prefix):
    """return a list of subpackages for the given directory"""
    result = []
    for package in os.listdir(directory):
        absfile = join(directory, package)
        if isdir(absfile):
            if exists(join(absfile, '')) or \
                   package in ('test', 'tests'):
                if prefix:
                    result.append('%s.%s' % (prefix, package))
                result += get_packages(absfile, result[-1])
    return result


packages = [modname] + get_packages(os.getcwd(), modname)

kwargs['packages'] = packages


where modname is imported from
distname = 'logilab-common'
modname = 'common'

What's interesting is there is no explicit unicode string definition within this package list gerneration, yet the final packages list looks like:

['logilab.common', u'logilab.common.test', u'', u'', u'logilab.common.ureports']
msg207230 - (view) Author: Boris FELD (Boris.FELD) * Date: 2014-01-03 15:27
I've the same problem today with package but only when I try to install one of my project which requires httpretty, if I try to install it directly it works like a charm.

pip install httpretty -> works
pip install mypkg -> doesn't works

Looks like HTTPretty is using __file__ variable in ( and pip seems to pass the file as unicode:

At the last line: "__file__=u'/home/lothiraldan/.virtualenvs/test/build/httpretty/'"
msg207236 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2014-01-03 19:28
It’s strange that this would happen when installing as a dependency and not when installing directly.  Pip can change faster than stdlib is released, could you report the bug to them and see if it’s possible to pass __file__ as a byte string?
msg207840 - (view) Author: Boris FELD (Boris.FELD) * Date: 2014-01-10 10:10
An issue has been opened in pip repository:
Date User Action Args
2014-01-10 10:10:47Boris.FELDsetmessages: + msg207840
2014-01-03 19:28:13eric.araujosetmessages: + msg207236
2014-01-03 15:27:18Boris.FELDsetnosy: + Boris.FELD
messages: + msg207230
2012-07-02 20:56:49eric.araujosettype: behavior
stage: needs patch
2012-02-06 00:56:06patrick.andrewsetmessages: + msg152715
2012-02-05 10:11:17eric.araujosetkeywords: + easy
assignee: tarek -> eric.araujo
messages: + msg152671

title: Lib/distutils/command/build_py fails when package string is unicode -> distutils’ build_py fails when package string is unicode
2012-02-04 20:41:07patrick.andrewcreate