classification
Title: distutils’ build_py fails when package string is unicode
Type: behavior Stage: resolved
Components: Distutils Versions: Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: eric.araujo Nosy List: Boris.FELD, eric.araujo, patrick.andrew, steve.dower, tarek
Priority: normal Keywords: easy

Created on 2012-02-04 20:41 by patrick.andrew, last changed 2021-02-03 18:30 by steve.dower. This issue is now closed.

Files
File name Uploaded Description Edit
patch-Lib-distutils-command-build_py.py patrick.andrew, 2012-02-04 20:41 build_py package name support patch
Messages (7)
msg152643 - (view) Author: Patrick Andrew (patrick.andrew) Date: 2012-02-04 20:41
When a package list is built using Unicode strings, distutils fails to build the package with a TypeError.

This patch alternatively checks for 'unicode' as the instance type and also prints the type in the type error for easier debugging.
msg152671 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-02-05 10:11
Thanks for the report and patch.  I think distutils was not written with Unicode in mind, or maybe even before Python had a unicode type.  Technically, http://docs.python.org/distutils/setupscript#additional-meta-data says that unicode is not allowed for metadata fields (nothing is said about py_modules, packages and the like), but we’ve fixed a couple of bugs related to unicode, so I think this is a reasonable request.  Can you post (part of) your failing setup script?
msg152715 - (view) Author: Patrick Andrew (patrick.andrew) Date: 2012-02-06 00:56
From py-logilab-common 0.57.1 port for FreeBSD. No patches applied:


package init file './test/__init__.py' not found (or not a regular file)
Traceback (most recent call last):
  File "setup.py", line 170, in <module>
    install()
  File "setup.py", line 166, in install
    **kwargs
  File "/usr/local/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/local/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python2.7/distutils/command/build.py", line 127, in run
    self.run_command(cmd_name)
  File "/usr/local/lib/python2.7/distutils/cmd.py", line 326, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python2.7/distutils/command/build_py.py", line 93, in run
    self.build_packages()
  File "/usr/local/lib/python2.7/distutils/command/build_py.py", line 372, in build_packages
    self.build_module(module, module_file, package)
  File "/usr/local/lib/python2.7/distutils/command/build_py.py", line 333, in build_module
    "'package' must be a string (dot-separated), list, or tuple")
TypeError: 'package' must be a string (dot-separated), list, or tuple
*** Error code 1

Stop in /usr/src/ports/devel/py-logilab-common.


This package's setup.py is auto-generating the packages list with the current working directory.


def get_packages(directory, prefix):
    """return a list of subpackages for the given directory"""
    result = []
    for package in os.listdir(directory):
        absfile = join(directory, package)
        if isdir(absfile):
            if exists(join(absfile, '__init__.py')) or \
                   package in ('test', 'tests'):
                if prefix:
                    result.append('%s.%s' % (prefix, package))
                else:
                    result.append(package)
                result += get_packages(absfile, result[-1])
    return result

...

packages = [modname] + get_packages(os.getcwd(), modname)

kwargs['packages'] = packages

setup(...
    **kwargs)

where modname is imported from __pkginfo__.py:
distname = 'logilab-common'
modname = 'common'


What's interesting is there is no explicit unicode string definition within this package list gerneration, yet the final packages list looks like:

['logilab.common', u'logilab.common.test', u'logilab.common.test.data', u'logilab.common.test.data.find_test', u'logilab.common.ureports']
msg207230 - (view) Author: Boris FELD (Boris.FELD) * Date: 2014-01-03 15:27
I've the same problem today with package https://pypi.python.org/pypi/httpretty/0.7.1 but only when I try to install one of my project which requires httpretty, if I try to install it directly it works like a charm.

pip install httpretty -> works
pip install mypkg -> doesn't works

Looks like HTTPretty is using __file__ variable in setup.py (https://github.com/gabrielfalcao/HTTPretty/blob/master/setup.py#L35) and pip seems to pass the file as unicode:

http://0bin.net/paste/dQfsSAmguWNYyY7w#0O/gcrWA44wKicfTdsGT4KqRYhbZLyhN9BUXNQD1XZA=

At the last line: "__file__=u'/home/lothiraldan/.virtualenvs/test/build/httpretty/setup.py'"
msg207236 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2014-01-03 19:28
It’s strange that this would happen when installing as a dependency and not when installing directly.  Pip can change faster than stdlib is released, could you report the bug to them and see if it’s possible to pass __file__ as a byte string?
msg207840 - (view) Author: Boris FELD (Boris.FELD) * Date: 2014-01-10 10:10
An issue has been opened in pip repository: https://github.com/pypa/pip/issues/1441
msg386427 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-02-03 18:30
Distutils is now deprecated (see PEP 632) and all tagged issues are being closed. From now until removal, only release blocking issues will be considered for distutils.

If this issue does not relate to distutils, please remove the component and reopen it. If you believe it still requires a fix, most likely the issue should be re-reported at https://github.com/pypa/setuptools
History
Date User Action Args
2021-02-03 18:30:36steve.dowersetstatus: open -> closed

nosy: + steve.dower
messages: + msg386427

resolution: out of date
stage: needs patch -> resolved
2014-01-10 10:10:47Boris.FELDsetmessages: + msg207840
2014-01-03 19:28:13eric.araujosetmessages: + msg207236
2014-01-03 15:27:18Boris.FELDsetnosy: + Boris.FELD
messages: + msg207230
2012-07-02 20:56:49eric.araujosettype: behavior
stage: needs patch
2012-02-06 00:56:06patrick.andrewsetmessages: + msg152715
2012-02-05 10:11:17eric.araujosetkeywords: + easy
assignee: tarek -> eric.araujo
messages: + msg152671

title: Lib/distutils/command/build_py fails when package string is unicode -> distutils’ build_py fails when package string is unicode
2012-02-04 20:41:07patrick.andrewcreate