diff -r 7d390c3a83c6 -r 5359c895f841 Doc/install/install.rst --- a/Doc/install/install.rst Sat Aug 20 14:18:25 2011 +0200 +++ b/Doc/install/install.rst Mon Aug 22 21:02:47 2011 +0800 @@ -1,1117 +1,1208 @@ -.. highlightlang:: none - -==================================== -Installing Python projects: overwiew -==================================== - -.. _packaging-install-intro: - -Introduction -============ - -Although Python's extensive standard library covers many programming needs, -there often comes a time when you need to add new functionality to your Python -installation in the form of third-party modules. This might be necessary to -support your own programming, or to support an application that you want to use -and that happens to be written in Python. - -In the past, there was little support for adding third-party modules to an -existing Python installation. With the introduction of the Python Distribution -Utilities (Distutils for short) in Python 2.0, this changed. However, not all -problems were solved; end-users had to rely on ``easy_install`` or -``pip`` to download third-party modules from PyPI, uninstall distributions or do -other maintenance operations. Packaging is a more complete replacement for -Distutils, in the standard library, with a backport named Distutils2 available -for older Python versions. - -This document is aimed primarily at people who need to install third-party -Python modules: end-users and system administrators who just need to get some -Python application running, and existing Python programmers who want to add -new goodies to their toolbox. You don't need to know Python to read this -document; there will be some brief forays into using Python's interactive mode -to explore your installation, but that's it. If you're looking for information -on how to distribute your own Python modules so that others may use them, see -the :ref:`packaging-index` manual. - - -.. _packaging-trivial-install: - -Best case: trivial installation -------------------------------- - -In the best case, someone will have prepared a special version of the module -distribution you want to install that is targeted specifically at your platform -and can be installed just like any other software on your platform. For example, -the module's developer might make an executable installer available for Windows -users, an RPM package for users of RPM-based Linux systems (Red Hat, SuSE, -Mandrake, and many others), a Debian package for users of Debian and derivative -systems, and so forth. - -In that case, you would use the standard system tools to download and install -the specific installer for your platform and its dependencies. - -Of course, things will not always be that easy. You might be interested in a -module whose distribution doesn't have an easy-to-use installer for your -platform. In that case, you'll have to start with the source distribution -released by the module's author/maintainer. Installing from a source -distribution is not too hard, as long as the modules are packaged in the -standard way. The bulk of this document addresses the building and installing -of modules from standard source distributions. - - -.. _packaging-distutils: - -The Python standard: Distutils ------------------------------- - -If you download a source distribution of a module, it will be obvious whether -it was packaged and distributed using Distutils. First, the distribution's name -and version number will be featured prominently in the name of the downloaded -archive, e.g. :file:`foo-1.0.tar.gz` or :file:`widget-0.9.7.zip`. Next, the -archive will unpack into a similarly-named directory: :file:`foo-1.0` or -:file:`widget-0.9.7`. Additionally, the distribution may contain a -:file:`setup.cfg` file and a file named :file:`README.txt` ---or possibly just -:file:`README`--- explaining that building and installing the module -distribution is a simple matter of issuing the following command at your shell's -prompt:: - - python setup.py install - -Third-party projects have extended Distutils to work around its limitations or -add functionality. After some years of near-inactivity in Distutils, a new -maintainer has started to standardize good ideas in PEPs and implement them in a -new, improved version of Distutils, called Distutils2 or Packaging. - - -.. _packaging-new-standard: - -The new standard: Packaging ---------------------------- - -The rules described in the first paragraph above apply to Packaging-based -projects too: a source distribution will have a name like -:file:`widget-0.9.7.zip`. One of the main differences with Distutils is that -distributions no longer have a :file:`setup.py` script; it used to cause a -number of issues. Now there is a unique script installed with Python itself:: - - pysetup install widget-0.9.7.zip - -Running this command is enough to build and install projects (Python modules or -packages, scripts or whole applications), without even having to unpack the -archive. It is also compatible with Distutils-based distributions. - -Unless you have to perform non-standard installations or customize the build -process, you can stop reading this manual ---the above command is everything you -need to get out of it. - -With :program:`pysetup`, you won't even have to manually download a distribution -before installing it; see :ref:`packaging-pysetup`. - - -.. _packaging-standard-install: - -Standard build and install -========================== - -As described in section :ref:`packaging-new-standard`, building and installing -a module distribution using Packaging usually comes down to one simple -command:: - - pysetup run install_dist - -This is a command that should be run in a terminal. On Windows, it is called a -command prompt and found in :menuselection:`Start --> Accessories`; Powershell -is a popular alternative. - - -.. _packaging-platform-variations: - -Platform variations -------------------- - -The setup command is meant to be run from the root directory of the source -distribution, i.e. the top-level subdirectory that the module source -distribution unpacks into. For example, if you've just downloaded a module -source distribution :file:`foo-1.0.tar.gz` onto a Unix system, the normal -steps to follow are these:: - - gunzip -c foo-1.0.tar.gz | tar xf - # unpacks into directory foo-1.0 - cd foo-1.0 - pysetup run install_dist - -On Windows, you'd probably download :file:`foo-1.0.zip`. If you downloaded the -archive file to :file:`C:\\Temp`, then it would unpack into -:file:`C:\\Temp\\foo-1.0`. To actually unpack the archive, you can use either -an archive manipulator with a graphical user interface (such as WinZip or 7-Zip) -or a command-line tool (such as :program:`unzip`, :program:`pkunzip` or, again, -:program:`7z`). Then, open a command prompt window and run:: - - cd c:\Temp\foo-1.0 - pysetup run install_dist - - -.. _packaging-splitting-up: - -Splitting the job up --------------------- - -Running ``pysetup run install_dist`` builds and installs all modules in one go. If you -prefer to work incrementally ---especially useful if you want to customize the -build process, or if things are going wrong--- you can use the setup script to -do one thing at a time. This is a valuable tool when different users will perform -separately the build and install steps. For example, you might want to build a -module distribution and hand it off to a system administrator for installation -(or do it yourself, but with super-user or admin privileges). - -For example, to build everything in one step and then install everything -in a second step, you aptly invoke two distinct Packaging commands:: - - pysetup run build - pysetup run install_dist - -If you do this, you will notice that invoking the :command:`install_dist` command -first runs the :command:`build` command, which ---in this case--- quickly -notices it can spare itself the work, since everything in the :file:`build` -directory is up-to-date. - -You may often ignore this ability to divide the process in steps if all you do -is installing modules downloaded from the Internet, but it's very handy for -more advanced tasks. If you find yourself in the need for distributing your own -Python modules and extensions, though, you'll most likely run many individual -Packaging commands. - - -.. _packaging-how-build-works: - -How building works ------------------- - -As implied above, the :command:`build` command is responsible for collecting -and placing the files to be installed into a *build directory*. By default, -this is :file:`build`, under the distribution root. If you're excessively -concerned with speed, or want to keep the source tree pristine, you can specify -a different build directory with the :option:`--build-base` option. For example:: - - pysetup run build --build-base /tmp/pybuild/foo-1.0 - -(Or you could do this permanently with a directive in your system or personal -Packaging configuration file; see section :ref:`packaging-config-files`.) -In the usual case, however, all this is unnecessary. - -The build tree's default layout looks like so:: - - --- build/ --- lib/ - or - --- build/ --- lib./ - temp./ - -where ```` expands to a brief description of the current OS/hardware -platform and Python version. The first form, with just a :file:`lib` directory, -is used for pure module distributions (module distributions that -include only pure Python modules). If a module distribution contains any -extensions (modules written in C/C++), then the second form, with two ```` -directories, is used. In that case, the :file:`temp.{plat}` directory holds -temporary files generated during the compile/link process which are not intended -to be installed. In either case, the :file:`lib` (or :file:`lib.{plat}`) directory -contains all Python modules (pure Python and extensions) to be installed. - -In the future, more directories will be added to handle Python scripts, -documentation, binary executables, and whatever else is required to install -Python modules and applications. - - -.. _packaging-how-install-works: - -How installation works ----------------------- - -After the :command:`build` command is run (whether explicitly or by the -:command:`install_dist` command on your behalf), the work of the :command:`install_dist` -command is relatively simple: all it has to do is copy the contents of -:file:`build/lib` (or :file:`build/lib.{plat}`) to the installation directory -of your choice. - -If you don't choose an installation directory ---i.e., if you just run -``pysetup run install_dist``\ --- then the :command:`install_dist` command -installs to the standard location for third-party Python modules. This location -varies by platform and depending on how you built/installed Python itself. On -Unix (and Mac OS X, which is also Unix-based), it also depends on whether the -module distribution being installed is pure Python or contains extensions -("non-pure"): - -+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ -| Platform | Standard installation location | Default value | Notes | -+=================+=====================================================+==================================================+=======+ -| Unix (pure) | :file:`{prefix}/lib/python{X.Y}/site-packages` | :file:`/usr/local/lib/python{X.Y}/site-packages` | \(1) | -+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ -| Unix (non-pure) | :file:`{exec-prefix}/lib/python{X.Y}/site-packages` | :file:`/usr/local/lib/python{X.Y}/site-packages` | \(1) | -+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ -| Windows | :file:`{prefix}\\Lib\\site-packages` | :file:`C:\\Python{XY}\\Lib\\site-packages` | \(2) | -+-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ - -Notes: - -(1) - Most Linux distributions include Python as a standard part of the system, so - :file:`{prefix}` and :file:`{exec-prefix}` are usually both :file:`/usr` on - Linux. If you build Python yourself on Linux (or any Unix-like system), the - default :file:`{prefix}` and :file:`{exec-prefix}` are :file:`/usr/local`. - -(2) - The default installation directory on Windows was :file:`C:\\Program - Files\\Python` under Python 1.6a1, 1.5.2, and earlier. - -:file:`{prefix}` and :file:`{exec-prefix}` stand for the directories that Python -is installed to, and where it finds its libraries at run-time. They are always -the same under Windows, and very often the same under Unix and Mac OS X. You -can find out what your Python installation uses for :file:`{prefix}` and -:file:`{exec-prefix}` by running Python in interactive mode and typing a few -simple commands. - -.. TODO link to Doc/using instead of duplicating - -To start the interactive Python interpreter, you need to follow a slightly -different recipe for each platform. Under Unix, just type :command:`python` at -the shell prompt. Under Windows (assuming the Python executable is on your -:envvar:`PATH`, which is the usual case), you can choose :menuselection:`Start --> Run`, -type ``python`` and press ``enter``. Alternatively, you can simply execute -:command:`python` at a command prompt (:menuselection:`Start --> Accessories`) -or in Powershell. - -Once the interpreter is started, you type Python code at the prompt. For -example, on my Linux system, I type the three Python statements shown below, -and get the output as shown, to find out my :file:`{prefix}` and :file:`{exec-prefix}`:: - - Python 3.3 (r32:88445, Apr 2 2011, 10:43:54) - Type "help", "copyright", "credits" or "license" for more information. - >>> import sys - >>> sys.prefix - '/usr' - >>> sys.exec_prefix - '/usr' - -A few other placeholders are used in this document: :file:`{X.Y}` stands for the -version of Python, for example ``3.2``; :file:`{abiflags}` will be replaced by -the value of :data:`sys.abiflags` or the empty string for platforms which don't -define ABI flags; :file:`{distname}` will be replaced by the name of the module -distribution being installed. Dots and capitalization are important in the -paths; for example, a value that uses ``python3.2`` on UNIX will typically use -``Python32`` on Windows. - -If you don't want to install modules to the standard location, or if you don't -have permission to write there, then you need to read about alternate -installations in section :ref:`packaging-alt-install`. If you want to customize your -installation directories more heavily, see section :ref:`packaging-custom-install`. - - -.. _packaging-alt-install: - -Alternate installation -====================== - -Often, it is necessary or desirable to install modules to a location other than -the standard location for third-party Python modules. For example, on a Unix -system you might not have permission to write to the standard third-party module -directory. Or you might wish to try out a module before making it a standard -part of your local Python installation. This is especially true when upgrading -a distribution already present: you want to make sure your existing base of -scripts still works with the new version before actually upgrading. - -The Packaging :command:`install_dist` command is designed to make installing module -distributions to an alternate location simple and painless. The basic idea is -that you supply a base directory for the installation, and the -:command:`install_dist` command picks a set of directories (called an *installation -scheme*) under this base directory in which to install files. The details -differ across platforms, so read whichever of the following sections applies to -you. - -Note that the various alternate installation schemes are mutually exclusive: you -can pass ``--user``, or ``--home``, or ``--prefix`` and ``--exec-prefix``, or -``--install-base`` and ``--install-platbase``, but you can't mix from these -groups. - - -.. _packaging-alt-install-user: - -Alternate installation: the user scheme ---------------------------------------- - -This scheme is designed to be the most convenient solution for users that don't -have write permission to the global site-packages directory or don't want to -install into it. It is enabled with a simple option:: - - pysetup run install_dist --user - -Files will be installed into subdirectories of :data:`site.USER_BASE` (written -as :file:`{userbase}` hereafter). This scheme installs pure Python modules and -extension modules in the same location (also known as :data:`site.USER_SITE`). -Here are the values for UNIX, including non-framework builds on Mac OS X: - -=============== =========================================================== -Type of file Installation directory -=============== =========================================================== -modules :file:`{userbase}/lib/python{X.Y}/site-packages` -scripts :file:`{userbase}/bin` -data :file:`{userbase}` -C headers :file:`{userbase}/include/python{X.Y}` -=============== =========================================================== - -Framework builds on Mac OS X use these paths: - -=============== =========================================================== -Type of file Installation directory -=============== =========================================================== -modules :file:`{userbase}/lib/python/site-packages` -scripts :file:`{userbase}/bin` -data :file:`{userbase}` -C headers :file:`{userbase}/include/python` -=============== =========================================================== - -And here are the values used on Windows: - -=============== =========================================================== -Type of file Installation directory -=============== =========================================================== -modules :file:`{userbase}\\Python{XY}\\site-packages` -scripts :file:`{userbase}\\Scripts` -data :file:`{userbase}` -C headers :file:`{userbase}\\Python{XY}\\Include` -=============== =========================================================== - -The advantage of using this scheme compared to the other ones described below is -that the user site-packages directory is under normal conditions always included -in :data:`sys.path` (see :mod:`site` for more information), which means that -there is no additional step to perform after running ``pysetup`` to finalize the -installation. - -The :command:`build_ext` command also has a ``--user`` option to add -:file:`{userbase}/include` to the compiler search path for header files and -:file:`{userbase}/lib` to the compiler search path for libraries as well as to -the runtime search path for shared C libraries (rpath). - - -.. _packaging-alt-install-home: - -Alternate installation: the home scheme ---------------------------------------- - -The idea behind the "home scheme" is that you build and maintain a personal -stash of Python modules. This scheme's name is derived from the concept of a -"home" directory on Unix, since it's not unusual for a Unix user to make their -home directory have a layout similar to :file:`/usr/` or :file:`/usr/local/`. -In spite of its name's origin, this scheme can be used by anyone, regardless -of the operating system. - -Installing a new module distribution in this way is as simple as :: - - pysetup run install_dist --home - -where you can supply any directory you like for the :option:`--home` option. On -Unix, lazy typists can just type a tilde (``~``); the :command:`install_dist` command -will expand this to your home directory:: - - pysetup run install_dist --home ~ - -To make Python find the distributions installed with this scheme, you may have -to :ref:`modify Python's search path ` or edit -:mod:`sitecustomize` (see :mod:`site`) to call :func:`site.addsitedir` or edit -:data:`sys.path`. - -The :option:`--home` option defines the base directory for the installation. -Under it, files are installed to the following directories: - -=============== =========================================================== -Type of file Installation directory -=============== =========================================================== -modules :file:`{home}/lib/python` -scripts :file:`{home}/bin` -data :file:`{home}` -C headers :file:`{home}/include/python` -=============== =========================================================== - -(Mentally replace slashes with backslashes if you're on Windows.) - - -.. _packaging-alt-install-prefix-unix: - -Alternate installation: Unix (the prefix scheme) ------------------------------------------------- - -The "prefix scheme" is useful when you wish to use one Python installation to -run the build command, but install modules into the third-party module directory -of a different Python installation (or something that looks like a different -Python installation). If this sounds a trifle unusual, it is ---that's why the -user and home schemes come before. However, there are at least two known cases -where the prefix scheme will be useful. - -First, consider that many Linux distributions put Python in :file:`/usr`, rather -than the more traditional :file:`/usr/local`. This is entirely appropriate, -since in those cases Python is part of "the system" rather than a local add-on. -However, if you are installing Python modules from source, you probably want -them to go in :file:`/usr/local/lib/python2.{X}` rather than -:file:`/usr/lib/python2.{X}`. This can be done with :: - - pysetup run install_dist --prefix /usr/local - -Another possibility is a network filesystem where the name used to write to a -remote directory is different from the name used to read it: for example, the -Python interpreter accessed as :file:`/usr/local/bin/python` might search for -modules in :file:`/usr/local/lib/python2.{X}`, but those modules would have to -be installed to, say, :file:`/mnt/{@server}/export/lib/python2.{X}`. This could -be done with :: - - pysetup run install_dist --prefix=/mnt/@server/export - -In either case, the :option:`--prefix` option defines the installation base, and -the :option:`--exec-prefix` option defines the platform-specific installation -base, which is used for platform-specific files. (Currently, this just means -non-pure module distributions, but could be expanded to C libraries, binary -executables, etc.) If :option:`--exec-prefix` is not supplied, it defaults to -:option:`--prefix`. Files are installed as follows: - -================= ========================================================== -Type of file Installation directory -================= ========================================================== -Python modules :file:`{prefix}/lib/python{X.Y}/site-packages` -extension modules :file:`{exec-prefix}/lib/python{X.Y}/site-packages` -scripts :file:`{prefix}/bin` -data :file:`{prefix}` -C headers :file:`{prefix}/include/python{X.Y}{abiflags}` -================= ========================================================== - -.. XXX misses an entry for platinclude - -There is no requirement that :option:`--prefix` or :option:`--exec-prefix` -actually point to an alternate Python installation; if the directories listed -above do not already exist, they are created at installation time. - -Incidentally, the real reason the prefix scheme is important is simply that a -standard Unix installation uses the prefix scheme, but with :option:`--prefix` -and :option:`--exec-prefix` supplied by Python itself as ``sys.prefix`` and -``sys.exec_prefix``. Thus, you might think you'll never use the prefix scheme, -but every time you run ``pysetup run install_dist`` without any other -options, you're using it. - -Note that installing extensions to an alternate Python installation doesn't have -anything to do with how those extensions are built: in particular, extensions -will be compiled using the Python header files (:file:`Python.h` and friends) -installed with the Python interpreter used to run the build command. It is -therefore your responsibility to ensure compatibility between the interpreter -intended to run extensions installed in this way and the interpreter used to -build these same extensions. To avoid problems, it is best to make sure that -the two interpreters are the same version of Python (possibly different builds, -or possibly copies of the same build). (Of course, if your :option:`--prefix` -and :option:`--exec-prefix` don't even point to an alternate Python installation, -this is immaterial.) - - -.. _packaging-alt-install-prefix-windows: - -Alternate installation: Windows (the prefix scheme) ---------------------------------------------------- - -Windows has a different and vaguer notion of home directories than Unix, and -since its standard Python installation is simpler, the :option:`--prefix` option -has traditionally been used to install additional packages to arbitrary -locations. :: - - pysetup run install_dist --prefix "\Temp\Python" - -to install modules to the :file:`\\Temp\\Python` directory on the current drive. - -The installation base is defined by the :option:`--prefix` option; the -:option:`--exec-prefix` option is not supported under Windows, which means that -pure Python modules and extension modules are installed into the same location. -Files are installed as follows: - -=============== ========================================================== -Type of file Installation directory -=============== ========================================================== -modules :file:`{prefix}\\Lib\\site-packages` -scripts :file:`{prefix}\\Scripts` -data :file:`{prefix}` -C headers :file:`{prefix}\\Include` -=============== ========================================================== - - -.. _packaging-custom-install: - -Custom installation -=================== - -Sometimes, the alternate installation schemes described in section -:ref:`packaging-alt-install` just don't do what you want. You might want to tweak -just one or two directories while keeping everything under the same base -directory, or you might want to completely redefine the installation scheme. -In either case, you're creating a *custom installation scheme*. - -To create a custom installation scheme, you start with one of the alternate -schemes and override some of the installation directories used for the various -types of files, using these options: - -====================== ======================= -Type of file Override option -====================== ======================= -Python modules ``--install-purelib`` -extension modules ``--install-platlib`` -all modules ``--install-lib`` -scripts ``--install-scripts`` -data ``--install-data`` -C headers ``--install-headers`` -====================== ======================= - -These override options can be relative, absolute, -or explicitly defined in terms of one of the installation base directories. -(There are two installation base directories, and they are normally the same ----they only differ when you use the Unix "prefix scheme" and supply different -``--prefix`` and ``--exec-prefix`` options; using ``--install-lib`` will -override values computed or given for ``--install-purelib`` and -``--install-platlib``, and is recommended for schemes that don't make a -difference between Python and extension modules.) - -For example, say you're installing a module distribution to your home directory -under Unix, but you want scripts to go in :file:`~/scripts` rather than -:file:`~/bin`. As you might expect, you can override this directory with the -:option:`--install-scripts` option and, in this case, it makes most sense to supply -a relative path, which will be interpreted relative to the installation base -directory (in our example, your home directory):: - - pysetup run install_dist --home ~ --install-scripts scripts - -Another Unix example: suppose your Python installation was built and installed -with a prefix of :file:`/usr/local/python`. Thus, in a standard installation, -scripts will wind up in :file:`/usr/local/python/bin`. If you want them in -:file:`/usr/local/bin` instead, you would supply this absolute directory for -the :option:`--install-scripts` option:: - - pysetup run install_dist --install-scripts /usr/local/bin - -This command performs an installation using the "prefix scheme", where the -prefix is whatever your Python interpreter was installed with ---in this case, -:file:`/usr/local/python`. - -If you maintain Python on Windows, you might want third-party modules to live in -a subdirectory of :file:`{prefix}`, rather than right in :file:`{prefix}` -itself. This is almost as easy as customizing the script installation directory ----you just have to remember that there are two types of modules to worry about, -Python and extension modules, which can conveniently be both controlled by one -option:: - - pysetup run install_dist --install-lib Site - -.. XXX Nothing is installed right under prefix in windows, is it?? - -The specified installation directory is relative to :file:`{prefix}`. Of -course, you also have to ensure that this directory is in Python's module -search path, such as by putting a :file:`.pth` file in a site directory (see -:mod:`site`). See section :ref:`packaging-search-path` to find out how to modify -Python's search path. - -If you want to define an entire installation scheme, you just have to supply all -of the installation directory options. Using relative paths is recommended here. -For example, if you want to maintain all Python module-related files under -:file:`python` in your home directory, and you want a separate directory for -each platform that you use your home directory from, you might define the -following installation scheme:: - - pysetup run install_dist --home ~ \ - --install-purelib python/lib \ - --install-platlib python/'lib.$PLAT' \ - --install-scripts python/scripts \ - --install-data python/data - -or, equivalently, :: - - pysetup run install_dist --home ~/python \ - --install-purelib lib \ - --install-platlib 'lib.$PLAT' \ - --install-scripts scripts \ - --install-data data - -``$PLAT`` doesn't need to be defined as an environment variable ---it will also -be expanded by Packaging as it parses your command line options, just as it -does when parsing your configuration file(s). (More on that later.) - -Obviously, specifying the entire installation scheme every time you install a -new module distribution would be very tedious. To spare you all that work, you -can store it in a Packaging configuration file instead (see section -:ref:`packaging-config-files`), like so:: - - [install_dist] - install-base = $HOME - install-purelib = python/lib - install-platlib = python/lib.$PLAT - install-scripts = python/scripts - install-data = python/data - -or, equivalently, :: - - [install_dist] - install-base = $HOME/python - install-purelib = lib - install-platlib = lib.$PLAT - install-scripts = scripts - install-data = data - -Note that these two are *not* equivalent if you override their installation -base directory when running the setup script. For example, :: - - pysetup run install_dist --install-base /tmp - -would install pure modules to :file:`/tmp/python/lib` in the first case, and -to :file:`/tmp/lib` in the second case. (For the second case, you'd probably -want to supply an installation base of :file:`/tmp/python`.) - -You may have noticed the use of ``$HOME`` and ``$PLAT`` in the sample -configuration file. These are Packaging configuration variables, which -bear a strong resemblance to environment variables. In fact, you can use -environment variables in configuration files on platforms that have such a notion, but -Packaging additionally defines a few extra variables that may not be in your -environment, such as ``$PLAT``. Of course, on systems that don't have -environment variables, such as Mac OS 9, the configuration variables supplied by -the Packaging are the only ones you can use. See section :ref:`packaging-config-files` -for details. - -.. XXX which vars win out eventually in case of clash env or Packaging? - -.. XXX need some Windows examples---when would custom installation schemes be - needed on those platforms? - - -.. XXX Move this section to Doc/using - -.. _packaging-search-path: - -Modifying Python's search path ------------------------------- - -When the Python interpreter executes an :keyword:`import` statement, it searches -for both Python code and extension modules along a search path. A default value -for this path is configured into the Python binary when the interpreter is built. -You can obtain the search path by importing the :mod:`sys` module and printing -the value of ``sys.path``. :: - - $ python - Python 2.2 (#11, Oct 3 2002, 13:31:27) - [GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-112)] on linux2 - Type "help", "copyright", "credits" or "license" for more information. - >>> import sys - >>> sys.path - ['', '/usr/local/lib/python2.3', '/usr/local/lib/python2.3/plat-linux2', - '/usr/local/lib/python2.3/lib-tk', '/usr/local/lib/python2.3/lib-dynload', - '/usr/local/lib/python2.3/site-packages'] - >>> - -The null string in ``sys.path`` represents the current working directory. - -The expected convention for locally installed packages is to put them in the -:file:`{...}/site-packages/` directory, but you may want to choose a different -location for some reason. For example, if your site kept by convention all web -server-related software under :file:`/www`. Add-on Python modules might then -belong in :file:`/www/python`, and in order to import them, this directory would -have to be added to ``sys.path``. There are several ways to solve this problem. - -The most convenient way is to add a path configuration file to a directory -that's already on Python's path, usually to the :file:`.../site-packages/` -directory. Path configuration files have an extension of :file:`.pth`, and each -line must contain a single path that will be appended to ``sys.path``. (Because -the new paths are appended to ``sys.path``, modules in the added directories -will not override standard modules. This means you can't use this mechanism for -installing fixed versions of standard modules.) - -Paths can be absolute or relative, in which case they're relative to the -directory containing the :file:`.pth` file. See the documentation of -the :mod:`site` module for more information. - -A slightly less convenient way is to edit the :file:`site.py` file in Python's -standard library, and modify ``sys.path``. :file:`site.py` is automatically -imported when the Python interpreter is executed, unless the :option:`-S` switch -is supplied to suppress this behaviour. So you could simply edit -:file:`site.py` and add two lines to it:: - - import sys - sys.path.append('/www/python/') - -However, if you reinstall the same major version of Python (perhaps when -upgrading from 3.3 to 3.3.1, for example) :file:`site.py` will be overwritten by -the stock version. You'd have to remember that it was modified and save a copy -before doing the installation. - -Alternatively, there are two environment variables that can modify ``sys.path``. -:envvar:`PYTHONHOME` sets an alternate value for the prefix of the Python -installation. For example, if :envvar:`PYTHONHOME` is set to ``/www/python``, -the search path will be set to ``['', '/www/python/lib/pythonX.Y/', -'/www/python/lib/pythonX.Y/plat-linux2', ...]``. - -The :envvar:`PYTHONPATH` variable can be set to a list of paths that will be -added to the beginning of ``sys.path``. For example, if :envvar:`PYTHONPATH` is -set to ``/www/python:/opt/py``, the search path will begin with -``['/www/python', '/opt/py']``. (Note that directories must exist in order to -be added to ``sys.path``; the :mod:`site` module removes non-existent paths.) - -Finally, ``sys.path`` is just a regular Python list, so any Python application -can modify it by adding or removing entries. - - -.. _packaging-config-files: - -Configuration files for Packaging -================================= - -As mentioned above, you can use configuration files to store personal or site -preferences for any option supported by any Packaging command. Depending on your -platform, you can use one of two or three possible configuration files. These -files will be read before parsing the command-line, so they take precedence over -default values. In turn, the command-line will override configuration files. -Lastly, if there are multiple configuration files, values from files read -earlier will be overridden by values from files read later. - -.. XXX "one of two or three possible..." seems wrong info. Below always 3 files - are indicated in the tables. - - -.. _packaging-config-filenames: - -Location and names of configuration files ------------------------------------------ - -The name and location of the configuration files vary slightly across -platforms. On Unix and Mac OS X, these are the three configuration files listed -in the order they are processed: - -+--------------+----------------------------------------------------------+-------+ -| Type of file | Location and filename | Notes | -+==============+==========================================================+=======+ -| system | :file:`{prefix}/lib/python{ver}/packaging/packaging.cfg` | \(1) | -+--------------+----------------------------------------------------------+-------+ -| personal | :file:`$HOME/.pydistutils.cfg` | \(2) | -+--------------+----------------------------------------------------------+-------+ -| local | :file:`setup.cfg` | \(3) | -+--------------+----------------------------------------------------------+-------+ - -Similarly, the configuration files on Windows ---also listed in the order they -are processed--- are these: - -+--------------+-------------------------------------------------+-------+ -| Type of file | Location and filename | Notes | -+==============+=================================================+=======+ -| system | :file:`{prefix}\\Lib\\packaging\\packaging.cfg` | \(4) | -+--------------+-------------------------------------------------+-------+ -| personal | :file:`%HOME%\\pydistutils.cfg` | \(5) | -+--------------+-------------------------------------------------+-------+ -| local | :file:`setup.cfg` | \(3) | -+--------------+-------------------------------------------------+-------+ - -On all platforms, the *personal* file can be temporarily disabled by -means of the `--no-user-cfg` option. - -Notes: - -(1) - Strictly speaking, the system-wide configuration file lives in the directory - where Packaging is installed. - -(2) - On Unix, if the :envvar:`HOME` environment variable is not defined, the - user's home directory will be determined with the :func:`getpwuid` function - from the standard :mod:`pwd` module. Packaging uses the - :func:`os.path.expanduser` function to do this. - -(3) - I.e., in the current directory (usually the location of the setup script). - -(4) - (See also note (1).) Python's default installation prefix is - :file:`C:\\Python`, so the system configuration file is normally - :file:`C:\\Python\\Lib\\packaging\\packaging.cfg`. - -(5) - On Windows, if the :envvar:`HOME` environment variable is not defined, - :envvar:`USERPROFILE` then :envvar:`HOMEDRIVE` and :envvar:`HOMEPATH` will - be tried. Packaging uses the :func:`os.path.expanduser` function to do this. - - -.. _packaging-config-syntax: - -Syntax of configuration files ------------------------------ - -All Packaging configuration files share the same syntax. Options defined in -them are grouped into sections, and each Packaging command gets its own section. -Additionally, there's a ``global`` section for options that affect every command. -Sections consist of one or more lines containing a single option specified as -``option = value``. - -For example, here's a complete configuration file that forces all commands to -run quietly by default:: - - [global] - verbose = 0 - -If this was the system configuration file, it would affect all processing -of any Python module distribution by any user on the current system. If it was -installed as your personal configuration file (on systems that support them), -it would affect only module distributions processed by you. Lastly, if it was -used as the :file:`setup.cfg` for a particular module distribution, it would -affect that distribution only. - -.. XXX "(on systems that support them)" seems wrong info - -If you wanted to, you could override the default "build base" directory and -make the :command:`build\*` commands always forcibly rebuild all files with -the following:: - - [build] - build-base = blib - force = 1 - -which corresponds to the command-line arguments:: - - pysetup run build --build-base blib --force - -except that including the :command:`build` command on the command-line means -that command will be run. Including a particular command in configuration files -has no such implication; it only means that if the command is run, the options -for it in the configuration file will apply. (This is also true if you run -other commands that derive values from it.) - -You can find out the complete list of options for any command using the -:option:`--help` option, e.g.:: - - pysetup run build --help - -and you can find out the complete list of global options by using -:option:`--help` without a command:: - - pysetup run --help - -See also the "Reference" section of the "Distributing Python Modules" manual. - -.. XXX no links to the relevant section exist. - - -.. _packaging-building-ext: - -Building extensions: tips and tricks -==================================== - -Whenever possible, Packaging tries to use the configuration information made -available by the Python interpreter used to run `pysetup`. -For example, the same compiler and linker flags used to compile Python will also -be used for compiling extensions. Usually this will work well, but in -complicated situations this might be inappropriate. This section discusses how -to override the usual Packaging behaviour. - - -.. _packaging-tweak-flags: - -Tweaking compiler/linker flags ------------------------------- - -Compiling a Python extension written in C or C++ will sometimes require -specifying custom flags for the compiler and linker in order to use a particular -library or produce a special kind of object code. This is especially true if the -extension hasn't been tested on your platform, or if you're trying to -cross-compile Python. - -.. TODO update to new setup.cfg - -In the most general case, the extension author might have foreseen that -compiling the extensions would be complicated, and provided a :file:`Setup` file -for you to edit. This will likely only be done if the module distribution -contains many separate extension modules, or if they often require elaborate -sets of compiler flags in order to work. - -A :file:`Setup` file, if present, is parsed in order to get a list of extensions -to build. Each line in a :file:`Setup` describes a single module. Lines have -the following structure:: - - module ... [sourcefile ...] [cpparg ...] [library ...] - - -Let's examine each of the fields in turn. - -* *module* is the name of the extension module to be built, and should be a - valid Python identifier. You can't just change this in order to rename a module - (edits to the source code would also be needed), so this should be left alone. - -* *sourcefile* is anything that's likely to be a source code file, at least - judging by the filename. Filenames ending in :file:`.c` are assumed to be - written in C, filenames ending in :file:`.C`, :file:`.cc`, and :file:`.c++` are - assumed to be C++, and filenames ending in :file:`.m` or :file:`.mm` are assumed - to be in Objective C. - -* *cpparg* is an argument for the C preprocessor, and is anything starting with - :option:`-I`, :option:`-D`, :option:`-U` or :option:`-C`. - -* *library* is anything ending in :file:`.a` or beginning with :option:`-l` or - :option:`-L`. - -If a particular platform requires a special library on your platform, you can -add it by editing the :file:`Setup` file and running ``pysetup run build``. -For example, if the module defined by the line :: - - foo foomodule.c - -must be linked with the math library :file:`libm.a` on your platform, simply add -:option:`-lm` to the line:: - - foo foomodule.c -lm - -Arbitrary switches intended for the compiler or the linker can be supplied with -the :option:`-Xcompiler` *arg* and :option:`-Xlinker` *arg* options:: - - foo foomodule.c -Xcompiler -o32 -Xlinker -shared -lm - -The next option after :option:`-Xcompiler` and :option:`-Xlinker` will be -appended to the proper command line, so in the above example the compiler will -be passed the :option:`-o32` option, and the linker will be passed -:option:`-shared`. If a compiler option requires an argument, you'll have to -supply multiple :option:`-Xcompiler` options; for example, to pass ``-x c++`` -the :file:`Setup` file would have to contain ``-Xcompiler -x -Xcompiler c++``. - -Compiler flags can also be supplied through setting the :envvar:`CFLAGS` -environment variable. If set, the contents of :envvar:`CFLAGS` will be added to -the compiler flags specified in the :file:`Setup` file. - - -.. _packaging-non-ms-compilers: - -Using non-Microsoft compilers on Windows ----------------------------------------- - -.. sectionauthor:: Rene Liebscher - - - -Borland/CodeGear C++ -^^^^^^^^^^^^^^^^^^^^ - -This subsection describes the necessary steps to use Packaging with the Borland -C++ compiler version 5.5. First you have to know that Borland's object file -format (OMF) is different from the format used by the Python version you can -download from the Python or ActiveState Web site. (Python is built with -Microsoft Visual C++, which uses COFF as the object file format.) For this -reason, you have to convert Python's library :file:`python25.lib` into the -Borland format. You can do this as follows: - -.. Should we mention that users have to create cfg-files for the compiler? -.. see also http://community.borland.com/article/0,1410,21205,00.html - -:: - - coff2omf python25.lib python25_bcpp.lib - -The :file:`coff2omf` program comes with the Borland compiler. The file -:file:`python25.lib` is in the :file:`Libs` directory of your Python -installation. If your extension uses other libraries (zlib, ...) you have to -convert them too. - -The converted files have to reside in the same directories as the normal -libraries. - -How does Packaging manage to use these libraries with their changed names? If -the extension needs a library (eg. :file:`foo`) Packaging checks first if it -finds a library with suffix :file:`_bcpp` (eg. :file:`foo_bcpp.lib`) and then -uses this library. In the case it doesn't find such a special library it uses -the default name (:file:`foo.lib`.) [#]_ - -To let Packaging compile your extension with Borland, C++ you now have to -type:: - - pysetup run build --compiler bcpp - -If you want to use the Borland C++ compiler as the default, you could specify -this in your personal or system-wide configuration file for Packaging (see -section :ref:`packaging-config-files`.) - - -.. seealso:: - - `C++Builder Compiler `_ - Information about the free C++ compiler from Borland, including links to the - download pages. - - `Creating Python Extensions Using Borland's Free Compiler `_ - Document describing how to use Borland's free command-line C++ compiler to build - Python. - - -GNU C / Cygwin / MinGW -^^^^^^^^^^^^^^^^^^^^^^ - -This section describes the necessary steps to use Packaging with the GNU C/C++ -compilers in their Cygwin and MinGW distributions. [#]_ For a Python interpreter -that was built with Cygwin, everything should work without any of these -following steps. - -Not all extensions can be built with MinGW or Cygwin, but many can. Extensions -most likely to not work are those that use C++ or depend on Microsoft Visual C -extensions. - -To let Packaging compile your extension with Cygwin, you have to type:: - - pysetup run build --compiler=cygwin - -and for Cygwin in no-cygwin mode [#]_ or for MinGW, type:: - - pysetup run build --compiler=mingw32 - -If you want to use any of these options/compilers as default, you should -consider writing it in your personal or system-wide configuration file for -Packaging (see section :ref:`packaging-config-files`.) - -Older Versions of Python and MinGW -"""""""""""""""""""""""""""""""""" -The following instructions only apply if you're using a version of Python -inferior to 2.4.1 with a MinGW inferior to 3.0.0 (with -:file:`binutils-2.13.90-20030111-1`). - -These compilers require some special libraries. This task is more complex than -for Borland's C++, because there is no program to convert the library. First -you have to create a list of symbols which the Python DLL exports. (You can find -a good program for this task at -http://www.emmestech.com/software/pexports-0.43/download_pexports.html). - -.. I don't understand what the next line means. --amk - (inclusive the references on data structures.) - -:: - - pexports python25.dll > python25.def - -The location of an installed :file:`python25.dll` will depend on the -installation options and the version and language of Windows. In a "just for -me" installation, it will appear in the root of the installation directory. In -a shared installation, it will be located in the system directory. - -Then you can create from these information an import library for gcc. :: - - /cygwin/bin/dlltool --dllname python25.dll --def python25.def --output-lib libpython25.a - -The resulting library has to be placed in the same directory as -:file:`python25.lib`. (Should be the :file:`libs` directory under your Python -installation directory.) - -If your extension uses other libraries (zlib,...) you might have to convert -them too. The converted files have to reside in the same directories as the -normal libraries do. - - -.. seealso:: - - `Building Python modules on MS Windows platform with MinGW `_ - Information about building the required libraries for the MinGW - environment. - - -.. rubric:: Footnotes - -.. [#] This also means you could replace all existing COFF-libraries with - OMF-libraries of the same name. - -.. [#] Check http://sources.redhat.com/cygwin/ and http://www.mingw.org/ for - more information. - -.. [#] Then you have no POSIX emulation available, but you also don't need - :file:`cygwin1.dll`. +.. highlightlang:: none + +==================================== +Installing Python projects: overwiew +==================================== + +.. _packaging-install-intro: + +Introduction +============ + +Although Python's extensive standard library covers many programming needs, +there often comes a time when you need to add new functionality to your Python +installation in the form of third-party modules. This might be necessary to +support your own programming, or to support an application that you want to use +and that happens to be written in Python. + +In the past, there was little support for adding third-party modules to an +existing Python installation. With the introduction of the Python Distribution +Utilities (Distutils for short) in Python 2.0, this changed. However, not all +problems were solved; end-users had to rely on ``easy_install`` or +``pip`` to download third-party modules from PyPI, uninstall distributions or do +other maintenance operations. Packaging is a more complete replacement for +Distutils, in the standard library, with a backport named Distutils2 available +for older Python versions. + +This document is aimed primarily at people who need to install third-party +Python modules: end-users and system administrators who just need to get some +Python application running, and existing Python programmers who want to add +new goodies to their toolbox. You don't need to know Python to read this +document; there will be some brief forays into using Python's interactive mode +to explore your installation, but that's it. If you're looking for information +on how to distribute your own Python modules so that others may use them, see +the :ref:`packaging-index` manual. + + +.. _packaging-trivial-install: + +Best case: trivial installation +------------------------------- + +In the best case, someone will have prepared a special version of the module +distribution you want to install that is targeted specifically at your platform +and can be installed just like any other software on your platform. For example, +the module's developer might make an executable installer available for Windows +users, an RPM package for users of RPM-based Linux systems (Red Hat, SuSE, +Mandrake, and many others), a Debian package for users of Debian and derivative +systems, and so forth. + +In that case, you would use the standard system tools to download and install +the specific installer for your platform and its dependencies. + +Of course, things will not always be that easy. You might be interested in a +module whose distribution doesn't have an easy-to-use installer for your +platform. In that case, you'll have to start with the source distribution +released by the module's author/maintainer. Installing from a source +distribution is not too hard, as long as the modules are packaged in the +standard way. The bulk of this document addresses the building and installing +of modules from standard source distributions. + + +.. _packaging-distutils: + +The Python standard: Distutils +------------------------------ + +If you download a source distribution of a module, it will be obvious whether +it was packaged and distributed using Distutils. First, the distribution's name +and version number will be featured prominently in the name of the downloaded +archive, e.g. :file:`foo-1.0.tar.gz` or :file:`widget-0.9.7.zip`. Next, the +archive will unpack into a similarly-named directory: :file:`foo-1.0` or +:file:`widget-0.9.7`. Additionally, the distribution may contain a +:file:`setup.cfg` file and a file named :file:`README.txt` ---or possibly just +:file:`README`--- explaining that building and installing the module +distribution is a simple matter of issuing the following command at your shell's +prompt:: + + python setup.py install + +Third-party projects have extended Distutils to work around its limitations or +add functionality. After some years of near-inactivity in Distutils, a new +maintainer has started to standardize good ideas in PEPs and implement them in a +new, improved version of Distutils, called Distutils2 or Packaging. + + +.. _packaging-new-standard: + +The new standard: Packaging +--------------------------- + +The rules described in the first paragraph above apply to Packaging-based +projects too: a source distribution will have a name like +:file:`widget-0.9.7.zip`. One of the main differences with Distutils is that +distributions no longer have a :file:`setup.py` script; it used to cause a +number of issues. Now there is a unique script installed with Python itself:: + + pysetup install widget-0.9.7.zip + +Running this command is enough to build and install projects (Python modules or +packages, scripts or whole applications), without even having to unpack the +archive. It is also compatible with Distutils-based distributions. + +Unless you have to perform non-standard installations or customize the build +process, you can stop reading this manual ---the above command is everything you +need to get out of it. + +With :program:`pysetup`, you won't even have to manually download a distribution +before installing it; see :ref:`packaging-pysetup`. + + +.. _packaging-standard-install: + +Standard build and install +========================== + +As described in section :ref:`packaging-new-standard`, building and installing +a module distribution using Packaging usually comes down to one simple +command:: + + pysetup run install_dist + +This is a command that should be run in a terminal. On Windows, it is called a +command prompt and found in :menuselection:`Start --> Accessories`; Powershell +is a popular alternative. + + +.. _packaging-platform-variations: + +Platform variations +------------------- + +The setup command is meant to be run from the root directory of the source +distribution, i.e. the top-level subdirectory that the module source +distribution unpacks into. For example, if you've just downloaded a module +source distribution :file:`foo-1.0.tar.gz` onto a Unix system, the normal +steps to follow are these:: + + gunzip -c foo-1.0.tar.gz | tar xf - # unpacks into directory foo-1.0 + cd foo-1.0 + pysetup run install_dist + +On Windows, you'd probably download :file:`foo-1.0.zip`. If you downloaded the +archive file to :file:`C:\\Temp`, then it would unpack into +:file:`C:\\Temp\\foo-1.0`. To actually unpack the archive, you can use either +an archive manipulator with a graphical user interface (such as WinZip or 7-Zip) +or a command-line tool (such as :program:`unzip`, :program:`pkunzip` or, again, +:program:`7z`). Then, open a command prompt window and run:: + + cd c:\Temp\foo-1.0 + pysetup run install_dist + + +.. _packaging-splitting-up: + +Splitting the job up +-------------------- + +Running ``pysetup run install_dist`` builds and installs all modules in one go. If you +prefer to work incrementally ---especially useful if you want to customize the +build process, or if things are going wrong--- you can use the setup script to +do one thing at a time. This is a valuable tool when different users will perform +separately the build and install steps. For example, you might want to build a +module distribution and hand it off to a system administrator for installation +(or do it yourself, but with super-user or admin privileges). + +For example, to build everything in one step and then install everything +in a second step, you aptly invoke two distinct Packaging commands:: + + pysetup run build + pysetup run install_dist + +If you do this, you will notice that invoking the :command:`install_dist` command +first runs the :command:`build` command, which ---in this case--- quickly +notices it can spare itself the work, since everything in the :file:`build` +directory is up-to-date. + +You may often ignore this ability to divide the process in steps if all you do +is installing modules downloaded from the Internet, but it's very handy for +more advanced tasks. If you find yourself in the need for distributing your own +Python modules and extensions, though, you'll most likely run many individual +Packaging commands. + + +.. _packaging-how-build-works: + +How building works +------------------ + +As implied above, the :command:`build` command is responsible for collecting +and placing the files to be installed into a *build directory*. By default, +this is :file:`build`, under the distribution root. If you're excessively +concerned with speed, or want to keep the source tree pristine, you can specify +a different build directory with the :option:`--build-base` option. For example:: + + pysetup run build --build-base /tmp/pybuild/foo-1.0 + +(Or you could do this permanently with a directive in your system or personal +Packaging configuration file; see section :ref:`packaging-config-files`.) +In the usual case, however, all this is unnecessary. + +The build tree's default layout looks like so:: + + --- build/ --- lib/ + or + --- build/ --- lib./ + temp./ + +where ```` expands to a brief description of the current OS/hardware +platform and Python version. The first form, with just a :file:`lib` directory, +is used for pure module distributions (module distributions that +include only pure Python modules). If a module distribution contains any +extensions (modules written in C/C++), then the second form, with two ```` +directories, is used. In that case, the :file:`temp.{plat}` directory holds +temporary files generated during the compile/link process which are not intended +to be installed. In either case, the :file:`lib` (or :file:`lib.{plat}`) directory +contains all Python modules (pure Python and extensions) to be installed. + +In the future, more directories will be added to handle Python scripts, +documentation, binary executables, and whatever else is required to install +Python modules and applications. + + +.. _packaging-how-install-works: + +How installation works +---------------------- + +After the :command:`build` command is run (whether explicitly or by the +:command:`install_dist` command on your behalf), the work of the :command:`install_dist` +command is relatively simple: all it has to do is copy the contents of +:file:`build/lib` (or :file:`build/lib.{plat}`) to the installation directory +of your choice. + +If you don't choose an installation directory ---i.e., if you just run +``pysetup run install_dist``\ --- then the :command:`install_dist` command +installs to the standard location for third-party Python modules. This location +varies by platform and depending on how you built/installed Python itself. On +Unix (and Mac OS X, which is also Unix-based), it also depends on whether the +module distribution being installed is pure Python or contains extensions +("non-pure"): + ++-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ +| Platform | Standard installation location | Default value | Notes | ++=================+=====================================================+==================================================+=======+ +| Unix (pure) | :file:`{prefix}/lib/python{X.Y}/site-packages` | :file:`/usr/local/lib/python{X.Y}/site-packages` | \(1) | ++-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ +| Unix (non-pure) | :file:`{exec-prefix}/lib/python{X.Y}/site-packages` | :file:`/usr/local/lib/python{X.Y}/site-packages` | \(1) | ++-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ +| Windows | :file:`{prefix}\\Lib\\site-packages` | :file:`C:\\Python{XY}\\Lib\\site-packages` | \(2) | ++-----------------+-----------------------------------------------------+--------------------------------------------------+-------+ + +Notes: + +(1) + Most Linux distributions include Python as a standard part of the system, so + :file:`{prefix}` and :file:`{exec-prefix}` are usually both :file:`/usr` on + Linux. If you build Python yourself on Linux (or any Unix-like system), the + default :file:`{prefix}` and :file:`{exec-prefix}` are :file:`/usr/local`. + +(2) + The default installation directory on Windows was :file:`C:\\Program + Files\\Python` under Python 1.6a1, 1.5.2, and earlier. + +:file:`{prefix}` and :file:`{exec-prefix}` stand for the directories that Python +is installed to, and where it finds its libraries at run-time. They are always +the same under Windows, and very often the same under Unix and Mac OS X. You +can find out what your Python installation uses for :file:`{prefix}` and +:file:`{exec-prefix}` by running Python in interactive mode and typing a few +simple commands. + +.. TODO link to Doc/using instead of duplicating + +To start the interactive Python interpreter, you need to follow a slightly +different recipe for each platform. Under Unix, just type :command:`python` at +the shell prompt. Under Windows (assuming the Python executable is on your +:envvar:`PATH`, which is the usual case), you can choose :menuselection:`Start --> Run`, +type ``python`` and press ``enter``. Alternatively, you can simply execute +:command:`python` at a command prompt (:menuselection:`Start --> Accessories`) +or in Powershell. + +Once the interpreter is started, you type Python code at the prompt. For +example, on my Linux system, I type the three Python statements shown below, +and get the output as shown, to find out my :file:`{prefix}` and :file:`{exec-prefix}`:: + + Python 3.3 (r32:88445, Apr 2 2011, 10:43:54) + Type "help", "copyright", "credits" or "license" for more information. + >>> import sys + >>> sys.prefix + '/usr' + >>> sys.exec_prefix + '/usr' + +A few other placeholders are used in this document: :file:`{X.Y}` stands for the +version of Python, for example ``3.2``; :file:`{abiflags}` will be replaced by +the value of :data:`sys.abiflags` or the empty string for platforms which don't +define ABI flags; :file:`{distname}` will be replaced by the name of the module +distribution being installed. Dots and capitalization are important in the +paths; for example, a value that uses ``python3.2`` on UNIX will typically use +``Python32`` on Windows. + +If you don't want to install modules to the standard location, or if you don't +have permission to write there, then you need to read about alternate +installations in section :ref:`packaging-alt-install`. If you want to customize your +installation directories more heavily, see section :ref:`packaging-custom-install`. + + +.. _packaging-alt-install: + +Alternate installation +====================== + +Often, it is necessary or desirable to install modules to a location other than +the standard location for third-party Python modules. For example, on a Unix +system you might not have permission to write to the standard third-party module +directory. Or you might wish to try out a module before making it a standard +part of your local Python installation. This is especially true when upgrading +a distribution already present: you want to make sure your existing base of +scripts still works with the new version before actually upgrading. + +The Packaging :command:`install_dist` command is designed to make installing module +distributions to an alternate location simple and painless. The basic idea is +that you supply a base directory for the installation, and the +:command:`install_dist` command picks a set of directories (called an *installation +scheme*) under this base directory in which to install files. The details +differ across platforms, so read whichever of the following sections applies to +you. + +Note that the various alternate installation schemes are mutually exclusive: you +can pass ``--user``, or ``--home``, or ``--prefix`` and ``--exec-prefix``, or +``--install-base`` and ``--install-platbase``, but you can't mix from these +groups. + + +.. _packaging-alt-install-user: + +Alternate installation: the user scheme +--------------------------------------- + +This scheme is designed to be the most convenient solution for users that don't +have write permission to the global site-packages directory or don't want to +install into it. It is enabled with a simple option:: + + pysetup run install_dist --user + +Files will be installed into subdirectories of :data:`site.USER_BASE` (written +as :file:`{userbase}` hereafter). This scheme installs pure Python modules and +extension modules in the same location (also known as :data:`site.USER_SITE`). +Here are the values for UNIX, including non-framework builds on Mac OS X: + +=============== =========================================================== +Type of file Installation directory +=============== =========================================================== +modules :file:`{userbase}/lib/python{X.Y}/site-packages` +scripts :file:`{userbase}/bin` +data :file:`{userbase}` +C headers :file:`{userbase}/include/python{X.Y}` +=============== =========================================================== + +Framework builds on Mac OS X use these paths: + +=============== =========================================================== +Type of file Installation directory +=============== =========================================================== +modules :file:`{userbase}/lib/python/site-packages` +scripts :file:`{userbase}/bin` +data :file:`{userbase}` +C headers :file:`{userbase}/include/python` +=============== =========================================================== + +And here are the values used on Windows: + +=============== =========================================================== +Type of file Installation directory +=============== =========================================================== +modules :file:`{userbase}\\Python{XY}\\site-packages` +scripts :file:`{userbase}\\Scripts` +data :file:`{userbase}` +C headers :file:`{userbase}\\Python{XY}\\Include` +=============== =========================================================== + +The advantage of using this scheme compared to the other ones described below is +that the user site-packages directory is under normal conditions always included +in :data:`sys.path` (see :mod:`site` for more information), which means that +there is no additional step to perform after running ``pysetup`` to finalize the +installation. + +The :command:`build_ext` command also has a ``--user`` option to add +:file:`{userbase}/include` to the compiler search path for header files and +:file:`{userbase}/lib` to the compiler search path for libraries as well as to +the runtime search path for shared C libraries (rpath). + + +.. _packaging-alt-install-home: + +Alternate installation: the home scheme +--------------------------------------- + +The idea behind the "home scheme" is that you build and maintain a personal +stash of Python modules. This scheme's name is derived from the concept of a +"home" directory on Unix, since it's not unusual for a Unix user to make their +home directory have a layout similar to :file:`/usr/` or :file:`/usr/local/`. +In spite of its name's origin, this scheme can be used by anyone, regardless +of the operating system. + +Installing a new module distribution in this way is as simple as :: + + pysetup run install_dist --home + +where you can supply any directory you like for the :option:`--home` option. On +Unix, lazy typists can just type a tilde (``~``); the :command:`install_dist` command +will expand this to your home directory:: + + pysetup run install_dist --home ~ + +To make Python find the distributions installed with this scheme, you may have +to :ref:`modify Python's search path ` or edit +:mod:`sitecustomize` (see :mod:`site`) to call :func:`site.addsitedir` or edit +:data:`sys.path`. + +The :option:`--home` option defines the base directory for the installation. +Under it, files are installed to the following directories: + +=============== =========================================================== +Type of file Installation directory +=============== =========================================================== +modules :file:`{home}/lib/python` +scripts :file:`{home}/bin` +data :file:`{home}` +C headers :file:`{home}/include/python` +=============== =========================================================== + +(Mentally replace slashes with backslashes if you're on Windows.) + + +.. _packaging-alt-install-prefix-unix: + +Alternate installation: Unix (the prefix scheme) +------------------------------------------------ + +The "prefix scheme" is useful when you wish to use one Python installation to +run the build command, but install modules into the third-party module directory +of a different Python installation (or something that looks like a different +Python installation). If this sounds a trifle unusual, it is ---that's why the +user and home schemes come before. However, there are at least two known cases +where the prefix scheme will be useful. + +First, consider that many Linux distributions put Python in :file:`/usr`, rather +than the more traditional :file:`/usr/local`. This is entirely appropriate, +since in those cases Python is part of "the system" rather than a local add-on. +However, if you are installing Python modules from source, you probably want +them to go in :file:`/usr/local/lib/python2.{X}` rather than +:file:`/usr/lib/python2.{X}`. This can be done with :: + + pysetup run install_dist --prefix /usr/local + +Another possibility is a network filesystem where the name used to write to a +remote directory is different from the name used to read it: for example, the +Python interpreter accessed as :file:`/usr/local/bin/python` might search for +modules in :file:`/usr/local/lib/python2.{X}`, but those modules would have to +be installed to, say, :file:`/mnt/{@server}/export/lib/python2.{X}`. This could +be done with :: + + pysetup run install_dist --prefix=/mnt/@server/export + +In either case, the :option:`--prefix` option defines the installation base, and +the :option:`--exec-prefix` option defines the platform-specific installation +base, which is used for platform-specific files. (Currently, this just means +non-pure module distributions, but could be expanded to C libraries, binary +executables, etc.) If :option:`--exec-prefix` is not supplied, it defaults to +:option:`--prefix`. Files are installed as follows: + +================= ========================================================== +Type of file Installation directory +================= ========================================================== +Python modules :file:`{prefix}/lib/python{X.Y}/site-packages` +extension modules :file:`{exec-prefix}/lib/python{X.Y}/site-packages` +scripts :file:`{prefix}/bin` +data :file:`{prefix}` +C headers :file:`{prefix}/include/python{X.Y}{abiflags}` +================= ========================================================== + +.. XXX misses an entry for platinclude + +There is no requirement that :option:`--prefix` or :option:`--exec-prefix` +actually point to an alternate Python installation; if the directories listed +above do not already exist, they are created at installation time. + +Incidentally, the real reason the prefix scheme is important is simply that a +standard Unix installation uses the prefix scheme, but with :option:`--prefix` +and :option:`--exec-prefix` supplied by Python itself as ``sys.prefix`` and +``sys.exec_prefix``. Thus, you might think you'll never use the prefix scheme, +but every time you run ``pysetup run install_dist`` without any other +options, you're using it. + +Note that installing extensions to an alternate Python installation doesn't have +anything to do with how those extensions are built: in particular, extensions +will be compiled using the Python header files (:file:`Python.h` and friends) +installed with the Python interpreter used to run the build command. It is +therefore your responsibility to ensure compatibility between the interpreter +intended to run extensions installed in this way and the interpreter used to +build these same extensions. To avoid problems, it is best to make sure that +the two interpreters are the same version of Python (possibly different builds, +or possibly copies of the same build). (Of course, if your :option:`--prefix` +and :option:`--exec-prefix` don't even point to an alternate Python installation, +this is immaterial.) + + +.. _packaging-alt-install-prefix-windows: + +Alternate installation: Windows (the prefix scheme) +--------------------------------------------------- + +Windows has a different and vaguer notion of home directories than Unix, and +since its standard Python installation is simpler, the :option:`--prefix` option +has traditionally been used to install additional packages to arbitrary +locations. :: + + pysetup run install_dist --prefix "\Temp\Python" + +to install modules to the :file:`\\Temp\\Python` directory on the current drive. + +The installation base is defined by the :option:`--prefix` option; the +:option:`--exec-prefix` option is not supported under Windows, which means that +pure Python modules and extension modules are installed into the same location. +Files are installed as follows: + +=============== ========================================================== +Type of file Installation directory +=============== ========================================================== +modules :file:`{prefix}\\Lib\\site-packages` +scripts :file:`{prefix}\\Scripts` +data :file:`{prefix}` +C headers :file:`{prefix}\\Include` +=============== ========================================================== + + +.. _packaging-custom-install: + +Custom installation +=================== + +Sometimes, the alternate installation schemes described in section +:ref:`packaging-alt-install` just don't do what you want. You might want to tweak +just one or two directories while keeping everything under the same base +directory, or you might want to completely redefine the installation scheme. +In either case, you're creating a *custom installation scheme*. + +To create a custom installation scheme, you start with one of the alternate +schemes and override some of the installation directories used for the various +types of files, using these options: + +====================== ======================= +Type of file Override option +====================== ======================= +Python modules ``--install-purelib`` +extension modules ``--install-platlib`` +all modules ``--install-lib`` +scripts ``--install-scripts`` +data ``--install-data`` +C headers ``--install-headers`` +====================== ======================= + +These override options can be relative, absolute, +or explicitly defined in terms of one of the installation base directories. +(There are two installation base directories, and they are normally the same +---they only differ when you use the Unix "prefix scheme" and supply different +``--prefix`` and ``--exec-prefix`` options; using ``--install-lib`` will +override values computed or given for ``--install-purelib`` and +``--install-platlib``, and is recommended for schemes that don't make a +difference between Python and extension modules.) + +For example, say you're installing a module distribution to your home directory +under Unix, but you want scripts to go in :file:`~/scripts` rather than +:file:`~/bin`. As you might expect, you can override this directory with the +:option:`--install-scripts` option and, in this case, it makes most sense to supply +a relative path, which will be interpreted relative to the installation base +directory (in our example, your home directory):: + + pysetup run install_dist --home ~ --install-scripts scripts + +Another Unix example: suppose your Python installation was built and installed +with a prefix of :file:`/usr/local/python`. Thus, in a standard installation, +scripts will wind up in :file:`/usr/local/python/bin`. If you want them in +:file:`/usr/local/bin` instead, you would supply this absolute directory for +the :option:`--install-scripts` option:: + + pysetup run install_dist --install-scripts /usr/local/bin + +This command performs an installation using the "prefix scheme", where the +prefix is whatever your Python interpreter was installed with ---in this case, +:file:`/usr/local/python`. + +If you maintain Python on Windows, you might want third-party modules to live in +a subdirectory of :file:`{prefix}`, rather than right in :file:`{prefix}` +itself. This is almost as easy as customizing the script installation directory +---you just have to remember that there are two types of modules to worry about, +Python and extension modules, which can conveniently be both controlled by one +option:: + + pysetup run install_dist --install-lib Site + +.. XXX Nothing is installed right under prefix in windows, is it?? + +The specified installation directory is relative to :file:`{prefix}`. Of +course, you also have to ensure that this directory is in Python's module +search path, such as by putting a :file:`.pth` file in a site directory (see +:mod:`site`). See section :ref:`packaging-search-path` to find out how to modify +Python's search path. + +If you want to define an entire installation scheme, you just have to supply all +of the installation directory options. Using relative paths is recommended here. +For example, if you want to maintain all Python module-related files under +:file:`python` in your home directory, and you want a separate directory for +each platform that you use your home directory from, you might define the +following installation scheme:: + + pysetup run install_dist --home ~ \ + --install-purelib python/lib \ + --install-platlib python/'lib.$PLAT' \ + --install-scripts python/scripts \ + --install-data python/data + +or, equivalently, :: + + pysetup run install_dist --home ~/python \ + --install-purelib lib \ + --install-platlib 'lib.$PLAT' \ + --install-scripts scripts \ + --install-data data + +``$PLAT`` doesn't need to be defined as an environment variable ---it will also +be expanded by Packaging as it parses your command line options, just as it +does when parsing your configuration file(s). (More on that later.) + +Obviously, specifying the entire installation scheme every time you install a +new module distribution would be very tedious. To spare you all that work, you +can store it in a Packaging configuration file instead (see section +:ref:`packaging-config-files`), like so:: + + [install_dist] + install-base = $HOME + install-purelib = python/lib + install-platlib = python/lib.$PLAT + install-scripts = python/scripts + install-data = python/data + +or, equivalently, :: + + [install_dist] + install-base = $HOME/python + install-purelib = lib + install-platlib = lib.$PLAT + install-scripts = scripts + install-data = data + +Note that these two are *not* equivalent if you override their installation +base directory when running the setup script. For example, :: + + pysetup run install_dist --install-base /tmp + +would install pure modules to :file:`/tmp/python/lib` in the first case, and +to :file:`/tmp/lib` in the second case. (For the second case, you'd probably +want to supply an installation base of :file:`/tmp/python`.) + +You may have noticed the use of ``$HOME`` and ``$PLAT`` in the sample +configuration file. These are Packaging configuration variables, which +bear a strong resemblance to environment variables. In fact, you can use +environment variables in configuration files on platforms that have such a notion, but +Packaging additionally defines a few extra variables that may not be in your +environment, such as ``$PLAT``. Of course, on systems that don't have +environment variables, such as Mac OS 9, the configuration variables supplied by +the Packaging are the only ones you can use. See section :ref:`packaging-config-files` +for details. + +.. XXX which vars win out eventually in case of clash env or Packaging? + +.. XXX need some Windows examples---when would custom installation schemes be + needed on those platforms? + + +.. XXX Move this section to Doc/using + +.. _packaging-search-path: + +Modifying Python's search path +------------------------------ + +When the Python interpreter executes an :keyword:`import` statement, it searches +for both Python code and extension modules along a search path. A default value +for this path is configured into the Python binary when the interpreter is built. +You can obtain the search path by importing the :mod:`sys` module and printing +the value of ``sys.path``. :: + + $ python + Python 2.2 (#11, Oct 3 2002, 13:31:27) + [GCC 2.96 20000731 (Red Hat Linux 7.3 2.96-112)] on linux2 + Type "help", "copyright", "credits" or "license" for more information. + >>> import sys + >>> sys.path + ['', '/usr/local/lib/python2.3', '/usr/local/lib/python2.3/plat-linux2', + '/usr/local/lib/python2.3/lib-tk', '/usr/local/lib/python2.3/lib-dynload', + '/usr/local/lib/python2.3/site-packages'] + >>> + +The null string in ``sys.path`` represents the current working directory. + +The expected convention for locally installed packages is to put them in the +:file:`{...}/site-packages/` directory, but you may want to choose a different +location for some reason. For example, if your site kept by convention all web +server-related software under :file:`/www`. Add-on Python modules might then +belong in :file:`/www/python`, and in order to import them, this directory would +have to be added to ``sys.path``. There are several ways to solve this problem. + +The most convenient way is to add a path configuration file to a directory +that's already on Python's path, usually to the :file:`.../site-packages/` +directory. Path configuration files have an extension of :file:`.pth`, and each +line must contain a single path that will be appended to ``sys.path``. (Because +the new paths are appended to ``sys.path``, modules in the added directories +will not override standard modules. This means you can't use this mechanism for +installing fixed versions of standard modules.) + +Paths can be absolute or relative, in which case they're relative to the +directory containing the :file:`.pth` file. See the documentation of +the :mod:`site` module for more information. + +A slightly less convenient way is to edit the :file:`site.py` file in Python's +standard library, and modify ``sys.path``. :file:`site.py` is automatically +imported when the Python interpreter is executed, unless the :option:`-S` switch +is supplied to suppress this behaviour. So you could simply edit +:file:`site.py` and add two lines to it:: + + import sys + sys.path.append('/www/python/') + +However, if you reinstall the same major version of Python (perhaps when +upgrading from 3.3 to 3.3.1, for example) :file:`site.py` will be overwritten by +the stock version. You'd have to remember that it was modified and save a copy +before doing the installation. + +Alternatively, there are two environment variables that can modify ``sys.path``. +:envvar:`PYTHONHOME` sets an alternate value for the prefix of the Python +installation. For example, if :envvar:`PYTHONHOME` is set to ``/www/python``, +the search path will be set to ``['', '/www/python/lib/pythonX.Y/', +'/www/python/lib/pythonX.Y/plat-linux2', ...]``. + +The :envvar:`PYTHONPATH` variable can be set to a list of paths that will be +added to the beginning of ``sys.path``. For example, if :envvar:`PYTHONPATH` is +set to ``/www/python:/opt/py``, the search path will begin with +``['/www/python', '/opt/py']``. (Note that directories must exist in order to +be added to ``sys.path``; the :mod:`site` module removes non-existent paths.) + +Finally, ``sys.path`` is just a regular Python list, so any Python application +can modify it by adding or removing entries. + +.. _packaging-install-develop-mode: + +Special installation - install a project in development mode +============================================================ + +Although we can install a project as above, but under normal circumstances, +it will often rebuilds or reinstalls the projects after some changes have been +madeto it during development. It's a little boring and will take time to copy +files. Thus a very important feature is to install a project in development +mode, which allows users to make the code apear as installed to Python and +other code can import it, but without copying any files. It's very useful +especially for developers using version control system to manage code. + +Installing a project in development mode usaully comes down to one simple command:: + + pysetup run develop + +As described in section :ref:`packaging-standard-install`, running this command +also depends on these three different platforms and the corresponding command line +interfaces. + +Using this command +------------------ + +This command also has platform variations, but it's too easy to use - just executing +the command line ``pysetup run develop`` under the project source directory. Look at +the examples shown in section :ref:`packaging-platform-variations`; they are also +very good examples to learn how to use ``develop`` command. + +Alternate installation +---------------------- + +When we say ``develop`` command is used to pseudo-install a project without rebuilding +and reinstalling the whole project and copying any files, we just mean it does not +rebuild and reinstall the pure :file:`.py` files in the project and copy them. If +there are some C extensions or similarly compiled files in current project, it still +needs to build them and copy them if you have made some changes to them. + +Actually to make our project importable even though we does not really install it, +we install a :file:`.pth` file under the specific directory and defaultly in site-packages. +And also for a development purpose, we install a :file:`.distinfo-link` file in the +specific directory. These are just development details, so the users of this command +can ignore them in the most circumstances, but we clarify it here because there are +still some special cases which need users know them, for example, the default +site-packages does not permit to write, the user/developer just wants to manage the +:file:`.pth` files in a specific directory, e.t.c. + +Thus, it's still necessary to offer users some ways to alternate the default +installation scheme. There are mainly two aspects that we need to consider to complete +a custom installation. + + + +Change the default installation directory +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +It's also easy to set the installation directory - there is an :option:`install-dir` option +offered. If this option is not directly specified on the command line or in a packaging +configuration file, the default installation location is used. Normally, this will be +the site-packages directory, but if you are using packaging configuration files, setting +things like :file:`{prefix}` or :file:`{install_lib}`, then those settings are taken into +account when computing the default staging area. + +We can set the installation directory simply by:: + + pysetup run develop --install-dir DIR + +Please pay attention to the section :ref:`packaging-custom-install`, it also shows good +use cases to complete a custom installation on different platforms and reminds the tips +that we should remember when using "develop" command. When this option is set, the +:file:`.pth` file and :file:`.distinfo-link` file will be installed under the specific +directory if there aren't other settings. And also please note here: the ``DIR`` should be +in Python's module search path. + + + +Manage the .pth files +^^^^^^^^^^^^^^^^^^^^^ + +After there are many projects are 'installed' with ``develop`` command, many :file:`.pth` +files will be created in default site-packages if we don't do some other settings. You +may want to manage these :file:`.pth` files in a specific directory or for some unknown +purpose you just want to install the :file:`.pth` file into a different direcotry, then +you can have a look at the :option:`install-pth` option. It's also very easy to use this +option:: + + pysetup run develop --install-pth=PTHDIR + +The do's and don'ts also refer to the section :ref:`packaging-custom-install`. Also +note ``PTHDIR`` should be on the Python's search path. + + +.. _packaging-config-files: + +Configuration files for Packaging +================================= + +As mentioned above, you can use configuration files to store personal or site +preferences for any option supported by any Packaging command. Depending on your +platform, you can use one of two or three possible configuration files. These +files will be read before parsing the command-line, so they take precedence over +default values. In turn, the command-line will override configuration files. +Lastly, if there are multiple configuration files, values from files read +earlier will be overridden by values from files read later. + +.. XXX "one of two or three possible..." seems wrong info. Below always 3 files + are indicated in the tables. + + +.. _packaging-config-filenames: + +Location and names of configuration files +----------------------------------------- + +The name and location of the configuration files vary slightly across +platforms. On Unix and Mac OS X, these are the three configuration files listed +in the order they are processed: + ++--------------+----------------------------------------------------------+-------+ +| Type of file | Location and filename | Notes | ++==============+==========================================================+=======+ +| system | :file:`{prefix}/lib/python{ver}/packaging/packaging.cfg` | \(1) | ++--------------+----------------------------------------------------------+-------+ +| personal | :file:`$HOME/.pydistutils.cfg` | \(2) | ++--------------+----------------------------------------------------------+-------+ +| local | :file:`setup.cfg` | \(3) | ++--------------+----------------------------------------------------------+-------+ + +Similarly, the configuration files on Windows ---also listed in the order they +are processed--- are these: + ++--------------+-------------------------------------------------+-------+ +| Type of file | Location and filename | Notes | ++==============+=================================================+=======+ +| system | :file:`{prefix}\\Lib\\packaging\\packaging.cfg` | \(4) | ++--------------+-------------------------------------------------+-------+ +| personal | :file:`%HOME%\\pydistutils.cfg` | \(5) | ++--------------+-------------------------------------------------+-------+ +| local | :file:`setup.cfg` | \(3) | ++--------------+-------------------------------------------------+-------+ + +On all platforms, the *personal* file can be temporarily disabled by +means of the `--no-user-cfg` option. + +Notes: + +(1) + Strictly speaking, the system-wide configuration file lives in the directory + where Packaging is installed. + +(2) + On Unix, if the :envvar:`HOME` environment variable is not defined, the + user's home directory will be determined with the :func:`getpwuid` function + from the standard :mod:`pwd` module. Packaging uses the + :func:`os.path.expanduser` function to do this. + +(3) + I.e., in the current directory (usually the location of the setup script). + +(4) + (See also note (1).) Python's default installation prefix is + :file:`C:\\Python`, so the system configuration file is normally + :file:`C:\\Python\\Lib\\packaging\\packaging.cfg`. + +(5) + On Windows, if the :envvar:`HOME` environment variable is not defined, + :envvar:`USERPROFILE` then :envvar:`HOMEDRIVE` and :envvar:`HOMEPATH` will + be tried. Packaging uses the :func:`os.path.expanduser` function to do this. + + +.. _packaging-config-syntax: + +Syntax of configuration files +----------------------------- + +All Packaging configuration files share the same syntax. Options defined in +them are grouped into sections, and each Packaging command gets its own section. +Additionally, there's a ``global`` section for options that affect every command. +Sections consist of one or more lines containing a single option specified as +``option = value``. + +For example, here's a complete configuration file that forces all commands to +run quietly by default:: + + [global] + verbose = 0 + +If this was the system configuration file, it would affect all processing +of any Python module distribution by any user on the current system. If it was +installed as your personal configuration file (on systems that support them), +it would affect only module distributions processed by you. Lastly, if it was +used as the :file:`setup.cfg` for a particular module distribution, it would +affect that distribution only. + +.. XXX "(on systems that support them)" seems wrong info + +If you wanted to, you could override the default "build base" directory and +make the :command:`build\*` commands always forcibly rebuild all files with +the following:: + + [build] + build-base = blib + force = 1 + +which corresponds to the command-line arguments:: + + pysetup run build --build-base blib --force + +except that including the :command:`build` command on the command-line means +that command will be run. Including a particular command in configuration files +has no such implication; it only means that if the command is run, the options +for it in the configuration file will apply. (This is also true if you run +other commands that derive values from it.) + +You can find out the complete list of options for any command using the +:option:`--help` option, e.g.:: + + pysetup run build --help + +and you can find out the complete list of global options by using +:option:`--help` without a command:: + + pysetup run --help + +See also the "Reference" section of the "Distributing Python Modules" manual. + +.. XXX no links to the relevant section exist. + + +.. _packaging-building-ext: + +Building extensions: tips and tricks +==================================== + +Whenever possible, Packaging tries to use the configuration information made +available by the Python interpreter used to run `pysetup`. +For example, the same compiler and linker flags used to compile Python will also +be used for compiling extensions. Usually this will work well, but in +complicated situations this might be inappropriate. This section discusses how +to override the usual Packaging behaviour. + + +.. _packaging-tweak-flags: + +Tweaking compiler/linker flags +------------------------------ + +Compiling a Python extension written in C or C++ will sometimes require +specifying custom flags for the compiler and linker in order to use a particular +library or produce a special kind of object code. This is especially true if the +extension hasn't been tested on your platform, or if you're trying to +cross-compile Python. + +.. TODO update to new setup.cfg + +In the most general case, the extension author might have foreseen that +compiling the extensions would be complicated, and provided a :file:`Setup` file +for you to edit. This will likely only be done if the module distribution +contains many separate extension modules, or if they often require elaborate +sets of compiler flags in order to work. + +A :file:`Setup` file, if present, is parsed in order to get a list of extensions +to build. Each line in a :file:`Setup` describes a single module. Lines have +the following structure:: + + module ... [sourcefile ...] [cpparg ...] [library ...] + + +Let's examine each of the fields in turn. + +* *module* is the name of the extension module to be built, and should be a + valid Python identifier. You can't just change this in order to rename a module + (edits to the source code would also be needed), so this should be left alone. + +* *sourcefile* is anything that's likely to be a source code file, at least + judging by the filename. Filenames ending in :file:`.c` are assumed to be + written in C, filenames ending in :file:`.C`, :file:`.cc`, and :file:`.c++` are + assumed to be C++, and filenames ending in :file:`.m` or :file:`.mm` are assumed + to be in Objective C. + +* *cpparg* is an argument for the C preprocessor, and is anything starting with + :option:`-I`, :option:`-D`, :option:`-U` or :option:`-C`. + +* *library* is anything ending in :file:`.a` or beginning with :option:`-l` or + :option:`-L`. + +If a particular platform requires a special library on your platform, you can +add it by editing the :file:`Setup` file and running ``pysetup run build``. +For example, if the module defined by the line :: + + foo foomodule.c + +must be linked with the math library :file:`libm.a` on your platform, simply add +:option:`-lm` to the line:: + + foo foomodule.c -lm + +Arbitrary switches intended for the compiler or the linker can be supplied with +the :option:`-Xcompiler` *arg* and :option:`-Xlinker` *arg* options:: + + foo foomodule.c -Xcompiler -o32 -Xlinker -shared -lm + +The next option after :option:`-Xcompiler` and :option:`-Xlinker` will be +appended to the proper command line, so in the above example the compiler will +be passed the :option:`-o32` option, and the linker will be passed +:option:`-shared`. If a compiler option requires an argument, you'll have to +supply multiple :option:`-Xcompiler` options; for example, to pass ``-x c++`` +the :file:`Setup` file would have to contain ``-Xcompiler -x -Xcompiler c++``. + +Compiler flags can also be supplied through setting the :envvar:`CFLAGS` +environment variable. If set, the contents of :envvar:`CFLAGS` will be added to +the compiler flags specified in the :file:`Setup` file. + + +.. _packaging-non-ms-compilers: + +Using non-Microsoft compilers on Windows +---------------------------------------- + +.. sectionauthor:: Rene Liebscher + + + +Borland/CodeGear C++ +^^^^^^^^^^^^^^^^^^^^ + +This subsection describes the necessary steps to use Packaging with the Borland +C++ compiler version 5.5. First you have to know that Borland's object file +format (OMF) is different from the format used by the Python version you can +download from the Python or ActiveState Web site. (Python is built with +Microsoft Visual C++, which uses COFF as the object file format.) For this +reason, you have to convert Python's library :file:`python25.lib` into the +Borland format. You can do this as follows: + +.. Should we mention that users have to create cfg-files for the compiler? +.. see also http://community.borland.com/article/0,1410,21205,00.html + +:: + + coff2omf python25.lib python25_bcpp.lib + +The :file:`coff2omf` program comes with the Borland compiler. The file +:file:`python25.lib` is in the :file:`Libs` directory of your Python +installation. If your extension uses other libraries (zlib, ...) you have to +convert them too. + +The converted files have to reside in the same directories as the normal +libraries. + +How does Packaging manage to use these libraries with their changed names? If +the extension needs a library (eg. :file:`foo`) Packaging checks first if it +finds a library with suffix :file:`_bcpp` (eg. :file:`foo_bcpp.lib`) and then +uses this library. In the case it doesn't find such a special library it uses +the default name (:file:`foo.lib`.) [#]_ + +To let Packaging compile your extension with Borland, C++ you now have to +type:: + + pysetup run build --compiler bcpp + +If you want to use the Borland C++ compiler as the default, you could specify +this in your personal or system-wide configuration file for Packaging (see +section :ref:`packaging-config-files`.) + + +.. seealso:: + + `C++Builder Compiler `_ + Information about the free C++ compiler from Borland, including links to the + download pages. + + `Creating Python Extensions Using Borland's Free Compiler `_ + Document describing how to use Borland's free command-line C++ compiler to build + Python. + + +GNU C / Cygwin / MinGW +^^^^^^^^^^^^^^^^^^^^^^ + +This section describes the necessary steps to use Packaging with the GNU C/C++ +compilers in their Cygwin and MinGW distributions. [#]_ For a Python interpreter +that was built with Cygwin, everything should work without any of these +following steps. + +Not all extensions can be built with MinGW or Cygwin, but many can. Extensions +most likely to not work are those that use C++ or depend on Microsoft Visual C +extensions. + +To let Packaging compile your extension with Cygwin, you have to type:: + + pysetup run build --compiler=cygwin + +and for Cygwin in no-cygwin mode [#]_ or for MinGW, type:: + + pysetup run build --compiler=mingw32 + +If you want to use any of these options/compilers as default, you should +consider writing it in your personal or system-wide configuration file for +Packaging (see section :ref:`packaging-config-files`.) + +Older Versions of Python and MinGW +"""""""""""""""""""""""""""""""""" +The following instructions only apply if you're using a version of Python +inferior to 2.4.1 with a MinGW inferior to 3.0.0 (with +:file:`binutils-2.13.90-20030111-1`). + +These compilers require some special libraries. This task is more complex than +for Borland's C++, because there is no program to convert the library. First +you have to create a list of symbols which the Python DLL exports. (You can find +a good program for this task at +http://www.emmestech.com/software/pexports-0.43/download_pexports.html). + +.. I don't understand what the next line means. --amk + (inclusive the references on data structures.) + +:: + + pexports python25.dll > python25.def + +The location of an installed :file:`python25.dll` will depend on the +installation options and the version and language of Windows. In a "just for +me" installation, it will appear in the root of the installation directory. In +a shared installation, it will be located in the system directory. + +Then you can create from these information an import library for gcc. :: + + /cygwin/bin/dlltool --dllname python25.dll --def python25.def --output-lib libpython25.a + +The resulting library has to be placed in the same directory as +:file:`python25.lib`. (Should be the :file:`libs` directory under your Python +installation directory.) + +If your extension uses other libraries (zlib,...) you might have to convert +them too. The converted files have to reside in the same directories as the +normal libraries do. + + +.. seealso:: + + `Building Python modules on MS Windows platform with MinGW `_ + Information about building the required libraries for the MinGW + environment. + + +.. rubric:: Footnotes + +.. [#] This also means you could replace all existing COFF-libraries with + OMF-libraries of the same name. + +.. [#] Check http://sources.redhat.com/cygwin/ and http://www.mingw.org/ for + more information. + +.. [#] Then you have no POSIX emulation available, but you also don't need + :file:`cygwin1.dll`. diff -r 7d390c3a83c6 -r 5359c895f841 Doc/library/packaging.install.rst --- a/Doc/library/packaging.install.rst Sat Aug 20 14:18:25 2011 +0200 +++ b/Doc/library/packaging.install.rst Mon Aug 22 21:02:47 2011 +0800 @@ -98,6 +98,14 @@ :file:`setup.py` script (in which case Distutils will be used under the hood to perform the installation). +.. function:: install_editable(path) + + Pseudo-install a project from a source directory, which will not copy any + files and need not rebuild or reinstall the whole project after you have + edited it. If the source directory contains a :file:`setup.py` using the + setuptools' develop. If a :file:`setup.cfg` is found, install using the + packaging's develop. + .. function:: remove(project_name, paths=None, auto_confirm=True) diff -r 7d390c3a83c6 -r 5359c895f841 Lib/packaging/command/__init__.py --- a/Lib/packaging/command/__init__.py Sat Aug 20 14:18:25 2011 +0200 +++ b/Lib/packaging/command/__init__.py Mon Aug 22 21:02:47 2011 +0800 @@ -22,6 +22,7 @@ 'install_data': 'packaging.command.install_data.install_data', 'install_distinfo': 'packaging.command.install_distinfo.install_distinfo', + 'develop': 'packaging.command.develop.develop', 'sdist': 'packaging.command.sdist.sdist', 'bdist': 'packaging.command.bdist.bdist', 'bdist_dumb': 'packaging.command.bdist_dumb.bdist_dumb', diff -r 7d390c3a83c6 -r 5359c895f841 Lib/packaging/command/cmd.py --- a/Lib/packaging/command/cmd.py Sat Aug 20 14:18:25 2011 +0200 +++ b/Lib/packaging/command/cmd.py Mon Aug 22 21:02:47 2011 +0800 @@ -318,9 +318,12 @@ cmd_obj.ensure_finalized() return cmd_obj - def get_reinitialized_command(self, command, reinit_subcommands=False): - return self.distribution.get_reinitialized_command( - command, reinit_subcommands) + def get_reinitialized_command(self, command, reinit_subcommands=False, **kw): + cmd_obj = self.distribution.get_reinitialized_command(command, + reinit_subcommands) + for k,v in kw.items(): + setattr(cmd_obj,k,v) #update command object with keywords + return cmd_obj def run_command(self, command): """Run some other command: uses the 'run_command()' method of diff -r 7d390c3a83c6 -r 5359c895f841 Lib/packaging/command/develop.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/Lib/packaging/command/develop.py Mon Aug 22 21:02:47 2011 +0800 @@ -0,0 +1,127 @@ +"""Install a project in development mode. + +This command can make your project importable without copying files, +through creating a .pth file in your site-packages dir or a specified +dir. +""" +import os +import sys +from packaging import logger +from packaging.dist import Distribution +from packaging.util import write_file, convert_path +from packaging.errors import PackagingOptionError +from packaging.command.cmd import Command +from packaging.command.install_distinfo import to_filename, safe_name + + +class develop(Command): + + description = "install the project in development mode" + + user_options = [ + ('uninstall', None, + "uninstall this project"), + ('install-scripts=', None, + "install scripts to DIR"), + ('no-scripts', None, + "don't install scripts"), + ('install-pth=', None, + "install the .pth file to a specific DIR which should be on sys.path"), + ('distinfo-path=', None, + "set the specified relative path to be used in the .distinfo-link file")] + + boolean_options = ['uninstall', 'no-scripts'] + + def initialize_options(self): + self.uninstall = None + self.install_pth = None + self.distinfo_path = None + self.install_dir = None + self.basename = None # the base name of .pth and .distinfo-link file + + def finalize_options(self): + di = self.get_finalized_command("install_distinfo") + + self.basename = to_filename(safe_name(di.distribution.metadata['Name'])) + + if self.install_dir is None: + self.set_undefined_options('install_dist', ('install_lib', 'install_dir')) + + if self.install_pth is None: + self.install_pth = self.install_dir + + self.pth_file = os.path.join(self.install_pth, self.basename + '.pth') + + sys_path = [normalize_path(path) for path in sys.path] + if normalize_path(self.install_pth) not in sys_path: + logger.warning("your specific 'install_pth' should be on sys.path") + + self.distinfo_link = os.path.join(self.install_dir, self.basename + '.distinfo-link') + + self.source_base = os.curdir # the source directory + if self.distinfo_path is None: + self.distinfo_path = os.path.abspath(self.source_base) + + target = normalize_path(self.source_base) + if normalize_path(os.path.join(self.install_dir, self.distinfo_path)) != target: + raise PackagingOptionError("'distinfo-path' must be a relative path from the install" + " directory to " + target) + + def run(self): + if self.uninstall: + self.uninstall_link() + else: + self.install_for_development() + + def install_for_development(self): + """Install the project in development mode.""" + + # Ensure metadata is up-to-date + # Install the .dist-info directory in-place + self.get_reinitialized_command('install_distinfo', distinfo_dir=os.curdir) + self.run_command('install_distinfo') + # Build extensions in-place if necessary + self.get_reinitialized_command('build_ext', inplace=1) + self.run_command('build_ext') + + # Create a .distinfo-link file in the installation dir, pointing to our .dist-info dir + logger.info("creating %r (link to %r)", self.distinfo_link, self.source_base) + + if not self.dry_run: + with open(self.distinfo_link, "w") as f: + f.write(self.distinfo_path + "\n") + + self.update_path_file() + + # xxx should fix: install wrapper scripts if 'no-scripts' is False + + def uninstall_link(self): + """Uninstall all the installed files - .pth and .distinfo-link file.""" + + if os.path.exists(self.pth_file): + logger.info("removing %r ", self.pth_file) + if not self.dry_run: + os.unlink(self.pth_file) + if os.path.exists(self.distinfo_link): + logger.info("removing %r ", self.distinfo_link) + if not self.dry_run: + os.unlink(self.distinfo_link) + # if wrapper scripts are installed, then should also remove these scripts here. + + def update_path_file(self): + """Updates or create a .pth file.""" + + self.distinfo_location = normalize_path(self.source_base) + + # Remove old .pth file and create new one if necessary + if os.path.islink(self.pth_file): + os.path.unlink(self.pth_file) + logger.info('removing old path file %r', self.pth_file) + + self.execute(write_file, (self.pth_file, [self.distinfo_location]), + "creating %r" % self.pth_file) + +def normalize_path(filename): + """Normalize a file/dir name for comparison purposes.""" + + return os.path.normcase(os.path.realpath(filename)) diff -r 7d390c3a83c6 -r 5359c895f841 Lib/packaging/install.py --- a/Lib/packaging/install.py Sat Aug 20 14:18:25 2011 +0200 +++ b/Lib/packaging/install.py Mon Aug 22 21:02:47 2011 +0800 @@ -90,6 +90,26 @@ except (IOError, os.error, PackagingError, CCompilerError) as msg: raise ValueError("Failed to install, " + str(msg)) +def _run_setuptools_develop(path): + cmd = '%s setup.py develop --record=%s --single-version-externally-managed' + record_file = os.path.join(path, 'RECORD') + + os.system(cmd % (sys.executable, record_file)) + if not os.path.exists(record_file): + raise ValueError('failed to install') + else: + egginfo_to_distinfo(record_file, remove_egginfo=True) + +def _run_packaging_develop(path): + # check for a valid setup.cfg + dist = Distribution() + dist.parse_config_files() + try: + dist.run_command('develop') + name = dist.metadata['name'] + return database.get_distribution(name) is not None + except (IOError, os.error, PackagingError, CCompilerError) as msg: + raise ValueError("Fail to install in development mode, " + str(msg)) def _install_dist(dist, path): """Install a distribution into a path. @@ -132,6 +152,28 @@ logger.warning('No project to install.') return False +def install_editable(path): + """Pseudo-install a project from a source directory, which will not copy + any files and need not rebuild or reinstall the whole project after you + have edited it. + + If the source directory contains a setup.py using the setuptools' develop. + If a setup.cfg is found, install using the packaging's develop . + + Returns True on success, False on Failure. + """ + path = os.path.abspath(path) + if os.path.isdir(path): + logger.info('Installing an editable version from source directory: %r', path) + return _run_develop_from_dir(path) + elif _is_archive_file(path): + logger.info('Installing an editable version from archive: %r', path) + _unpacked_dir = tempfile.mkdtemp() + shutil.unpack_archive(path, _unpacked_dir) + return _run_develop_from_archive(_unpacked_dir) + else: + logger.warning('No projects to install as editable.') + return False def _run_install_from_archive(source_dir): # XXX need a better way @@ -142,12 +184,23 @@ break return _run_install_from_dir(source_dir) +def _run_develop_from_archive(source_dir): + for item in os.listdir(source_dir): + fullpath = os.path.join(source_dir, item) + if os.path.isdir(fullpath): + source_dir = fullpath + break + return _run_develop_from_dir(source_dir) install_methods = { 'packaging': _run_packaging_install, 'setuptools': _run_setuptools_install, 'distutils': _run_distutils_install} +develop_methods = { + 'packaging': _run_packaging_develop, + 'setuptools': _run_setuptools_develop} + def _run_install_from_dir(source_dir): old_dir = os.getcwd() @@ -166,6 +219,20 @@ finally: os.chdir(old_dir) +def _run_develop_from_dir(source_dir): + old_dir = os.getcwd() + os.chdir(source_dir) + develop_method = get_develop_method(source_dir) + try: + func = develop_methods[develop_method] + try: + func(source_dir) + return True + except ValueError as err: + logger.info(str(err)) + return False + finally: + os.chdir(old_dir) def install_dists(dists, path, paths=None): """Install all distributions provided in dists, with the given prefix. diff -r 7d390c3a83c6 -r 5359c895f841 Lib/packaging/run.py --- a/Lib/packaging/run.py Sat Aug 20 14:18:25 2011 +0200 +++ b/Lib/packaging/run.py Mon Aug 22 21:02:47 2011 +0800 @@ -1,693 +1,692 @@ -"""Main command line parser. Implements the pysetup script.""" - -import os -import re -import sys -import getopt -import logging - -from packaging import logger -from packaging.dist import Distribution -from packaging.util import _is_archive_file, generate_setup_py -from packaging.command import get_command_class, STANDARD_COMMANDS -from packaging.install import install, install_local_project, remove -from packaging.database import get_distribution, get_distributions -from packaging.depgraph import generate_graph -from packaging.fancy_getopt import FancyGetopt -from packaging.errors import (PackagingArgError, PackagingError, - PackagingModuleError, PackagingClassError, - CCompilerError) - - -command_re = re.compile(r'^[a-zA-Z]([a-zA-Z0-9_]*)$') - -common_usage = """\ -Actions: -%(actions)s - -To get more help on an action, use: - - pysetup action --help -""" - -create_usage = """\ -Usage: pysetup create - or: pysetup create --help - -Create a new Python project. -""" - -generate_usage = """\ -Usage: pysetup generate-setup - or: pysetup generate-setup --help - -Generate a setup.py script for backward-compatibility purposes. -""" - - -graph_usage = """\ -Usage: pysetup graph dist - or: pysetup graph --help - -Print dependency graph for the distribution. - -positional arguments: - dist installed distribution name -""" - -install_usage = """\ -Usage: pysetup install [dist] - or: pysetup install [archive] - or: pysetup install [src_dir] - or: pysetup install --help - -Install a Python distribution from the indexes, source directory, or sdist. - -positional arguments: - archive path to source distribution (zip, tar.gz) - dist distribution name to install from the indexes - scr_dir path to source directory - -""" - -metadata_usage = """\ -Usage: pysetup metadata [dist] [-f field ...] - or: pysetup metadata [dist] [--all] - or: pysetup metadata --help - -Print metadata for the distribution. - -positional arguments: - dist installed distribution name - -optional arguments: - -f metadata field to print - --all print all metadata fields -""" - -remove_usage = """\ -Usage: pysetup remove dist [-y] - or: pysetup remove --help - -Uninstall a Python distribution. - -positional arguments: - dist installed distribution name - -optional arguments: - -y auto confirm distribution removal -""" - -run_usage = """\ -Usage: pysetup run [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] - or: pysetup run --help - or: pysetup run --list-commands - or: pysetup run cmd --help -""" - -list_usage = """\ -Usage: pysetup list dist [dist ...] - or: pysetup list --help - or: pysetup list --all - -Print name, version and location for the matching installed distributions. - -positional arguments: - dist installed distribution name - -optional arguments: - --all list all installed distributions -""" - -search_usage = """\ -Usage: pysetup search [project] [--simple [url]] [--xmlrpc [url] [--fieldname value ...] --operator or|and] - or: pysetup search --help - -Search the indexes for the matching projects. - -positional arguments: - project the project pattern to search for - -optional arguments: - --xmlrpc [url] wether to use the xmlrpc index or not. If an url is - specified, it will be used rather than the default one. - - --simple [url] wether to use the simple index or not. If an url is - specified, it will be used rather than the default one. - - --fieldname value Make a search on this field. Can only be used if - --xmlrpc has been selected or is the default index. - - --operator or|and Defines what is the operator to use when doing xmlrpc - searchs with multiple fieldnames. Can only be used if - --xmlrpc has been selected or is the default index. -""" - -global_options = [ - # The fourth entry for verbose means that it can be repeated. - ('verbose', 'v', "run verbosely (default)", True), - ('quiet', 'q', "run quietly (turns verbosity off)"), - ('dry-run', 'n', "don't actually do anything"), - ('help', 'h', "show detailed help message"), - ('no-user-cfg', None, 'ignore pydistutils.cfg in your home directory'), - ('version', None, 'Display the version'), -] - -negative_opt = {'quiet': 'verbose'} - -display_options = [ - ('help-commands', None, "list all available commands"), -] - -display_option_names = [x[0].replace('-', '_') for x in display_options] - - -def _parse_args(args, options, long_options): - """Transform sys.argv input into a dict. - - :param args: the args to parse (i.e sys.argv) - :param options: the list of options to pass to getopt - :param long_options: the list of string with the names of the long options - to be passed to getopt. - - The function returns a dict with options/long_options as keys and matching - values as values. - """ - optlist, args = getopt.gnu_getopt(args, options, long_options) - optdict = {} - optdict['args'] = args - for k, v in optlist: - k = k.lstrip('-') - if k not in optdict: - optdict[k] = [] - if v: - optdict[k].append(v) - else: - optdict[k].append(v) - return optdict - - -class action_help: - """Prints a help message when the standard help flags: -h and --help - are used on the commandline. - """ - - def __init__(self, help_msg): - self.help_msg = help_msg - - def __call__(self, f): - def wrapper(*args, **kwargs): - f_args = args[1] - if '--help' in f_args or '-h' in f_args: - print(self.help_msg) - return - return f(*args, **kwargs) - return wrapper - - -@action_help(create_usage) -def _create(distpatcher, args, **kw): - from packaging.create import main - return main() - - -@action_help(generate_usage) -def _generate(distpatcher, args, **kw): - generate_setup_py() - logger.info('The setup.py was generated') - - -@action_help(graph_usage) -def _graph(dispatcher, args, **kw): - name = args[1] - dist = get_distribution(name, use_egg_info=True) - if dist is None: - logger.warning('Distribution not found.') - return 1 - else: - dists = get_distributions(use_egg_info=True) - graph = generate_graph(dists) - print(graph.repr_node(dist)) - - -@action_help(install_usage) -def _install(dispatcher, args, **kw): - # first check if we are in a source directory - if len(args) < 2: - # are we inside a project dir? - if os.path.isfile('setup.cfg') or os.path.isfile('setup.py'): - args.insert(1, os.getcwd()) - else: - logger.warning('No project to install.') - return 1 - - target = args[1] - # installing from a source dir or archive file? - if os.path.isdir(target) or _is_archive_file(target): - return not install_local_project(target) - else: - # download from PyPI - return not install(target) - - -@action_help(metadata_usage) -def _metadata(dispatcher, args, **kw): - opts = _parse_args(args[1:], 'f:', ['all']) - if opts['args']: - name = opts['args'][0] - dist = get_distribution(name, use_egg_info=True) - if dist is None: - logger.warning('%r not installed', name) - return 1 - elif os.path.isfile('setup.cfg'): - logger.info('searching local dir for metadata') - dist = Distribution() # XXX use config module - dist.parse_config_files() - else: - logger.warning('no argument given and no local setup.cfg found') - return 1 - - metadata = dist.metadata - - if 'all' in opts: - keys = metadata.keys() - else: - if 'f' in opts: - keys = (k for k in opts['f'] if k in metadata) - else: - keys = () - - for key in keys: - if key in metadata: - print(metadata._convert_name(key) + ':') - value = metadata[key] - if isinstance(value, list): - for v in value: - print(' ', v) - else: - print(' ', value.replace('\n', '\n ')) - - -@action_help(remove_usage) -def _remove(distpatcher, args, **kw): - opts = _parse_args(args[1:], 'y', []) - if 'y' in opts: - auto_confirm = True - else: - auto_confirm = False - - retcode = 0 - for dist in set(opts['args']): - try: - remove(dist, auto_confirm=auto_confirm) - except PackagingError: - logger.warning('%r not installed', dist) - retcode = 1 - - return retcode - - -@action_help(run_usage) -def _run(dispatcher, args, **kw): - parser = dispatcher.parser - args = args[1:] - - commands = STANDARD_COMMANDS # + extra commands - - if args == ['--list-commands']: - print('List of available commands:') - cmds = sorted(commands) - - for cmd in cmds: - cls = dispatcher.cmdclass.get(cmd) or get_command_class(cmd) - desc = getattr(cls, 'description', - '(no description available)') - print(' %s: %s' % (cmd, desc)) - return - - while args: - args = dispatcher._parse_command_opts(parser, args) - if args is None: - return - - # create the Distribution class - # need to feed setup.cfg here ! - dist = Distribution() - - # Find and parse the config file(s): they will override options from - # the setup script, but be overridden by the command line. - - # XXX still need to be extracted from Distribution - dist.parse_config_files() - - for cmd in dispatcher.commands: - dist.run_command(cmd, dispatcher.command_options[cmd]) - - # XXX this is crappy - return dist - - -@action_help(list_usage) -def _list(dispatcher, args, **kw): - opts = _parse_args(args[1:], '', ['all']) - dists = get_distributions(use_egg_info=True) - if 'all' in opts or opts['args'] == []: - results = dists - listall = True - else: - results = (d for d in dists if d.name.lower() in opts['args']) - listall = False - - number = 0 - for dist in results: - print('%r %s (from %r)' % (dist.name, dist.version, dist.path)) - number += 1 - - if number == 0: - if listall: - logger.info('Nothing seems to be installed.') - else: - logger.warning('No matching distribution found.') - return 1 - else: - logger.info('Found %d projects installed.', number) - - -@action_help(search_usage) -def _search(dispatcher, args, **kw): - """The search action. - - It is able to search for a specific index (specified with --index), using - the simple or xmlrpc index types (with --type xmlrpc / --type simple) - """ - #opts = _parse_args(args[1:], '', ['simple', 'xmlrpc']) - # 1. what kind of index is requested ? (xmlrpc / simple) - logger.error('not implemented') - return 1 - - -actions = [ - ('run', 'Run one or several commands', _run), - ('metadata', 'Display the metadata of a project', _metadata), - ('install', 'Install a project', _install), - ('remove', 'Remove a project', _remove), - ('search', 'Search for a project in the indexes', _search), - ('list', 'List installed releases', _list), - ('graph', 'Display a graph', _graph), - ('create', 'Create a project', _create), - ('generate-setup', 'Generate a backward-comptatible setup.py', _generate), -] - - -class Dispatcher: - """Reads the command-line options - """ - def __init__(self, args=None): - self.verbose = 1 - self.dry_run = False - self.help = False - self.cmdclass = {} - self.commands = [] - self.command_options = {} - - for attr in display_option_names: - setattr(self, attr, False) - - self.parser = FancyGetopt(global_options + display_options) - self.parser.set_negative_aliases(negative_opt) - # FIXME this parses everything, including command options (e.g. "run - # build -i" errors with "option -i not recognized") - args = self.parser.getopt(args=args, object=self) - - # if first arg is "run", we have some commands - if len(args) == 0: - self.action = None - else: - self.action = args[0] - - allowed = [action[0] for action in actions] + [None] - if self.action not in allowed: - msg = 'Unrecognized action "%s"' % self.action - raise PackagingArgError(msg) - - self._set_logger() - self.args = args - - # for display options we return immediately - if self.help or self.action is None: - self._show_help(self.parser, display_options_=False) - - def _set_logger(self): - # setting up the logging level from the command-line options - # -q gets warning, error and critical - if self.verbose == 0: - level = logging.WARNING - # default level or -v gets info too - # XXX there's a bug somewhere: the help text says that -v is default - # (and verbose is set to 1 above), but when the user explicitly gives - # -v on the command line, self.verbose is incremented to 2! Here we - # compensate for that (I tested manually). On a related note, I think - # it's a good thing to use -q/nothing/-v/-vv on the command line - # instead of logging constants; it will be easy to add support for - # logging configuration in setup.cfg for advanced users. --merwok - elif self.verbose in (1, 2): - level = logging.INFO - else: # -vv and more for debug - level = logging.DEBUG - - # setting up the stream handler - handler = logging.StreamHandler(sys.stderr) - handler.setLevel(level) - logger.addHandler(handler) - logger.setLevel(level) - - def _parse_command_opts(self, parser, args): - # Pull the current command from the head of the command line - command = args[0] - if not command_re.match(command): - raise SystemExit("invalid command name %r" % (command,)) - self.commands.append(command) - - # Dig up the command class that implements this command, so we - # 1) know that it's a valid command, and 2) know which options - # it takes. - try: - cmd_class = get_command_class(command) - except PackagingModuleError as msg: - raise PackagingArgError(msg) - - # XXX We want to push this in packaging.command - # - # Require that the command class be derived from Command -- want - # to be sure that the basic "command" interface is implemented. - for meth in ('initialize_options', 'finalize_options', 'run'): - if hasattr(cmd_class, meth): - continue - raise PackagingClassError( - 'command %r must implement %r' % (cmd_class, meth)) - - # Also make sure that the command object provides a list of its - # known options. - if not (hasattr(cmd_class, 'user_options') and - isinstance(cmd_class.user_options, list)): - raise PackagingClassError( - "command class %s must provide " - "'user_options' attribute (a list of tuples)" % cmd_class) - - # If the command class has a list of negative alias options, - # merge it in with the global negative aliases. - _negative_opt = negative_opt.copy() - - if hasattr(cmd_class, 'negative_opt'): - _negative_opt.update(cmd_class.negative_opt) - - # Check for help_options in command class. They have a different - # format (tuple of four) so we need to preprocess them here. - if (hasattr(cmd_class, 'help_options') and - isinstance(cmd_class.help_options, list)): - help_options = cmd_class.help_options[:] - else: - help_options = [] - - # All commands support the global options too, just by adding - # in 'global_options'. - parser.set_option_table(global_options + - cmd_class.user_options + - help_options) - parser.set_negative_aliases(_negative_opt) - args, opts = parser.getopt(args[1:]) - - if hasattr(opts, 'help') and opts.help: - self._show_command_help(cmd_class) - return - - if (hasattr(cmd_class, 'help_options') and - isinstance(cmd_class.help_options, list)): - help_option_found = False - for help_option, short, desc, func in cmd_class.help_options: - if hasattr(opts, help_option.replace('-', '_')): - help_option_found = True - if hasattr(func, '__call__'): - func() - else: - raise PackagingClassError( - "invalid help function %r for help option %r: " - "must be a callable object (function, etc.)" - % (func, help_option)) - - if help_option_found: - return - - # Put the options from the command line into their official - # holding pen, the 'command_options' dictionary. - opt_dict = self.get_option_dict(command) - for name, value in vars(opts).items(): - opt_dict[name] = ("command line", value) - - return args - - def get_option_dict(self, command): - """Get the option dictionary for a given command. If that - command's option dictionary hasn't been created yet, then create it - and return the new dictionary; otherwise, return the existing - option dictionary. - """ - d = self.command_options.get(command) - if d is None: - d = self.command_options[command] = {} - return d - - def show_help(self): - self._show_help(self.parser) - - def print_usage(self, parser): - parser.set_option_table(global_options) - - actions_ = [' %s: %s' % (name, desc) for name, desc, __ in actions] - usage = common_usage % {'actions': '\n'.join(actions_)} - - parser.print_help(usage + "\nGlobal options:") - - def _show_help(self, parser, global_options_=True, display_options_=True, - commands=[]): - # late import because of mutual dependence between these modules - from packaging.command.cmd import Command - - print('Usage: pysetup [options] action [action_options]') - print() - if global_options_: - self.print_usage(self.parser) - print() - - if display_options_: - parser.set_option_table(display_options) - parser.print_help( - "Information display options (just display " + - "information, ignore any commands)") - print() - - for command in commands: - if isinstance(command, type) and issubclass(command, Command): - cls = command - else: - cls = get_command_class(command) - if (hasattr(cls, 'help_options') and - isinstance(cls.help_options, list)): - parser.set_option_table(cls.user_options + cls.help_options) - else: - parser.set_option_table(cls.user_options) - - parser.print_help("Options for %r command:" % cls.__name__) - print() - - def _show_command_help(self, command): - if isinstance(command, str): - command = get_command_class(command) - - desc = getattr(command, 'description', '(no description available)') - print('Description:', desc) - print() - - if (hasattr(command, 'help_options') and - isinstance(command.help_options, list)): - self.parser.set_option_table(command.user_options + - command.help_options) - else: - self.parser.set_option_table(command.user_options) - - self.parser.print_help("Options:") - print() - - def _get_command_groups(self): - """Helper function to retrieve all the command class names divided - into standard commands (listed in - packaging.command.STANDARD_COMMANDS) and extra commands (given in - self.cmdclass and not standard commands). - """ - extra_commands = [cmd for cmd in self.cmdclass - if cmd not in STANDARD_COMMANDS] - return STANDARD_COMMANDS, extra_commands - - def print_commands(self): - """Print out a help message listing all available commands with a - description of each. The list is divided into standard commands - (listed in packaging.command.STANDARD_COMMANDS) and extra commands - (given in self.cmdclass and not standard commands). The - descriptions come from the command class attribute - 'description'. - """ - std_commands, extra_commands = self._get_command_groups() - max_length = max(len(command) - for commands in (std_commands, extra_commands) - for command in commands) - - self.print_command_list(std_commands, "Standard commands", max_length) - if extra_commands: - print() - self.print_command_list(extra_commands, "Extra commands", - max_length) - - def print_command_list(self, commands, header, max_length): - """Print a subset of the list of all commands -- used by - 'print_commands()'. - """ - print(header + ":") - - for cmd in commands: - cls = self.cmdclass.get(cmd) or get_command_class(cmd) - description = getattr(cls, 'description', - '(no description available)') - - print(" %-*s %s" % (max_length, cmd, description)) - - def __call__(self): - if self.action is None: - return - - for action, desc, func in actions: - if action == self.action: - return func(self, self.args) - return -1 - - -def main(args=None): - old_level = logger.level - old_handlers = list(logger.handlers) - try: - dispatcher = Dispatcher(args) - if dispatcher.action is None: - return - return dispatcher() - except KeyboardInterrupt: - logger.info('interrupted') - return 1 - except (IOError, os.error, PackagingError, CCompilerError) as exc: - logger.exception(exc) - return 1 - finally: - logger.setLevel(old_level) - logger.handlers[:] = old_handlers - - -if __name__ == '__main__': - sys.exit(main()) +"""Main command line parser. Implements the pysetup script.""" + +import os +import re +import sys +import getopt +import logging + +from packaging import logger +from packaging.dist import Distribution +from packaging.util import _is_archive_file, generate_setup_py +from packaging.command import get_command_class, STANDARD_COMMANDS +from packaging.install import install, install_local_project, remove +from packaging.database import get_distribution, get_distributions +from packaging.depgraph import generate_graph +from packaging.fancy_getopt import FancyGetopt +from packaging.errors import (PackagingArgError, PackagingError, + PackagingModuleError, PackagingClassError, + CCompilerError) + + +command_re = re.compile(r'^[a-zA-Z]([a-zA-Z0-9_]*)$') + +common_usage = """\ +Actions: +%(actions)s + +To get more help on an action, use: + + pysetup action --help +""" + +create_usage = """\ +Usage: pysetup create + or: pysetup create --help + +Create a new Python project. +""" + +generate_usage = """\ +Usage: pysetup generate-setup + or: pysetup generate-setup --help + +Generate a setup.py script for backward-compatibility purposes. +""" + + +graph_usage = """\ +Usage: pysetup graph dist + or: pysetup graph --help + +Print dependency graph for the distribution. + +positional arguments: + dist installed distribution name +""" + +install_usage = """\ +Usage: pysetup install [dist] + or: pysetup install [archive] + or: pysetup install [src_dir] + or: pysetup install --help + +Install a Python distribution from the indexes, source directory, or sdist. + +positional arguments: + archive path to source distribution (zip, tar.gz) + dist distribution name to install from the indexes + scr_dir path to source directory + +""" + +metadata_usage = """\ +Usage: pysetup metadata [dist] [-f field ...] + or: pysetup metadata [dist] [--all] + or: pysetup metadata --help + +Print metadata for the distribution. + +positional arguments: + dist installed distribution name + +optional arguments: + -f metadata field to print + --all print all metadata fields +""" + +remove_usage = """\ +Usage: pysetup remove dist [-y] + or: pysetup remove --help + +Uninstall a Python distribution. + +positional arguments: + dist installed distribution name + +optional arguments: + -y auto confirm distribution removal +""" + +run_usage = """\ +Usage: pysetup run [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] + or: pysetup run --help + or: pysetup run --list-commands + or: pysetup run cmd --help +""" + +list_usage = """\ +Usage: pysetup list dist [dist ...] + or: pysetup list --help + or: pysetup list --all + +Print name, version and location for the matching installed distributions. + +positional arguments: + dist installed distribution name + +optional arguments: + --all list all installed distributions +""" + +search_usage = """\ +Usage: pysetup search [project] [--simple [url]] [--xmlrpc [url] [--fieldname value ...] --operator or|and] + or: pysetup search --help + +Search the indexes for the matching projects. + +positional arguments: + project the project pattern to search for + +optional arguments: + --xmlrpc [url] wether to use the xmlrpc index or not. If an url is + specified, it will be used rather than the default one. + + --simple [url] wether to use the simple index or not. If an url is + specified, it will be used rather than the default one. + + --fieldname value Make a search on this field. Can only be used if + --xmlrpc has been selected or is the default index. + + --operator or|and Defines what is the operator to use when doing xmlrpc + searchs with multiple fieldnames. Can only be used if + --xmlrpc has been selected or is the default index. +""" + +global_options = [ + # The fourth entry for verbose means that it can be repeated. + ('verbose', 'v', "run verbosely (default)", True), + ('quiet', 'q', "run quietly (turns verbosity off)"), + ('dry-run', 'n', "don't actually do anything"), + ('help', 'h', "show detailed help message"), + ('no-user-cfg', None, 'ignore pydistutils.cfg in your home directory'), + ('version', None, 'Display the version'), +] + +negative_opt = {'quiet': 'verbose'} + +display_options = [ + ('help-commands', None, "list all available commands"), +] + +display_option_names = [x[0].replace('-', '_') for x in display_options] + + +def _parse_args(args, options, long_options): + """Transform sys.argv input into a dict. + + :param args: the args to parse (i.e sys.argv) + :param options: the list of options to pass to getopt + :param long_options: the list of string with the names of the long options + to be passed to getopt. + + The function returns a dict with options/long_options as keys and matching + values as values. + """ + optlist, args = getopt.gnu_getopt(args, options, long_options) + optdict = {} + optdict['args'] = args + for k, v in optlist: + k = k.lstrip('-') + if k not in optdict: + optdict[k] = [] + if v: + optdict[k].append(v) + else: + optdict[k].append(v) + return optdict + + +class action_help: + """Prints a help message when the standard help flags: -h and --help + are used on the commandline. + """ + + def __init__(self, help_msg): + self.help_msg = help_msg + + def __call__(self, f): + def wrapper(*args, **kwargs): + f_args = args[1] + if '--help' in f_args or '-h' in f_args: + print(self.help_msg) + return + return f(*args, **kwargs) + return wrapper + + +@action_help(create_usage) +def _create(distpatcher, args, **kw): + from packaging.create import main + return main() + + +@action_help(generate_usage) +def _generate(distpatcher, args, **kw): + generate_setup_py() + logger.info('The setup.py was generated') + + +@action_help(graph_usage) +def _graph(dispatcher, args, **kw): + name = args[1] + dist = get_distribution(name, use_egg_info=True) + if dist is None: + logger.warning('Distribution not found.') + return 1 + else: + dists = get_distributions(use_egg_info=True) + graph = generate_graph(dists) + print(graph.repr_node(dist)) + + +@action_help(install_usage) +def _install(dispatcher, args, **kw): + # first check if we are in a source directory + if len(args) < 2: + # are we inside a project dir? + if os.path.isfile('setup.cfg') or os.path.isfile('setup.py'): + args.insert(1, os.getcwd()) + else: + logger.warning('No project to install.') + return 1 + + target = args[1] + # installing from a source dir or archive file? + if os.path.isdir(target) or _is_archive_file(target): + return not install_local_project(target) + else: + # download from PyPI + return not install(target) + +@action_help(metadata_usage) +def _metadata(dispatcher, args, **kw): + opts = _parse_args(args[1:], 'f:', ['all']) + if opts['args']: + name = opts['args'][0] + dist = get_distribution(name, use_egg_info=True) + if dist is None: + logger.warning('%r not installed', name) + return 1 + elif os.path.isfile('setup.cfg'): + logger.info('searching local dir for metadata') + dist = Distribution() # XXX use config module + dist.parse_config_files() + else: + logger.warning('no argument given and no local setup.cfg found') + return 1 + + metadata = dist.metadata + + if 'all' in opts: + keys = metadata.keys() + else: + if 'f' in opts: + keys = (k for k in opts['f'] if k in metadata) + else: + keys = () + + for key in keys: + if key in metadata: + print(metadata._convert_name(key) + ':') + value = metadata[key] + if isinstance(value, list): + for v in value: + print(' ', v) + else: + print(' ', value.replace('\n', '\n ')) + + +@action_help(remove_usage) +def _remove(distpatcher, args, **kw): + opts = _parse_args(args[1:], 'y', []) + if 'y' in opts: + auto_confirm = True + else: + auto_confirm = False + + retcode = 0 + for dist in set(opts['args']): + try: + remove(dist, auto_confirm=auto_confirm) + except PackagingError: + logger.warning('%r not installed', dist) + retcode = 1 + + return retcode + + +@action_help(run_usage) +def _run(dispatcher, args, **kw): + parser = dispatcher.parser + args = args[1:] + + commands = STANDARD_COMMANDS # + extra commands + + if args == ['--list-commands']: + print('List of available commands:') + cmds = sorted(commands) + + for cmd in cmds: + cls = dispatcher.cmdclass.get(cmd) or get_command_class(cmd) + desc = getattr(cls, 'description', + '(no description available)') + print(' %s: %s' % (cmd, desc)) + return + + while args: + args = dispatcher._parse_command_opts(parser, args) + if args is None: + return + + # create the Distribution class + # need to feed setup.cfg here ! + dist = Distribution() + + # Find and parse the config file(s): they will override options from + # the setup script, but be overridden by the command line. + + # XXX still need to be extracted from Distribution + dist.parse_config_files() + + for cmd in dispatcher.commands: + dist.run_command(cmd, dispatcher.command_options[cmd]) + + # XXX this is crappy + return dist + + +@action_help(list_usage) +def _list(dispatcher, args, **kw): + opts = _parse_args(args[1:], '', ['all']) + dists = get_distributions(use_egg_info=True) + if 'all' in opts or opts['args'] == []: + results = dists + listall = True + else: + results = (d for d in dists if d.name.lower() in opts['args']) + listall = False + + number = 0 + for dist in results: + print('%r %s (from %r)' % (dist.name, dist.version, dist.path)) + number += 1 + + if number == 0: + if listall: + logger.info('Nothing seems to be installed.') + else: + logger.warning('No matching distribution found.') + return 1 + else: + logger.info('Found %d projects installed.', number) + + +@action_help(search_usage) +def _search(dispatcher, args, **kw): + """The search action. + + It is able to search for a specific index (specified with --index), using + the simple or xmlrpc index types (with --type xmlrpc / --type simple) + """ + #opts = _parse_args(args[1:], '', ['simple', 'xmlrpc']) + # 1. what kind of index is requested ? (xmlrpc / simple) + logger.error('not implemented') + return 1 + + +actions = [ + ('run', 'Run one or several commands', _run), + ('metadata', 'Display the metadata of a project', _metadata), + ('install', 'Install a project', _install), + ('remove', 'Remove a project', _remove), + ('search', 'Search for a project in the indexes', _search), + ('list', 'List installed releases', _list), + ('graph', 'Display a graph', _graph), + ('create', 'Create a project', _create), + ('generate-setup', 'Generate a backward-comptatible setup.py', _generate), +] + + +class Dispatcher: + """Reads the command-line options + """ + def __init__(self, args=None): + self.verbose = 1 + self.dry_run = False + self.help = False + self.cmdclass = {} + self.commands = [] + self.command_options = {} + + for attr in display_option_names: + setattr(self, attr, False) + + self.parser = FancyGetopt(global_options + display_options) + self.parser.set_negative_aliases(negative_opt) + # FIXME this parses everything, including command options (e.g. "run + # build -i" errors with "option -i not recognized") + args = self.parser.getopt(args=args, object=self) + + # if first arg is "run", we have some commands + if len(args) == 0: + self.action = None + else: + self.action = args[0] + + allowed = [action[0] for action in actions] + [None] + if self.action not in allowed: + msg = 'Unrecognized action "%s"' % self.action + raise PackagingArgError(msg) + + self._set_logger() + self.args = args + + # for display options we return immediately + if self.help or self.action is None: + self._show_help(self.parser, display_options_=False) + + def _set_logger(self): + # setting up the logging level from the command-line options + # -q gets warning, error and critical + if self.verbose == 0: + level = logging.WARNING + # default level or -v gets info too + # XXX there's a bug somewhere: the help text says that -v is default + # (and verbose is set to 1 above), but when the user explicitly gives + # -v on the command line, self.verbose is incremented to 2! Here we + # compensate for that (I tested manually). On a related note, I think + # it's a good thing to use -q/nothing/-v/-vv on the command line + # instead of logging constants; it will be easy to add support for + # logging configuration in setup.cfg for advanced users. --merwok + elif self.verbose in (1, 2): + level = logging.INFO + else: # -vv and more for debug + level = logging.DEBUG + + # setting up the stream handler + handler = logging.StreamHandler(sys.stderr) + handler.setLevel(level) + logger.addHandler(handler) + logger.setLevel(level) + + def _parse_command_opts(self, parser, args): + # Pull the current command from the head of the command line + command = args[0] + if not command_re.match(command): + raise SystemExit("invalid command name %r" % (command,)) + self.commands.append(command) + + # Dig up the command class that implements this command, so we + # 1) know that it's a valid command, and 2) know which options + # it takes. + try: + cmd_class = get_command_class(command) + except PackagingModuleError as msg: + raise PackagingArgError(msg) + + # XXX We want to push this in packaging.command + # + # Require that the command class be derived from Command -- want + # to be sure that the basic "command" interface is implemented. + for meth in ('initialize_options', 'finalize_options', 'run'): + if hasattr(cmd_class, meth): + continue + raise PackagingClassError( + 'command %r must implement %r' % (cmd_class, meth)) + + # Also make sure that the command object provides a list of its + # known options. + if not (hasattr(cmd_class, 'user_options') and + isinstance(cmd_class.user_options, list)): + raise PackagingClassError( + "command class %s must provide " + "'user_options' attribute (a list of tuples)" % cmd_class) + + # If the command class has a list of negative alias options, + # merge it in with the global negative aliases. + _negative_opt = negative_opt.copy() + + if hasattr(cmd_class, 'negative_opt'): + _negative_opt.update(cmd_class.negative_opt) + + # Check for help_options in command class. They have a different + # format (tuple of four) so we need to preprocess them here. + if (hasattr(cmd_class, 'help_options') and + isinstance(cmd_class.help_options, list)): + help_options = cmd_class.help_options[:] + else: + help_options = [] + + # All commands support the global options too, just by adding + # in 'global_options'. + parser.set_option_table(global_options + + cmd_class.user_options + + help_options) + parser.set_negative_aliases(_negative_opt) + args, opts = parser.getopt(args[1:]) + + if hasattr(opts, 'help') and opts.help: + self._show_command_help(cmd_class) + return + + if (hasattr(cmd_class, 'help_options') and + isinstance(cmd_class.help_options, list)): + help_option_found = False + for help_option, short, desc, func in cmd_class.help_options: + if hasattr(opts, help_option.replace('-', '_')): + help_option_found = True + if hasattr(func, '__call__'): + func() + else: + raise PackagingClassError( + "invalid help function %r for help option %r: " + "must be a callable object (function, etc.)" + % (func, help_option)) + + if help_option_found: + return + + # Put the options from the command line into their official + # holding pen, the 'command_options' dictionary. + opt_dict = self.get_option_dict(command) + for name, value in vars(opts).items(): + opt_dict[name] = ("command line", value) + + return args + + def get_option_dict(self, command): + """Get the option dictionary for a given command. If that + command's option dictionary hasn't been created yet, then create it + and return the new dictionary; otherwise, return the existing + option dictionary. + """ + d = self.command_options.get(command) + if d is None: + d = self.command_options[command] = {} + return d + + def show_help(self): + self._show_help(self.parser) + + def print_usage(self, parser): + parser.set_option_table(global_options) + + actions_ = [' %s: %s' % (name, desc) for name, desc, __ in actions] + usage = common_usage % {'actions': '\n'.join(actions_)} + + parser.print_help(usage + "\nGlobal options:") + + def _show_help(self, parser, global_options_=True, display_options_=True, + commands=[]): + # late import because of mutual dependence between these modules + from packaging.command.cmd import Command + + print('Usage: pysetup [options] action [action_options]') + print() + if global_options_: + self.print_usage(self.parser) + print() + + if display_options_: + parser.set_option_table(display_options) + parser.print_help( + "Information display options (just display " + + "information, ignore any commands)") + print() + + for command in commands: + if isinstance(command, type) and issubclass(command, Command): + cls = command + else: + cls = get_command_class(command) + if (hasattr(cls, 'help_options') and + isinstance(cls.help_options, list)): + parser.set_option_table(cls.user_options + cls.help_options) + else: + parser.set_option_table(cls.user_options) + + parser.print_help("Options for %r command:" % cls.__name__) + print() + + def _show_command_help(self, command): + if isinstance(command, str): + command = get_command_class(command) + + desc = getattr(command, 'description', '(no description available)') + print('Description:', desc) + print() + + if (hasattr(command, 'help_options') and + isinstance(command.help_options, list)): + self.parser.set_option_table(command.user_options + + command.help_options) + else: + self.parser.set_option_table(command.user_options) + + self.parser.print_help("Options:") + print() + + def _get_command_groups(self): + """Helper function to retrieve all the command class names divided + into standard commands (listed in + packaging.command.STANDARD_COMMANDS) and extra commands (given in + self.cmdclass and not standard commands). + """ + extra_commands = [cmd for cmd in self.cmdclass + if cmd not in STANDARD_COMMANDS] + return STANDARD_COMMANDS, extra_commands + + def print_commands(self): + """Print out a help message listing all available commands with a + description of each. The list is divided into standard commands + (listed in packaging.command.STANDARD_COMMANDS) and extra commands + (given in self.cmdclass and not standard commands). The + descriptions come from the command class attribute + 'description'. + """ + std_commands, extra_commands = self._get_command_groups() + max_length = max(len(command) + for commands in (std_commands, extra_commands) + for command in commands) + + self.print_command_list(std_commands, "Standard commands", max_length) + if extra_commands: + print() + self.print_command_list(extra_commands, "Extra commands", + max_length) + + def print_command_list(self, commands, header, max_length): + """Print a subset of the list of all commands -- used by + 'print_commands()'. + """ + print(header + ":") + + for cmd in commands: + cls = self.cmdclass.get(cmd) or get_command_class(cmd) + description = getattr(cls, 'description', + '(no description available)') + + print(" %-*s %s" % (max_length, cmd, description)) + + def __call__(self): + if self.action is None: + return + + for action, desc, func in actions: + if action == self.action: + return func(self, self.args) + return -1 + + +def main(args=None): + old_level = logger.level + old_handlers = list(logger.handlers) + try: + dispatcher = Dispatcher(args) + if dispatcher.action is None: + return + return dispatcher() + except KeyboardInterrupt: + logger.info('interrupted') + return 1 + except (IOError, os.error, PackagingError, CCompilerError) as exc: + logger.exception(exc) + return 1 + finally: + logger.setLevel(old_level) + logger.handlers[:] = old_handlers + + +if __name__ == '__main__': + sys.exit(main()) diff -r 7d390c3a83c6 -r 5359c895f841 Lib/packaging/tests/test_command_develop.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/Lib/packaging/tests/test_command_develop.py Mon Aug 22 21:02:47 2011 +0800 @@ -0,0 +1,179 @@ +"""Tests for packaging.command.develop.""" + +import os +import sys + +from packaging.dist import Distribution +from packaging.command.develop import develop, normalize_path +from packaging.tests import unittest, support +from packaging.errors import PackagingOptionError + +class DevelopTestCase(support.TempdirManager, + support.LoggingCatcher, + unittest.TestCase): + def test_pth_file_installation(self): + # test if the .pth file can be succesfully generated and installed + projdir, dist = self.create_dist() + oldcwd = os.getcwd() + try: + cmd = develop(dist) + os.chdir(projdir) + cmd.ensure_finalized() + cmd.run() + finally: + os.chdir(oldcwd) + + pth_file = cmd.pth_file + self.assertIsNotNone(pth_file) + self.assertTrue(os.path.exists(pth_file)) + # check if it writes the right path + with open(pth_file) as f: + written_path = f.readline().rstrip() + self.assertEqual(written_path, normalize_path(projdir)) + + # .pth and .distinfo-link files are not in temp dir, so it needs clean work here + distinfo_link_file = os.path.join(cmd.install_dir, cmd.basename+'.distinfo-link') + self._clean([pth_file, distinfo_link_file]) + + def test_distinfo_link_file_installation(self): + # test if the .distinfo-link file is installed succesfully and with correct content + projdir, dist = self.create_dist(name='xx',version='1.2.3') + oldcwd = os.getcwd() + try: + cmd = develop(dist) + os.chdir(projdir) + cmd.ensure_finalized() + cmd.run() + finally: + os.chdir(oldcwd) + + distinfo_link_file = cmd.distinfo_link + self.assertIsNotNone(distinfo_link_file) + self.assertTrue(os.path.exists(distinfo_link_file)) + # check if it writes the right .dist-info directory path + with open(distinfo_link_file) as f: + written_path = f.readline().rstrip() + directory_name = 'xx-1.2.3.dist-info' + distinfo_dir = os.path.join(cmd.install_dir, written_path, directory_name) + self.assertTrue(os.path.exists(distinfo_dir)) + + # then clean + pth_file = os.path.join(cmd.install_dir, 'xx.pth') + self._clean([pth_file, distinfo_link_file]) + + def test_distinfo_dir_installation(self): + # test if the .dist-info directory is installed under correct directory + projdir, dist = self.create_dist(name='xxx', version='1.2.3') + distinfo_name = 'xxx-1.2.3.dist-info' + # the default installation path + distinfo_dir = os.path.join(projdir, distinfo_name) + # should not exist at first + self.assertFalse(os.path.exists(distinfo_dir)) + + oldcwd = os.getcwd() + try: + cmd = develop(dist) + os.chdir(projdir) + cmd.ensure_finalized() + cmd.run() + finally: + os.chdir(oldcwd) + + # should be created then + self.assertTrue(os.path.exists(distinfo_dir)) + + # then clean + pth_file = os.path.join(cmd.install_dir, 'xxx.pth') + distinfo_link_file = os.path.join(cmd.install_dir, 'xxx.distinfo-link') + self._clean([pth_file, distinfo_link_file]) + + def test_distinfo_path(self): + # test the distinfo_path option + projdir,dist = self.create_dist(name='def') + cmd = develop(dist) + # making sure the distinfo_path option is there + options = [name for name,short,label in cmd.user_options] + self.assertIn('distinfo-path=', options) + + # should be None at first + self.assertIsNone(cmd.distinfo_path) + + # an absolute path or a wrong relative path, should raise exception ; + # but we only need to check one + cmd.distinfo_path = self.mkdtemp() + os.chdir(projdir) + self.assertRaises(PackagingOptionError, cmd.finalize_options) + + # set the 'install_dir' option value: + # we construct the 'install_dir' value in a tmp dir + tmpdist = Distribution() + tmpcmd = develop(tmpdist) + tmpcmd.ensure_finalized() + cmd.install_dir = tmpcmd.install_dir + + # create a correct relative path and assign it to distinfo_path + p = cmd.install_dir.replace(os.sep, '/') + p = '../'*(p.count('/')+1) + cmd.distinfo_path = os.path.join(p, projdir) + + oldcwd = os.getcwd() + try: + os.chdir(projdir) + cmd.ensure_finalized() + cmd.run() + finally: + os.chdir(oldcwd) + + # we can check if the 'distinfo-path' option is set a right value + # through check the .distinfo-link file is successfully generated or not + self.assertIsNotNone(cmd.distinfo_link) + self.assertTrue(os.path.exists(cmd.distinfo_link)) + + # clean work + pth_file = os.path.join(cmd.install_dir, 'def.pth') + self._clean([pth_file, cmd.distinfo_link]) + + def test_uninstall(self): + # test the uninstall option + projdir, dist = self.create_dist() + cmd = develop(dist) + options = [name for name,short,label in cmd.user_options] + self.assertIn('uninstall', options) + # uninstall should be false at first + self.assertFalse(cmd.uninstall) + + oldcwd = os.getcwd() + try: + os.chdir(projdir) + cmd.ensure_finalized() + cmd.run() + finally: + os.chdir(oldcwd) + + # .pth and .distinfo-link should be created then + self.assertIsNotNone(cmd.pth_file) + self.assertIsNotNone(cmd.distinfo_link) + self.assertTrue(os.path.exists(cmd.pth_file)) + self.assertTrue(os.path.exists(cmd.distinfo_link)) + + # set it True + cmd.uninstall = True + cmd.run() + + # should remove them + self.assertFalse(os.path.exists(cmd.pth_file)) + self.assertFalse(os.path.exists(cmd.distinfo_link)) + # xxx should also check the wrapper scripts if features supported + + def _clean(self, paths=[]): + # clean function to remove generated files in python's + # default site-package + for file in paths: + if os.path.isfile(file): + os.unlink(file) + +def test_suite(): + return unittest.makeSuite(DevelopTestCase) + +if __name__ == '__main__': + unittest.main(defaultTest="test_suite") diff -r 7d390c3a83c6 -r 5359c895f841 Lib/packaging/util.py --- a/Lib/packaging/util.py Sat Aug 20 14:18:25 2011 +0200 +++ b/Lib/packaging/util.py Mon Aug 22 21:02:47 2011 +0800 @@ -1380,6 +1380,21 @@ else: raise InstallationException('Cannot detect install method') +def get_develop_method(path): + """Check if the project is based on packaging or setuptools + + :param path: path to source directory containing a setup.cfg file, + or setup.py + + Returns a string representing the best develop method to use. + """ + if is_packaging(path): + return "packaging" + elif is_setuptools(path): + return "setuptools" + else: + raise InstallationException('Cannot detect develop method') + # XXX to be replaced by shutil.copytree def copy_tree(src, dst, preserve_mode=True, preserve_times=True,