diff -r 1892f4567b07 Doc/install/install.rst --- a/Doc/install/install.rst Sat Feb 25 17:26:44 2012 +0100 +++ b/Doc/install/install.rst Mon Feb 27 13:14:28 2012 +0100 @@ -752,6 +752,97 @@ be added to ``sys.path``; the :mod:`site Finally, ``sys.path`` is just a regular Python list, so any Python application can modify it by adding or removing entries. +.. _packaging-install-develop-mode: + +Special installation - install a project in development mode +============================================================ + +Although we can install a project as above, but under normal circumstances, +it will often rebuilds or reinstalls the projects after some changes have been +madeto it during development. It's a little boring and will take time to copy +files. Thus a very important feature is to install a project in development +mode, which allows users to make the code apear as installed to Python and +other code can import it, but without copying any files. It's very useful +especially for developers using version control system to manage code. + +Installing a project in development mode usaully comes down to one simple command:: + + pysetup run develop + +As described in section :ref:`packaging-standard-install`, running this command +also depends on these three different platforms and the corresponding command line +interfaces. + +Using this command +------------------ + +This command also has platform variations, but it's too easy to use - just executing +the command line ``pysetup run develop`` under the project source directory. Look at +the examples shown in section :ref:`packaging-platform-variations`; they are also +very good examples to learn how to use ``develop`` command. + +Alternate installation +---------------------- + +When we say ``develop`` command is used to pseudo-install a project without rebuilding +and reinstalling the whole project and copying any files, we just mean it does not +rebuild and reinstall the pure :file:`.py` files in the project and copy them. If +there are some C extensions or similarly compiled files in current project, it still +needs to build them and copy them if you have made some changes to them. + +Actually to make our project importable even though we does not really install it, +we install a :file:`.pth` file under the specific directory and defaultly in site-packages. +And also for a development purpose, we install a :file:`.distinfo-link` file in the +specific directory. These are just development details, so the users of this command +can ignore them in the most circumstances, but we clarify it here because there are +still some special cases which need users know them, for example, the default +site-packages does not permit to write, the user/developer just wants to manage the +:file:`.pth` files in a specific directory, e.t.c. + +Thus, it's still necessary to offer users some ways to alternate the default +installation scheme. There are mainly two aspects that we need to consider to complete +a custom installation. + + + +Change the default installation directory +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +It's also easy to set the installation directory - there is an :option:`install-dir` option +offered. If this option is not directly specified on the command line or in a packaging +configuration file, the default installation location is used. Normally, this will be +the site-packages directory, but if you are using packaging configuration files, setting +things like :file:`{prefix}` or :file:`{install_lib}`, then those settings are taken into +account when computing the default staging area. + +We can set the installation directory simply by:: + + pysetup run develop --install-dir DIR + +Please pay attention to the section :ref:`packaging-custom-install`, it also shows good +use cases to complete a custom installation on different platforms and reminds the tips +that we should remember when using "develop" command. When this option is set, the +:file:`.pth` file and :file:`.distinfo-link` file will be installed under the specific +directory if there aren't other settings. And also please note here: the ``DIR`` should be +in Python's module search path. + + + +Manage the .pth files +^^^^^^^^^^^^^^^^^^^^^ + +After there are many projects are 'installed' with ``develop`` command, many :file:`.pth` +files will be created in default site-packages if we don't do some other settings. You +may want to manage these :file:`.pth` files in a specific directory or for some unknown +purpose you just want to install the :file:`.pth` file into a different direcotry, then +you can have a look at the :option:`install-pth` option. It's also very easy to use this +option:: + + pysetup run develop --install-pth=PTHDIR + +The do's and don'ts also refer to the section :ref:`packaging-custom-install`. Also +note ``PTHDIR`` should be on the Python's search path. + .. _packaging-config-files: diff -r 1892f4567b07 Doc/library/packaging.install.rst --- a/Doc/library/packaging.install.rst Sat Feb 25 17:26:44 2012 +0100 +++ b/Doc/library/packaging.install.rst Mon Feb 27 13:14:28 2012 +0100 @@ -98,6 +98,14 @@ Public functions :file:`setup.py` script (in which case Distutils will be used under the hood to perform the installation). +.. function:: install_develop(path) + + Pseudo-install a project from a source directory, which will not copy any + files and need not rebuild or reinstall the whole project after you have + edited it. If the source directory contains a :file:`setup.py` using the + setuptools' develop. If a :file:`setup.cfg` is found, install using the + packaging's develop. + .. function:: remove(project_name, paths=None, auto_confirm=True) diff -r 1892f4567b07 Lib/packaging/command/__init__.py --- a/Lib/packaging/command/__init__.py Sat Feb 25 17:26:44 2012 +0100 +++ b/Lib/packaging/command/__init__.py Mon Feb 27 13:14:28 2012 +0100 @@ -9,7 +9,7 @@ from packaging.util import resolve_name STANDARD_COMMANDS = [ # packaging - 'check', 'test', + 'check', 'test', 'develop', # building 'build', 'build_py', 'build_ext', 'build_clib', 'build_scripts', 'clean', # installing diff -r 1892f4567b07 Lib/packaging/command/develop.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/Lib/packaging/command/develop.py Mon Feb 27 13:14:28 2012 +0100 @@ -0,0 +1,128 @@ +"""Install a project in development mode. + +This command can make your project importable without copying files, +through creating a .pth file in your site-packages dir or a specified +dir. +""" +import os +import sys +from packaging import logger +from packaging.dist import Distribution +from packaging.util import write_file, convert_path +from packaging.errors import PackagingOptionError +from packaging.command.cmd import Command + + +class develop(Command): + + description = "install the project in development mode" + + user_options = [ + ('uninstall', None, + "uninstall this project"), + ('install-scripts=', None, + "install scripts to DIR"), + ('no-scripts', None, + "don't install scripts"), + ('install-pth=', None, + "install the .pth file to a specific DIR which should be on sys.path"), + ('distinfo-path=', None, + "set the specified relative path to be used in the .distinfo-link file")] + + boolean_options = ['uninstall', 'no-scripts'] + + def initialize_options(self): + self.uninstall = None + self.install_pth = None + self.distinfo_path = None + self.install_dir = None + self.basename = None # the base name of .pth and .distinfo-link file + + def finalize_options(self): + di = self.get_finalized_command("install_distinfo") + + self.basename = di.distribution.metadata['Name'] # FIXME make filesafe + + if self.install_dir is None: + self.set_undefined_options('install_dist', ('install_lib', 'install_dir')) + + if self.install_pth is None: + self.install_pth = self.install_dir + + self.pth_file = os.path.join(self.install_pth, self.basename + '.pth') + + sys_path = [normalize_path(path) for path in sys.path] + if normalize_path(self.install_pth) not in sys_path: + logger.warning("your specific 'install_pth' should be on sys.path") + + self.distinfo_link = os.path.join(self.install_dir, self.basename + '.distinfo-link') + + self.source_base = os.curdir # the source directory + if self.distinfo_path is None: + self.distinfo_path = os.path.abspath(self.source_base) + + target = normalize_path(self.source_base) + if normalize_path(os.path.join(self.install_dir, self.distinfo_path)) != target: + raise PackagingOptionError("'distinfo-path' must be a relative path from the install" + " directory to " + target) + + def run(self): + if self.uninstall: + self.uninstall_link() + else: + self.install_for_development() + + def install_for_development(self): + """Install the project in development mode.""" + + # Ensure metadata is up-to-date + # Install the .dist-info directory in-place + cmd = self.reinitialize_command('install_distinfo') + cmd.distinfo_dir = os.curdir + self.run_command('install_distinfo') + # Build extensions in-place if necessary + cmd = self.reinitialize_command('build_ext') + cmd.inplace = True + self.run_command('build_ext') + + # Create a .distinfo-link file in the installation dir, pointing to our .dist-info dir + logger.info("creating %r (link to %r)", self.distinfo_link, self.source_base) + + if not self.dry_run: + with open(self.distinfo_link, "w") as f: + f.write(self.distinfo_path + "\n") + + self.update_path_file() + + # xxx should fix: install wrapper scripts if 'no-scripts' is False + + def uninstall_link(self): + """Uninstall all the installed files - .pth and .distinfo-link file.""" + + if os.path.exists(self.pth_file): + logger.info("removing %r ", self.pth_file) + if not self.dry_run: + os.unlink(self.pth_file) + if os.path.exists(self.distinfo_link): + logger.info("removing %r ", self.distinfo_link) + if not self.dry_run: + os.unlink(self.distinfo_link) + # if wrapper scripts are installed, then should also remove these scripts here. + + def update_path_file(self): + """Updates or create a .pth file.""" + + self.distinfo_location = normalize_path(self.source_base) + + # Remove old .pth file and create new one if necessary + if os.path.islink(self.pth_file): + os.path.unlink(self.pth_file) + logger.info('removing old path file %r', self.pth_file) + + self.execute(write_file, (self.pth_file, [self.distinfo_location]), + "creating %r" % self.pth_file) + +def normalize_path(filename): + """Normalize a file/dir name for comparison purposes.""" + + return os.path.normcase(os.path.realpath(filename)) diff -r 1892f4567b07 Lib/packaging/install.py --- a/Lib/packaging/install.py Sat Feb 25 17:26:44 2012 +0100 +++ b/Lib/packaging/install.py Mon Feb 27 13:14:28 2012 +0100 @@ -17,8 +17,7 @@ from sysconfig import get_config_var, ge from packaging import logger from packaging.dist import Distribution -from packaging.util import (_is_archive_file, ask, get_install_method, - egginfo_to_distinfo) +from packaging.util import _is_archive_file, ask, get_install_method from packaging.pypi import wrapper from packaging.version import get_version_predicate from packaging.database import get_distributions, get_distribution @@ -31,7 +30,7 @@ from packaging import database __all__ = ['install_dists', 'install_from_infos', 'get_infos', 'remove', - 'install', 'install_local_project'] + 'install', 'install_local_project', 'install_develop'] def _move_files(files, destination): @@ -90,6 +89,16 @@ def _run_packaging_install(path): except (IOError, os.error, PackagingError, CCompilerError) as msg: raise ValueError("Failed to install, " + str(msg)) +def _run_packaging_develop(path): + # check for a valid setup.cfg + dist = Distribution() + dist.parse_config_files() + try: + dist.run_command('develop') + name = dist.metadata['name'] + return database.get_distribution(name) is not None + except (IOError, os.error, PackagingError, CCompilerError) as msg: + raise ValueError("Fail to install in development mode, " + str(msg)) def _install_dist(dist, path): """Install a distribution into a path. @@ -132,6 +141,25 @@ def install_local_project(path): logger.warning('No project to install.') return False +def install_develop(path): + """Pseudo-install a project from a source directory, which will not copy + any files and need not rebuild or reinstall the whole project after you + have edited it. + + Returns True on success, False on Failure. + """ + path = os.path.abspath(path) + if os.path.isdir(path): + logger.info('Installing an editable version from source directory: %r', path) + return _run_develop_from_dir(path) + elif _is_archive_file(path): + logger.info('Installing an editable version from archive: %r', path) + _unpacked_dir = tempfile.mkdtemp() + shutil.unpack_archive(path, _unpacked_dir) + return _run_develop_from_archive(_unpacked_dir) + else: + logger.warning('No projects to install as editable.') + return False def _run_install_from_archive(source_dir): # XXX need a better way @@ -142,12 +170,22 @@ def _run_install_from_archive(source_dir break return _run_install_from_dir(source_dir) +def _run_develop_from_archive(source_dir): + for item in os.listdir(source_dir): + fullpath = os.path.join(source_dir, item) + if os.path.isdir(fullpath): + source_dir = fullpath + break + return _run_develop_from_dir(source_dir) install_methods = { 'packaging': _run_packaging_install, 'setuptools': _run_setuptools_install, 'distutils': _run_distutils_install} +develop_methods = { + 'packaging': _run_packaging_develop} + def _run_install_from_dir(source_dir): old_dir = os.getcwd() @@ -166,6 +204,20 @@ def _run_install_from_dir(source_dir): finally: os.chdir(old_dir) +def _run_develop_from_dir(source_dir): + old_dir = os.getcwd() + os.chdir(source_dir) + develop_method = get_develop_method(source_dir) + try: + func = develop_methods[develop_method] + try: + func(source_dir) + return True + except ValueError as err: + logger.info(str(err)) + return False + finally: + os.chdir(old_dir) def install_dists(dists, path, paths=None): """Install all distributions provided in dists, with the given prefix. diff -r 1892f4567b07 Lib/packaging/tests/test_command_develop.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/Lib/packaging/tests/test_command_develop.py Mon Feb 27 13:14:28 2012 +0100 @@ -0,0 +1,179 @@ +"""Tests for packaging.command.develop.""" + +import os +import sys + +from packaging.dist import Distribution +from packaging.command.develop import develop, normalize_path +from packaging.tests import unittest, support +from packaging.errors import PackagingOptionError + +class DevelopTestCase(support.TempdirManager, + support.LoggingCatcher, + unittest.TestCase): + def test_pth_file_installation(self): + # test if the .pth file can be succesfully generated and installed + projdir, dist = self.create_dist() + oldcwd = os.getcwd() + try: + cmd = develop(dist) + os.chdir(projdir) + cmd.ensure_finalized() + cmd.run() + finally: + os.chdir(oldcwd) + + pth_file = cmd.pth_file + self.assertIsNotNone(pth_file) + self.assertTrue(os.path.exists(pth_file)) + # check if it writes the right path + with open(pth_file) as f: + written_path = f.readline().rstrip() + self.assertEqual(written_path, normalize_path(projdir)) + + # .pth and .distinfo-link files are not in temp dir, so it needs clean work here + distinfo_link_file = os.path.join(cmd.install_dir, cmd.basename+'.distinfo-link') + self._clean([pth_file, distinfo_link_file]) + + def test_distinfo_link_file_installation(self): + # test if the .distinfo-link file is installed succesfully and with correct content + projdir, dist = self.create_dist(name='xx',version='1.2.3') + oldcwd = os.getcwd() + try: + cmd = develop(dist) + os.chdir(projdir) + cmd.ensure_finalized() + cmd.run() + finally: + os.chdir(oldcwd) + + distinfo_link_file = cmd.distinfo_link + self.assertIsNotNone(distinfo_link_file) + self.assertTrue(os.path.exists(distinfo_link_file)) + # check if it writes the right .dist-info directory path + with open(distinfo_link_file) as f: + written_path = f.readline().rstrip() + directory_name = 'xx-1.2.3.dist-info' + distinfo_dir = os.path.join(cmd.install_dir, written_path, directory_name) + self.assertTrue(os.path.exists(distinfo_dir)) + + # then clean + pth_file = os.path.join(cmd.install_dir, 'xx.pth') + self._clean([pth_file, distinfo_link_file]) + + def test_distinfo_dir_installation(self): + # test if the .dist-info directory is installed under correct directory + projdir, dist = self.create_dist(name='xxx', version='1.2.3') + distinfo_name = 'xxx-1.2.3.dist-info' + # the default installation path + distinfo_dir = os.path.join(projdir, distinfo_name) + # should not exist at first + self.assertFalse(os.path.exists(distinfo_dir)) + + oldcwd = os.getcwd() + try: + cmd = develop(dist) + os.chdir(projdir) + cmd.ensure_finalized() + cmd.run() + finally: + os.chdir(oldcwd) + + # should be created then + self.assertTrue(os.path.exists(distinfo_dir)) + + # then clean + pth_file = os.path.join(cmd.install_dir, 'xxx.pth') + distinfo_link_file = os.path.join(cmd.install_dir, 'xxx.distinfo-link') + self._clean([pth_file, distinfo_link_file]) + + def test_distinfo_path(self): + # test the distinfo_path option + projdir,dist = self.create_dist(name='def') + cmd = develop(dist) + # making sure the distinfo_path option is there + options = [name for name,short,label in cmd.user_options] + self.assertIn('distinfo-path=', options) + + # should be None at first + self.assertIsNone(cmd.distinfo_path) + + # an absolute path or a wrong relative path, should raise exception ; + # but we only need to check one + cmd.distinfo_path = self.mkdtemp() + os.chdir(projdir) + self.assertRaises(PackagingOptionError, cmd.finalize_options) + + # set the 'install_dir' option value: + # we construct the 'install_dir' value in a tmp dir + tmpdist = Distribution() + tmpcmd = develop(tmpdist) + tmpcmd.ensure_finalized() + cmd.install_dir = tmpcmd.install_dir + + # create a correct relative path and assign it to distinfo_path + p = cmd.install_dir.replace(os.sep, '/') + p = '../'*(p.count('/')+1) + cmd.distinfo_path = os.path.join(p, projdir) + + oldcwd = os.getcwd() + try: + os.chdir(projdir) + cmd.ensure_finalized() + cmd.run() + finally: + os.chdir(oldcwd) + + # we can check if the 'distinfo-path' option is set a right value + # through check the .distinfo-link file is successfully generated or not + self.assertIsNotNone(cmd.distinfo_link) + self.assertTrue(os.path.exists(cmd.distinfo_link)) + + # clean work + pth_file = os.path.join(cmd.install_dir, 'def.pth') + self._clean([pth_file, cmd.distinfo_link]) + + def test_uninstall(self): + # test the uninstall option + projdir, dist = self.create_dist() + cmd = develop(dist) + options = [name for name,short,label in cmd.user_options] + self.assertIn('uninstall', options) + # uninstall should be false at first + self.assertFalse(cmd.uninstall) + + oldcwd = os.getcwd() + try: + os.chdir(projdir) + cmd.ensure_finalized() + cmd.run() + finally: + os.chdir(oldcwd) + + # .pth and .distinfo-link should be created then + self.assertIsNotNone(cmd.pth_file) + self.assertIsNotNone(cmd.distinfo_link) + self.assertTrue(os.path.exists(cmd.pth_file)) + self.assertTrue(os.path.exists(cmd.distinfo_link)) + + # set it True + cmd.uninstall = True + cmd.run() + + # should remove them + self.assertFalse(os.path.exists(cmd.pth_file)) + self.assertFalse(os.path.exists(cmd.distinfo_link)) + # xxx should also check the wrapper scripts if features supported + + def _clean(self, paths=[]): + # clean function to remove generated files in python's + # default site-package + for file in paths: + if os.path.isfile(file): + os.unlink(file) + +def test_suite(): + return unittest.makeSuite(DevelopTestCase) + +if __name__ == '__main__': + unittest.main(defaultTest="test_suite") diff -r 1892f4567b07 Lib/packaging/util.py --- a/Lib/packaging/util.py Sat Feb 25 17:26:44 2012 +0100 +++ b/Lib/packaging/util.py Mon Feb 27 13:14:28 2012 +0100 @@ -1328,6 +1328,19 @@ def get_install_method(path): else: raise InstallationException('Cannot detect install method') +def get_develop_method(path): + """Only support packaging new-style develop command here. + + :param path: path to source directory containing a setup.cfg file, + or setup.py + + Returns a representing string. + """ + if is_packaging(path): + return "packaging" + else: + raise InstallationException('Cannot detect develop method') + # XXX to be replaced by shutil.copytree def copy_tree(src, dst, preserve_mode=True, preserve_times=True,