classification
Title: Packaging: add a 'develop' command
Type: enhancement Stage: resolved
Components: Distutils2 Versions: Python 3.3
process
Status: closed Resolution: out of date
Dependencies: 12279 Superseder:
Assigned To: eric.araujo Nosy List: Peter.Waller, alex.gronholm, alexis, carljm, chris, eric.araujo, higery, meatballhat, michael.mulich, mu_mind, tarek
Priority: normal Keywords: gsoc, patch

Created on 2010-05-09 01:19 by meatballhat, last changed 2014-03-13 04:23 by eric.araujo. This issue is now closed.

Files
File name Uploaded Description Edit
aa68d35988bb.diff eric.araujo, 2011-06-16 14:12 review
2750cd9e2111.diff higery, 2011-07-12 13:46 review
af7d14ff129b.diff higery, 2011-08-18 14:30 review
5359c895f841.diff eric.araujo, 2011-08-22 23:40 review
adb2cb19ca9b.diff higery, 2011-10-12 15:02 review
fe817128d2fc.diff eric.araujo, 2012-02-27 12:23 review
Repositories containing patches
https://bitbucket.org/higery/cpython#developwork
Messages (40)
msg105359 - (view) Author: Dan Buch (meatballhat) Date: 2010-05-09 01:19
make a 'develop' command in distutils2 a la setuptools

(unless, of course, this has already been unilaterally refused via distutils-sig or other communication)
msg105748 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-05-14 19:37
I have started to draft a similar proposal¹ but it seems that the wiki is not a communication medium as good as the bug tracker or the mailing lists (probably because of technical issues such as the lack of watchlist and e-mail notifications as much as because of Python culture). The discussion can happen here or on the mailing list unless Tarek firmly disagrees with the idea.

¹ http://wiki.python.org/moin/Distutils/Proposals/DevelopmentDistributions
msg115200 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2010-08-30 06:28
One part of develop that we could add is the creation of a .pth file in a directory on sys.path (with Python >= 2.6, --user gives that for free). IMNSHO, this should be implemented as an option to the install_lib* command, named --link-only or something better.

*(Not install, since .pth does not handle scripts, headers or data. The purpose of this command/option is to make modules and packages importable, more would be trickier.)

(“pythonx.y setup.py install_lib --user --link-only” is long, I agree. This problem is solved with copy/paste, shell history, shell function, or we could steal the aliases system from setuptools.)

I have a suspicion the install command (or a subcommand) already has most of what it takes to do this. I’m still trying to understand the extra_path argument (#901727), which may or may not give us 90% of what’s needed to implement this feature.
msg132709 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-03-31 21:46
See also http://groups.google.com/group/the-fellowship-of-the-packaging/browse_thread/thread/10506b8326a7b047
msg137843 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-06-07 15:54
FTR, Xu Dehai (higery) is working on this for GSoC.  We discussed the requirements on the mailing list: http://groups.google.com/group/the-fellowship-of-the-packaging/browse_thread/thread/ae196efc4956b9e2
This message in particular defines clearly the requirements: http://groups.google.com/group/the-fellowship-of-the-packaging/msg/9ef937cf2a90a882
msg138444 - (view) Author: Peter Waller (Peter.Waller) Date: 2011-06-16 14:43
Hi - Great to see this functionality coming. There is one feature of it that I would really like to see fixed, which is currently broken in setuptools/distribute - I'm sorry if this is the wrong forum for this note, but I wanted to add it to the discussion somewhere.

That feature is the "package_dir" argument to setup(). If the sources aren't in the root directory, the package doesn't function correctly with `python setup.py develop`

Here is a reference to an issue filed against distribute:

https://bitbucket.org/tarek/distribute/issue/177/setuppy-develop-doesnt-support-package_dir-arg-to

Is there any possibility of seeing this work correctly? A lot of packages use it, and for them, `develop` is currently broken.

Apologies if this feature is implemented and I missed it, but I see no reference to "package_dir" in the patch, so I would be (pleasantly) surprised if it was implemented. I would be happy to provide a testcase on request.
msg138447 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-06-16 15:08
Yes, this should of course be supported.

Note that in packaging, packages_dir has been simplified to packages_root, which means that all modules and packages must be in the same directory.
msg138602 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-06-18 21:28
Higery asked me why I recommended/asked that the .dist-info directory should be built in the build directory: “don't you feel it's a bit strange because we actually don't build the source except for some .c/.cpp files, but create a build directory?”

There are a few converging reasons for that.  Maybe a develop expert user will disagree with me, but I think that together they make a good case.

The first reason is symmetry.  The test command (http://docs.python.org/dev/packaging/commandref#test) needs to have the Python code in the build directory on sys.path, so that if you use build-time 2to3 conversion (currently undocumented :( apart from a line in http://docs.python.org/dev/packaging/setupcfg#command-sections), the tests need to find the 3.x code in the build dir, not the 2.x code in the project dir.  I think that since we already put the build dir on sys.path to run tests, we might as well follow the same idead and put the dist-info dir in the build dir.  The test command will then automatically have access to dist-info (see #12302), and other commands and tools can learn to look for dist-info in the build dir.

The second reason is that I think we need to separate build_distinfo from install_distinfo (#12279).  If we do this, then the new command has to write in the build dir, just like all other build_* commands (with possible exceptions).

The third reason is that I don’t like that every project that uses setuptools needs to add ProjectName.egg-info in the VCS ignore file (like .hgignore); if we use the build dir, our generated files are already covered by the ignore of the build dir.  (This makes me think that we may want to move MANIFEST to the build dir.)

I hope my reasons are clearer now; let me know if I did not explain something clearly enough, or if you disagree :)
msg140128 - (view) Author: Michael Mulich (michael.mulich) * Date: 2011-07-11 15:17
After looking over the use cases, these are my findings and questions:

* Cases 2, 3, 5 and 6 are strongly related. I'd suggest you condense them into a single use case. I agree with case 2 and 6 most, but have questions:
** Why wouldn't one simply use a virtualenv? -- Case 5 touches on this topic, but if we are installing in-place, who cares if can place a development package in the global site-packages directory?
** After the package has been installed in-place (using the develop command), how does one identify it as an in development project (or in development mode)? -- Case 3 and 6 touch on this topic (case 3 is a little vague at this time), but doesn't explain what type of action is intended. So if we install in-place (aka, develop), how does the python interpreter find the package? Are we using PYTHONPATH at this point (which would be contradict a requirement in  case 6)?

* Case 4 is a be unclear. Is Carl, the actor, pulling unreleased remote changes (hg pull --update) for these mercurial server plugins then running the develop command on them? 

* Case 1 is good and very clear, but I'd consider it a feature rather than required. Perhaps it should not be focused on first (priority). Thoughts?
msg140146 - (view) Author: Carl Meyer (carljm) * Date: 2011-07-11 16:43
Can someone post a link here to the page of use cases that Michael just reviewed? I think the link came through on the Fellowship mailing list, but I'm not quickly finding it...
msg140147 - (view) Author: Alexis Metaireau (alexis) * Date: 2011-07-11 16:55
Carl, I believe that's this one: http://wiki.python.org/moin/UsecasesOfDevelop
msg140151 - (view) Author: Carl Meyer (carljm) * Date: 2011-07-11 17:14
On 07/11/2011 09:17 AM, Michael Mulich wrote:
> * Cases 2, 3, 5 and 6 are strongly related. I'd suggest you condense them into a single use case. I agree with case 2 and 6 most, but have questions:
> ** Why wouldn't one simply use a virtualenv? 

I don't know. I don't consider case 3 useful, because I don't consider
"I don't want to use a virtualenv" (without some clearer technical
justification) to be a prejudice the develop feature needs to support;
especially if supporting it essentially means re-implementing a
less-capable version of virtualenv within the develop command.

> -- Case 5 touches on this topic, but if we are installing in-place, who cares if can place a development package in the global site-packages directory?

Several of these stories make the assumption that even the "in-place"
installation will require placing a file in the installation location (a
.pth file, if we follow the current setuptools implementation strategy).
I think this is probably true, given the requirements in case 6 (which I
agree with). So if you want an in-place install that's globally
accessible, you'd need write access to global site-packages.

> ** After the package has been installed in-place (using the develop command), how does one identify it as an in development project (or in development mode)? -- Case 3 and 6 touch on this topic (case 3 is a little vague at this time), but doesn't explain what type of action is intended. So if we install in-place (aka, develop), how does the python interpreter find the package? Are we using PYTHONPATH at this point (which would be contradict a requirement in  case 6)?

These use cases (probably intentionally) don't touch on specific
implementation strategies, but as I mentioned there's an implicit
assumption that a .pth file is the most likely strategy.

> * Case 4 is a be unclear. Is Carl, the actor, pulling unreleased remote changes (hg pull --update) for these mercurial server plugins then running the develop command on them? 

Right, although the requirement for that story is that you don't have to
re-run the develop command after every pull; if you develop-install it
once, you can simply pull more code changes in and they'll immediately
be available. I've added a line to that story to make it more clear.

> * Case 1 is good and very clear, but I'd consider it a feature rather than required. Perhaps it should not be focused on first (priority). Thoughts?

I agree that's a second-level feature (or, perhaps more accurately, a
bug in the existing setuptools feature that I was hoping could be
addressed in the d2 version), not a primary requirement.
msg140166 - (view) Author: Michael Mulich (michael.mulich) * Date: 2011-07-12 01:14
On Mon, Jul 11, 2011 at 1:14 PM, Carl Meyer <report@bugs.python.org> wrote:
>> * Cases 2, 3, 5 and 6 are strongly related.
> I don't know. I don't consider case 3 useful, because I don't consider
> "I don't want to use a virtualenv" (without some clearer technical
> justification) to be a prejudice the develop feature needs to support;
> especially if supporting it essentially means re-implementing a
> less-capable version of virtualenv within the develop command.

I think your later comments about the use of .pth files solves the
issue for all four cases. We simply make reference in a .pth file in
one of the approved site locations. For example, in case two we would
write a .pth entry to site.USER_SITE. Sound about right?

>> -- Case 5
> Several of these stories make the assumption that even the "in-place"
> installation will require placing a file in the installation location (a
> .pth file, if we follow the current setuptools implementation strategy).
> I think this is probably true, given the requirements in case 6 (which I
> agree with). So if you want an in-place install that's globally
> accessible, you'd need write access to global site-packages.

Basically write the .pth entry for the build to a site (the standard
lib module) recognized location.

>> * Case 4
> Right, although the requirement for that story is that you don't have to
> re-run the develop command after every pull; if you develop-install it
> once, you can simply pull more code changes in and they'll immediately
> be available. I've added a line to that story to make it more clear.

Ah, this case impacts the decision being made in issue 12279
(http://bugs.python.org/issue12279). The decision is to write a RECORD
file or not. We wouldn't write a RECORD if you want to be able to
update without rerunning the develop command. But this would be
invalid based on PEP 376 guidelines. Please post your thoughts about
this in that issue.

The wiki page has been edited to note what the develop command will
write to the file system. I'll restate it here as well...

The develop command writes three pieces of information to the filesystem:
 1. It calls upon the build action(s) to build the package relative to
the package's root directory.
 2. It calls the [build|install]_distinfo action to write the
.dist-info metadata inside the build directory. (see also Issue 12279)
 3. It adds the build directory's path to a .pth file.

Thanks Carl
msg140182 - (view) Author: higery (higery) Date: 2011-07-12 13:34
>
> ** After the package has been installed in-place (using the develop
> command), how does one identify it as an in development project (or in
> development mode)? -- Case 3 and 6 touch on this topic (case 3 is a little
> vague at this time), but doesn't explain what type of action is intended. So
> if we install in-place (aka, develop), how does the python interpreter find
> the package? Are we using PYTHONPATH at this point (which would be
> contradict a requirement in  case 6)?
>

There is an .egg-link file that will be used by pkg_resources to find the
develop-installed packages, so my current implementation of develop command
in packaging module also adds a .distinfo-link file in the site-packages
which will be used to identify a project is installed in development mode or
not.
msg140184 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-07-12 13:37
For now, you should not worry about pkg_resources.  Write a simple pure-packaging implementation compatible with packaging; the setuptools and distribute developers will see if they want to add forward compatibility with our system.
msg140186 - (view) Author: higery (higery) Date: 2011-07-12 13:38
2011/7/12 Michael Mulich <report@bugs.python.org>

>
> Michael Mulich <michael.mulich@gmail.com> added the comment:
>
> The wiki page has been edited to note what the develop command will
> write to the file system. I'll restate it here as well...
>
> The develop command writes three pieces of information to the filesystem:
>  1. It calls upon the build action(s) to build the package relative to
> the package's root directory.
>  2. It calls the [build|install]_distinfo action to write the
> .dist-info metadata inside the build directory. (see also Issue 12279)
>  3. It adds the build directory's path to a .pth file.
>

You are right, what you listed above are also the things done by the
'develop' command of my current implementation. In addition, as I replied
earlier, we can also add a .distinfo-link file  more than the .pth file.
msg140194 - (view) Author: Michael Mulich (michael.mulich) * Date: 2011-07-12 14:10
On Tue, Jul 12, 2011 at 9:39 AM, higery <report@bugs.python.org> wrote:
>> The develop command writes three pieces of information to the filesystem:
>>  1. It calls upon the build action(s) to build the package relative to
>> the package's root directory.
>>  2. It calls the [build|install]_distinfo action to write the
>> .dist-info metadata inside the build directory. (see also Issue 12279)
>>  3. It adds the build directory's path to a .pth file.
>>
>
> You are right, what you listed above are also the things done by the
> 'develop' command of my current implementation. In addition, as I replied
> earlier, we can also add a .distinfo-link file  more than the .pth file.

I don't like the idea of a .distinfo-link file. Would it even be
necessary if we already have a .pth entry?

We should probably just use one of these files, either .distinfo-link
or .pth. The .pth implementation has the least impact on code base and
is already implemented. If we add support for a .distinfo-link, we
would then need to modify database module to support that extension.
msg140195 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-07-12 14:18
Oh, I just realized that one thing I insisted on was wrong.

I pushed for the modules to be built in the build dir, as well as the dist-info dir, so that the build dir can be added to sys.path to let both import and packaging.database find the files.  But this breaks one important develop feature: editions to the code should be visible immediately, without having to re-run develop or build.
msg140197 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-07-12 14:31
Ah, higery’s code already has an answer for me: it writes *two* paths in the .pth file, one to the build dir (so that .dist-info is found) and one to the modules root (for modules, built in place).  Anyone sees a problem with that?  (For example huge sys.path.)

In this scheme, when Python modules are edited, changes are visible instantly, when C modules are edited, a call to build_ext is required, and when the metadata is edited, build_distinfo is required.  Does that sound good?
msg140198 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-07-12 14:48
I’ve reviewed the last patch.  It looks like the code only installs to the global site-packages, and there is no support to install to the user site-packages or to another arbitrary location.

On Windows, normal users seem to be able to write to the global site-packages (see #12260), but on other OSes with a proper rights model <wink> that won’t do.  Luckily, PEP 370 brings us user site-packages (currently poorly documented, see #8617 and #10745), but only for 2.6, 2.7 and 3.x.  It looks like Tarek is ready to drop 2.4 compatibility for distutils2, so the question is: what to do under 2.5?

Generally, I don’t see why develop could not install to any directory.  We want a default invocation without options to Just Work™, finding a writable directory already on sys.path and writing into it, but that doesn’t exclude letting the user do what they want.
msg140205 - (view) Author: Carl Meyer (carljm) * Date: 2011-07-12 19:04
> Ah, higery’s code already has an answer for me: it writes *two* paths in the .pth file, one to the build dir (so that .dist-info is found) and one to the modules root (for modules, built in place).  Anyone sees a problem with that?  (For example huge sys.path.)
> 
> In this scheme, when Python modules are edited, changes are visible instantly, when C modules are edited, a call to build_ext is required, and when the metadata is edited, build_distinfo is required.  Does that sound good?

That sounds reasonable to me. I'm not worried about that making sys.path
too long: whatever we do we aren't going to challenge buildout in that
department ;-)
msg140208 - (view) Author: Carl Meyer (carljm) * Date: 2011-07-12 19:30
> I’ve reviewed the last patch.  It looks like the code only installs
> to the global site-packages, and there is no support to install to
> the user site-packages or to another arbitrary location.
> 
> On Windows, normal users seem to be able to write to the global
> site-packages (see #12260), but on other OSes with a proper rights
> model <wink> that won’t do.  Luckily, PEP 370 brings us user
> site-packages (currently poorly documented, see #8617 and #10745),
> but only for 2.6, 2.7 and 3.x.  It looks like Tarek is ready to drop
> 2.4 compatibility for distutils2, so the question is: what to do
> under 2.5?
> 
> Generally, I don’t see why develop could not install to any
> directory.  We want a default invocation without options to Just
> Work™, finding a writable directory already on sys.path and writing
> into it, but that doesn’t exclude letting the user do what they
> want.

I don't see why the installation-location-finding for develop should be
any different than for a normal "pysetup install". Does "pysetup
install" install to global site-packages by default, or try to find
somewhere it can install without additional privileges? Whatever it does
by default, develop should do the same. If "develop" can install to
arbitrary locations, then "install" should be able to as well (though I
don't really see the value in "arbitrary locations", since you then have
to set up PYTHONPATH manually anyway). There is no reason for them to
have different features in this area, it just adds confusion.

Certainly "develop" should support PEP 370, ideally with the same
command-line flag as a regular install.
msg140600 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-07-18 16:18
[Carl]
> there's an implicit assumption that a .pth file is the most likely
> strategy.
If you have other ideas, please share them.

[another message]
> I don't see why the installation-location-finding for develop should
> be any different than for a normal "pysetup install".
It’s only a technical limitation for now: the develop command is currently a standalone command, so it has to decide where to write stuff.  If it were an option to install_dist instead of a standalone command, then it would have paths processing already written.

Higery changed his code recently to get paths from the install_dist command instead of requiring site-packages.  (You can read the reviews, if you don’t mind style comments mixed with more important issues.)

> Does "pysetup install" install to global site-packages by default, or
> try to find somewhere it can install without additional privileges?
The install action can have a different behavior than the install_dist command.  develop is only a command now, and I agree it should behave like install_dist (which it now does).

> (though I don't really see the value in "arbitrary locations", since
> you then have to set up PYTHONPATH manually anyway).
We don’t know what people do, what with /opt installs and plugins and whatever, so there’s just no value in not allowing any path for install.

> Certainly "develop" should support PEP 370, ideally with the same
> command-line flag as a regular install.
Yes.
msg140757 - (view) Author: Carl Meyer (carljm) * Date: 2011-07-20 17:53
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> Éric Araujo <merwok@netwok.org> added the comment:
> 
> [Carl]
>> there's an implicit assumption that a .pth file is the most likely
>> strategy.
> If you have other ideas, please share them.

No, I think that's the most promising strategy. The "implicit
assumption" comment was not criticism, just explanation for Michael.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4nF2wACgkQ1j/fhc23WEDvlwCeK3Y+MJGyb3uoEzYzJWaSCrTy
WewAoI7UdW+nqP2SEtquvQXCndXX57VO
=UFOY
-----END PGP SIGNATURE-----
msg140848 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-07-22 00:44
Great.  I’ve been re-reading some old threads in the spirit of “pth files are evil”, so I wondered whether we could/should avoid them, but I think the criticism was directed against pth files edited after the initial installation, and pth files abusing the format (import causes exec), IOW setuptools’ pth files; our current ideas avoid these two problems.
msg142329 - (view) Author: higery (higery) Date: 2011-08-18 14:51
Through discussing inside or outside the mailing list on this bug tracker, current 'develop' has been made a kind of command other than an action. But there still isn't a consensus of the concrete implemention way, so I keep this command name as 'develop' and the simple usage of this command is :

pysetup run develop

BTW, most developers in this list have agreed that 'develop' is a kind of install command, so I also add an entry funtion 'install_editable' for it, like 'install_local_project' for the 'install_dist' command. In addition, current patch also supports several ways to run the 'develop' which is learned from the source code of install.py. When installing, there are three ways, '_run_packaging_install', '_run_setuptools_install', and '_run_distutils_install', so here for develop command, there are two ways, '_run_packaging_develop' and '_run_setuptools_develop'. You can know more through the source code and docs in this patch.

BTW, should way convert the .egg-info directory to .dist-info directory?
msg142332 - (view) Author: Alexis Metaireau (alexis) * Date: 2011-08-18 15:12
_run_setuptools_install is only intended to support setuptools setup.py, converting .egg-info to .dist-info, internally. IMO, you should not care about the differences between setuptools/distutils1/setuptools at this level, as it should be taken care at the install level.

When installing old setuptools-based setup.py, the .egg-info should be gone, replaced by a shiny new .dist-info folder. I'm not sure why you're talking about it in the context of develop, can you clarify this?
msg142334 - (view) Author: Alexis Metaireau (alexis) * Date: 2011-08-18 15:14
IOW, in my opinion, support for setuptools develop command is not needed in packaging core, and still be taken care directly be the users wanting to run python setup.py develop: I don't see any reason to make it avaible on the stdlib.
msg142340 - (view) Author: higery (higery) Date: 2011-08-18 15:30
Alexis Metaireau <alexis@notmyidea.org> added the comment:
>>I'm not sure why you're talking about it in the context of develop, can you clarify this?

My consideration is : if in Packaging we always convert .egg-info directory to .dist-info directory, then my two different kinds of ways(setuptools' develop or packaging's develop) for the 'develop' command are unnecessary, we can just retain the new-style packaging's develop, which mean '_run_packaging_develop' function here.
msg142341 - (view) Author: Alexis Metaireau (alexis) * Date: 2011-08-18 15:34
Yep, packaging is not keeping the .egginfo directories, or at least does not plan to keep them (It should be the case currently but I haven't checked recently) in the upcoming release, so I would go on removing support for setuptools' develop command here :)
msg142347 - (view) Author: higery (higery) Date: 2011-08-18 15:54
>>IOW, in my opinion, support for setuptools develop command is not needed in packaging core

Then do you also mean support that for setuptools install is also not necessary in packaging core?

>>and still be taken care directly be the users wanting to run python setup.py develop: I don't see any reason to make it avaible on the stdlib.

I think the current implemention way of Packaging 'install' command just offers an executing router to run a proper 'install', which users doesn't need to know. It also the same to 'develop'. BTW, I think it's ok if setuptools is already installed when a user try to run 'python setup.py develop', if not, python will report error. In addition, Packaging use a different way to issue commands - cd to the source directory, and use 'pysetup run develop' to do the job. Old setup.py will be supported only by issuing commands under the source directory containing the setup.py, and if setuptools exists, Packaging will dispatch the job to it.
msg142351 - (view) Author: Alexis Metaireau (alexis) * Date: 2011-08-18 16:05
On 08/18/2011 05:54 PM, higery wrote:
> Then do you also mean support that for setuptools install is also not necessary in packaging core?

setuptools install is only supported in packaging because it's a widely 
used thing, and many python distributions are currently packaged using 
setuptools features in their setup.py

 > I think the current implemention way of Packaging 'install' command 
just offers an executing router to run a proper 'install', which users 
doesn't need to know.

What packaging.install does is, if the project has been packaged using 
setuptools, relying on it to install the projects, while not letting it 
take care of the dependencies. Then, the .egginfo is converted to a 
.distinfo, so the way setuptools distributions are installed is *not* 
compatible with setuptools installation procedure.

It is important to note that setuptools will *not* be a dependency for 
packaging, but rather the end user will be prompted to install it if 
it's not present and he wants to install a setuptools based project (or 
if what he's trying to install relies on setuptools based projects).

The problem with the develop command seems different to my eyes: develop 
is used by developers, not by end users. Thus, proposing them to keep 
using the "old" setuptools develop command in packaging may seem like 
encouraging them to continue using setuptools.

Rather, I would prefer to say: "well, if you want to use setuptools' 
develop command, you can continue to do so, but the way to go would 
probably to repackage your project using shiny new standards". That's 
one of the handles we have to help the transition to packaging, so 
taking this occasion seems important to me.
msg142383 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-08-18 17:48
> [...] so I keep this command name as 'develop' and the simple usage
> of this command is :
> pysetup run develop
So far, so good.

> BTW, most developers in this list have agreed that 'develop' is a
> kind of install command, so I also add an entry funtion
> 'install_editable' for it,
That can be useful.  On the other hand, pip uses subprocesses instead of calling Python functions, to make sure an error in the called code does not stop pip.  I still think it’s a nice function to have.  The name should be changed: “editable” is not used in our current terminology, so install_develop would be IMO better.

I also think that “pysetup install --develop path/to/project” would be a nice high-level way to use develop, in the future.

About supporting setup.py develop: I think too that it is not needed.  Supporting setup.py install is a needed feature for *users*, as Alexis said; conversely, having a packaging-only develop feature can help seduce *developers* to use the new standards and tools.  My vote is for removing setup.py develop support.
msg145017 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-10-06 14:57
higery, can you give us a status update?  Do you have the time to address current reviews or would you like me to make an updated patch?  I’d like to incorporate this command as soon as possible to let people play with it, and then we’ll see about integration with the install action.
msg145261 - (view) Author: higery (higery) Date: 2011-10-09 15:18
>> higery, can you give us a status update?  Do you have the time to ...

Sorry to reply to you so late, I will read carefully these reviews asap and try to make an updated patch before 12th.
msg145399 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2011-10-12 15:56
Thanks for the updated patch.  There are things that should be removed (for example functions like get_develop_method, given that we only support packaging-based projects), and there are a few things to clean up in the tests.  What’s more practical for you, a review or a patch?
msg145455 - (view) Author: higery (higery) Date: 2011-10-13 15:21
>> What’s more practical for you, a review or a patch?

I think a review is better(thanks for your time:) ), because some changes maybe needed after I have finished all the issues against the develop command.
msg154469 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2012-02-27 12:23
Merged yesterday and created this patch.  Will refresh my memories by reading all messages again and review.
msg162738 - (view) Author: Alex Grönholm (alex.gronholm) * Date: 2012-06-13 22:52
Python 3.3 is entering beta soon. The develop command is a must have, especially now that virtualenv is part of the official Python distribution. Can someone summarize what still needs to be done to get this feature merged?
msg213351 - (view) Author: Éric Araujo (eric.araujo) * (Python committer) Date: 2014-03-13 04:23
This is handled by setuptools/pip.  It cannot be converted to a distutils feature request, as distutils does not handle dependencies.
History
Date User Action Args
2014-03-13 04:23:10eric.araujosetstatus: open -> closed
resolution: out of date
messages: + msg213351

stage: needs patch -> resolved
2012-06-13 22:52:23alex.gronholmsetnosy: + alex.gronholm
messages: + msg162738
2012-02-27 12:23:33eric.araujosetfiles: + fe817128d2fc.diff

dependencies: - Add **kwargs to reinitialize_command
messages: + msg154469
2011-12-01 13:28:48chrissetnosy: + chris
2011-10-18 01:41:31mu_mindsetnosy: + mu_mind
2011-10-13 15:21:46higerysetmessages: + msg145455
2011-10-12 15:56:42eric.araujosetmessages: + msg145399
2011-10-12 15:02:19higerysetfiles: + adb2cb19ca9b.diff
2011-10-09 15:18:08higerysetmessages: + msg145261
2011-10-06 14:57:08eric.araujosetmessages: + msg145017
2011-08-22 23:40:36eric.araujosetfiles: + 5359c895f841.diff
2011-08-18 17:48:19eric.araujosetmessages: + msg142383
2011-08-18 16:05:21alexissetmessages: + msg142351
2011-08-18 15:54:16higerysetmessages: + msg142347
2011-08-18 15:34:33alexissetmessages: + msg142341
2011-08-18 15:30:01higerysetmessages: + msg142340
2011-08-18 15:14:24alexissetmessages: + msg142334
2011-08-18 15:12:23alexissetmessages: + msg142332
2011-08-18 14:51:42higerysetmessages: + msg142329
2011-08-18 14:30:38higerysetfiles: + af7d14ff129b.diff
2011-07-22 00:44:56eric.araujosetmessages: + msg140848
2011-07-20 17:53:56carljmsetmessages: + msg140757
2011-07-18 16:18:46eric.araujosetmessages: + msg140600
2011-07-12 19:30:34carljmsetmessages: + msg140208
2011-07-12 19:04:59carljmsetmessages: + msg140205
2011-07-12 14:48:16eric.araujosetmessages: + msg140198
2011-07-12 14:31:44eric.araujosetmessages: + msg140197
2011-07-12 14:18:07eric.araujosetmessages: + msg140195
2011-07-12 14:10:49michael.mulichsetmessages: + msg140194
2011-07-12 13:49:10eric.araujosetfiles: - b1b9da3b3d20.diff
2011-07-12 13:46:46higerysetfiles: + 2750cd9e2111.diff
2011-07-12 13:41:15higerysetfiles: - unnamed
2011-07-12 13:38:59higerysetfiles: + unnamed

messages: + msg140186
2011-07-12 13:37:07eric.araujosetmessages: + msg140184
2011-07-12 13:36:06eric.araujosetfiles: - unnamed
2011-07-12 13:34:37higerysetfiles: + unnamed

messages: + msg140182
2011-07-12 01:14:45michael.mulichsetmessages: + msg140166
2011-07-11 17:14:03carljmsetmessages: + msg140151
2011-07-11 16:55:41alexissetmessages: + msg140147
2011-07-11 16:43:18carljmsetmessages: + msg140146
2011-07-11 15:17:18michael.mulichsetmessages: + msg140128
2011-07-08 11:24:51eric.araujosetfiles: + b1b9da3b3d20.diff
2011-06-18 21:28:17eric.araujosetnosy: + carljm
messages: + msg138602
2011-06-16 16:16:18eric.araujosetdependencies: + Add **kwargs to reinitialize_command
2011-06-16 15:08:32eric.araujosetmessages: + msg138447
2011-06-16 14:43:29Peter.Wallersetmessages: + msg138444
2011-06-16 14:12:13eric.araujosetfiles: + aa68d35988bb.diff
keywords: + patch
2011-06-16 14:11:27eric.araujosethgrepos: + hgrepo29
2011-06-15 15:07:45Peter.Wallersetnosy: + Peter.Waller
2011-06-09 22:41:24michael.mulichsetnosy: + michael.mulich
2011-06-07 15:54:54eric.araujosetnosy: + higery
messages: + msg137843

dependencies: + Add build_distinfo command to packaging
stage: needs patch
2011-06-06 16:23:06eric.araujosetkeywords: + gsoc
assignee: tarek -> eric.araujo
title: add a 'develop' command -> Packaging: add a 'develop' command
versions: + Python 3.3, - 3rd party
2011-03-31 21:46:24eric.araujosetmessages: + msg132709
2010-09-30 00:00:32eric.araujosetversions: + 3rd party, - Python 2.6, Python 2.5, Python 3.1, Python 2.7, Python 3.2
2010-08-30 06:28:07eric.araujosetnosy: + alexis
messages: + msg115200
2010-05-14 19:37:43eric.araujosetnosy: + eric.araujo

messages: + msg105748
versions: + Python 2.6, Python 2.5, Python 3.1, Python 2.7, Python 3.2
2010-05-09 01:19:04meatballhatcreate