The devguide recommends using hg update to switch between branches in one repository.  This is only practical if you build Python in a custom (sub)directory, otherwise you’d need to either do the configure-make-test dance when merging/porting patches, or skip testing.  I’ve always used one clone per Python version, where I can keep the compiled artifacts; it makes it easy to see what a file looks like in any of the three versions, easy to work on different things in the different repos (like fixing something in 3.2 that you noticed while you were adding something to 3.3), it is cheap thanks to hardlinks, fast because you run hg pull instead of hg update to merge a patch from 3.2, and is just the simplest thing that works.  I don’t think anyone uses the one-clone-with-update approach, so I think we should rewrite the instructions to talk about one clone per version.

+1

I agree, for reasons stated. Having to skip over the wrong instructions made getting started a bit harder for me. I also think TortoiseHG should get more than just a mention. The initial re-creation of the recent DAG for each branch when starting up Workbench takes some time, but having the graph makes it immediately obvious whether the repository is in a proper state -- two heads for default and 2.7, three for 3.2 -- both before starting work after pulling from py.org and again when ready to push changes back.

Are you referring to http://docs.python.org/devguide/committing.html#forward-porting ?

+1

It is important to note that if you have a 'cpython' as a clone (which pulls and pushed to hg.python.org) and from it you clone '2.7' and '3.2' (as I think it's the most common setup) you first have to pull from cpython and then on the other 2.

Anyhow, is anyone up for preparing a patch? else I'll happily work on it in the coming days.

Here's the proposed patch.

I'm sorry but probably 'hg diff' went too wild, and since I moved the "Using several working copies" as the first sub-paragraph of "Forward-Porting" it seems it got confused, so the diff is not so clean. I've also changed quite some part of "Porting Within *" sections - please look at them carefully since it's not that evident.

A brief list is:

- change in commands to reflect the multi-clone setup
- add of "Assuming all your clones are in the same directory"
- add of "The import is possible..." sentence

-   $ hg clone py3k py3.2
-   $ cd py3.2
-   $ hg update 3.2
+   $ hg clone py3k py3.2 -u 3.2

While the second command is more concise, I find the first easier to understand, so maybe you could leave both the first time (proposing the concise one as an alternative), and then use just the second.
It's also possible to do "hg clone py3k#3.2 py3.2".

There are a few differences between the process you describe and the one I personally use (e.g. I first commit on 2.7, export/import on 3.2, merge with 3.3).  Also since I cloned 2.7 using py3k#2.7, I don't think I would be able to graft a py3 changeset in 2.7 (but it should work the other way around).

Hi Ezio, thanks for the review: how about this new version of the patch? I've left the more "verbose" version in the 3.2 example, while used the more compact way for the 2.7 .

Re first committing to 2.7 then 3.2 and then merge, that would work for doc patches, but probably what happens most of the time is the commit in 3.2 and merge on default, which should be covered.

Any other comments?

It might be worth mentioning and/or linking to some of the branching tips  discussed in the FAQ.  For example, using `hg share` instead of `hg clone` eliminates the need to pull between repositories/working directories:

http://docs.python.org/devguide/faq.html#how-do-i-avoid-repeated-pulls-and-pushes-between-my-local-repositories

I didn't know the FAQ had hg workflow tips until a recent discussion on core-mentorship.

Sandro, I find your patch is really a regression:

-* Then clone it to create another local repository which is then used to
-  checkout branch 3.2::
+* Then clone it to create another local repository for the 3.2 branch::
 
This makes the terminology ambiguous. The old sentence is worded exaclty like that, because it is important to make the distinction between repository and working copy contents. You do *not* create a repository for a single branch; instead you clone the *whole* repository and then checkout the desired branch. If you don't distinguish the concepts clearly, hg beginners will get a wrong idea of what happens.

+  (this command is equivalent to those used to create a local 3.2 branch, but
+  it's more concise.)

This is wrong too. You *don't* create a local 3.2 branch, you checkout the existing 3.2 branch. This is totally different.

+you have a patch that should also be applied to Python 3.2. To properly port
+the patch to both versions of Python, you should first apply the patch to
+Python 3.2::

Please replace "apply the patch to Python 3.2" with "apply the patch to the 3.2 branch".

+With the patch now committed, you want to merge the patch up into Python 3.3.

a.k.a in the default branch.

+This will push changes in both the Python 3.2 and Python 3.3 branches to
+hg.python.org.

I would prefer "the 3.2 and default branches".

+The import is
+possible because changesets are unique per repository, and so in ``py2.7`` you
+also have all of those of the other branches.

I find the wording of this sentence confusing and I don't think it brings any useful information.
(yes, changeset ids are unique, this is a property of Mercurial and doesn't deserve mentioning specifically here)

I think the aim of that part of the devguide is to give one clear, simple, working way to operate on different branches at the same time. Additional workflows can be presented, but probably in another place (like the FAQ indeed). What others thing about this?

> I think the aim of that part of the devguide is to give one clear, simple, working way to operate on different branches at the same time.

The change can be as simple as adding a sentence along the lines of, "Also see the version control section of the FAQ <http://docs.python.org/devguide/faq.html#version-control> for some suggestions for alternative workflows."

You could mention "hg out" where you talk about pushing all the changes.

About hg graft, I would assume that most of the patches grafted from 3.x don't work right away on 2.7, or if the graft work the code might still need to be changed.  How do you deal with this situation?  Is there a --no-commit option for graft?  (I haven't used graft yet)

> Re first committing to 2.7 then 3.2 and then merge

FWIW now that 2.7 is "getting old" I agree that committing on 3.x first and possibly backporting to 2.7 later is more common.

> Any other comments?

[Disclaimer: This is not directly related to your patch but about other things mentioned in the same page. Here I'm describing my workflow, it might or might not be better than the one suggested by the devguide, but if it is feel free to include the relevant bits in the patch]
That page suggest to push from e.g. 3.2 to py3k.  I usually do the opposite, i.e. I always pull the 3.2 changes in py3k.  I guess it's just a matter of preferences, but iiuc if you push and there's a conflict the push will fail, whereas the pull will usually work better.  Not using push myself I can't tell how often that happens and if that's a problem at all.
I also have all the clones configured to pull and push from h.p.o.  As soon as I "enter" a clone I "hg pull -u" the changes from h.p.o, and if I have to pull some local changesets too I also "hg pull -u ../py3.2".  When I'm done I "hg push" from py3k (or sometimes from 2.7, if the changes don't apply to 3.x).  I also "hg pull -u" just before committing, in case someone pushed something in the meanwhile.
This means that:
 * I don't have to chain-pulling following 2.7<-3.2<-default, and thus I don't need the "simple script" suggested by the devguide;
 * If I have uncommitted changes in one of the clone, I can deal with them one by one, and not when I run some scripts that tries to update everything at once;
 * I download the same changes from h.p.o 2-3 times;
 * I have to do a bit more typing;

This can be summarized with:
$ cd py32
$ hg pull -u
$ hg import --no-c patch.diff
$ # review, run tests, etc.
$ hg ci -m '...'
$ cd ../py3k
$ hg pull -u
$ hg pull -u ../3.2
$ hg merge 3.2
$ # solve conflicts, run tests, etc.
$ hg ci -m '...'
$ hg push

FWIW I agree with Antoine’s comments.

FTR I now switched to hg share, and while I think it's a better option for committers that work on several branches on a daily basis, it might not be the same for contributors that usually prepare patches against default.

"hg graft" should also be mentioned.  I now use "hg graft 2.7" instead of "hg export 2.7 | hg import -" to copy changeset from 2.7 to 3.2 (and then merge on 3.3/3.x).

Note that "hg graft" is already mentioned/discussed in the devguide here:

http://docs.python.org/devguide/committing.html#porting-between-major-versions

The attached patch adds a couple of section about the single and multiple clones approaches. The patch is still incomplete, because the rest of the page should be adapted to the new content (in particular the old sections should be removed, and the whole structure revisited), but I wanted some early feedback about the ones I added.

The idea is that contributors can use the single clone approach, and since they don't need to commit/merge, they don't need more instructions then the ones provided already in the setup.rst page.
Committers are better off with multiple clones, and the best way to do it is IMHO with the share extension, so that's what I described.
Ideally this section should be followed by a FAQ-like list that explains how to deal with conflicts, head merges, null merges, long-term features and similar.  Some of these things are already there; some extra things are also there, and they should probably be moved or removed.

Content-wise the patch looks pretty good.  I agree with the recommendations.  A couple suggestions though: I would break up the 20 lines of command-line commands.  Right now that chunk is a bit too long to grasp meaningfully.  My suggestion would be to break it up into two sections with text in between: one for applying a patch to 2.x or 3.x, and the other for merging (forward-porting) from 3.x to 3.y, with a textual explanation of what the subsequent chunk of commands will do.

I would also state (or link to) something about forward-porting from 3.x to 3.y and that 2.7 should be kept separate (both of which I think the current patch assumes knowledge of).  I would also say (or link to) something about pushing all branches simultaneously.

Lastly, might it be worth explicitly dividing the Mercurial stuff into separate sections for (1) everyone, and (2) committers?  Putting the committer-specific stuff (e.g. instructions on merging and pushing changes) in a separate section will simplify things for the general contributor.

> I would break up the 20 lines of command-line commands.

I would have to find a compromise for this, because on one hand it's convenient to have all the commands in a single place (so it's easy to get an overview), but on the other hand that block includes several logically different operation (importing, grafting, merging) that would deserve their own "FAQ".

> I would also state (or link to) something about forward-porting from
> 3.x to 3.y and that 2.7 should be kept separate (both of which I
> think the current patch assumes knowledge of).

This is currently explained, but should be reorganized and integrated with my patch.

> I would also say (or link to) something about pushing all branches 
> simultaneously.

This happens automatically, so I don't think it deserves more than a short mention somewhere.

> Lastly, might it be worth explicitly dividing the Mercurial stuff
> into separate sections for (1) everyone, and (2) committers? 

ISTM that all the things that contributors need to know are already explained somewhere else in the devguide.  This includes cloning, switching branches, and applying and generating patches.
What I'm described here is mostly aimed to developers, since contributors don't have to commit/merging/grafting and dealing with all the related things.

One thing that occurred to me is that it is often or usually not sufficient to go from 2.7 to 3.2 and on forward because applying a patch made against the default branch loses information if first applied to an earlier branch.  The given workflow assumes no loss of information and so should probably note this constraint.

I usually craft my patch against the default branch.  If applying to 2.7 or 3.2, etc. loses information (which has been more often the case for me), then instead of merging I null-merge and reapply the original patch.  Should the recommended workflow cover this possibility?

It can probably be added to the list of "FAQs", or mentioned together with null merges.

Here is another iteration of the patch.
I removed some of the old material, added half of the FAQs and the title and outline for the other FAQs.  If the structure looks good I'll proceed.

I just noticed that the devguide already contains a section in the FAQ about Mercurial.  Would it make sense to move/keep questions for committers only in the committing.rst page, and leave the generic question in faq.rst?

Since this patch is on the longer side, would it be possible to use the "Remote hg repo" feature so Rietveld will work?  I assume this is possible for devguide patches.

Regarding the FAQ, it seems preferable to me to keep all questions in the FAQ, even if non-committers might not need to know the answer.  It seems like it would be difficult to draw that line anyways, unless the question is about the Python commit process itself -- in which case it's probably not a generic Mercurial question anyways.  We also have the option of creating a new subsection of the FAQ if it makes sense to do so (e.g. a subsection about Committing).

> Since this patch is on the longer side, would it be possible to use 
> the "Remote hg repo" feature so Rietveld will work?

I was actually thinking about doing this for the devguide and for the peps repos, using the "components" to determine what repo should be used.  I'm not sure the remote hg repo supports non-cpython repos.

> Regarding the FAQ, it seems preferable to me to keep all questions in 
> the FAQ, even if non-committers might not need to know the answer.

What I was doing was converting the current prose in smaller FAQ-like sections, so that it's easier to navigate and it's in the same page where "committing" is explained.  Having all the "committers FAQs" here has the advantage that it's easier to search and that regular contributors can skip the whole page.  OTOH some committers might want to still look at the regular FAQs for answers to more generic Mercurial questions.

> I'm not sure the remote hg repo supports non-cpython repos.

Can you try?  I would be surprised if it didn't.

> What I was doing was converting the current prose in smaller FAQ-like sections

Okay, then it sounds like they're more like sections that fit into the natural flow of the main body of text.  That's fine with me.  I got the impression you were moving questions in the current FAQ.  It is good, though, to have the main instructions be smaller and easier to read through, with sections describing short-cuts and other non-essential info separated off into linkable chunks (in committers.rst or elsewhere).

> Can you try?  I would be surprised if it didn't.

IIRC it does a diff between the head of the cpython repo and the head of the linked one.

> Okay, then it sounds like they're more like sections that fit into
> the natural flow of the main body of text. 

Yes, the idea is that the first few FAQs describe the "regular" workflow in order (committing, merging, pushing), but than I was planning to mention a few other cases, e.g. null-merges or merge conflicts.  Some of these might already be in the FAQs.

FWIW I uploaded the rst file to make reviews easier.  I moved two sections that were not related to Mercurial to the top (Handling Others' Code and Contributor Licensing Agreements).  What I'm working on is the "Working with Mercurial" part.  Everything in there is either new or an adaptation of the previous text.  I also removed a couple of things (e.g. the differences between svnmerge and hg merge).

Sorry, my apologies for the mess-ups!

Okay, it looks like you can't do it.  It failed with a "repository is unrelated" error.

FTR I created a bitbucket clone of the devguide to make reviews easier:
Here you can find the rendered output: https://bitbucket.org/ezio_melotti/devguide/src/default/committing.rst?at=default
Here you can find the patches I applied and comment on them:  https://bitbucket.org/ezio_melotti/devguide/commits/all

I just talked to Ezio on IRC about this.  I think it will be a lot easier to review and see what's going on with these changes if they are put forth in smaller, bite-sized pieces and committed incrementally.  It's harder to understand and have a dialogue about a large set of changes all at once.

I gave him some suggestions for first commits in this direction and offered to help more with breaking things up into smaller pieces if he needs it.  I also have an interest in simplifying our presentation/organization and in adding more information about using Mercurial more effectively (e.g. issue 16930 and issue 16931).

I added more comments on bitbucket to indicate the new parts, the ones that have been moved, and the ones that have been removed.  That should make reviews easier.

I still haven't looked at the original FAQs (in faqs.rst), and still have to decide what should go where.  The general idea is that committers-specific FAQs should go in committing.rst, and the rest should go in faqs.rst.
Another option is to keep in committing.rst only the "4 fundamental FAQs" (i.e. "In what branches should I commit?", "In what order should I merge my commits?", "How do I merge between different branches (within the same major version)?", "How do I port a changeset between the two major Python versions (2.x and 3.x)?"), and move all the other stubs (including the one about null merges) on faqs.rst, possibly grouping them in two different sections: committers and contributors.

I think it could cause confusion to have FAQs spread across two different files because then we'll have to distinguish between two FAQs in our hyperlinks, and it won't be obvious which FAQ page contains which questions.

I would recommend creating a new section in our existing FAQ called something like "Using Mercurial with CPython," optionally subdividing that into a committer section and an "everyone" section.  The original section can be morphed into a section answering generic questions about Mercurial (not necessarily specific to CPython or the patch workflow).

If a question is important enough to be on the same page as the rest of committing.rst, it can be added as a normal section of that page (what is the need to give all of those sections FAQ-style titles?).

I used FAQ-style titles to make easier to find the solution starting from a problem ("how do I do X?") and because I was planning to add more, but I can revert them to normal titles.
So that would result in 4 sections:
 * Active branches (was "In what branches should I commit?");
 * Merging order (was "In what order should I merge my commits?");
 * Merging between different branches (within the same major version) (was "How do I merge between different branches (within the same major version)?");
 * Porting changesets between the two major Python versions (2.x and 3.x) (was "How do I port a changeset between the two major Python versions (2.x and 3.x)?");

At the end of committing.rst I can add a link to the "committers faqs" in faqs.rst, and add more FAQs there.  Everything about null merges, head merges, merge conflicts, long-term development branches etc. will go to faqs.rst.

The combination of the last two suggestions sounds good to me.

I made a separate repo to implement this and divided the previous commits in smaller ones: https://bitbucket.org/ezio_melotti/devguide-14468
If this is OK, I'll start regrouping the FAQs in faqs.rst and then move there the section about long-term development of features.

I now divided and regrouped the FAQs in two sections, one "for everyone" and one "for core developers".
I'm not sure what to do with the "long-term development of features" section.  One one hand I would prefer to move it away from committing.rst, but on the other hand it's a bit too long to be a single FAQ.  For now I left it there, but it could also be moved to the faq.rst page and possibly be broken in different subsections or separate FAQs.

I think what I have done so far can be committed, even though there are more improvements to do -- but those can be done later, so they shouldn't block this.  Please review the commits on bitbucket, and if they are OK I will collapse them and apply the resulting patch.

If there are no comments I'll commit this during the weekend.

I would like to request again that these changes be proposed and committed in smaller chunks.

I have comments of a basic nature that would be much easier to discuss and address in the context of smaller patches.  Otherwise, the discussion isn't manageable because too much is up in the air at once.  I was waiting for smaller patches before raising these comments.  Also, some of my suggestions above weren't addressed.

On the repo I created (https://bitbucket.org/ezio_melotti/devguide-14468) there are separate commits that include smaller changes.  The patches attached to this issue are outdated.

My understanding is that the commits are all or nothing.  In other words, they all need to be reviewed before any are committed.  I would like for the patches to be reviewed and committed individually so the review process is more manageable and people can participate more easily.  As I commented above, I offered suggestions on how to do this.

Also, I would be happy to start that process using the work that you have done.

Yes, I was planning to collapse them in a single patch and push only that.  I don't think they can be further divided.  In the moment I add the paragraph about setting up the multiple repos using "hg share" the old description becomes unnecessary, the instructions needs to be updated, and the FAQs don't need to mention hg share anymore.
I made all these things in separate commits to make it easy to review, but they have to be committed together (unless you prefer to have intermediate commits that contained duplicated informations).

I see ways that the changes can be proposed and committed incrementally.  I can start proposing those.

I'm not sure what's the point though.  The repo on bitbucket provides an easy way to review individual changes, and pushing everything at once prevents history pollution (assuming you consider 8 related changesets as pollution).  That's similar to the process we follow for feature branches, except that this is trivial enough that I didn't create a branch.

The point is to allow a manageable discussion to take place around points of contention while making forward commit progress along the way.  I have suggestions right now, but the current massive batch of changes makes it too unwieldy.  The incremental commits I'm proposing wouldn't be "pollutive."  Each would be forward progress towards what we want with a commit message explaining the change.

> The point is to allow a manageable discussion to take place around 
> points of contention

That can be done as (possibly inline) comments on bitbucket on the individual commits.

> while making forward commit progress along the way.

I'm not sure how you intend to do that, but feel free to propose what you have in mind.  As I see it, there's a bunch of text written assuming that there is a single clone or multiple not shared clones, and all this needs to be replaced and updated.  You can probably extract a couple of chunks, but that won't make a difference IMHO.

> the current massive batch of changes makes it too unwieldy.

Maybe you are overestimating it.  It's really not so massive: it's 8 changesets, 2 just remove stuff, other 2 just move existing content without adding/removing anything (so there's nothing much to review here).  Of the remaining 4 changesets, the first only adds 85 lines, the other 3 change about 20 lines each.

Let’s stay friends here :)  Why not remove the existing patches here, export the 8 changesets as patches, attach them here, let people comment?  (I’m not keen on having discussion outside of our system)

Because this is a patch for the devguide, so we cannot use Rietveld for the review.

I extracted the diffs and attached them to the issue.
The first 6 patches update committing.rst:
[1]: https://bitbucket.org/ezio_melotti/devguide-14468/commits/c2fca99bdb7212c4815d9fe6b0c869bb4358886a
[2]: https://bitbucket.org/ezio_melotti/devguide-14468/commits/37e287725636bb9c4b82ae864e38a97e8b332809
[3]: https://bitbucket.org/ezio_melotti/devguide-14468/commits/55055f30dd933c3f05be1581fd7c6cf2ec97c09c
[4]: https://bitbucket.org/ezio_melotti/devguide-14468/commits/389488fc431dca256b3534ca658a4e4f
[5]: https://bitbucket.org/ezio_melotti/devguide-14468/commits/8dc7283e53de97692a76b9f363a80303
[6]: https://bitbucket.org/ezio_melotti/devguide-14468/commits/3c5c77f2fa75d3c8bcb459d0c6060354

The last two patches update faqs.rst:
[7]: https://bitbucket.org/ezio_melotti/devguide-14468/commits/7a6f9b28a9d14e1e40988c76712a78a6a6014d28
[8]: https://bitbucket.org/ezio_melotti/devguide-14468/commits/5bf313aec51c326673a63206f0d7d0c7b40d6e86

I also updated two patches that includes the changes of patches 1-6 and 7-8.
I'm planning to commit these two last patches, but I can also make 8 separate commits or even a single one.

Thanks for waiting and for posting the patches here.  I think the second patch "2-move_two_sections.diff" should be committed now, along with making "Working with Mercurial" a higher-level header (as it is done in the aggregate patch).  This will separate the Mercurial-specific info from the non-Mercurial info in committing.rst, which is a good change in any case.

I will try to voice my high-level comments on the rest of the patches by this weekend, and/or suggest a second incremental change.

Looking at patch 1. On Windows, ~/.hgrc is $HOME$/mercurial.ini.
On my win7 machine, that translates to C:/Users/Terry/mercurial.ini.
I think it would be different on xp. But with TortoiseHG/Workbench, one should better use the Settings dialogs than edit directly.

I have [extensions]\nshare = set in that file but I don't think that is actually used as I created the clones with hg update rather than hg share. I will see what I have to do when I recreate the clones with hg share instead.

I could mention mercurial.ini too, but I don't think the devguide should cover tortoisehg.  All the commands should work fine on Windows too.

> I don't think that is actually used as I created the clones with
> hg update rather than hg share.

Do you you mean s/update/clone/?

For reasons I stated above, I think it will help to break this issue into smaller, self-contained parts as we go -- even if some of the issues turn out to be short-lived.

For example, an issue can be created for adding to the docs a section on how to set up multiple clones.  This way, we can separate the discussion and treatment of telling people how to use multiple clones from telling people how to set them up.  I think it will be simpler to address and commit the latter first.  The current "1-add_clones_setup.diff" patch mixes things by including info on applying, committing, and merging a patch under the section on setting things up.

> The current "1-add_clones_setup.diff" patch mixes things by including 
> info on applying, committing, and merging a patch under the section on 
> setting things up.

The goal of that section is to provide an overview of all the steps that a committer has to follow, starting from the beginning (getting a clone) till the end (the patch is applied on all the branches and pushed).
The "Clones setup" section is not about "how to set up a clone", but "how do I do these steps depending on the specific setup I'm using" (single or multiple clones).


> For example, an issue can be created for adding to the docs a section on how to set up multiple clones. 

This is not the goal of this issue.
How to create one/multiple clones is already covered in:
 1) http://docs.python.org/devguide/#quick-start (single clone)
 2) http://docs.python.org/devguide/setup.html#checkout (single clone)
 3) http://docs.python.org/devguide/committing.html#using-several-working-copies (multiple clones without share)
 4) http://docs.python.org/devguide/faq.html#i-want-to-keep-a-separate-working-copy-per-development-branch-is-it-possible (multiple clones without share)
 5) http://docs.python.org/devguide/faq.html#how-do-i-avoid-repeated-pulls-and-pushes-between-my-local-repositories (just a link to the share extension)


After all the patches are applied:
 * the quick start and the setup page will still explain the single-clone approach (intended for contributors, but valid also for developers);
 * the committing.rst page will only explain the multiple shared clones approach (for developers) and just link to 2) for the single-clone approach;
 * the faq will still explain the multiple non-shared clones approach in 4) as an alternative, but also suggest the shared version and link to committing.rst;

The reason why (almost) all the patches should be applied together is that once the multiple shared clones approach is explained as the default approach (patch 1), the instructions about the different branches (patch 3), merging between the same major version (patch 4) and between major versions (patch 5) must be updated.  At the same time, the section about using multiple unshared clones is no longer necessary so it has to be removed (patch 6).  Finally there's no reason to mention the share extension in the faq (patch 8).

Patch 2 could indeed committed separately, and patch 7 reflects the separation between single-clone/contributors and multiple-clones/developers by dividing the FAQs in two groups.

After this is done I was planning to update the "developers" FAQs, but this issue is currently blocking that.

> Do you you mean s/update/clone/? duh, yes
> I don't think the devguide should cover tortoisehg.

Given the obnoxiousness of Command Prompt, and how foreign it is to working on Windows, I think maybe there should be an addendum *somewhere*, but I don't expect anyone else to write it. (Goodness, there is a devguide chapter for emacs editing.) The easiest way to run direct hg commands is within the command pane of Workbench.

About  "run `make patchcheck`". The current Committing page gives
"(or ./python.exe Tools/scripts/patchcheck.py on Windows)" as the windows equivalent. './' does not work on Windows. '.\' will, but I do not know if it is needed. I presume the command should be given from within a particular repository. In its top directory, 'python' will run the installed python, 'PCbuild\python.exe' or 'PCbuild\python_d.exe' (backslash required here, though not in the patchcheck path) is needed to run the python built from that repository.

> The "Clones setup" section is not about "how to set up a clone", but "how do I do these steps depending on the specific setup I'm using" (single or multiple clones).

Then the section should be called something like "Cloning approaches" rather than "Clones setup."  Neverthless, I think it would be good to have a separate subsection dedicated just to the setting up portion of multiple clones with the share extension (with an appropriate title).

>> For example, an issue can be created for adding to the docs a section on how to set up multiple clones.
>
> This is not the goal of this issue.

My remarks were about setting up multiple clones with the share extension.  I was suggesting that a separate issue be created to address that aspect of the current issue.  It's okay to add information on how to set up multiple clones with the share extension without initially saying everything about how to use this approach (just as the share extension is already linked to without including such instructions).

As someone who previously tried but failed to setup shared clones, to avoid the apparently senseless copying of multiple copies of .hg/store, because of the inadequate disjointed doc, I like patch 1. I am using it now to redo my setup and batch file. I think how to set it up (10 lines) *should* be followed by how to use it. One thing that is missing though, which I think is or was in a current or previous version, is to put all the pull and update commands in a batch file. If the shared approach somehow mandates that they be interspersed with edit, merge, and commit, that would be a big negative.

> I think it would be good to have a separate subsection dedicated just 
> to the setting up portion of multiple clones with the share extension 

I'm not sure that's necessary though, given how simple it is (enable share extension, "hg share source dest", repeat).

The ideas are:
  1) have a straightforward from-beginning-to-end overview in committing.rst, followed by a few sections that explain more in detail the most common steps (e.g. how to merge between different versions, how to solve merge conflicts, etc.);
  2) list alternative approaches (e.g. multiple unshared clones) and "advanced" steps (e.g. deal with merge conflicts) in the FAQs;

If you think all the approaches should be listed somewhere, we can always add a new FAQ later, but it shouldn't block this issue (and it shouldn't be in committing.rst).

> One thing that is missing though, which I think is or was in a
> current or previous version, is to put all the pull and update
> commands in a batch file.

The point of the share extension is that is no longer necessary to pull/push changesets between clones (the updates still are though).

I meant one pull and multiple updates (instead of the multiple 'pull -u's I have now).

As discussed above and because this comment thread is getting very long, I'm going to start proposing smaller issues off of this one.  In this way we can start committing as we reach agreement, and hash out any disagreements in more focused contexts around smaller patches.  I will copy the nosy list unless I hear otherwise from anyone.

> Patch 2 could indeed committed separately,

I created issue 17284 for this.

> I'm going to start proposing smaller issues off of this one

I still fail to understand what are you trying to achieve.  If you want to further dissect my patches, hand picking chunks and reorder them in order to obtain even more patches, I'll just close this issue and let you do what you think it's best -- but I'm not going to spend more time on this.

Otherwise I will commit my patches as they are, either all together or one by one or anything in the middle -- you pick the granularity you like most.  After that it's done you can open other issues to further improve what we have.

I haven't really followed the discussion here, but both patches look ok to me (the committing.rst one, especially, is a welcome cleanup).

> I still fail to understand what are you trying to achieve.

My goal is to reach consensus on changes and have them committed.  In its current form, I don't agree with the patch.  The length of the comment thread and the length of the patch has discouraged me from raising most of my comments, because I'm fairly certain my comments will lead to back-and-forth discussion which will make the thread even longer.  That's why I want to break things up.

I want to be clear that I'm not against the goals of the patch, so I'm not trying to "block" or stalemate anything.   I just have concerns I would like to discuss which is what the review process is supposed to allow.  Needless to say, I know what it feels like to have a patch reviewed in detail (which I'm not necessarily planning to do).  On the plus side, it's much better than comments being against the patch outright in any form.

Perfect is the enemy of good. I vote for just committing it, then Chris can
propose additional fixes/changes in a subsequent patch.
On 26 Feb 2013 09:52, "Chris Jerdonek" <report@bugs.python.org> wrote:

>
> Chris Jerdonek added the comment:
>
> > I still fail to understand what are you trying to achieve.
>
> My goal is to reach consensus on changes and have them committed.  In its
> current form, I don't agree with the patch.  The length of the comment
> thread and the length of the patch has discouraged me from raising most of
> my comments, because I'm fairly certain my comments will lead to
> back-and-forth discussion which will make the thread even longer.  That's
> why I want to break things up.
>
> I want to be clear that I'm not against the goals of the patch, so I'm not
> trying to "block" or stalemate anything.   I just have concerns I would
> like to discuss which is what the review process is supposed to allow.
>  Needless to say, I know what it feels like to have a patch reviewed in
> detail (which I'm not necessarily planning to do).  On the plus side, it's
> much better than comments being against the patch outright in any form.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue14468>
> _______________________________________
>

> Perfect is the enemy of good. I vote for just committing it, then
> Chris can propose additional fixes/changes in a subsequent patch.

Agreed.

Chris, any preference about the number of commits (8 separate commits, 1-6 + 7-8, 2 on its own)?

For the record, I don't recall *any* changes being made to any of the patches in response to mine or others' comments, other than dividing them up.  So we're not talking about perfection.  If they're going to be committed as is, it might as well be one patch.  I hope there isn't a double standard when it comes to proposing subsequent changes.

ISTM, committing changes to the devguide is fundamentally different from committing a change to Python itself.  The devguide has a much smaller and focused audience, does not have compatibility considerations, it's continuously releasable etc etc.  So there's no need to have exactly the same stringent criteria for changes to the devguide.  That doesn't mean that there shouldn't be any review of those changes or that the process devolves to a change storm.  But, surely at this point, it would be easier to get meaningful additional review after the current set of changes are committed rather than continually redoing a large set of patches.  Then others are free to propose their own further changes for review.  I guess the lesson I would take from this is to limit the size and scope of individual changes to the devguide where possible, e.g. "ship" frequently.  This doesn't need to be difficult.

OK, I'll commit 1-6 and 7-8 then.  After that we can further improve things starting from there.

> it would be easier to get meaningful additional review after the 
> current set of changes are committed rather than continually redoing a 
> large set of patches.

And that's the reason why.  I'm willing to discuss and accept changes, but I would rather doing it after this is committed than having to modify existing patches (especially if the changes are minor).

> But, surely at this point, it would be easier to get meaningful additional review after the current set of changes are committed rather than continually redoing a large set of patches.

This was my reason for asking early on that the changes be proposed and committed individually, so that the whole set of patches doesn't have to be redone after each round of comments:

http://bugs.python.org/issue14468#msg179849

But the bulk of the discussion wound up being about this request rather than on the content of the patches themselves.  I've never had a problem breaking up my own issues/patches when asked.

For various reasons, there is a phenomenon that the larger the patch, the less relative scrutiny it tends to undergo (with the exception of PEP's where the process is different), which is the opposite of what it should be.  I'd like for us to try to avoid that.

This was a somewhat major rewrite.  The fact that we had no rietveld available made things more complicated on the reviewer side, and splitting the patch in smaller chunks made things more complicated on my side.

FWIW I have 2 major doc issues up next: #4153 and #14097.  Much like this issue, I'd have to go through the existing content and make several adjustments, that not necessarily stand on their own.  Having a single patch and update that based on the comments received on rietveld seems to work there though.

New changeset a50e537c5914 by Ezio Melotti in branch 'default':
#14468: document the use of the share extension as the suggested approach for core developers.
http://hg.python.org/devguide/rev/a50e537c5914

New changeset 3e213eaf85a6 by Ezio Melotti in branch 'default':
#14468: regroup the "Version control" FAQs in two sections: "for everyone" and "for committers".
http://hg.python.org/devguide/rev/3e213eaf85a6

New changeset ec43cf291255 by Ezio Melotti in branch 'default':
#14468: update FAQs about multiple clones and share extension.
http://hg.python.org/devguide/rev/ec43cf291255

While multiple, spaced, commits might have been better, if the first had been made at least 6 months ago, I agree that the best thing *now* was to commit these as a base for further improvements. There are 19 other open devguide issues to review and close as out-of-date, a bad idea, or a good idea with an improvement made. I think the core-mentorship list would be one place to get more opinions if more are needed.

"I think the core-mentorship list would be one place to get more opinions if more are needed."

Keep in mind that the core-mentorship list is a closed list so any discussions there would take place without direct participation of the (many?) core developers, like me, not currently on the list.  Soliciting opinions is one thing but, in general, I don't think it is a good idea to encourage side, non-public discussions of the development process.

I have no intention of excluding you ;-). Please comment on any or all of the other 19. But if an issue is stuck with inadequate or conflicting developer opinion, getting supplemental opinions from target users, as in "There is a proposal to change 'abc' to 'xyz'. Which is clearer to you?" might be one way to move forward.

I went through all the messages on this issue, and I'll address them here.  There's enough material for two new issues (advanced FAQs, and improvements about Windows docs), and a few minor issues that can be discussed and fixed as part of this issues before closing it.


Chris in msg178927:

> I would break up the 20 lines of command-line commands.

Those are supposed to provide a complete overview, the individual commands are explained with inline comments, and more in details in the following section.  Is that OK?

> I would also state (or link to) something about forward-porting from
> 3.x to 3.y and that 2.7 should be kept separate

This should be covered now.

> I would also say (or link to) something about pushing all branches
> simultaneously.

This is not stated explicitly, but the fact that there's a single "hg push" at end implies it.  It might be mentioned somewhere together with "hg out".

> Lastly, might it be worth explicitly dividing the Mercurial stuff into separate sections for (1) everyone, and (2) committers?

This is done, explicitly in the FAQs, and more implicitly in committing.rst.  Committing.rst is written mainly for developers, whereas other parts of the devguide are more generic and valid for everyone.


Chris in msg178983:

> If applying to 2.7 or 3.2, etc. loses information (which has been 
> more often the case for me), then instead of merging I null-merge and 
> reapply the original patch.

This could be a FAQ.


Ezio in msg179853:

> At the end of committing.rst I can add a link to the "committers 
> faqs" in faqs.rst, and add more FAQs there.  Everything about null
> merges, head merges, merge conflicts, long-term development branches 
> etc. will go to faqs.rst.

This still needs to be done, and should be covered in a separate "Add 'advanced' mercurial FAQs for committers." issue.


Terry in msg182625:

> On Windows, ~/.hgrc is $HOME$/mercurial.ini.

This could either be mentioned right there, or if there are multiple occurrences of it, we could add a FAQs that briefly explains what .hgrc is and where to find it on Linux and Windows and link to it.

> with TortoiseHG/Workbench, one should better use the Settings dialogs 
> than edit directly.

This could also be mentioned in the aforementioned FAQ.


Terry in msg182633:

> Given the obnoxiousness of Command Prompt, and how foreign it is to working on > Windows, I think maybe there should be an addendum *somewhere*, 
> About  "run `make patchcheck`". The current Committing page gives
> "(or ./python.exe Tools/scripts/patchcheck.py on Windows)" as the windows
> equivalent. './' does not work on Windows. '.\' will, [...]

I suggest to create a new issue like "Improve Windows instructions in the devguide" and mention all these issues.

Ezio, did you delete the section on null-merging in your commits?  I don't see it in the devguide anymore.

Yes, it should be added back in the FAQs.

New changeset 50e726533391 by Ezio Melotti in branch 'default':
#14468: move a paragraph and link to the list of branches.
http://hg.python.org/devguide/rev/50e726533391

The attached patch adds 3 new FAQs:
 * How do I solve merge conflicts?
 * How do I make a null merge?
 * I got "abort: push creates new remote heads!" while pushing, what do I do?

It also replaces the overly generic "How do I find out which revisions need merging?".
The content of "How do I list the files in conflict after a merge?" and "How I mark a file resolved after I have resolved merge conflicts?" has been summarized in "How do I solve merge conflicts?" too, so I removed them.

For the null merge entry, /filed/files/.

The "create new remote heads". is really needed. I handled a situation wrong again today. Question: commit to 3.2, merge forward without change, push, and message is '... new remote head in 3.2'. Is 3.2 really the only branch with a head conflict? Or would one appear in 3.3 as soon as 3.2 heads are merged and merge is merged forward to 3.3?

> Question: commit to 3.2, merge forward without change, push, and message is '... new
> remote head in 3.2'. Is 3.2 really the only branch with a head conflict?

I don't remember the details of the error message, but you can use "hg heads" to verify that.  In general this shouldn't happen, because if someone committed something on 3.2, he should have also merged and pushed it on 3.3/default[0].  It could happen if you forget to update your 3.2 clone but you have updated 3.3 and default clones.

> Or would one appear in 3.3 as soon as 3.2 heads are merged and merge
> is merged forward to 3.3?

Heads on a branch don't appear magically.  The only two ways to create another head on a branch are:
  1) make a commit on a branch that is not updated;
  2) pull a changeset after you made a commit;

If the branch is updated, a new commit will replace the previous head, which will be its parent.

To answer your question, assume you have one head on default, one head on 3.3, and two heads on 3.2.
The first thing you have to do is merge the heads of 3.2, so that you end up with one head per branch.  This created a new changesets on 3.2, but 3.3 and default are unaffected.
This new changeset -- like all the other changesets on 3.2 -- must be merged with 3.3/default, so you merge branches as you would do with any other changeset you committed on 3.2.

In the worst case scenario, you just made commits on all the four branches, and someone else did the same and pushed before you.  Your push fails because it creates new heads.
You pull the new changesets and end up with 2 heads for each branch.
To solve this you have to do:
 1) 2.7$ hg merge (merge heads on 2.7);
 2) 3.2$ hg merge (merge heads on 3.2);
 3) 3.3$ hg merge (merge heads on 3.3);
 4) 3.x$ hg merge (merge heads on default);
 5) 3.3$ hg merge 3.2 (merge 3.2 with 3.3);
 6) 3.x$ hg merge 3.3 (merge 3.3 with default);

I'm not sure this answers your questions -- I tried to explain it using different approaches, but the end result wasn't much clearer, so I discarded them.  The concepts themselves are not particularly complicated or difficult to understand, however they are not really easy to explain either :)
Using "hg glog" or http://hg.python.org/cpython/graph/ (or the one integrated in TortoiseHG) should help.

[0] note that this is not currently enforced. #15917 has a patch to fix this.

I now understand what I should have done. After pulling and updating,

1. Make sure there is only 1 head per branch by using 'hg heads <branch>*. Merge all but only pairs revealed by that command.

* 'hg heads' without giving a branch lists all heads in all branches, including 2.6 and 3.1, making it harder to find pairs in one branch. This is especially true in the Workbench command window which has room to see only one head at a time. So I recommend you at least mention the option of adding the branch (with 3.4 being 'default').

Comment: while 2 heads for 2.7 or default are visibly obvious in the dag (at least they were the one time I had such), 2 heads for 3.2 and 3.3 seems not to be, at least when merged forwards. As far as I know, the only way to tell which nodes hg considers to be heads (in particular, which node is the already pushed head) is to use the 'heads' command.

(My big mistake was to *assume* -- without checking 'heads' -- that the 3.2 heads, when merged forward, must have become 2 heads in 3.3 ...)

2. Merge forward as needed. I think a version of your 6 step display would be helpful. It was for me.

The attached patch fixes the typo and mentions ``hg heads <branch>``.

> I think a version of your 6 step display would be helpful. It was for me.

The FAQ already describes the general approach (merge heads in each branch and then merge branches as usual).  The worst-case scenario that those 6 steps address is something that doesn't really happen often (and it's also a bit intimidating), so I preferred to leave it out and just describe a more common example.

New changeset 7a707540391e by Ezio Melotti in branch 'default':
#14468: add FAQs about merge conflicts, null merges, and heads merges.
http://hg.python.org/devguide/rev/7a707540391e

I created #17570 about the Windows-related improvements, so this can finally be closed.