Under guidance from zware during Pycon sprints, I've changed the Doc/ Makefile to automatically create a virtual environment and install Sphinx, all as part of the `make html` command.

I messed up the PR through a failed rebase (trying to rebase my PR on top of upstream). I closed the PR as a result.  I have now fixed up my feature branch, but I have not resubmitted the PR.  Since the PR was left alone for many months, I'm ok with leaving things as is, and close this issue?

You should be able to force-push your branch (`git push -f origin auto-venv-docbuilder`, or replace `origin` with the correct remote name) to fix the existing PR.

Sorry I haven't gotten back to this previously; time to do a review and remembering that the PR exists have not coincided.

No worries. I've made a new PR 4346. The old one was unsalvagable I'm afraid. Too many other people got added to the notifications list as a result of my incorrect rebase.  The new one is fine.

New changeset d8d6b9122134f040cd5a4f15f40f6c9e3386db4d by Zachary Ware (Caleb Hattingh) in branch 'master':
bpo-30487: automatically create a venv and install Sphinx when running make (GH-4346)
https://github.com/python/cpython/commit/d8d6b9122134f040cd5a4f15f40f6c9e3386db4d

I don't think this is a good idea. It has already caused problems with one buildbot (Issue32149) and will likely break other build scripts.  As the Doc Makefile stood previous to this commit, the Doc builds could take advantage of either a system-installed or user-supplied Sphinx and blurb without having to use the venv step.  Unless there are major objections, I am going to at least temporarily revert it.

Hi Ned

It's still supposed to allow both. It sounds like it's not working properly. I'll have a look. FYI, I worked on this for Zach Ware who is the primary stakeholder for this feature. 

Rgds
Caleb

> On 28 Nov 2017, at 7:12 AM, Ned Deily <report@bugs.python.org> wrote:
> 
> 
> Ned Deily <nad@python.org> added the comment:
> 
> I don't think this is a good idea. It has already caused problems with one buildbot (Issue32149) and will likely break other build scripts.  As the Doc Makefile stood previous to this commit, the Doc builds could take advantage of either a system-installed or user-supplied Sphinx and blurb without having to use the venv step.  Unless there are major objections, I am going to at least temporarily revert it.
> 
> ----------
> nosy: +ned.deily
> resolution: fixed -> 
> stage: resolved -> needs patch
> status: closed -> open
> 
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue30487>
> _______________________________________

One problem is that the venv can't always be automatically built in all environments, as a recent Python 3 needs to be available in the right location.

Go ahead with the revert, Ned.

It looks like the check for an existing sphinx-build passes, and so no new venv is made, but this also means that blurb doesn't get installed.  I was concerned about this, but figured that at least the buildbots would create new envs each time, and this would only be an issue that a user with an odd configuration might have.

Sorry for the trouble.

The feature was simpler when it was only sphinx.  Now that blurb is there too, the logic for checking what is and isn't already present becomes a bit complex to reason through.

"Now that blurb is there too, the logic for checking what is and isn't already present becomes a bit complex to reason through."

Yeah, it is a bit complicated.  There's also the issue of trying to use a make recipe to ensure a "venv" exists and is up-to-date.  I don't necessarily want to abandon the idea (and sorry I wasn't around to review the proposed change initially) but let's at least hold off until 370a3 is out the door.

Yep, sounds good.

New changeset 122fc136b34e11906466851e77bb6959946467ee by Ned Deily in branch 'master':
Revert "bpo-30487: automatically create a venv and install Sphinx when running make (GH-4346)" (#4592)
https://github.com/python/cpython/commit/122fc136b34e11906466851e77bb6959946467ee

I believe this can be closed as resolved?  Thanks.

@cheryl.sabella I am ok with closing this, but the original motivation for this work was from @zack.ware so he should weigh in.

I am not going to work on this any further for the forseeable future (I've got my hands full already with the asyncio docs I'm trying to write in #34831).