I've already updated system setuptools 36.0.1 with `pip3 install --upgrade pip setuptools wheel` but when I create a new enviro with `python3 -m venv foo` it will have an old/outdated setuptools 28.8.0 from embedded ensurepip.

Why? I think it should install the already-updated one!

$ pip3.6 list
pip (9.0.1)
setuptools (32.2.0)
wheel (0.29.0)

$ pip3.6 install --upgrade pip setuptools wheel
$ pip3.6 list
pip (9.0.1)
setuptools (36.0.1)
wheel (0.29.0)

$ python3 -m venv foo
$ source foo/bin/activate
(foo) $ pip list
pip (9.0.1)
setuptools (28.8.0)

You're certainly right. venv uses ensurepip to install pip when creating a virtual environment and does not access the internet/upgrade any packages. ensurepip was specifically designed to use the bundled version of setuptools. The bundled version of setuptools was up to date as of the final Python 3.6.0 beta, though it seems like it could be due for an upgrade.

For now, you can either run `pip install --upgrade setuptools` after activating your environment (recommended) or give your environment access to your local site packages that has the upgraded setuptools using `python3 -m venv foo --system-site-packages` (not recommended, as this will include ALL global packages you have installed). Alternatively, virtualenv (https://virtualenv.pypa.io/en/stable/) maintains an updated version of setuptools more frequently, though it must be installed as an additional package.

It should also be noted that wheel is not installed by venv and must also be installed after the environment creation if needed.

Indeed, Emily is correct: this is expected behaviour for system Python versions that don't patch their ensurepip modules, as the default pip (et al) used in virtual environments created with the venv module generally only gets upgraded when upgrading to a new maintenance release of CPython.

Some redistributors do patch the ensurepip module to seed virtual environments based on the installed system level packages, so if your redistributor is one that already does so (e.g. Fedora), then you may have a downstream bug to file, and if they don't, then you may have a downstream feature request to ask them to switch to that model rather than using the files bundled by upstream.

In the meantime, the two main approaches folks adopt to make sure that they always have the latest versions of these components installed by default are:

- using the latest version of the third party `virtualenv` module for venv management (as that gets updated independently of the CPython maintenance release cycle)
- running "python -m pip install --upgrade pip setuptools wheel" immediately after creating each virtual environment

It's not obvious from your discussion, Nick, *why* venv won't use an upgraded system pip if it has been manually upgraded.  There's no need for internet access in that case (which is the argument for using the bundled pip when running ensurepip for the first time).  For that matter, there's no obvious relationship between venv and ensurepip, other than ensurepip making sure there is at least one version of pip for venv to use.

So I agree with the OP that this behavior is quite surprising.  As well as annoying.  Perhaps there is a good reason for it that I'm not seeing?

The upstream supported install path for bootstrapping the venv is from a wheel file. The CPython source tree contains the necessary wheels, and, by default, upgrading the system pip doesn't touch them.

The only distro I can speak authoritatively for when it comes to doing things differently is Fedora, and what we do is:

- add a circular dependency between the system Python & the system pip
- add a circular dependency between the system Python & Fedora's "rewheel" utility
- patch CPython to remove the bundled wheel files
- patch ensurepip to "rewheel" the system pip and install that into freshly bootstrapped virtual environments

The end result works, but it really isn't pretty from a distro maintenance perspective, so not every distro is going to want to do it (Fedora has the luxury of Red Hat's Python maintenance team also maintaining the Fedora system Python).

It also isn't currently practical for upstream to make it work that way by default due to:

* the implied external dependency on the system pip (which not every distro is going to have)
* the need to recreate a wheel from an already installed package (which isn't officially supported by distutils-sig as a valid thing to do)
* the fact some distro's unbundle pip's vendored dependencies, so rewheeling just pip isn't necessarily sufficient (it is on Fedora, but that's only because we leave the vendored dependencies in place and trust pip to issue new updates promptly when necessary)
* the fact that not every distro lays down the distutils-metadata needed to get tools like rewheel to work (Debian's equivalent, dirtbike, uses deb metadata instead)

OK, so the key here is venv's dependency on a pip wheel.  That makes sense, but is certainly counterintuitive.  One would naively think that having a copy of PIP installed would make it easy to link that into the venv, but obviously it is a lot more complex than that, given pip's dependencies.  Thanks for the explanation.

Emily's answer sort of covered the "It's not as simple as it might seem" aspect, whereby the bit that's genuinely tricky is to provide access to the system pip(/setuptools/wheel) without providing accessing to the system site packages in general.

If the latter is actually considered desirable, then the most suitable venv configuration is likely to be:

    python3 -m venv --system-site-packages --without-pip

as that will give you a fresh venv to install your own additions into, but use the system pip package by default, rather than automatically bootstrapping a venv specific instance of it.

We don't currently discuss that kind of hybrid configuration in the docs, so it may be appropriate to reopen this as a docs issue, and add a new "How do I use system level Python packages from a virtual environment?" question to https://docs.python.org/3/installing/#how-do-i

A quick note on how to include system packages in the "How Do I..?" section could be helpful, though does not necessarily solve the current issue. I would be hesitant to instruct people to use the system-site-packages flag simply to have an updated version of setuptools, even with a strong word of caution. From my experience, most people generally don't have a clean pip environment outside of a virtual environment for various reasons and this could cause more confusion and undesired outcomes by encouraging use use of the system-site-packages flag.

As a side note, I do feel that the existing documentation for the system-site-packages and without-pip flags is adequate as-is (that was one of the first things I tried when debugging this issue).

Nick, could you answer a couple of questions for me? 
1) I've read that (for Python 3.5 or greater) that venv is recommended over virtualenv. Why? (Not challenging anything, just trying to understand from the perspective of someone who has used virtualenv for years and never looked back).
2) Could there be an additional argument for venv that triggers an upgrade to setuptools on virtual environment creation? Alternatively, I've looked into extending venv.EnvBuilder. We could add an example of extending this to bootstrap environment building to include a setuptools upgrade on creation for those who deem this important.

diff --git a/Lib/venv/__init__.py b/Lib/venv/__init__.py
index 716129d139..757e8de4e7 100644
--- a/Lib/venv/__init__.py
+++ b/Lib/venv/__init__.py
@@ -243,6 +243,9 @@ class EnvBuilder:
         cmd = [context.env_exe, '-Im', 'ensurepip', '--upgrade',
                                                     '--default-pip']
         subprocess.check_output(cmd, stderr=subprocess.STDOUT)
+        cmd= [context.env_exe, '-Im', 'pip', 'install', '--upgrade',
+                                'pip', 'setuptools']
+        subprocess.check_output(cmd, stderr=subprocess.STDOUT)
 
     def setup_scripts(self, context):
         """

Crude, but effective. Food for thought?

Curtis: by design, ``ensurepip`` isn't permitted to assume internet access (since we want both it and venv to work without warnings or errors on systems with no internet access). This was covered in the original PEP: https://www.python.org/dev/peps/pep-0453/#implementation-strategy

The default presence of `setuptools` and `wheel` in `venv` created virtual environments at all is also an implementation detail that's likely to change once PEP 517 is implemented (again, as per the original design in PEP 453).

Emily: for 3.5+, venv is recommended just because it avoids the extra 3rd party dependency, and integrates slightly more nicely with the interpreter internals. As soon as you need greater assurances of cross-version consistency though, virtualenv is the better choice. (Donald Stufft actually has a virtualenv refactoring pending that makes it just a thin shell around venv on 3.5+, but pursuing that's on hold for now due to other priorities)

For the auto-upgrade aspect, environment managers like `pew` offer more flexibility to automate that kind of thing in a way that's consistent across Python versions. It *might* make sense to offer a `--upgrade-pip` flag for `venv` creation, but it's not entirely clear how much benefit that really offers over just running `python -m pip install --upgrade pip` inside the venv yourself. That one's probably worth filing as a separate RFE though, as it's probably the single most common post-creation script for new virtual environments.