Before the next version is released, I'd like to push this one line modification to reduce the risk of sniffing Python development password when people upload packages to PyPI by using https:// communication channel by default.

Distutils doesn't validate PyPI server certificate, so this change doesn't prevent from MITM attacks, but at least it makes package submissions over wireless channels and public networks safer.

Taking into account that people still release packages for Python 2.5+ (AppEngine), I'd like to see this fix backported to at least Python 2.6

This should probably be discussed on catalog-SIG, not the CPython bug tracker.

Oops, nevermind that, thought this was suggesting a change to PyPI itself, not distutils.

The distutils version that exposes this vulnerability is shipped with Python, that's why it is here.

Have you tried registering/uploading stuff through https ? if it's working as is I am +1 for this change in the next 2.7.x. 

For 2.6 don't know if it's worth a security patch. Martin, Barry, any opinion ?

Yes. I've just uploaded http://pypi.python.org/pypi/review/r749 through https://pypi.python.org/pypi

I'm +0 on changing 2.6. Many people use setuptools or distribute, so the impact of this change was higher if those were modified instead of modifying 2.6 - in particular since few people are expected to pick up these security releases, anyway.

> Distutils doesn't validate PyPI server certificate, so this change
> doesn't prevent from MITM attacks, but at least it makes package
> submissions over wireless channels and public networks safer.

Is that so? It's been a while, but I think e.g. ettercap is a highly
automated tool for MITM attacks that isn't very hard to use.

Given that 2.6.7 is rc2 with a final release scheduled in 2 days, I don't want to apply this to 2.6 right now.  Can you guarantee this won't regress for anybody?  If so, then I'm also +0 for 2.6 after the 2.6.7 release.

On Wed, Jun 1, 2011 at 10:30 AM, Stefan Krah <report@bugs.python.org> wrote:
>
>> Distutils doesn't validate PyPI server certificate, so this change
>> doesn't prevent from MITM attacks, but at least it makes package
>> submissions over wireless channels and public networks safer.
>
> Is that so? It's been a while, but I think e.g. ettercap is a highly
> automated tool for MITM attacks that isn't very hard to use.

This patch won't help against properly baited ettercap, but will
prevent transit sniffing of weakly protected passwords.
--
anatoly t.

On Wed, Jun 1, 2011 at 2:06 PM, Barry A. Warsaw <report@bugs.python.org> wrote:
>
> Barry A. Warsaw <barry@python.org> added the comment:
>
> Given that 2.6.7 is rc2 with a final release scheduled in 2 days, I don't want to apply this to 2.6 right now.  Can you guarantee this won't regress for anybody?  If so, then I'm also +0 for 2.6 after the 2.6.7 release.

Adding catalog-sig to CC. I can guarantee this for Windows. I'll be
near Linux box tomorrow and will try upload to PyPI from there. It
still will be more authoritative if more than one person can test
upload to PyPI with this patch on different systems.
--
anatoly t.

On Wed, Jun 1, 2011 at 11:14 AM, anatoly techtonik <techtonik@gmail.com> wrote:
> Adding catalog-sig to CC. I can guarantee this for Windows. I'll be
> near Linux box tomorrow and will try upload to PyPI from there. It
> still will be more authoritative if more than one person can test
> upload to PyPI with this patch on different systems.

The interesting case will be for a build that doesn't include SSL support.

  -Fred

On Wed, Jun 1, 2011 at 5:17 PM, Fred Drake <fdrake@acm.org> wrote:
> On Wed, Jun 1, 2011 at 11:14 AM, anatoly techtonik <techtonik@gmail.com> wrote:
>> Adding catalog-sig to CC. I can guarantee this for Windows. I'll be
>> near Linux box tomorrow and will try upload to PyPI from there. It
>> still will be more authoritative if more than one person can test
>> upload to PyPI with this patch on different systems.
>
> The interesting case will be for a build that doesn't include SSL support.

Yeah.. We do have in packaging a way to test against a PyPI server
that gets launched on a real socket, so what we could do is:

1/ add a test that tries to upload and register via ssh
2/ make sure it fallbacks to http if _ssl is not found

>
>
>  -Fred
>
> --
> Fred L. Drake, Jr.    <fdrake at acm.org>
> "Give me the luxuries of life and I will willingly do without the necessities."
>    --Frank Lloyd Wright
> _______________________________________________
> Catalog-SIG mailing list
> Catalog-SIG@python.org
> http://mail.python.org/mailman/listinfo/catalog-sig
>

Patch needs to update the default server in packaging too.

I think there should be a warning that the connection is unauthenticated
(i.e. not secure). Users tend to be upset if they see 'https' and later
find out that no certificates were verified.


A reasonably secure alternative is to publish the pypi server
certificate in a couple of places (python-dev, www.python.org).
Then the user can import the certificate into the browser while
on a trusted connection and henceforth do all uploading etc.
via the browser.

If users use a browser to do secure uploading, there is no need to publish the certificate. It is signed by a trusted CA (cacert), so you just need to make sure your browser knows about the cacert certificate.

Sorry for the delay. I've just uploaded http://pypi.python.org/pypi/ctypesgen/0.r125 from Ubuntu using python2.6 with patched distutils module to https://pypi.python.org/pypi and can confirm it works without problems on Linux.

So I can guarantee there won't be any regress in 99.9% of cases and <0.1% of cases that are left, risk of breaking these flaky setups doesn't outweight the need to close this security hole.

I can fill separate bug for packaging, server certificate validation and support for non-SSL builds of python (for which there is workaround with upload -r <server> option that I've just discovered), but it is an additional delay and the only reason I invested some time into this issue is because I saw RC announcements.


P.S. To Martin. CAcert is not trusted authority on Windows, and all browsers warn about it, so we need to distribute its root certificate with Python if we want to validate ours.

> I can fill separate bug for packaging, server certificate validation
> and support for non-SSL builds of python (for which there is
> workaround with upload -r <server> option that I've just discovered),
> but it is an additional delay and the only reason I invested some
> time into this issue is because I saw RC announcements.

This effort was wasted - there is zero chance that this can go into
a release candidate. Only fundamental flaws (such as being unable to
compile Python on a major platform, or serious regressions) should be
considered for changes between a rc and the final release.

I believe that's a very personal judgement. For me exposing core Python development accounts is a fundamental flaw. The more accounts are collected, the more real are attacks through PyPI package injection.

> I think there should be a warning that the connection is
> unauthenticated (i.e. not secure). Users tend to be upset if they see
> 'https' and later find out that no certificates were verified.

Thanks Stephan, that was on my mind but I forgot it.  I’m -1 on using https if no validation is performed.

> I believe that's a very personal judgement.

Not really; it’s an explanation of our release rules, exposed by one of the older developers.

> For me exposing core Python development accounts is a fundamental
> flaw.

What is a core Python development account?

On Sat, Jun 4, 2011 at 5:33 PM, Éric Araujo <report@bugs.python.org> wrote:>
>> I think there should be a warning that the connection is
>> unauthenticated (i.e. not secure). Users tend to be upset if they see
>> 'https' and later find out that no certificates were verified.
>
> Thanks Stephan, that was on my mind but I forgot it.  I’m -1 on using https if no validation is performed.

It will be more professional if you could also explain why. Thanks.

>> I believe that's a very personal judgement.
> Not really; it’s an explanation of our release rules, exposed by one of the older developers.

Release rules should be clear enough not to require explanation.

>> For me exposing core Python development accounts is a fundamental
>> flaw.

> What is a core Python development account?

'core' is not the best word here, so it needs an explanation. Any
account on PyPI that uploads packages used for in enterprise
deployment schemes imposes a danger. Potential target are identified
using 'popularity package/developer activity' rating to reduce the
risk. These are the primary targets for an attack, which I called
'core'. 'primary' would be a better name probably.

>> Thanks Stephan, that was on my mind but I forgot it.  I’m -1 on
>> using https if no validation is performed.
> It will be more professional if you could also explain why.

If you make an HTTPS connection without checking the certificate, what security does it add?

> > Not really; it’s an explanation of our release rules, exposed by
>> one of the older developers.
> Release rules should be clear enough not to require explanation.

Explanations make them clear.

> Any account on PyPI that uploads packages used for in enterprise
> deployment schemes imposes a danger.

Sidenote: I don’t want to give less security to non-enterprise users.

Anyway, I understand your point now: insecure upload and download are vulnerable to MITM attacks, and encouraging HTTPS use (through default value + docs) would help against that.  I am supportive of a patch, but it doesn’t mean the release process should not be followed.  See also #11357 and #8561 about download security.

On Mon, Jun 6, 2011 at 11:50 AM, Éric Araujo <report@bugs.python.org> wrote:
> If you make an HTTPS connection without checking the certificate, what security does it add?

I'm in favor of cert checking, myself.

> If you make an HTTPS connection without checking the certificate, what 
> security does it add?

Well, it does prevent the most trivial class of attacks (sniffing).
That said, Python has support for certificate checking, especially in 3.2+, so you should use that. You could e.g. bundle the CACert root certificate with the distribution.

This simple patch slipped off 2.7.2. Why?

Because it’s not finished.

What is left?

Certificate checking.

That's the issue12358.

CVE-2013-1754  Man-in-the-middle vulnerability in package upload feature of Python's distutils

Please notice that a redesign of PyPI and package security is ongoing in catalog-sig.

New changeset f86d46a580d8 by Benjamin Peterson in branch 'default':
use the HTTPS for pypi upload
http://hg.python.org/cpython/rev/f86d46a580d8

Benjamin, you committed a change to use HTTPS instead of HTTP.  In this bug report, we were having a discussion about the false/incomplete security that this provides if there is no certificate checking.  What are your thoughts on that?

This is true, but if we get proper certificate checking, this should automatically work correctly then.

I’m not sure what “this” refers to (in “This is true” and “this should automatically work correctly”).

My only concern is to avoid giving a false sense of security, so my initial stance was all-or-nothing.  However with the recent trend of incremental improvements to the PyPI ecosystem, I think it’s important to do what we can and keep the momentum, so I’m okay with the commit—I just wanted to make sure that committing half a fix was intentional.  You probably know more about SSL than me and you’re the RM, so let’s ship this. :)

By "this", I meant the change I made. It was made in consultation with Richard Jones (added to nosy) at the PyCon sprints.

Using HTTPS without a Certificate prevents passive attacks but not active attacks. It puts things in a _better_ situation but not the ideal situation.

This should have been backported to Python 2. I expect some related attacks on EuroPython.

I would +! backporting this, but It's not massively required since it only protects against passive attacks.

It would however make things a little nicer.

If somebody sponsor my visit to EuroPython, I will dedicate some time to prepare a demo uploading rogue packages using sniffed credentials over WiFi without owner's consent. After moving to CDN no upload logs are available, so it is even more secure for attacker to do this stuff.

Uploading always hits the backend servers and thus has the same logging as before

Merely switching to HTTPS only provides protections against passive attacks. You need verification to protect against active attacks (which are simple and easy to do as well). Like I said, not a bad move and I'd be in agreement on doing it but if the powers that be decide it's too big of a change it's not going to massively decrease the security.

How come that this CVE is still present in just released 2.7.6?

Donald assesses that porting the changeset to 2.7 would “make things a little nicer”, as it protects from passive attacks only.  The change is small.  What do people think?

Well, passive attacks are the easiest to mount by a casual attacker, so I think this is important to get in.

How about:

- load ca cert from default verify locations
- try connect with CERT_REQUIRED
- print warning when cert validation fails and try again with CERT_NONE
- match hostname otherwise

At least this warns the user about the issue. Is there way to distinguish between CA missing and other failures?

Antoine Pitrou <report@bugs.python.org> schrieb:
>
>Antoine Pitrou added the comment:
>
>Well, passive attacks are the easiest to mount by a casual attacker, so
>I think this is important to get in.
>
>----------
>
>_______________________________________
>Python tracker <report@bugs.python.org>
><http://bugs.python.org/issue12226>
>_______________________________________

New changeset 32a39ec6bd75 by Antoine Pitrou in branch '2.7':
Issue #12226: HTTPS is now used by default when connecting to PyPI.
http://hg.python.org/cpython/rev/32a39ec6bd75

New changeset 2b5cd6d4d149 by Antoine Pitrou in branch '3.2':
Issue #12226: HTTPS is now used by default when connecting to PyPI.
http://hg.python.org/cpython/rev/2b5cd6d4d149

New changeset e5a9755c967c by Antoine Pitrou in branch '3.3':
Issue #12226: HTTPS is now used by default when connecting to PyPI.
http://hg.python.org/cpython/rev/e5a9755c967c

New changeset 9839aa0e5b28 by Antoine Pitrou in branch 'default':
Null merge (#12226 already fixed on default)
http://hg.python.org/cpython/rev/9839aa0e5b28

Closing as fixed, and opening a new issue for cert checking.