This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: docs.python.org is prone to political blocking in Russia
Type: behavior Stage:
Components: Documentation Versions:
process
Status: closed Resolution: third party
Dependencies: Superseder:
Assigned To: docs@python Nosy List: Alexander.Patrakov, christian.heimes, docs@python, dstufft, ezio.melotti, georg.brandl, jcea, lemburg
Priority: normal Keywords:

Created on 2014-08-18 10:57 by Alexander.Patrakov, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (15)
msg225487 - (view) Author: Alexander Patrakov (Alexander.Patrakov) Date: 2014-08-18 10:57
I could not access http://docs.python.org/ from work today. Upon investigation, it appears that the ISP has blocked all sites on the IP address 185.31.17.175, because of the court order unrelated to docs.python.org. Many other ISPs in Russia also don't know any methods of site blocking except by IP address.

Even if the bad site removes the offending materials, the situation is doomed to repeat itself in the future.

To avoid this, please obtain an IPv4 or IPv6 address for docs.python.org that is not shared with any site not controlled by PSF.
msg225488 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2014-08-18 11:58
Dmitry Chestnykh has pointed me to https://antizapret.info/site.php?id=5903
msg225491 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2014-08-18 12:48
Just a FYI, I've let Fastly know about this.
msg225492 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-08-18 13:43
Not much more we can do from here.
msg225495 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2014-08-18 14:56
It looks like the IP address is being used by some viruses/trojans:

https://www.virustotal.com/en/ip-address/185.31.17.175/information/

It may help using e.g. b.global-ssl.fastly.net as CNAME for docs.python.org (e.g. by adding it to the /etc/hosts).
msg225556 - (view) Author: Alexander Patrakov (Alexander.Patrakov) Date: 2014-08-20 07:17
The site is now accessible. But this case is going to repeat itself.
msg225557 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-08-20 07:28
We know, but this will happen to any sites that have content hosted by a CDN such as Fastly.

In this specific case, you can download the docs or build them yourself for offline usage.  Our Mercurial server hg.python.org is (obviously :) not hosted on a CDN, so this scenario can't happen there.
msg225559 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2014-08-20 07:39
See also #21072.
msg225560 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2014-08-20 08:07
On 20.08.2014 09:28, Georg Brandl wrote:
> 
> We know, but this will happen to any sites that have content hosted by a CDN such as Fastly.

I think we should have additional fallback domains setup
that go to frontend.python.org and then also get mapped to
the right backend server in order to be able to easily
work around this.

This would also help with some other issues such as Fastly
cache invalidation not working properly (which most likely is due
to the website not providing the right information for this work
rather than Fastly's fault).
msg225571 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2014-08-20 14:38
> I think we should have additional fallback domains setup
> that go to frontend.python.org and then also get mapped to
> the right backend server in order to be able to easily
> work around this.

I'm not sure it's worth it tbh. It's certainly going to be error prone to store
the frontend configuration in two locations (Fastly and front.python.org).
There might be additional things we can do on the Fastly side of things, I know
the have the ability to have dedicated IP addresses in all their DCs (typically
this is in use for providing EV SSL certs and the like). We don't have that
and generally this costs like $1500/month for the SSL cert stuff. I can ping
them about it and see if they have any other ideas.

> This would also help with some other issues such as Fastly
> cache invalidation not working properly (which most likely is due
> to the website not providing the right information for this work
> rather than Fastly's fault).

Right, it's basically that there is no cache invalidation as far as I know.
Actually invalidating the cache pretty much always works in my experience.
msg225575 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2014-08-20 17:14
I've heard back from Fastly!

Specific to this particular incident, they've identified a few places where their own internal procedures fell short and they've rectified them. Specifically:

1. Their ticketing software saw the notifications from the Russian government/ISP as spam, most likely due to the Cryllic character set. This lead them to miss seeing the reports initially until later. They've resolved this by whitelisting those domains.
2. They notified the customers and then told the Russian government/ISP that the owner of the content as been notified. Instead they are going to ensure that the content in the future.

In the long term they are evaluating their own policies for how they host customer sites which allow user uploaded content (since those types of sites are the most likely to have these kinds of issues) and determining if it makes sense for them to require dedicated IP addresses for those customers.

For now I think Fastly has sufficiently handled the issue to not require some sort of backup system to need to be put in place. They are going to let me know how they are going to handle it long term and what, if any changes, we can make in our use of their service to help isolate from those kinds of issues.
msg225577 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-08-20 17:56
Very nice, thanks for the update.
msg225580 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2014-08-20 18:44
On 20.08.2014 19:14, Donald Stufft wrote:
> 
> For now I think Fastly has sufficiently handled the issue to not require some sort of backup system to need to be put in place. They are going to let me know how they are going to handle it long term and what, if any changes, we can make in our use of their service to help isolate from those kinds of issues.

Sounds good.
msg225596 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2014-08-21 00:41
I just heard back from Fastly again. They are going to donate a dedicated IP
address setup on top of the rest of the stuff they are already donating to us.
It's not setup yet and the exact details are not sorted out yet. This should
more or less eliminate this problem (unless of course one of our own services
get banned somewhere).

I believe the sticker cost of this is $1500/month (Normally this is used for
custom SSL certificates, not sure if it'd be cheaper w/o that) on top of the
~$12k/month they already donate to the PSF. So that's incredibly awesome IMO
and a big thanks to them!
msg226807 - (view) Author: Donald Stufft (dstufft) * (Python committer) Date: 2014-09-12 04:30
Just to close the gap on this, most of the PSF web properties that go through Fastly have been switched over to a set of IP addresses that are dedicated to the PSF. So if someone does an IP ban they are blocking us.

I just made the switch in DNS so it'll take hold as that propagates.
History
Date User Action Args
2022-04-11 14:58:07adminsetgithub: 66420
2014-09-12 04:30:45dstufftsetmessages: + msg226807
2014-08-21 00:41:53dstufftsetmessages: + msg225596
2014-08-20 18:44:04lemburgsetmessages: + msg225580
2014-08-20 17:56:05georg.brandlsetmessages: + msg225577
2014-08-20 17:14:16dstufftsetmessages: + msg225575
2014-08-20 14:38:32dstufftsetmessages: + msg225571
2014-08-20 14:35:50jceasetnosy: + jcea
2014-08-20 08:07:49lemburgsetmessages: + msg225560
2014-08-20 07:39:57ezio.melottisetnosy: + ezio.melotti
messages: + msg225559
2014-08-20 07:28:10georg.brandlsetmessages: + msg225557
2014-08-20 07:17:20Alexander.Patrakovsetmessages: + msg225556
2014-08-18 14:56:16lemburgsetnosy: + lemburg
messages: + msg225495
2014-08-18 13:43:08georg.brandlsetstatus: open -> closed

nosy: + georg.brandl
messages: + msg225492

resolution: third party
2014-08-18 12:48:51dstufftsetnosy: + dstufft
messages: + msg225491
2014-08-18 11:58:12christian.heimessetnosy: + christian.heimes
messages: + msg225488
2014-08-18 10:57:24Alexander.Patrakovcreate