With the code as it stands, calls to shutdown that happen before
serve_forever enters its loop will deadlock, and there's no simple way
for the user to avoid this. The attached patch prevents the deadlock and
allows multiple serve_forever..shutdown cycles, but it's pretty
complicated. I could make it a lot simpler by making shutdown permanent:
any later serve_forever calls would return immediately.

A third choice would be to add a .serve_in_thread function that returns
a token that can be used to shut down exactly that loop, instead of
putting .shutdown() on the server. Any opinions?

>With the code as it stands, calls to shutdown that happen before
>serve_forever enters its loop will deadlock, and there's no simple way
>for the user to avoid this. The attached patch prevents the deadlock and
>allows multiple serve_forever..shutdown cycles, but it's pretty
>complicated. I could make it a lot simpler by making shutdown permanent:
>any later serve_forever calls would return immediately.

Never thought of using the SocketServer taht way, wouldn't the person
doing this bunch of shutdown()s and serve_forever()s be better off using
handle_request() on a loop instead ?

>A third choice would be to add a .serve_in_thread function that returns
>a token that can be used to shut down exactly that loop, instead of
>putting .shutdown() on the server. Any opinions?

I don't think I understand this part, what loop do you refer to ?

After a bit of investigation, this issue seems to be exactly why test_httpservers sometimes hangs.
The patch looks complicated to me, though; I don't think we really have to support the multiple shutdowns case.

Here is a simpler patch. It also fixes the wrong use of a lock instead of an event in test_httpservers. With this patch, test_httpservers runs forever without freezing.

By the way, getting rid of poll_interval for a file descriptor is easy under Unix, but wouldn't work under Windows (where select() only takes sockets, not arbitrary file descriptors).

Fixed with a test in r80484 (trunk), r80486 (2.6), r80487 (py3k), r80491 (3.1).