classification
Title: test_poplib hangs with the changes done in PR
Type: Stage:
Components: Tests Versions: Python 3.8, Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: christian.heimes, inada.naoki, jayyin11043, skip.montanaro
Priority: normal Keywords: patch

Created on 2018-03-18 22:18 by jayyin11043, last changed 2018-04-09 19:59 by skip.montanaro.

Pull Requests
URL Status Linked Edit
PR 6428 closed inada.naoki, 2018-04-09 10:04
Messages (7)
msg314064 - (view) Author: Jay Yin (jayyin11043) * Date: 2018-03-18 22:18
my test hangs locally on my computer with the changes I've done in bpo-32642 but doesn't hang on TravisCI, anyone able to help with checking what's wrong here (sounds like another edge case with my env but I could be wrong)

the trace for the command https://pastebin.com/q4FKnPZH
msg315118 - (view) Author: Inada Naoki (inada.naoki) * (Python committer) Date: 2018-04-09 10:09
There are some resource leaks:

* When error occurred in setUp() function, server thread is not stopped.  It leaked threads and sockets for the server.
* When error occurred in server thread's run() method, asyncore.close_all() is not called.

These leaks makes dangling threads.  It caused hang in test_poplib shutdown process.

Additionally, the error is not printed in these cases.  It makes harder to fix real problem.
msg315120 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-04-09 10:11
This is likely a duplicate of BPO #32706 and #32753
msg315121 - (view) Author: Inada Naoki (inada.naoki) * (Python committer) Date: 2018-04-09 10:19
@Christian

Precisely, my PR doesn't fix #32706 and #32753.
It fixes only hang caused by them and print stack trace for them.
msg315141 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2018-04-09 17:15
Just in case this would be useful to others, git bisect led me to this commit as the source of the problem:

61d478c71c5341cdc54e6bfb4ace4252852fd972 is the first bad commit
commit 61d478c71c5341cdc54e6bfb4ace4252852fd972
Author: Christian Heimes <christian@python.org>
Date:   Sat Jan 27 15:51:38 2018 +0100

    bpo-31399: Let OpenSSL verify hostname and IP address (#3462)
    
    bpo-31399: Let OpenSSL verify hostname and IP
    
    The ssl module now uses OpenSSL's X509_VERIFY_PARAM_set1_host() and
    X509_VERIFY_PARAM_set1_ip() API to verify hostname and IP addresses.
    
    * Remove match_hostname calls
    * Check for libssl with set1_host, libssl must provide X509_VERIFY_PARAM_set1_host()
    * Add documentation for OpenSSL 1.0.2 requirement
    * Don't support OpenSSL special mode with a leading dot, e.g. ".example.org" matches "www.example.org". It's not standard conform.
    * Add hostname_checks_common_name
    
    Signed-off-by: Christian Heimes <christian@python.org>

:040000 040000 09f4c8a18941f926c1f79e2f07dfd7731edf74c2 798ae3fab880e3365c490d56935901a4c18fd3bd M	Doc
:040000 040000 f35a1b4e2144050dfc4c7dbb02b7d92e3de1c2d0 7af4e4a8e82eafc6e7f2f4392c951213777f12c4 M	Lib
:040000 040000 e287c27f4db61beae905c21b96666194bb930c13 59e6fc63d296f7fe3187ff0987a04c9999e2d1d0 M	Misc
:040000 040000 95250f4c1009b5f10ec392c186908698d78f9cd3 160ebeb2c63d1ca305c7e37a0b555900b5084d20 M	Modules
:040000 040000 8adc7f664a8eaa667f4540f537304d7f9de672d4 41ea621f5433077f6dd776f4c2c51f7a8e4cab0f M	PC
:100644 100644 a6f4488cc99941abaa8b6fb7c53079b7cc292f9f ba0a7624cfcd0129e73bc218510f398001514df8 M	setup.py
msg315142 - (view) Author: Christian Heimes (christian.heimes) * (Python committer) Date: 2018-04-09 17:44
Thanks for you confirmation. I already knew which commit lead to the issue. :)

The problem didn't "pop" up in my local tests or on Travis CI. I can only reproduce it locally, when I run the test in a tight loop and put extra load on the other CPU cores.
msg315147 - (view) Author: Skip Montanaro (skip.montanaro) * (Python triager) Date: 2018-04-09 19:59
@christian.heimes I figured you probably didn't need it, but I did that work before I discovered this ticket. And as a nice bonus, I got to learn git bisect. :-)
History
Date User Action Args
2018-04-09 19:59:12skip.montanarosetmessages: + msg315147
2018-04-09 17:44:12christian.heimessetmessages: + msg315142
2018-04-09 17:15:01skip.montanarosetnosy: + skip.montanaro
messages: + msg315141
2018-04-09 10:19:10inada.naokisetmessages: + msg315121
2018-04-09 10:11:51christian.heimessetnosy: + christian.heimes
messages: + msg315120
2018-04-09 10:09:06inada.naokisetnosy: + inada.naoki

messages: + msg315118
stage: patch review ->
2018-04-09 10:04:53inada.naokisetkeywords: + patch
stage: patch review
pull_requests: + pull_request6126
2018-03-18 22:18:03jayyin11043create