msg384106 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) *  |
Date: 2020-12-31 09:36 |
Since yesterday ALL PRs are blocked by failing test_nntplib.
For example https://github.com/python/cpython/runs/1629664606?check_suite_focus=true:
======================================================================
ERROR: test_descriptions (test.test_nntplib.NetworkedNNTP_SSLTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/runner/work/cpython/cpython/Lib/test/test_nntplib.py", line 250, in wrapped
meth(self)
File "/home/runner/work/cpython/cpython/Lib/test/test_nntplib.py", line 99, in test_descriptions
desc = descs[self.GROUP_NAME]
KeyError: 'comp.lang.python'
======================================================================
FAIL: test_description (test.test_nntplib.NetworkedNNTP_SSLTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/runner/work/cpython/cpython/Lib/test/test_nntplib.py", line 250, in wrapped
meth(self)
File "/home/runner/work/cpython/cpython/Lib/test/test_nntplib.py", line 85, in test_description
self.assertIn("Python", desc)
AssertionError: 'Python' not found in ''
----------------------------------------------------------------------
|
msg384111 - (view) |
Author: Ross Rhodes (trrhodes) * |
Date: 2020-12-31 12:30 |
Server is returning '215 Newsgroup descriptions in form "group description"', but an empty list of lines, so it's reaching the 'nothing' case in _getdescriptions: https://github.com/python/cpython/blob/master/Lib/nntplib.py#L660
|
msg384112 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2020-12-31 12:33 |
Right now, it works for me:
$ ./python -m test -u all test_nntplib -v -m test_descriptions
...
test_descriptions (test.test_nntplib.NNTPv1Tests) ... ok
test_descriptions (test.test_nntplib.NNTPv2Tests) ... ok
test_descriptions (test.test_nntplib.NetworkedNNTPTests) ... ok
skipped "<class 'test.test_nntplib.NetworkedNNTP_SSLTests'> got [SSL: DH_KEY_TOO_SMALL] dh key too small (_ssl.c:1122) connecting to 'nntp.aioe.org'"
...
Tests result: SUCCESS
|
msg384114 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) *  |
Date: 2020-12-31 12:57 |
On my computer tests are passed. They are only failed on CI.
|
msg384122 - (view) |
Author: Marc-Andre Lemburg (lemburg) *  |
Date: 2020-12-31 14:45 |
FWIW, I'm getting the same errors in PR https://github.com/python/cpython/pull/23140
Checking on the server that's being used, the newsgroup description is empty indeed: https://news.aioe.org/index.php?id=statistics-about-groups&group=comp.lang.python
|
msg384155 - (view) |
Author: Dong-hee Na (corona10) *  |
Date: 2021-01-01 08:46 |
== CPython 3.10.0a3+ (heads/master:c8a7b8fa1b, Jan 1 2021, 08:28:41) [GCC 7.5.0]
== Linux-5.4.0-1031-azure-x86_64-with-glibc2.27 little-endian
== cwd: /home/corona10/cpython/build/test_python_66899æ
== CPU count: 2
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 0.00 Run tests sequentially
0:00:00 load avg: 0.00 [1/1] test_nntplib
test_descriptions (test.test_nntplib.NNTPv1Tests) ... ok
test_descriptions (test.test_nntplib.NNTPv2Tests) ... ok
test_descriptions (test.test_nntplib.NetworkedNNTPTests) ... ok
test_descriptions (test.test_nntplib.NetworkedNNTP_SSLTests) ... ERROR
======================================================================
ERROR: test_descriptions (test.test_nntplib.NetworkedNNTP_SSLTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/corona10/cpython/Lib/test/test_nntplib.py", line 250, in wrapped
meth(self)
File "/home/corona10/cpython/Lib/test/test_nntplib.py", line 99, in test_descriptions
desc = descs[self.GROUP_NAME]
KeyError: 'comp.lang.python'
----------------------------------------------------------------------
Ran 4 tests in 4.734s
FAILED (errors=1)
test test_nntplib failed
test_nntplib failed
== Tests result: FAILURE ==
1 test failed:
test_nntplib
Total duration: 4.8 sec
Tests result: FAILURE
My test machine success to reproduce this issue.
(Pdb) descs
{}
There might be 4 ways to solve this issue.
1. Re-create group and skip test while descs is empty
2. Update test to use other NNTP server which supports SSL
|
msg384156 - (view) |
Author: Dong-hee Na (corona10) *  |
Date: 2021-01-01 08:46 |
4 ways / 2 ways
|
msg384157 - (view) |
Author: Dong-hee Na (corona10) *  |
Date: 2021-01-01 08:56 |
I submit the patch to use group_name which is mentioned from https://news.aioe.org/manual/aioe-hierarchy/
IMHO, it will be not changed frequently than what we used before.
|
msg384159 - (view) |
Author: Serhiy Storchaka (serhiy.storchaka) *  |
Date: 2021-01-01 10:31 |
The test was passed on my computer because test.test_nntplib.NetworkedNNTP_SSLTests was skipped because of the configuration of SSL on Ubuntu 20.04 (see issue41561).
After using custom openssl.cnf I get the same errors.
$ OPENSSL_CONF=~/py/openssl.cnf ./python -m test -vuall test_nntplib -m 'test_description*'
The NNTP server nntp.aioe.org is used instead of snews.gmane.org since 2010 (45ca9874955b1eaf3d941b7a278b594fad50d4ee). Sadly there is no reference to any discussion, maybe Antoine remember some details. What other alternatives were considered? How to handle similar breakage of external servers?
|
msg384161 - (view) |
Author: Dong-hee Na (corona10) *  |
Date: 2021-01-01 11:31 |
> How to handle similar breakage of external servers?
For tests with the external environment,
How about skipping tests if servers are not available?
It is assumed that if there is a change related to NNTP, it is well checked in the CI and review.
This is because, at this point, reviewers and patch submitters check whether tests with external servers are passed.
The reason for testing with the outside is to confirm that the whole operation works well.
If external servers are not available, we can emit warning messages so that we can recognize when an issue occurs.
Other than that, I can think of local server testing or mocking testing, but I am not sure if it is easy way for the SSL environment.
|
msg384163 - (view) |
Author: Dong-hee Na (corona10) *  |
Date: 2021-01-01 12:07 |
> For tests with the external environment,
How about skipping tests if servers are not available?
Hmm, I change my mind this is not a good idea :(
|
msg384164 - (view) |
Author: Christian Heimes (christian.heimes) *  |
Date: 2021-01-01 12:57 |
Does this issue mean that I should include nntplib in PEP 594 again?
|
msg384169 - (view) |
Author: Marc-Andre Lemburg (lemburg) *  |
Date: 2021-01-01 13:58 |
On 01.01.2021 13:57, Christian Heimes wrote:
>
> Does this issue mean that I should include nntplib in PEP 594 again?
The test fails because it was relying on an external news server's
configuratoin. This doesn't mean the code itself is broken, just the
test, so clearly: no.
|
msg384171 - (view) |
Author: Dong-hee Na (corona10) *  |
Date: 2021-01-01 14:20 |
New changeset ec3165320e81ac87edcb85c86c452528ddbaec1c by Dong-hee Na in branch 'master':
bpo-42794: Update test_nntplib to use offical group name for testing (GH-24037)
https://github.com/python/cpython/commit/ec3165320e81ac87edcb85c86c452528ddbaec1c
|
msg384176 - (view) |
Author: miss-islington (miss-islington) |
Date: 2021-01-01 14:40 |
New changeset 381f3e4bfd4b1c440f7cb3025972fe0acd0406fc by Miss Islington (bot) in branch '3.8':
bpo-42794: Update test_nntplib to use offical group name for testing (GH-24037)
https://github.com/python/cpython/commit/381f3e4bfd4b1c440f7cb3025972fe0acd0406fc
|
msg384177 - (view) |
Author: miss-islington (miss-islington) |
Date: 2021-01-01 14:42 |
New changeset b20d5e5ce95248e0fa77c5d7bf8f6f5b1231fa53 by Miss Islington (bot) in branch '3.9':
bpo-42794: Update test_nntplib to use offical group name for testing (GH-24037)
https://github.com/python/cpython/commit/b20d5e5ce95248e0fa77c5d7bf8f6f5b1231fa53
|
msg384184 - (view) |
Author: Christian Heimes (christian.heimes) *  |
Date: 2021-01-01 15:20 |
> The test fails because it was relying on an external news server's
> configuratoin. This doesn't mean the code itself is broken, just the
> test, so clearly: no.
Broken and unstable tests are one of two reasons why I listed nntplib in my PEP, https://www.python.org/dev/peps/pep-0594/#nntplib .
Q.E.D.
|
msg384185 - (view) |
Author: Marc-Andre Lemburg (lemburg) *  |
Date: 2021-01-01 16:43 |
Thanks for the patch, Dong-hee Na.
Christian: Tests relying on external resources will always have such issues. This doesn't mean that the code which is being tested is outdated or broken. It's just an issue with the tests. Perhaps we ought to disable external tests for NNTP on CI or ask the PSF to setup a news server for testing purposes (just like we have for SSL tests), e.g. news.pythontest.net mirroring comp.lang.python.
|
msg384197 - (view) |
Author: Ned Deily (ned.deily) *  |
Date: 2021-01-01 18:37 |
New changeset 8200ee66697601a8766f234d6eb8e4c8735216fd by Miss Islington (bot) in branch '3.7':
bpo-42794: Update test_nntplib to use offical group name for testing (GH-24037) (GH-24041)
https://github.com/python/cpython/commit/8200ee66697601a8766f234d6eb8e4c8735216fd
|
msg384198 - (view) |
Author: Ned Deily (ned.deily) *  |
Date: 2021-01-01 18:42 |
New changeset 546baba63a446e261d0248338f9034e56eccfc46 by Miss Islington (bot) in branch '3.6':
bpo-42794: Update test_nntplib to use offical group name for testing (GH-24037) (GH-24042)
https://github.com/python/cpython/commit/546baba63a446e261d0248338f9034e56eccfc46
|
msg384202 - (view) |
Author: STINNER Victor (vstinner) *  |
Date: 2021-01-01 19:25 |
Thanks Dong-hee Na for the fix!
We have many tests using external services, like test_ssl that Christian may know ;-)
https://pythondev.readthedocs.io/infra.html#services-used-by-unit-tests
|
|
Date |
User |
Action |
Args |
2022-04-11 14:59:39 | admin | set | github: 86960 |
2021-01-05 15:35:44 | serhiy.storchaka | set | pull_requests:
+ pull_request22950 |
2021-01-01 19:25:55 | vstinner | set | status: open -> closed resolution: fixed messages:
+ msg384202
stage: resolved |
2021-01-01 18:42:19 | ned.deily | set | messages:
+ msg384198 |
2021-01-01 18:37:42 | ned.deily | set | nosy:
+ ned.deily messages:
+ msg384197
|
2021-01-01 16:43:04 | lemburg | set | messages:
+ msg384185 stage: patch review -> (no value) |
2021-01-01 15:20:49 | christian.heimes | set | messages:
+ msg384184 |
2021-01-01 14:42:46 | miss-islington | set | messages:
+ msg384177 |
2021-01-01 14:40:21 | miss-islington | set | messages:
+ msg384176 |
2021-01-01 14:21:09 | miss-islington | set | pull_requests:
+ pull_request22883 |
2021-01-01 14:21:01 | miss-islington | set | pull_requests:
+ pull_request22882 |
2021-01-01 14:20:55 | miss-islington | set | pull_requests:
+ pull_request22881 |
2021-01-01 14:20:47 | miss-islington | set | nosy:
+ miss-islington
pull_requests:
+ pull_request22880 stage: patch review |
2021-01-01 14:20:45 | corona10 | set | messages:
+ msg384171 |
2021-01-01 13:58:19 | lemburg | set | messages:
+ msg384169 |
2021-01-01 12:57:44 | christian.heimes | set | nosy:
+ christian.heimes messages:
+ msg384164
|
2021-01-01 12:07:48 | corona10 | set | messages:
+ msg384163 |
2021-01-01 11:31:46 | corona10 | set | messages:
+ msg384161 |
2021-01-01 10:31:02 | serhiy.storchaka | set | files:
+ openssl.cnf nosy:
+ pitrou messages:
+ msg384159
|
2021-01-01 08:56:43 | corona10 | set | messages:
+ msg384157 stage: patch review -> (no value) |
2021-01-01 08:54:30 | corona10 | set | keywords:
+ patch stage: patch review pull_requests:
+ pull_request22877 |
2021-01-01 08:46:20 | corona10 | set | messages:
+ msg384156 |
2021-01-01 08:46:06 | corona10 | set | messages:
+ msg384155 |
2021-01-01 08:36:12 | corona10 | set | nosy:
+ corona10
|
2020-12-31 14:45:49 | lemburg | set | nosy:
+ lemburg messages:
+ msg384122
|
2020-12-31 12:57:49 | serhiy.storchaka | set | messages:
+ msg384114 |
2020-12-31 12:33:40 | vstinner | set | nosy:
+ vstinner messages:
+ msg384112
|
2020-12-31 12:30:06 | trrhodes | set | nosy:
+ trrhodes messages:
+ msg384111
|
2020-12-31 09:36:38 | serhiy.storchaka | create | |