classification
Title: Gentoo Refleaks 3.7: test_smtplib has dangling threads
Type: behavior Stage: resolved
Components: email Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: barry, miss-islington, pablogsal, r.david.murray, scorphus, serhiy.storchaka, vstinner
Priority: normal Keywords: patch

Created on 2018-07-27 10:02 by pablogsal, last changed 2018-09-08 01:22 by pablogsal. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 8554 merged scorphus, 2018-07-29 14:59
PR 9108 merged pablogsal, 2018-09-07 21:50
PR 9111 merged miss-islington, 2018-09-07 22:04
PR 9112 merged pablogsal, 2018-09-07 23:12
Messages (15)
msg322481 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-07-27 10:02
The Gentoo Refleaks 3.7 buildbot is reporting that test_smtplib may be leaving with dangling threads:

https://buildbot.python.org/all/#builders/114/builds/179

FAILED (errors=7)
Warning -- threading._dangling was modified by test_smtplib
  Before: <_weakrefset.WeakSet object at 0xb6918bcc>
  After:  <_weakrefset.WeakSet object at 0xb68e761c> 
test test_smtplib failed
1 test failed again:
    test_smtplib
== Tests result: FAILURE then FAILURE ==
msg322482 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-07-27 10:04
New failure on AMD64 Windows8.1 Refleaks 3.6:

https://buildbot.python.org/all/#builders/64/builds/301
msg322488 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2018-07-27 12:42
Ah, sorry.  we just modernized test_smtplib's __main__ section, and I think we lost the thread cleanup code that used to be run around the test suite.  We need to add the thread cleanup decorators.
msg322493 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-07-27 14:06
I think it may be something extra as test_send_message_error_on_non_ascii_addrs_if_no_smtputf8 is failing when you run the test with -R:

./python -W error -m test test_smtplib  -R 3:3

Warning -- threading._dangling was modified by test_smtplib
  Before: <_weakrefset.WeakSet object at 0x10ed7dd70>
  After:  <_weakrefset.WeakSet object at 0x10f619830>
test test_smtplib failed -- Traceback (most recent call last):
  File "/Users/pgalindo3/cpython/Lib/test/test_smtplib.py", line 1232, in test_send_message_error_on_non_ascii_addrs_if_no_smtputf8
    smtp.send_message(msg))
  File "/Users/pgalindo3/cpython/Lib/unittest/case.py", line 743, in assertRaises
    return context.handle('assertRaises', args, kwargs)
  File "/Users/pgalindo3/cpython/Lib/unittest/case.py", line 178, in handle
    callable_obj(*args, **kwargs)
TypeError: 'dict' object is not callable

test_smtplib failed

== Tests result: FAILURE ==

1 test failed:
    test_smtplib

Total duration: 16 sec 566 ms
Tests result: FAILURE
msg322494 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-07-27 14:12
The TypeError is because the test is misusing the assertRaises API:

         self.assertRaises(smtplib.SMTPNotSupportedError,
                          smtp.send_message(msg))

should be 

         self.assertRaises(smtplib.SMTPNotSupportedError,
                          smtp.send_message, msg)
msg322500 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2018-07-27 15:32
That's not the code I thought I merged.  I'll have to take a look at what actually got merged.
msg322501 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-07-27 15:48
That was from commit 48ed88a93bb0bbeaae9a4cfaa533e4edf13bcb51
msg322502 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-07-27 15:50
Oh, sorry, my bad. That was precisely what commit 48ed88a93bb0bbeaae9a4cfaa533e4edf13bcb51 changed. The problem with running with -R still happens:

======================================================================
ERROR: test_name_field_not_included_in_envelop_addresses (test.test_smtplib.SMTPSimTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/pgalindo3/cpython/Lib/test/test_smtplib.py", line 1100, in test_name_field_not_included_in_envelop_addresses
    self.assertDictEqual(smtp.send_message(message), {})
  File "/Users/pgalindo3/cpython/Lib/smtplib.py", line 967, in send_message
    rcpt_options)
  File "/Users/pgalindo3/cpython/Lib/smtplib.py", line 861, in sendmail
    (code, resp) = self.mail(from_addr, esmtp_opts)
  File "/Users/pgalindo3/cpython/Lib/smtplib.py", line 532, in mail
    'SMTPUTF8 not supported by server')
smtplib.SMTPNotSupportedError: SMTPUTF8 not supported by server

----------------------------------------------------------------------
Ran 60 tests in 1.365s

FAILED (errors=7)
Warning -- threading._dangling was modified by test_smtplib
  Before: <_weakrefset.WeakSet object at 0x10f445e90>
  After:  <_weakrefset.WeakSet object at 0x10fe75ad0>
test test_smtplib failed
test_smtplib failed

== Tests result: FAILURE ==

1 test failed:
    test_smtplib

Total duration: 2 sec 920 ms
Tests result: FAILURE
msg322509 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2018-07-27 18:06
That's due to another bug, which will get fixed by #32814, unless you'd like to generate a PR with the this fix in it (fixing the mutable default for smtp_options).
msg322544 - (view) Author: Pablo Santiago Blum de Aguiar (scorphus) * Date: 2018-07-28 11:24
This is just to note that I'm investigating this issue as part of the CPython sprint in EuroPython.
msg323780 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-08-20 08:27
See also issue32657.
msg324804 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-09-07 22:04
New changeset d5fbe9b1a3d65ceeb9159c5ba999ee966a945f76 by Pablo Galindo (Pablo Aguiar) in branch 'master':
bpo-34246: Use no mutable default args in smtplib (GH-8554)
https://github.com/python/cpython/commit/d5fbe9b1a3d65ceeb9159c5ba999ee966a945f76
msg324809 - (view) Author: miss-islington (miss-islington) Date: 2018-09-07 22:29
New changeset 9835696ec4c57a9a30f1c11cfb4c5d3e121bf97c by Miss Islington (bot) in branch '3.7':
bpo-34246: Use no mutable default args in smtplib (GH-8554)
https://github.com/python/cpython/commit/9835696ec4c57a9a30f1c11cfb4c5d3e121bf97c
msg324811 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-09-07 23:15
New changeset 5b7a2cb5caeb7df68e637f45a98632cbc84a51bf by Pablo Galindo in branch 'master':
bpo-34246: Make sure test_smtplib always cleans resources when finished (GH-9108)
https://github.com/python/cpython/commit/5b7a2cb5caeb7df68e637f45a98632cbc84a51bf
msg324817 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2018-09-08 01:20
New changeset eb6ab73f93c8b883a8d75a83560e2b4c59170d95 by Pablo Galindo in branch '3.6':
[3.6] bpo-34246: Use no mutable default args in smtplib (GH-8554) (#9112)
https://github.com/python/cpython/commit/eb6ab73f93c8b883a8d75a83560e2b4c59170d95
History
Date User Action Args
2018-09-08 01:22:13pablogsalsetstatus: open -> closed
stage: patch review -> resolved
2018-09-08 01:20:31pablogsalsetmessages: + msg324817
2018-09-07 23:15:25pablogsalsetmessages: + msg324811
2018-09-07 23:12:17pablogsalsetpull_requests: + pull_request8566
2018-09-07 22:29:32miss-islingtonsetnosy: + miss-islington
messages: + msg324809
2018-09-07 22:04:58miss-islingtonsetpull_requests: + pull_request8565
2018-09-07 22:04:51pablogsalsetmessages: + msg324804
2018-09-07 21:50:36pablogsalsetpull_requests: + pull_request8562
2018-09-04 09:34:44vstinnersetnosy: + vstinner
2018-08-20 08:27:10serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg323780
2018-08-16 10:29:29serhiy.storchakasetnosy: + barry
components: + email
2018-07-29 14:59:36scorphussetkeywords: + patch
stage: needs patch -> patch review
pull_requests: + pull_request8070
2018-07-28 11:24:46scorphussetnosy: + scorphus
messages: + msg322544
2018-07-27 18:06:09r.david.murraysetmessages: + msg322509
2018-07-27 15:50:01pablogsalsetmessages: + msg322502
2018-07-27 15:48:02pablogsalsetmessages: + msg322501
2018-07-27 15:32:20r.david.murraysetmessages: + msg322500
2018-07-27 14:12:47pablogsalsetmessages: + msg322494
2018-07-27 14:06:33pablogsalsetmessages: + msg322493
2018-07-27 12:42:02r.david.murraysetversions: + Python 3.6, Python 3.7, Python 3.8
nosy: + r.david.murray

messages: + msg322488

type: behavior
stage: needs patch
2018-07-27 10:04:02pablogsalsetmessages: + msg322482
2018-07-27 10:02:13pablogsalcreate