The enclosed patch does three things:
1. enables SMTP_SSL working: the _get_socket method was setting
   self.sock instead of returning the socket to the caller, which
   did reset self.sock to None
2. replace home-grown SSLFakeFile() with calls to ssl.socket's makefile()
   calls both in the starttls and in the SMTP_SSL cases
3. shutdown sockets before closing them, to avoid server-side piling and
   connection refused on connection-limited servers
The last change is just a cosmetical refactoring, but it really helps
the SMTP_SSL case: default_port should really be a class attribute,
instead of being set at __init__ time.

I've reworked the patch into a series, like haypo requested for
poplib and imaplib.

With the closure of 4066 all the tests in the test patch pass, so I'm
lowering the piority of this ticket.

I haven't reviewed the other patches, but the tests in the test patch
appear to be doing tests in the setup method, which doesn't seem like a
good idea.

I still have to apply Catucci's patch (or a modification of) after every Ubuntu installation or upgrade.  Otherwise, I get

  ...
  File "/usr/lib/python2.7/smtplib.py", line 752, in __init__
    SMTP.__init__(self, host, port, local_hostname, timeout)
  File "/usr/lib/python2.7/smtplib.py", line 239, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/lib/python2.7/smtplib.py", line 295, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib/python2.7/smtplib.py", line 757, in _get_socket
    new_socket = socket.create_connection((host, port), timeout)
  File "/usr/lib/python2.7/socket.py", line 571, in create_connection
    raise err
socket.error: [Errno 111] Connection refused

But isn't it that #4066 should have solved the critical part this issue pair 4066/4470?

Torsten, can you provide a clear, failing unittest for this?

No, I don't know how to do that.  All I can provide is a minimal version of my code that triggers the above mentioned traceback.  It is:

import smtplib

s = smtplib.SMTP_SSL("relay-auth.rwth-aachen.de")
s.login("***", "***")
s.sendmail("bronger@physik.rwth-aachen.de", "bronger.randys@googlemail.com"], "Hello")

I hope it helps to understand what I mean.

Sorry, it must be:

import smtplib

s = smtplib.SMTP_SSL("relay-auth.rwth-aachen.de")
s.login("***", "***")
s.sendmail("bronger@physik.rwth-aachen.de", ["bronger.randys@googlemail.com"], "Hello")

(A bracket was missing.)

According to your traceback you should be seeing the error in the first line (the creation of the SMTP_SSL object).  If I run that line at the python prompt of python2.7.1, I get your connection failure.  If I run it using 2.7 tip (or 3.3 tip), the connection succeeds.  I know there have been other fixes in this area, but I don't know which one makes the difference.

Torsten, can you test with 2.7.2 and/or 2.7 tip?

I'm not sure what is left to do in this issue.  Do you have an opinion, Lorenzo?

On Fri, 17 Jun 2011, R. David Murray wrote:

RDM> 
RDM> R. David Murray <rdmurray@bitdance.com> added the comment:
RDM> 
RDM> According to your traceback you should be seeing the error in the 
RDM> first line (the creation of the SMTP_SSL object).  If I run that line 
RDM> at the python prompt of python2.7.1, I get your connection failure.  
RDM> If I run it using 2.7 tip (or 3.3 tip), the connection succeeds.  I 
RDM> know there have been other fixes in this area, but I don't know which 
RDM> one makes the difference.
RDM> 
RDM> Torsten, can you test with 2.7.2 and/or 2.7 tip?
RDM> 
RDM> I'm not sure what is left to do in this issue.  Do you have an 
RDM> opinion, Lorenzo?
RDM> 

Torsten, would you mind letting us know both the python and the .deb 
version by running 

$ python2.7 --version

and

$ dpkg -l python2.7

At least in debian's python2.7_2.7.1-8, default_port is still an instance 
attribute instead of a class attribute. 

If the ubuntu situation is the same, the missing patch is 
smtplib_01_default_port.diff , and you could work-around the problem by 
explicitly calling s = smtplib.SMTP_SSL("relay-auth.rwth-aachen.de", 465).

The patch got in with:

changeset:   69931:bcf04ced5ef1
branch:      2.7
parent:      69915:7c3a20b5943a
user:        Antoine Pitrou <solipsis@pitrou.net>
date:        Sat May 07 19:59:33 2011 +0200
summary:     Issue #11927: SMTP_SSL now uses port 465 by default as 
documented.  Patch by Kasun Herath.

The last hunk, which fixes LMTP is still missing.

@@ -776,8 +777,9 @@
     authentication, but your mileage might vary."""

     ehlo_msg = "lhlo"
+    default_port = LMTP_PORT

-    def __init__(self, host = '', port = LMTP_PORT, local_hostname = 
None):
+    def __init__(self, host = '', port = 0, local_hostname = None):
         """Initialize a new instance."""
         SMTP.__init__(self, host, port, local_hostname)

Have a nice week-end, yours

	lorenzo

+-------------------------+----------------------------------------------+
|   Lorenzo M.  Catucci   | Centro di Calcolo e Documentazione           |
| catucci@ccd.uniroma2.it | Università degli Studi di Roma "Tor Vergata" |
|                         | Via O. Raimondo 18 ** I-00173 ROMA  ** ITALY |
|  Tel. +39 06 7259 2255  | Fax. +39 06 7259 2125                        |
+-------------------------+----------------------------------------------+

Most of the problems in this issue were solved already so it could almost be closed:
* patch 1 was addressed in #11927
* patch 2 was addressed in #4066
* patches 3 and 4 were addressed in #11893

Torsten's problem was addressed by bcf04ced5ef1.

> I'm not sure what is left to do in this issue.

The only patch remaining is patch 5. I attached an updated version against tip of default branch. My patch mimics shutdown in imaplib.py in that it silences ENOTCONN. However I don't have a test that fails without the shutdown and I don't know if checking ENOTCONN is really needed. I tried to get shutdown to produce ENOTCONN by using Postfix as a server with smtpd_timeout=5s, connecting to it and waiting idle for more than 5 seconds before doing close(). In the Postfix logs I see that Postfix disconnects after 5 seconds of inactivity but doing shutdown afterwards doesn't trigger any exception so the ENOTCONN part remains unexercised.

My patch also adds shutdown method and SHUT_RDWR constant to mock_socket.py since otherwise test_smtplib fails.

(Added Antoine to nosy because he reviewed the patches for #11927 and #11893)

My Python version is "Python 2.7.1+" and the package is called "python2.7 2.7.1-5ubuntu2" (Ubuntu Natty).

Just finished testing both 2.7 and default branches' socket close behaviour, and it seems 05 is not strictly needed.

I'd still prefer if smtplib_05_shutdown_socket_v2.patch since, this way the REMOTE socket close will be unconditionally correct, instead of being dependent on GC artifacts.

I'd still prefer if smtplib_05_shutdown_socket_v2.patch could get in,
                                                       ^^^^^^^^^^^^^^
since, this way the REMOTE socket close will be unconditionally correct,
instead of being dependent on GC artifacts.