classification
Title: Add CCC command support to ftplib
Type: enhancement Stage: resolved
Components: Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: giampaolo.rodola Nosy List: giampaolo.rodola, haypo, pitrou, python-dev
Priority: normal Keywords: patch

Created on 2011-05-21 18:09 by giampaolo.rodola, last changed 2011-06-30 16:36 by giampaolo.rodola. This issue is now closed.

Files
File name Uploaded Description Edit
ftplib-ccc.patch giampaolo.rodola, 2011-05-21 18:08 review
Messages (10)
msg136459 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2011-05-21 18:08
In FTPS, CCC command can be used to switch back to a clear-text control connection.
This can be useful to take advantage of firewalls that know how to handle NAT with non-secure FTP without opening fixed ports.
Patch in attachment includes lib, test and doc changes.
msg136463 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-05-21 18:19
> This can be useful to take advantage of firewalls that know how to
> handle NAT with non-secure FTP without opening fixed ports.

Can you explain this a little more?

+            resp = self.voidcmd('CCC')
+            self.sock = self.sock.unwrap()
+            return resp

Should the response be checked before calling unwrap()?
msg136465 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2011-05-21 19:08
voidcmd() expects a response code starting with "2" and automatically raises an exception otherwise.
A server succesfully accepting the CCC command is supposed to first reply with 220 and then shutdown() the SSL layer.
Client is supposed to do the same: first receive the 2xx response and then make the shutdown().
msg139250 - (view) Author: Roundup Robot (python-dev) Date: 2011-06-27 09:17
New changeset d2eacbbdaf57 by Giampaolo Rodola' in branch 'default':
Issue 12139: add CCC command support to FTP_TLS class to revert the SSL connection back to clear-text.
http://hg.python.org/cpython/rev/d2eacbbdaf57
msg139252 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-06-27 09:57
http://www.python.org/dev/buildbot/all/builders/x86%20Ubuntu%20Shared%203.x/builds/4043/steps/test/logs/stdio

======================================================================
ERROR: test_ccc (test.test_ftplib.TestTLS_FTPClass)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_ftplib.py", line 890, in test_ccc
    self.client.sendcmd('noop')
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/ftplib.py", line 261, in sendcmd
    return self.getresp()
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/ftplib.py", line 236, in getresp
    raise error_proto(resp)
ftplib.error_proto:  Ôxé֮º¸qhёøcÞÅ\³9úӅ#ï•å200 noop ok
msg139253 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-06-27 09:58
http://www.python.org/dev/buildbot/all/builders/x86%20Tiger%203.x/builds/2792/steps/test/logs/stdio

======================================================================
ERROR: test_ccc (test.test_ftplib.TestTLS_FTPClass)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/test_ftplib.py", line 890, in test_ccc
    self.client.sendcmd('noop')
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/ftplib.py", line 261, in sendcmd
    return self.getresp()
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/ftplib.py", line 226, in getresp
    resp = self.getmultiline()
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/ftplib.py", line 212, in getmultiline
    line = self.getline()
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/ftplib.py", line 199, in getline
    line = self.file.readline()
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/socket.py", line 279, in readinto
    return self._sock.recv_into(b)
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/ssl.py", line 392, in recv_into
    return socket.recv_into(self, buffer, nbytes, flags)
socket.timeout: timed out
msg139254 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2011-06-27 10:01
Hmm... Reopening. I'll look into this later.
msg139493 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2011-06-30 15:21
The test is still failing on many buildbots.

x86 Ubuntu Shared 3.x:

======================================================================
ERROR: test_ccc (test.test_ftplib.TestTLS_FTPClass)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_ftplib.py", line 890, in test_ccc
    self.client.sendcmd('noop')
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/ftplib.py", line 261, in sendcmd
    return self.getresp()
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/ftplib.py", line 236, in getresp
    raise error_proto(resp)
ftplib.error_proto:  žeÊc¯ŸöZÖ8N´†ýùô

"PPC Tiger 3.x", "AMD64 Leopard 3.x", "x86 FreeBSD 6.4 3.x" and "x86 Tiger 3.x":

======================================================================
ERROR: test_ccc (test.test_ftplib.TestTLS_FTPClass)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86-2/build/Lib/test/test_ftplib.py", line 890, in test_ccc
    self.client.sendcmd('noop')
  File "/Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86-2/build/Lib/ftplib.py", line 261, in sendcmd
    return self.getresp()
  File "/Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86-2/build/Lib/ftplib.py", line 226, in getresp
    resp = self.getmultiline()
  File "/Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86-2/build/Lib/ftplib.py", line 212, in getmultiline
    line = self.getline()
  File "/Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86-2/build/Lib/ftplib.py", line 199, in getline
    line = self.file.readline()
  File "/Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86-2/build/Lib/socket.py", line 279, in readinto
    return self._sock.recv_into(b)
  File "/Users/pythonbuildbot/buildarea/3.x.hansen-osx-x86-2/build/Lib/ssl.py", line 392, in recv_into
    return socket.recv_into(self, buffer, nbytes, flags)
socket.timeout: timed out
msg139499 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2011-06-30 16:25
The culprit here is the dummy FTP test server which does not complete the SSL shutdown procedure successfully.

I believe it's better if we remove the self.client.sendcmd('noop') line.

That's there in order to check that client and server can still "talk" after reverting the connection back to clear-text, but making the server doing this right is quite complicated, see for example:
http://code.google.com/p/pyftpdlib/source/browse/trunk/pyftpdlib/contrib/handlers.py?spec=svn868&r=861#196

I don't want to include and maintain that monster into test_ftplib.py so I think I'll just remove that line from the tests.
msg139501 - (view) Author: Roundup Robot (python-dev) Date: 2011-06-30 16:34
New changeset 0cef8cac71c8 by Giampaolo Rodola' in branch 'default':
Issue 12139: ftplib - remove 'post CCC' test to fix various buildot failures due to dummy test server not properly handling SSL shutdown(), see http://bugs.python.org/msg139499
http://hg.python.org/cpython/rev/0cef8cac71c8
History
Date User Action Args
2011-06-30 16:36:03giampaolo.rodolasetstatus: open -> closed
2011-06-30 16:34:52python-devsetmessages: + msg139501
2011-06-30 16:25:26giampaolo.rodolasetmessages: + msg139499
2011-06-30 15:21:46hayposetmessages: + msg139493
2011-06-29 10:40:16pitrousetstatus: pending -> open
resolution: fixed
2011-06-27 10:01:29giampaolo.rodolasetstatus: closed -> pending
resolution: fixed -> (no value)
messages: + msg139254
2011-06-27 09:58:22hayposetmessages: + msg139253
2011-06-27 09:58:00hayposetnosy: + haypo
messages: + msg139252
2011-06-27 09:18:39giampaolo.rodolasetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2011-06-27 09:17:58python-devsetnosy: + python-dev
messages: + msg139250
2011-05-21 19:08:16giampaolo.rodolasetmessages: + msg136465
2011-05-21 18:19:44pitrousetnosy: + pitrou
messages: + msg136463
2011-05-21 18:09:00giampaolo.rodolacreate