When attempting to store a file via FTP, the following error occurs. See 
attached code.

Traceback (most recent call last):
  File "C:\Documents and 
Settings\rdevaughn\Desktop\HTTP\src\FTP_directory.py", line 14, in 
<module>
    ftp.storlines("STOR source.txt",f)
  File "C:\Python30\lib\ftplib.py", line 477, in storlines
    if buf[-1] in B_CRLF: buf = buf[:-1]
TypeError: Type str doesn't support the buffer AP

Oh yes, FTP.storlines() fails if the file is a text file. Here is a
patch. My patch encodes each line with self.encoding, which is latin1 by
default.

Does this still pass the test suite? in test_ftplib, storlines() is given 
a BytesIO object.

@amaury.forgeotdarc: Oops, no, it doesn't.

The new patch includes a new test (for the unicode file).

Attached is a patch which fixes FTP_TLS class as well and changes the test server a little bit.
Shouldn't retrlines() suffer the same issue?

To me this isn't a bug, and the patch introduces incorrect behaviour. If you want to store data on an FTP server, you have to provide binary data, not text data. The FTP class is not supposed to guess in which charset your data should be encoded.

(the "encoding" argument on the FTP class is meant for protocol commands (such as file names), not for file contents)

I agree with Antoine. Maybe it would make sense to check the file object passed to storlines() and raise an exception if mode != binary.

I do not agree with Antoine.  For binary transfer, another method is called, and it does work fine.  However, storelines would be called for ASCII mode, in which encoding and decoding should be done by the FTP program.  ASCII mode would translate files, so a file loaded on a PC may not be identical to the same file when loaded on a Mac.

> I do not agree with Antoine.  For binary transfer, another method is
> called, and it does work fine.  However, storelines would be called
> for ASCII mode, in which encoding and decoding should be done by the
> FTP program.  ASCII mode would translate files, so a file loaded on a
> PC may not be identical to the same file when loaded on a Mac.

Well if it's really "ASCII" mode, it should refuse any character > 127,
in which case I would agree to hardcode ascii as an encoding.

But, in any case, storing "lines" has a meaning in binary mode, as
witnessed by the fact that binary files in Python 3 have a readline()
method. This means storlines should still succeed for binary input, even
if it is made compatible with text input.

I would personally argue that the text transfer mode in FTP is an
ill-conceived feature from the start and ftplib should do whatever is
reasonable nowadays, rather than what looked reasonable 30 years ago.

Just my 2 cents to add that FTP ASCII mode should consists in nothing but replacing os.sep in "\r\n" before sending file data.
The opposite operation must be done by the receiving peer which has to read the data from the socket and replace "\r\n" with os.sep.
That should be all: no further conversion should be done against the data.

There seems to be a lot of confusion on this thread.  So I'll add mine and hopefully clear things up a bit.    In short I think aymanhs has it right.

This is what I get when trying to send a simple text file on a Windows Xp PC using python 3.x
-----------------------------------------------------------------
    ftp.storlines('STOR ' + localFile, fd)                  # Send the file by issuing the STOR command.
  File "C:\Python30\lib\ftplib.py", line 477, in storlines
    if buf[-1] in B_CRLF: buf = buf[:-1]
TypeError: Type str doesn't support the buffer API

-----------------------------------------------------------------

getting up on my soap box....  :-)

FTP has two modes binary and ascii.  When doing binary you expect your data to be sent as is with NO changes.  When doing ascii, you expect to send a text file with your line endings to be adjusted if they are not the same on the two platforms involved in the FTP transfer (MAC, Linux, PC, etc.).  

So I do think it's an bug.  Every other FTP program I've used, and it's a long list, has two distinct modes ASCII and Binary.  When using ASCII, the line endings are translated.  I don't know enough to comment on whether the ascii code for every character is supposed to be 127 or less.  But I can tell you that I've never had another FTP program fail like Python 3 is failing for me trying to what seems like a simple transfer of a text file (expecting the line endings to be adjusted).  

I think the goal here should be (must be) to make it work like it is expected to work, based on the expectations of millions of people who have been using FTP programs which have come before python for several decades.  The other finer points are really not important.

stepping off the soap box.

One other observation on this issue.

I just 'backleveled' the python on my PC from 3.x to 2.7.  My test script works fine on 2.7.

- Dan

My other 2 cents worth.

Actually that is my main issue.  Things used to work in 2.x, and
"suddenly" refused to work after my 3.x upgrade.

I'm not saying it worked means it is correct, but the Exception being
thrown does not look right.  I think we should always have core
libraries work out of the box, and have workarounds even if it's not
"our" error.

 Ayman

> But I can tell you that I've never had another FTP program fail like
> Python 3 is failing for me trying to what seems like a simple transfer
> of a text file (expecting the line endings to be adjusted).  

Python 3 is not failing, you are just using it the wrong way. Open your
file in binary mode (as opposed to text mode) and it should work. "Text
files" in Python 3 are files decoded to unicode while reading (and
encoded from unicode while writing); they are much more than simply
binary files with adaptive line endings. This is more or less documented
here (although I agree it would perhaps need a better introductory
paragraph):
http://docs.python.org/py3k/library/io.html

pitrou> The FTP class is not supposed to guess in which charset 
pitrou> your data should be encoded.
pitrou> (the "encoding" argument on the FTP class is meant 
pitrou> for protocol commands (such as file names), not for file
pitrou> contents)

I agree, my patch is completly wrong :-) You should open your file in binary mode, not in text (unicode) mode. The issue is more a documentation bug, than a bug in ftplib behaviour.

Here is a new patch: very short patch on the doc (just add "binary" to "open binary file object *file*").

ftplib_doc.patch is so simple I'm assuming this has slipped under the radar, so would someone like to do the honours and commit the patch please.

Apparently the doc has already been changed and mentions that a binary file is necessary. The unit test is legitimate and can be committed though (Victor, you want to do it?).

New changeset c78dfc6ce37a by Victor Stinner in branch '3.3':
Close #6822: ftplib.FTP.storlines() expects a binary file, not a text file
http://hg.python.org/cpython/rev/c78dfc6ce37a

New changeset 9328e2b8a397 by Victor Stinner in branch 'default':
(Merge 3.3) Close #6822: ftplib.FTP.storlines() expects a binary file, not a text file
http://hg.python.org/cpython/rev/9328e2b8a397