This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: "What's New" should say VERY CLEARLY that the type file is gone
Type: Stage:
Components: Documentation Versions: Python 3.0, Python 3.1
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: LambertDW, MLModel, benjamin.peterson, georg.brandl, loewis, rhettinger
Priority: normal Keywords:

Created on 2009-03-18 22:26 by MLModel, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (13)
msg83783 - (view) Author: Mitchell Model (MLModel) Date: 2009-03-18 22:26
MAIN POINT

The Python 3 "What's New" should SCREAM that the type file is gone, not 
just that people should use the function open() to open files, since 
that has been a recommendation for quite a while.

EXPLANATION

In multiple readings of the Python 3 "What's New" I blew right past the 
"Removed file. Use open().", since I've been using open() instead of 
file() for a long time. I didn't notice that unlike the preceding 
several lines, there were no parentheses after "file" and that this line 
was literally saying there was no longer a type called "file".

OBSERVATIONS

(1) If the line is meant to say that you can no longer call file() as a 
function -- which would be strange if it were still a type -- then it is 
missing its parentheses.

(2) If the line is meant to say that there is no longer a file type, as 
it apparently means to say since in fact -- and to my great surprise -- 
there really IS no type called "file" in Python 3 (I discovered that 
doing a dir(file) to check whether file provided method function I 
thought it did instead of taking the time to look it up.) then there is 
a grammatical problem with the line since a (n old) type shouldn't be 
equated to a function call.

(3) I predict that anyone who has more than a passing acquaintance with 
Python 2 will be similarly shocked when they find out that what they get 
back from open() is a _io.TextIOWrapper (and, by the way, that they have 
to import _io or at least _io.TextIOWrapper to be able to do a dir on 
it). Likewise for help(file) and help(_io.TextIOWrapper). There should 
be a very prominent statement that as part of the reimplementation of 
the io system, the type file has been replaced by _io.TextIOWrapper.

RECOMMENDATION

The line
    "Removed file. Use open()."
should be replaced with:
    "The type file has been removed; use open() to open a file."
or possibly:
    "The type file has been replaced by _ioTextIOWrapper; use open() to 
open a file; open returns an instance of _ioTextIOWrapper."
msg83784 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-03-18 22:30
I'm working on the whatsnew updates and will make sure this is clear.
msg83801 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-03-19 06:26
> The Python 3 "What's New" should SCREAM that the type file is gone

I don't think the documentation should ever SCREAM.

>     "The type file has been replaced by _ioTextIOWrapper; use open() to 
> open a file; open returns an instance of _ioTextIOWrapper."

That would be an incorrect statement: there are multiple types that
replace the 2.x file type. See the documentation of the io module for
details.
msg83809 - (view) Author: David W. Lambert (LambertDW) Date: 2009-03-19 14:30
# With py3Krc1 it took me days to figure out how to
# replace my base class file.   Granted, there were
# issues with io module at the time.  Following met
# my need.


import io

class File(io.TextIOWrapper):

    '''Open a text file with read access, providing...'''

    def __init__(self,name):
        super().__init__(open(name).buffer)
msg83812 - (view) Author: Mitchell Model (MLModel) Date: 2009-03-19 15:32
At 11:03 AM -0400 3/19/09, Mitchell L Model wrote:
>>Martin v. Löwis <martin@v.loewis.de> added the comment:
>>
>>> The Python 3 "What's New" should SCREAM that the type file is gone
>>
>>I don't think the documentation should ever SCREAM.

No, of course not. I was being extreme. The problem with the laconic alternative
is that it sets traps for people. It reminds me of wrestling Stroustrop's first book
on C++ where all this weird behavior happened and when you went back to the
book you could find one sentence whose third-level implication was consistent
with the explanation. What I'm saying is that it's not enough to say to use open().
Don't you think people ever referred to the type file directly? As in help(file) or
dir(file) or file.somemethod(receiver, arg)? Or even storing a file method in a
dictionary for dispatch? It doesn't matter whether any of these were a good idea,
I claim they were quite ingrained in Python 2 user's minds. The changes in the
sequence types are quite substantial, but they are explained in detail. This isn't.

In reviewing the "What's New" for the purpose of this reponse I noticed an
earlier mention of sys.stdin, sys.stdout, and sys.stderr, where it says they
are now instances of TextIOBase. (They are actually instances of TextIOWrapper,
just like the result of open().) So why can't the documentation of open() say
that it returns an instance of TextIOBase too?

At least add to the paragraph of PEP 3116 that "the old file type is gone".
Maybe that's better than discussing it in conjunction with open(), in which
case the statement using open() instead of file should read that open replaces
"file()", not "file".

It just seems like a very wide trap to never mention that the whole file type is gone.
It does not follow from the two facts that one should use open() instead of file()
and that the IO system has been rewritten, that there is no file type. Even if
whatever open returns has the same interface as the old file type.

>>
>> >     "The type file has been replaced by _ioTextIOWrapper; use open() to
>> > open a file; open returns an instance of _ioTextIOWrapper."
>>
>>That would be an incorrect statement: there are multiple types that
>>replace the 2.x file type. See the documentation of the io module for
>>details.

OK, I didn't follow the technical details through and, as I said above, I hadn't
noticed the earlier mention of TextIOBase for sys.stdout, etc. So just say
something like "The type file is gone, replaced by classes in the IO module.
Use open() to open a file." And maybe add that it returns an instance of
TextIOBase or TextIOWrapper, whichever is deemed more appropriate.

Whatever else it says with regard to the above comments the "What's New"
really needs to explicitly say that the file type is gone.

And I apologize for screaming in my entry. I was just so incredibly shocked when
I figured out what was going on.
-- 

        --- Mitchell
msg83813 - (view) Author: David W. Lambert (LambertDW) Date: 2009-03-19 15:36
#OOPS!  I forgot the subtlety.
#I must also retain the stream
#else it gets collected.
#Nasty.


import io

class file(io.TextIOWrapper):

    '''condensing code for this list without test is a no no!'''

    def __init__(self,name):
        self.stream = open(name)   # SAVE THE STREAM!
        super().__init__(self.stream.buffer)
msg83823 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-03-19 22:38
> Don't you think people ever referred to the type file directly? As in help(file)

Yes, certainly. There is no immediate replacement for that; read the io
documentation.

> dir(file)

Likewise

> file.somemethod(receiver, arg)?

No. Why would anybody want to do that?

> Or even storing a file method in a dictionary for dispatch?

Likewise: what's the use case? Are you sure people store unbound methods
in a dictionary? Why not bound methods?

> I claim they were quite ingrained in Python 2 user's minds.

Partially, perhaps. It's an unfortunate mistake that file ever became
a builtin - that should have been avoided (and py3k corrects it).

> In reviewing the "What's New" for the purpose of this reponse I noticed an
> earlier mention of sys.stdin, sys.stdout, and sys.stderr, where it says they
> are now instances of TextIOBase. (They are actually instances of TextIOWrapper,
> just like the result of open().) 

Notice that sys.stdin/stdout/stderr are indeed instances of TextIOBase:

py> import sys,io
py> isinstance(sys.stdout, io.TextIOBase)
True

> So why can't the documentation of open() say
> that it returns an instance of TextIOBase too?

Because it would be incorrect to say so:

py> isinstance(open("/etc/passwd","rb"), io.TextIOBase)
False

> It just seems like a very wide trap to never mention that the whole file type is gone.

Well, Raymond promised to address this issue in his rewrite.

> OK, I didn't follow the technical details through and, as I said above, I hadn't
> noticed the earlier mention of TextIOBase for sys.stdout, etc. So just say
> something like "The type file is gone, replaced by classes in the IO module.
> Use open() to open a file." 

That would be correct to say. We'll see what Raymond comes up with.

> And maybe add that it returns an instance of
> TextIOBase or TextIOWrapper, whichever is deemed more appropriate.

Neither. It would be correct to say that it returns an instance of
io.IOBase (which isn't really a class, but an ABC).
msg83824 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-03-19 22:48
> class file(io.TextIOWrapper):
> 
>     '''condensing code for this list without test is a no no!'''
> 
>     def __init__(self,name):
>         self.stream = open(name)   # SAVE THE STREAM!
>         super().__init__(self.stream.buffer)

I don't know what this is supposed to achieve, but it looks incorrect.
I would write it as

py> class file(io.TextIOWrapper):
...   def __init__(self, name):
...     super().__init__(io.BufferedReader(io.FileIO(name, "r")))
...

Your version creates a separate TextIOWrapper for no apparent reason.
msg83827 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2009-03-19 23:01
It might be helpful to provide a table in the open() docs saying what
classes in io exactly are returned for different modes.
msg83830 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2009-03-19 23:15
Unassigning.  This appears to have evolved beyond the original request.
msg83854 - (view) Author: David W. Lambert (LambertDW) Date: 2009-03-20 06:00
My file class extends text files with seek or read through condition or 
pattern, providing an awk like pattern{action} task separation.

If I allow a pre-existing stream into my constructor (subprocess.Popen 
my favorite) I still suffer the same garbage collection problem.

class file(io.TextIOWrapper):
    'add condition matching to a stream'
    def __init__(self,stream_or_name):
        a = stream_or_name
        buffer = (a.buffer if isinstance(a, io.TextIOWrapper)
               else io.BufferedReader(io.FileIO(a, 'r')))
        super().__init__(buffer)

Use this on a stream whose reference count goes to zero causes
ValueError: I/O operation on closed file.  Increasing stream's reference 
count by saving it with the object corrects it.

I appreciate your considerations.
Dave.
msg83855 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-03-20 06:10
> I appreciate your considerations.

David, this discussions is certainly out of scope for this issue
(which is about the "What's new" document), so I will not consider
it further.
msg83996 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2009-03-23 02:50
Fixed in r70536.
History
Date User Action Args
2022-04-11 14:56:46adminsetgithub: 49763
2009-03-23 02:50:10benjamin.petersonsetstatus: open -> closed
resolution: fixed
messages: + msg83996
2009-03-20 06:10:30loewissetmessages: + msg83855
2009-03-20 06:00:51LambertDWsetmessages: + msg83854
2009-03-19 23:15:59rhettingersetassignee: rhettinger ->
messages: + msg83830
2009-03-19 23:01:54benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg83827
2009-03-19 22:48:00loewissetmessages: + msg83824
2009-03-19 22:38:37loewissetmessages: + msg83823
title: "What's New" should say VERY CLEARLY that the type file is gone -> "What's New" should say VERY CLEARLY that the type file is gone
2009-03-19 15:36:51LambertDWsetmessages: + msg83813
2009-03-19 15:32:35MLModelsetmessages: + msg83812
title: "What's New" should say VERY CLEARLY that the type file is gone -> "What's New" should say VERY CLEARLY that the type file is gone
2009-03-19 14:30:16LambertDWsetnosy: + LambertDW
messages: + msg83809
2009-03-19 06:26:31loewissetnosy: + loewis
title: "What's New" should say VERY CLEARLY that the type file is gone -> "What's New" should say VERY CLEARLY that the type file is gone
messages: + msg83801
2009-03-18 22:30:46rhettingersetassignee: georg.brandl -> rhettinger

messages: + msg83784
nosy: + rhettinger
2009-03-18 22:26:15MLModelcreate