This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: IO buffering behaviour not properly documented
Type: Stage:
Components: Documentation, IO Versions: Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: pitrou Nosy List: georg.brandl, pakal, pitrou
Priority: normal Keywords: patch

Created on 2009-12-19 15:59 by pakal, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
io-open-doc.patch pitrou, 2009-12-19 18:49
Messages (7)
msg96607 - (view) Author: Pascal Chambon (pakal) * Date: 2009-12-19 15:59
Hello,
It seems there is an important difference between the doc of the IO
module, and its implementation so far (until todcay trunk revision 76805)

"buffering is an optional integer used to set the buffering policy. By
default full buffering is on. Pass 0 to switch buffering off (only
allowed in binary mode), 1 to set line buffering, and an integer > 1 for
full buffering."
--> actually full buffering only occurs if a negative buffering
parameter is given, else it seems th current value i kept as the buffer
size - eg. if we give "3", buffering will be 3 bytes...
This seems a fine behaviour to me, so this implementation could just be
documented as is.

-----------
Only case of full buffering in the C iomodule :
    if (buffering < 0) {
        buffering = DEFAULT_BUFFER_SIZE;
#ifdef HAVE_STRUCT_STAT_ST_BLKSIZE
        {
            struct stat st;
            long fileno;
            PyObject *res = PyObject_CallMethod(raw, "fileno", NULL);
            if (res == NULL)
                goto error;

            fileno = PyInt_AsLong(res);
            Py_DECREF(res);
            if (fileno == -1 && PyErr_Occurred())
                goto error;

            if (fstat(fileno, &st) >= 0)
                buffering = st.st_blksize;
        }
#endif
    }
msg96608 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-12-19 16:08
"Full buffering" means exactly what you discovered it means - enable a
buffer of a given number of bytes (3, 4096 or anything else). I'm not
sure what you thought it meant? That the file was buffered in its
entirety, regardless of its size?
msg96616 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2009-12-19 17:12
The docs have a different wording, which I suggest copying to the docstring:

   *buffering* is an optional integer used to set the buffering policy.  By
   default full buffering is on.  Pass 0 to switch buffering off (only
allowed
   in binary mode), 1 to set line buffering, and an integer > 1 to
indicate the
   size of the buffer.

But now the question remains what the default is -- "full buffering" is
only meaningful with a specified buffer size.  The implementation seems
to default to line buffering.
msg96630 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-12-19 18:07
> But now the question remains what the default is -- "full buffering" is
> only meaningful with a specified buffer size.  The implementation seems
> to default to line buffering.

"full" buffering actually uses a default or custom buffer size when you
don't specify it. 4096 or 8192 usually (yes there's a heuristic :-)).

"full" buffering is the default for binary streams, as well as for text
streams which aren't a tty. text streams which are tty default to line
buffering.

(I admit full buffering is a confusing name; what could we use instead?
fixed-size buffering? chunk buffering?)
msg96640 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-12-19 18:49
Here is a possible patch for the Doc. I suppose the docstrings need
updating too?
msg96644 - (view) Author: Pascal Chambon (pakal) * Date: 2009-12-19 20:38
Yep, I knew "full buffering" didn't mean "fill my RAM until crash" :p 
The only visible problem was the interpretation of positive/negative
buffering value, which wasn't the same in doc and in code. But the patch
seems to fix up the plot prettily well B-)
Thanks
msg96647 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-12-19 21:10
Thank you, fixed in a lot of revisions.
History
Date User Action Args
2022-04-11 14:56:55adminsetgithub: 51794
2009-12-19 21:10:17pitrousetstatus: open -> closed
resolution: fixed
messages: + msg96647
2009-12-19 20:38:35pakalsetmessages: + msg96644
2009-12-19 18:49:34pitrousetfiles: + io-open-doc.patch
keywords: + patch
messages: + msg96640
2009-12-19 18:07:15pitrousetmessages: + msg96630
2009-12-19 17:12:05georg.brandlsetassignee: georg.brandl -> pitrou
messages: + msg96616
2009-12-19 16:08:55pitrousetnosy: + pitrou
messages: + msg96608
2009-12-19 15:59:10pakalcreate