print() function for some reason buffers output on Windows if end!='\n'. 

In attached examples "Processing.. " string is shown in Python 3.2 only after the actual processing is finished, while in Python 2.6 it is shown before.

printtest2.py displays directly "Processing.. " on Windows, but not on Linux. It looks like stdout is not buffered on Windows, which looks like a bug to bug :-) I think that it is safer to always call sys.stdout.flush() to ensure that your message is directly displayed. With Python 2, you can use -u flag (unbuffered output) to avoid the explicit flush, but this is very inefficient (slow).

Python 3 uses line buffers, even with python3 -u, for better performances. If you want to see directly "Processing.. ", as Python 2, call sys.stdout.flush().

It is not a regression, it is a choice to be efficient.

From my perspective it is a regression on Windows and a bug in Linux version of Python 2.x, which unfortunately can not be fixed, because of 2.x release process.

If the fact that print statement doesn't output anything when called is not a bug - then should be documented long ago.

> From my perspective it is a regression on Windows and a bug in Linux
> version of Python 2.x, which unfortunately can not be fixed,
> because of 2.x release process.

Line buffering is used by default on most operating systems (ok, maybe not Windows, which looks strange to me) and is not specific to Python.

Yes, Python has subtle differences on different operating systems, but at least it looks like Python 3 has the same behaviour on Linux and Windows ;-)

> If the fact that print statement doesn't output anything when called
> is not a bug - then should be documented long ago.

Can you please write a patch for the doc? Reopen the issue if you have a patch.

How about making print() user-friendly with flushing after every call, and if you want explicitly want speed - use buffered sys.stdout.write/flush()?

> How about making print() user-friendly with flushing after every call,
> and if you want explicitly want speed - use buffered
> sys.stdout.write/flush()?

This is exactly the -u option of Python 2: use it if you would like a completly unbuffered sys.stdout in a portable way.

In Python 3, it is only useful to flush at each line, even if the output is not a console, but a pipe (e.g. output redirected to a file). But nobody asked yet to have a fully unbuffered sys.stdout in Python 3.

You must realize that the most common use case for print(..., end!='\n') is when you want to notify user about intermediate progress of a very long operation.

Making documentation for simple print() statement overloaded with low level buffering details makes language seem overly complicated for new users.

We are talking about different things here:

- When python is run from a console, sys.stdout is line buffered. sys.stdout.write() flushes if there is a carriage return. No need to change anything here.

- print() could call file.flush() if file.isatty(), *after* the multiple calls to file.write().

Python 3.2, WinXP, IDLE edit window, F5 Run:
'Processing ...' appears immediately, 'Done' 3 sec later.
The only difference between printtest2/3 is where 'Done' appears.

Behavior is same when pasting into interactive interpreter -- has to be since the first two prints are executed before the sleep. I presume interpreter flushes before or after printing next prompt. IDLE must do same even when running from editor even when not printing prompts.

Anatoly, I gather you ran files by some other method.

I disagree with closing this yet. Print was designed to be a simplified wrapper around sys.stdout.write (and now, any file.write). Requiring that one import sys to use print goes against that.

I have always experienced and expected Python's print to screen to be immediately visible. I thought that was pretty standard in other languages with a print-to-screen separate from general file-write. When printing to screen, efficiency is, I believe, a non-issue. Writing to files or the net is a whole different ballgame.

Amaury's idea of checking isatty seems good. Otherwise, print should gain an optional, no-default flush=True/False parameter (no default because behavior currently varies.) Until then, non-flushing *should* be documented. The current doc for print does not seem to address the issue either way.

Amaury, I though 'run from console' more or less meant 'isatty', so I am not sure I understand your comment.

The attached patch calls "if file.isatty(): file.flush()" at the end of the print function:
- only when an "end" argument was specified
- errors in file.isatty() are ignored (and then no flush occurs)

> Python 3.2, WinXP, IDLE edit window, F5 Run:
> 'Processing ...' appears immediately, 'Done' 3 sec later.

Terry, IDLE is completely different, its sys.stdout completely bypasses the new io stack, and there is no buffering...

amaury> When python is run from a console, sys.stdout is line buffered.
amaury> sys.stdout.write() flushes if there is a carriage return.
amaury> No need to change anything here.

Anatoly would like a flush after all calls to print().

> print() could call file.flush() if file.isatty(), *after* the multiple
> calls to file.write().

I vote +0 to change print(), call sys.stdout.flush(), if:

 - file option is not used (and so, sys.stdout is used)
 - sys.stdout is a TTY
 - end option is used (fast heuristic to check if print will write a newline or not, a better one whould be to check if end contains a newline character or not, but we had to check for \n and/or \r, for a little gain)

But I don't want to change print() for print(text, file=file), because it would make Python slower and print(... file=file) is not used to an interactive prompt or to display informations to the user.

> Behavior is same when pasting into interactive interpreter ...
> I presume interpreter flushes before or after printing next prompt.

Did you wrote all commands on the same line? Python does change stdout buffer in interactive mode:
------------
    if (Py_UnbufferedStdioFlag) {
#ifdef HAVE_SETVBUF
        setvbuf(stdin,  (char *)NULL, _IONBF, BUFSIZ);
        setvbuf(stdout, (char *)NULL, _IONBF, BUFSIZ);
        setvbuf(stderr, (char *)NULL, _IONBF, BUFSIZ);
#else /* !HAVE_SETVBUF */
        setbuf(stdin,  (char *)NULL);
        setbuf(stdout, (char *)NULL);
        setbuf(stderr, (char *)NULL);
#endif /* !HAVE_SETVBUF */
    }
    else if (Py_InteractiveFlag) {
#ifdef MS_WINDOWS
        /* Doesn't have to have line-buffered -- use unbuffered */
        /* Any set[v]buf(stdin, ...) screws up Tkinter :-( */
        setvbuf(stdout, (char *)NULL, _IONBF, BUFSIZ);
#else /* !MS_WINDOWS */
#ifdef HAVE_SETVBUF
        setvbuf(stdin,  (char *)NULL, _IOLBF, BUFSIZ);
        setvbuf(stdout, (char *)NULL, _IOLBF, BUFSIZ);
#endif /* HAVE_SETVBUF */
#endif /* !MS_WINDOWS */
        /* Leave stderr alone - it should be unbuffered anyway. */
    }
#ifdef __VMS
    else {
        setvbuf (stdout, (char *)NULL, _IOLBF, BUFSIZ);
    }
#endif /* __VMS */
------------
(it doesn't check if stdout is a TTY or not, but I don't think that it is very useful to use the interactive mode outside a TTY)

> I have always experienced and expected Python's print to screen
> to be immediately visible. I thought that was pretty standard
> in other languages with a print-to-screen separate from
> general file-write.

Did you try Perl, Ruby, bash and other languages? I know that at least the C language requires an explicit call to fflush(stdout). I always used that.

> Terry, IDLE is completely different, its sys.stdout completely
> bypasses the new io stack, and there is no buffering...

As I wrote: "unbuffered mode" is not implemented for TextIOWrapper. So even with python3 -u, sys.stdout.write("abc") doesn't flush immediatly into the underlying FileIO.

I completely agree that file/socket output should be left alone. Flushing char by char to either is a bit insane. The two interactive to screen use cases I can think of are text progress meters, mentioned by Anatoly, such as :
  Working .... (1 dot printed at intervals)
and timed text like

import time
for c in 'Similated 10 cps teletype output':
    print(c,end='')
    time.sleep(.1)
print()

which works fine from IDLE and whose non-functioning when started otherwise would puzzle any beginner and many beyond.

I've tried to switch to Python 3 once more and stumbled upon this problem once more.

Seems like this regression got stale. Last Victor's proposal seems reasonable for me. Should we open a new, more clear bug report and close this one?

No, I don't think so. Another issue will not magically create more time for anyone.

On Mon, Jan 9, 2012 at 2:03 PM, Terry J. Reedy <report@bugs.python.org>wrote:

>
> Terry J. Reedy <tjreedy@udel.edu> added the comment:
>
> No, I don't think so. Another issue will not magically create more time
> for anyone.
>

But anyone will waste less time to get to the outcome of discussion.
-- 
anatoly t.

> You must realize that the most common use case for print(..., end!='\n') is when you want
> to notify user about intermediate progress of a very long operation.

References needed.

> Making documentation for simple print() statement overloaded with low level buffering details makes language seem overly complicated for new users.

Why don't anybody require references for that?

The current doc says

"print([object, ...], *, sep=' ', end='\n', file=sys.stdout) 
Print object(s) to the stream file, separated by sep and followed by end. sep, end and file, if present, must be given as keyword arguments.

All non-keyword arguments are converted to strings like str() does and written to the stream, separated by sep and followed by end. Both sep and end must be strings; they can also be None, which means to use the default values. If no object is given, print() will just write end.

The file argument must be an object with a write(string) method; if it is not present or None, sys.stdout will be used."

(The bit about None, said twice, could be factored out and said once after the second sentence.)

This is exactly what print does and Guido today (Python ideas) said that is what it should do and that "Apps that need flushing should call flush()." So a code change is rejected.

The issue title was incorrect. The print function does not do any buffering. The file object it writes to may. Even sys.stdout may or may not. 

We could add at the end a sentence or two something like

"Output buffering is determined by *file*. Call file.flush() to ensure, for instance, immediate appearance on a screen."

New changeset bc043cef94f2 by Terry Jan Reedy in branch '3.2':
Closes #11633 Clarify print buffering.
http://hg.python.org/cpython/rev/bc043cef94f2

New changeset fb0d61fd1753 by Terry Jan Reedy in branch 'default':
Merge with 3.2
http://hg.python.org/cpython/rev/fb0d61fd1753

#13761 proposes to add flush=False param with option for True.

I agree with the python-ideas message that ``sys.stdout.flush()`` is surprising / possibly misleading and should be ``file.flush()``.  If the other bug report about adding a flush argument is rejected, please consider this.  Thanks :)

New changeset 4a767054551b by Terry Jan Reedy in branch '3.2':
#11633 At least 2 people prefer earlier revision.
http://hg.python.org/cpython/rev/4a767054551b

New changeset 22688f5f9d0f by Terry Jan Reedy in branch 'default':
Merge #11633 At least 2 people prefer earlier revision.
http://hg.python.org/cpython/rev/22688f5f9d0f

Thank you sir.  Should the doc edit be backported to the 2.7 docs, with a mention that it’s only on unix?

Putting the wording into 2.7 might be nice, but I thought it was in bugfix only mode.

Regarding UNIX only, I'd avoid it; any file may be buffered in almost any way on any platform. Saying an explicit flush call may be necessary for immediate output is _not_ UNIX only and would be very misleading. Remembering that ~UNIX != Windows.

Telling users to explicitly call flush to ensure immediate output where that is necessary ensures portable coding (or ought to, user pigheadedness discounted:-)

Bug fixes include doc improvements, so 2.7 is fair game.

Thanks for your suggestion to not mention specific platforms.  Let’s just backport the 3.2 text.

New changeset 8935a33773b9 by Terry Jan Reedy in branch '2.7':
#11633 about buffering of print
http://hg.python.org/cpython/rev/8935a33773b9