classification
Title: Buffering bug after calling curses function
Type: behavior Stage:
Components: Interpreter Core Versions: Python 2.7
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: Trundle, Wes.McKinney, akuchling, fperez, ned.deily, pitrou
Priority: normal Keywords:

Created on 2010-10-19 04:05 by Wes.McKinney, last changed 2010-10-19 22:02 by ned.deily. This issue is now closed.

Messages (5)
msg119115 - (view) Author: Wes McKinney (Wes.McKinney) Date: 2010-10-19 04:05
We tracked a bug originating in IPython to the Python interpreter itself, seems to be present in 2.6.x and 2.7.x but not 3.1.x. This is on Ubuntu Linux 10.04, does not seem to occur in OS X 10.6.

Reference: http://article.gmane.org/gmane.comp.python.ipython.user/5336

> cat bufferbug.py
"""Strange bug in buffering of sys.stdout after calling curses functions.
"""
import time
import curses

def bug():
   curses.initscr()
   curses.endwin()

def f(n=2):
   s = 0.75
   for i in range(n):
       print i
       time.sleep(s)
   print i+1

if __name__ == '__main__':
   f()
   print 'Calling bug() now!'
   bug()
   f()

####
msg119161 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2010-10-19 20:07
The problem is reproducible with the python2 versions I have access to, that is, on Mac OS X and Debian Linux, and likewise not with any of the python3 3.1 and 3.2 versions.  It is most likely due to the underlying ncurses library interacting with the libc stdio, maybe changing the buffer size of the stdout file. The output is flushed as expected if the program is changed to use the python2.7 io module to write to stdout, like python3 does, rather than print which uses the 'file' object.  As a workaround on python2, it looks like the expected behavior can be restored by closing and reopening sys.stdout with something like this:

   f()
   stdout=sys.stdout
   print('Calling bug() now!')
   bug()
   fn = sys.stdout.fileno()
   sys.stdout.close()
   sys.stdout = os.fdopen(fn,'a')
   f()
msg119163 - (view) Author: Andreas Stührk (Trundle) Date: 2010-10-19 20:26
That behaviour is indeed caused by ncurses as it changes the buffer size of stdio. The rationale behind that is explained in a comment in ncurses/tinfo/setbuf.c (http://codesearch.google.com/codesearch/p?hl=en#5KTrgOW2hXs/pub/nslu2/sources/ncurses-5.4.tar.gz%7CqZkYBenqdbw/ncurses-5.4/ncurses/tinfo/setbuf.c&l=46)

That behaviour is also mentioned in the man-page of ncurses and one can disable it using the NCURSES_NO_SETBUF environment variable. If I set that environment variable, the example behaves as expected on my machine.
msg119164 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2010-10-19 20:44
Thanks for NCURSES_NO_SETBUF ("when it doubt read the man page"!).  Since the problem does not seem to exist in python3 and this is likely been an issue for a long time and apparently not previously reported and there are various workarounds, I think this could be closed as "wont fix".  Any objections?
msg119170 - (view) Author: Fernando Perez (fperez) Date: 2010-10-19 21:47
No problem for us (IPython) if you mark it as won't fix. I've just applied the environment workaround you guys suggested:

http://github.com/ipython/ipython/commit/147b245d2ead0e15d2c17b7bb760a03126660fb7

Thanks a lot for that tip!  That leaves us in good shape.
History
Date User Action Args
2010-10-19 22:02:17ned.deilysetstatus: open -> closed
resolution: wont fix
2010-10-19 21:47:08fperezsetnosy: + fperez
messages: + msg119170
2010-10-19 20:44:11ned.deilysetmessages: + msg119164
2010-10-19 20:26:41Trundlesetnosy: + Trundle
messages: + msg119163
2010-10-19 20:07:50ned.deilysetnosy: + akuchling, pitrou, ned.deily

messages: + msg119161
versions: - Python 2.6
2010-10-19 04:05:46Wes.McKinneycreate