classification
Title: curses addch broken on Python3.3a1
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Nicholas.Cole, cben, georg.brandl, gpolo, haypo, inigoserna, jcea, ned.deily, phep, pitrou, python-dev, r.david.murray
Priority: release blocker Keywords:

Created on 2012-03-07 20:44 by Nicholas.Cole, last changed 2012-09-09 20:12 by haypo. This issue is now closed.

Messages (19)
msg155118 - (view) Author: Nicholas Cole (Nicholas.Cole) Date: 2012-03-07 20:44
The following code works on Python versions prior to 3.3a1:

import curses

def test_screen(screen):
    screen.addch(5,5, curses.ACS_HLINE)
    screen.refresh()

curses.wrapper(test_screen)

On python3.3, the program produces the following traceback:


Traceback (most recent call last):
  File "/tmp/p.py", line 7, in <module>
    curses.wrapper(test_screen)
  File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/curses/__init__.py", line 94, in wrapper
    return func(stdscr, *args, **kwds)
  File "/tmp/p.py", line 4, in test_screen
    screen.addch(5,5, curses.ACS_HLINE)
OverflowError: byte doesn't fit in chtype
msg155125 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2012-03-07 22:08
Marking as release blocker since this is a regression.  Added people from the other curses issue as being likely to be interested in this one.
msg155142 - (view) Author: Roundup Robot (python-dev) Date: 2012-03-08 01:08
New changeset 861a5f3e7453 by Victor Stinner in branch 'default':
Close #14223: curses.addch() is no more limited to the range 0-255 when the
http://hg.python.org/cpython/rev/861a5f3e7453
msg155143 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2012-03-08 01:10
I introduced the arbitrary limit when I added support of the curses Unicode API (libncursesw). The limit is useless if libncursesw is used, whereas it blocks legal "characters" like KEY_xxx constants. So I just removed the check and it should not be ok.

The real issue is that Python curses is not linked to libncursesw on Mac OS X!
msg169579 - (view) Author: Nicholas Cole (Nicholas.Cole) Date: 2012-08-31 20:19
I'm reopening this bug because I've noticed that in Python3.3rc1, although trying to print curses.ACS_HLINE and other such characters no long cause an Exception, only blank characters are printed to the screen.

Tested with both Terminal.App and xterm running under XQuartz.

Here is some minimal test code, that will print a line on Python2.x but a blank screen on Python3.3rc1.
msg169580 - (view) Author: Nicholas Cole (Nicholas.Cole) Date: 2012-08-31 20:20
import curses

def test_screen(screen):
    screen.addch(5,5, curses.ACS_HLINE)
    screen.addch(5,6, curses.ACS_HLINE)
    screen.refresh()
    curses.napms(2000)

curses.wrapper(test_screen)
msg169628 - (view) Author: Roundup Robot (python-dev) Date: 2012-09-01 13:08
New changeset 27b5bd5f0e4c by Victor Stinner in branch 'default':
Close #14223: Fix window.addch(curses.ACS_HLINE)
http://hg.python.org/cpython/rev/27b5bd5f0e4c
msg169630 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2012-09-01 13:24
> I'm reopening this bug because I've noticed that in Python3.3rc1,
> although trying to print curses.ACS_HLINE and other such characters
> no long cause an Exception, only blank characters are printed to
> the screen.

If the Python curses module is compiled in Unicode mode (if curses.unget_wch() is present), addch(int) calls the C function wadd_wch() or mvwadd_wch(), instead of waddch() or mvwaddch(). It looks like the *_wch() family does not support characters like curses.ACS_HLINE.

My commit fixes the issue.

@Nicholas.Cole: Can you please try the last development version (default branch)?

@georg.brandl: Can you please include the important fix 27b5bd5f0e4c in Python 3.3 final?
msg169631 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2012-09-01 13:25
(keep the issue open until the fix is included in Georg's repository)
msg169752 - (view) Author: Nicholas Cole (Nicholas.Cole) Date: 2012-09-03 08:48
Unless I'm getting the build process wrong (possible, because I haven't tried testing fixes before), this fix isn't working for me.
msg169756 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2012-09-03 12:02
> Unless I'm getting the build process wrong (possible, because
> I haven't tried testing fixes before), this fix isn't working for me.

Oh did you compile Python? Did you install it?

Do you have libncursesw? => do you have the function curses.unget_wch?

What is the version of your libcurses[w] library? There are issues on libncurses 5.7. I think Ned Bat changed the script to compile Python on Mac to use a builtin copy of libncurses 5.9. See:
http://bugs.python.org/issue14225#msg163323
msg169759 - (view) Author: Nicholas Cole (Nicholas.Cole) Date: 2012-09-03 12:57
> Unless I'm getting the build process wrong (possible, because
> I haven't tried testing fixes before), this fix isn't working for me.

Oh did you compile Python? Did you install it?

Yes, I tried compiling it.  

#./configure MACOSX_DEPLOYMENT_TARGET=10.8 --enable-framework --with-universal-archs="64-bit" CFLAGS="-arch x86_64" LDFLAGS="-arch x86_64" && make -j6 && make install

Do you have libncursesw? => do you have the function curses.unget_wch?

yes...

Python 3.3.0rc1+ (default, Sep  3 2012, 09:45:35) 
[GCC 4.2.1 Compatible Apple Clang 4.0 ((tags/Apple/clang-421.0.57))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import curses
>>> curses.unget_wch()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
_curses.error: must call initscr() first
>>> curses.version
b'2.2'
>>> 


What is the version of your libcurses[w] library? There are issues on libncurses 5.7. I think Ned Bat changed the script to compile Python on Mac to use a builtin copy of libncurses 5.9. See:
http://bugs.python.org/issue14225#msg163323

How do I check which version it was built against?  The messages from ./configure just say that the tests pass, not which version it has found.
msg169764 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2012-09-03 14:15
> How do I check which version it was built against?

On Linux, I use:

$ ldd /usr/lib/python2.7/lib-dynload/_curses.so |grep curses
        libncursesw.so.5 => /lib/i386-linux-gnu/libncursesw.so.5 (0xb76d0000)
$ ls -l /lib/i386-linux-gnu/libncursesw.so.5
lrwxrwxrwx 1 root root 18 nov.  18  2011
/lib/i386-linux-gnu/libncursesw.so.5 -> libncursesw.so.5.9

=> version 5.9
msg169765 - (view) Author: Nicholas Cole (Nicholas.Cole) Date: 2012-09-03 14:36
On Mon, Sep 3, 2012 at 3:15 PM, STINNER Victor <report@bugs.python.org> wrote:
>
> STINNER Victor added the comment:
>
>> How do I check which version it was built against?
>
> On Linux, I use:
>
> $ ldd /usr/lib/python2.7/lib-dynload/_curses.so |grep curses
>         libncursesw.so.5 => /lib/i386-linux-gnu/libncursesw.so.5 (0xb76d0000)
> $ ls -l /lib/i386-linux-gnu/libncursesw.so.5
> lrwxrwxrwx 1 root root 18 nov.  18  2011
> /lib/i386-linux-gnu/libncursesw.so.5 -> libncursesw.so.5.9
>
> => version 5.9

I get Version 5.4, which seems very out of date (I'm running the
latest release of OS X) but:

nicholas$ ls -l /usr/lib/libncurses.
libncurses.5.4.dylib  libncurses.5.dylib    libncurses.dylib
NPSC:mnpyscreen nicholas$ ls -l /usr/lib/libncurses.dylib
lrwxr-xr-x  1 root  wheel  20 25 Jul 18:32 /usr/lib/libncurses.dylib
-> libncurses.5.4.dylib

nicholas$ ldd /usr/lib/python2.7/lib-dynload/_curses.so |grep curses
/usr/lib/python2.7/lib-dynload/_curses.so:
	/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current
version 5.4.0)

Can that possibly be right??
msg169781 - (view) Author: Ned Deily (ned.deily) * (Python committer) Date: 2012-09-03 16:57
> Can that possibly be right??

Yes. That's the version of ncurses that Apple ships with current versions of OS X. You can build it yourself and install into /usr/local and rebuild python.  For ncurses 5.9, something like:

./configure  --enable-widec --enable-shared --with-shared --without-normal
&& make && sudo make install

should work.  FWIW, when I run your test script with tip, I now do see the two horizontal bars.
msg170032 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2012-09-08 05:53
Done in d6d632f254ee.
msg170083 - (view) Author: Roundup Robot (python-dev) Date: 2012-09-09 09:18
New changeset d6d632f254ee by Victor Stinner in branch 'default':
Close #14223: Fix window.addch(curses.ACS_HLINE)
http://hg.python.org/cpython/rev/d6d632f254ee
msg170118 - (view) Author: Nicholas Cole (Nicholas.Cole) Date: 2012-09-09 17:24
I've just tried the OS X build of 3.3rc2, and this has been fixed.  Thank you!
msg170132 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2012-09-09 20:12
> I've just tried the OS X build of 3.3rc2, and this has been fixed.  Thank you!

Great!
History
Date User Action Args
2012-09-09 20:12:34hayposetmessages: + msg170132
2012-09-09 17:24:39Nicholas.Colesetmessages: + msg170118
2012-09-09 09:18:51python-devsetmessages: + msg170083
2012-09-08 05:53:42georg.brandlsetstatus: open -> closed
resolution: fixed
messages: + msg170032
2012-09-03 16:57:50ned.deilysetmessages: + msg169781
2012-09-03 14:36:35Nicholas.Colesetmessages: + msg169765
2012-09-03 14:15:01hayposetmessages: + msg169764
2012-09-03 12:57:33Nicholas.Colesetmessages: + msg169759
2012-09-03 12:31:27zehasetnosy: - zeha
2012-09-03 12:02:32hayposetnosy: + ned.deily
messages: + msg169756
2012-09-03 10:43:13schodetsetnosy: - schodet
2012-09-03 08:48:20Nicholas.Colesetmessages: + msg169752
2012-09-01 13:25:03hayposetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg169631
2012-09-01 13:24:08hayposetnosy: + georg.brandl
messages: + msg169630
2012-09-01 13:08:56python-devsetstatus: open -> closed
resolution: fixed
messages: + msg169628
2012-08-31 20:20:27Nicholas.Colesetmessages: + msg169580
2012-08-31 20:19:55Nicholas.Colesetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg169579
2012-03-08 01:10:59hayposetmessages: + msg155143
2012-03-08 01:08:48python-devsetstatus: open -> closed
resolution: fixed
messages: + msg155142

stage: needs patch -> resolved
2012-03-07 22:08:20r.david.murraysetpriority: normal -> release blocker

nosy: + jcea, cben, pitrou, haypo, gpolo, r.david.murray, inigoserna, phep, zeha, schodet, python-dev
messages: + msg155125

type: behavior
stage: needs patch
2012-03-07 20:44:53Nicholas.Colecreate