classification
Title: failure of test_colors_funcs in test_curses with ncurses 6.1
Type: behavior Stage:
Components: Tests Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Jeffrey.Kintscher, xdegaye, yan12125
Priority: normal Keywords:

Created on 2019-04-14 19:32 by xdegaye, last changed 2019-05-24 03:17 by Jeffrey.Kintscher.

Files
File name Uploaded Description Edit
extended_pair_content.c Jeffrey.Kintscher, 2019-05-20 01:07 ncurses extended_pair_content() test program
Messages (5)
msg340228 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2019-04-14 19:32
ncurses version: 6.1
TERM: screen-256color

$  ./python -m test -u curses test_curses
Run tests sequentially
0:00:00 load avg: 0.55 [1/1] test_curses
test test_curses failed -- Traceback (most recent call last):
  File "/path/to/Lib/test/test_curses.py", line 285, in test_colors_funcs
    curses.pair_content(curses.COLOR_PAIRS - 1)
OverflowError: signed short integer is greater than maximum

test_curses failed

== Tests result: FAILURE ==


Not sure if the following is relevant.

In /usr/include/ncurses.h:

    NCURSES_WRAPPED_VAR(int, COLOR_PAIRS);
    ...
    #define COLOR_PAIRS NCURSES_PUBLIC_VAR(COLOR_PAIRS())
    ...
    extern NCURSES_EXPORT_VAR(int) COLOR_PAIRS;

ncurses 6.1 release notes [1] says:

    The TERMINAL structure in <term.h> is now opaque. Doing that allowed making the structure larger, to hold the extended numeric data.
    ...
    The new data in TERMINAL holds the same information as TERMTYPE, but with larger numbers (“int” versus “short”). It is named TERMTYPE2.

[1] https://www.gnu.org/software/ncurses/
msg340253 - (view) Author: Chih-Hsuan Yen (yan12125) * Date: 2019-04-15 09:55
I asked on bug-ncurses mailing list and Thomas Dickey suggests "improving the python curses binding to handle the newer terminal descriptions". Looks like that requires non-trivial efforts. On the other hand, I've also found a workaround:

$ TERM=xterm python -m test -u curses -v test_curses

(or anything without -256color suffix for $TERM)

How about setting TERM=xterm in tests and documenting that CPython does not support new terminal descriptions for now?

[1] http://lists.gnu.org/archive/html/bug-ncurses/2019-04/msg00003.html
msg342885 - (view) Author: Jeffrey Kintscher (Jeffrey.Kintscher) * Date: 2019-05-20 01:07
The test fails because curses.pair_content(curses.COLOR_PAIRS-1) validates its parameter against the limits for signed short (max 32767) while curses.COLOR_PAIRS-1 has the value 65535.

Unfortunately, re-plumbing curses.pair_content() to use signed integers instead of signed shorts and replacing the underlying ncurses API call from pair_content() to extended_pair_content() doesn't fix the problem because extended_pair_content() still fails when passed 65535. Tracing into the ncurses 6.1 source code, I found that start_color() clamps the maximum number of color pairs at SHRT_MAX (32767) regardless of the number of color pairs supported by the terminal.
msg342956 - (view) Author: Jeffrey Kintscher (Jeffrey.Kintscher) * Date: 2019-05-20 21:40
I posted a bug report to the bug-ncurses mailing list:

https://lists.gnu.org/archive/html/bug-ncurses/2019-05/msg00022.html
msg343342 - (view) Author: Jeffrey Kintscher (Jeffrey.Kintscher) * Date: 2019-05-24 03:17
I created issue #36982 to track the extended color changes since they broader than this issue.
History
Date User Action Args
2019-05-24 03:17:43Jeffrey.Kintschersetmessages: + msg343342
2019-05-20 21:40:34Jeffrey.Kintschersetmessages: + msg342956
2019-05-20 01:07:14Jeffrey.Kintschersetfiles: + extended_pair_content.c
nosy: + Jeffrey.Kintscher
messages: + msg342885

2019-04-15 09:55:52yan12125setnosy: + yan12125
messages: + msg340253
2019-04-14 19:32:58xdegayecreate