classification
Title: Having escape sequences (like color codes) in the sys.ps1 messes up readline's input line calculations
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.3, Python 2.7
process
Status: closed Resolution: works for me
Dependencies: Superseder:
Assigned To: Nosy List: georg.brandl, lonetwin
Priority: normal Keywords:

Created on 2014-01-23 09:09 by lonetwin, last changed 2014-01-23 14:43 by lonetwin. This issue is now closed.

Messages (7)
msg208890 - (view) Author: Steve (lonetwin) Date: 2014-01-23 09:09
If you change the sys.ps1 to have some color, you end up messing up readline's input line calculations and the characters in the line you are typing might not get displayed properly. This behaviour is easier to demonstrate/reporduce than explain ...:

On a python promt with readline enabled (for instance, on any linux box):

[steve@localhost ~]$ python
Python 2.7.5 (default, Nov 12 2013, 16:18:42) 
[GCC 4.8.2 20131017 (Red Hat 4.8.2-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.ps1 = '\033[31m>>> \033[0m'
>>> # doing a reverse-i-search (Ctrl+r) on the next line will mess up the prompt display
... 
>>> h)`sys': sys.ps1 = '\033[31m>>> \033[0m'
(...press Ctrl-L to clear...)
(reverse-i-search)`': (press enter)
>>> arch)`':

I did some (amateur) investigation and the root cause might be similar to what's reported at:
http://trac.haskell.org/haskeline/ticket/78
https://groups.google.com/forum/#!topic/gnu.bash.bug/5P0gWzzLVOU

FWIW, I bumped into this issue while customizing my .pythonrc[1] and although it might be a silly small annoyance, it would be really nice if this bug could be fixed.

[1] https://gist.github.com/lonetwin/5902720
msg208891 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-01-23 09:11
Thanks for the report.

I think it's your obligation to surround nonprinting escape sequences with \x01 and \x02, with that change it works for me.
msg208896 - (view) Author: Steve (lonetwin) Date: 2014-01-23 09:25
wow, that was super quick ! Yes, indeed. Surrounding the escape sequences with '\0x1' & '\0x2' fixes this issue. Thanks !
msg208897 - (view) Author: Steve (lonetwin) Date: 2014-01-23 09:26
...of course I meant \x01 and \x02, like you suggested
msg208905 - (view) Author: Steve (lonetwin) Date: 2014-01-23 10:10
Although surrounding the escapes with \x01 and \x02 worked to correctly remove the message while doing a reverse-i-search (thus preserving the original line), the input line measurement still appears to be incorrect as demonstated with the following actions:

[steve@localhost ~]$ python
Python 2.7.5 (default, Nov 12 2013, 16:18:42) 
[GCC 4.8.2 20131017 (Red Hat 4.8.2-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.ps1 = '\x01\033[32m>>> \033[0m\x02'
>>> # on the next line, press the up arrow to get this line and then backspace, over the entire prompt
... 
#<-- backspaced up until here
... 


...again, not a big deal, just annoying when deleting word with a repeated 'alt+backspace' or ctrl+w
msg208910 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2014-01-23 10:43
I think you're using the escapes correctly: you must only escape nonprinting characters. Try

>>> sys.ps1 = '\x01\033[32m\x02>>> \x01\033[0m\x02'
msg208948 - (view) Author: Steve (lonetwin) Date: 2014-01-23 14:43
Hi Georg,

Thanks again for the responses and your help. After a bit of research, I discovered the /reasons/ behind needing the \001 and \002 escapes. Thought I'd log the links here for posterity sake:

  - To color something in on a color capable terminal console you just need to use the "\033[<color code>m" escape sequence. This would be sufficient[1]
  - However readline messes up the line width calculation because it measures the escape sequences as a characters too. To avoid this you have to wrap the escape sequences within \001 and \002.[2]
  - On some terminal applications (like the one I am using - terminator[3]), if you add the \001 and \002 escapes to color text which is *not* interpreted by readline, (for instance if you have a single function to color text and you want to use it to color both your sys.ps1 and output text), the \001 and \002 codes will get printed out using a representation (like a unicode 'box'[4]). So, one would have to workaround that in the text coloring function.

[1] http://en.wikipedia.org/wiki/ANSI_escape_code#Colors
[2] bugs.python.org/issue17337/ and
    http://stackoverflow.com/questions/9468435/look-how-to-fix-column-calculation-in-python-readline-if-use-color-prompt
[3] http://gnometerminator.blogspot.sg/p/introduction.html
[4] http://en.wikipedia.org/wiki/Control_character#Display
History
Date User Action Args
2014-01-23 14:43:56lonetwinsetmessages: + msg208948
2014-01-23 10:43:45georg.brandlsetstatus: open -> closed

messages: + msg208910
2014-01-23 10:10:17lonetwinsetstatus: closed -> open

messages: + msg208905
2014-01-23 09:26:54lonetwinsetmessages: + msg208897
2014-01-23 09:25:49lonetwinsetmessages: + msg208896
2014-01-23 09:11:24georg.brandlsetstatus: open -> closed

nosy: + georg.brandl
messages: + msg208891

resolution: works for me
2014-01-23 09:09:10lonetwincreate