classification
Title: doctest pdb readline broken
Type: behavior Stage: patch review
Components: Library (Lib) Versions: Python 3.1, Python 3.2, Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: belopolsky, georg.brandl, ncoghlan, ping, sriram, wosc
Priority: normal Keywords: patch

Created on 2009-04-09 11:25 by wosc, last changed 2010-07-30 10:00 by georg.brandl. This issue is now closed.

Files
File name Uploaded Description Edit
doctest-readline.patch wosc, 2009-06-05 14:59
readline.patch sriram, 2009-12-27 18:33 svn diff for pdb.py and doctest.py (Python 2.6.4)
pdbreadline.patch sriram, 2010-04-27 17:52 svn diff of pdb, doctest and test_doctest
pdb_doctest_readline.patch sriram, 2010-05-04 18:39 svn diff of pdb, doctest and test_doctest
Messages (7)
msg85808 - (view) Author: Wolfgang Schnerring (wosc) Date: 2009-04-09 11:25
When pdb is called from inside a doctest under python2.5, the readline
keys do not work anymore -- like they did just fine in 2.4.

Steps to reproduce: 
1. Create two files, foo.txt and foo.py, like so:
$ cat > foo.txt
>>> import pdb; pdb.set_trace()
$ cat > foo.py
import doctest; doctest.testfile('foo.txt')
2. run it
$ python2.5 foo.py
3. If I now press Ctrl-P, I get "^P" printed on the prompt, instead of
going back in the readline history. Likewise, the arrow keys print
escape sequences instead of moving the cursor.
msg88948 - (view) Author: Wolfgang Schnerring (wosc) Date: 2009-06-05 14:59
I've tracked down the reason by diffing pdb.py and cmd.py between 2.4
and 2.5:
It turns out that pdb.Pdb in 2.5 changes the way it handles input
depending on whether an explicit output was provided, more precisely, it
disables readline in that case. I don't understand what's going on here,
but there is a simple, non-intrusive fix on the doctest side, see the
attached patch.

Unfortunately, I can't imagine how to write a test to check this behaviour.
msg92878 - (view) Author: Sriram (sriram) Date: 2009-09-19 21:13
Hi,

How about changing pdb's behavior, that it disables readline only if the 
passed stream is not the stdout stream?

Also when looking at doctest module, I found that bdb's trace_dispatch 
was overridden to set the debugger's output stream to something other 
than stdout. I presume, that since prior to 2.5, pdb never took the 
stdin and stdout arguments and that overridden method was needed to make 
pdb use a stream other than stdout, but now that 2.5 allows us to pass 
the output stream in init method of pdb, that function is no longer 
needed.

Thanks
msg96916 - (view) Author: Sriram (sriram) Date: 2009-12-27 18:33
Hi,

This is the first bug am working in python, kindly excuse my mistakes, 
if any.

As far as I can understand, the pdb disabled readline when an explicit 
stdin or stdout is passed, to allow remote debugging.

I found this in Python 2.5.4 Release log. 
"""
Patch #721464: pdb.Pdb instances can now be given explicit stdin and
  stdout arguments, making it possible to redirect input and output
  for remote debugging.
"""

Now in doctest.py since we pass the stdout argument (which is always 
sys.stdout) to pdb.py, readline is always disabled when pdb is invoked 
from doctest.py.

One fix I can think of is to have pdb disable use of readline, not if 
any output stream is passed but only if a output stream other than 
sys.stdout is passed. I infact believe, this will preserve the 
functionality of pdb.py that existed before release of version 2.5.4


The above fix would still not solve the problem because before we pass 
the output stream to pdb.py, we override sys.stdout in doctest to 
doctest's spoofout. So in pdb.py, sys.stdout will not point to real 
sys.stdout. This can be fixed by overriding sys.stdout after we 
initialize the debugger.

Please find the patch for doctest and pdb as an attachment. 

Thanks
Sriram
msg104340 - (view) Author: Sriram (sriram) Date: 2010-04-27 17:52
Hi,

I believe this behaviour can be tested if we can prove that Cmd's cmdloop uses raw_input to get the data as against self.stdin.readline().

To test it, ideally I would have liked to override sys.stdin with a fake input stream and pass the list of args (like it's done in test_doctest.py). I can then pass the character codes for Up and Down Arrows in our fake stream and check if the raw_input can use readline and move through the fake input stream but that won't be possible because the C implementation of raw_input uses the readline functionality if only both stdin and stdout are from a terminal.

So alternatively, if we can just ensure that doctest's pdb (_OutputREdirectingPdb) has use_rawinput as 1, we can be assured that readline will be used.

I have attached the svn diff with trunk. Please review and comment

Thanks
Sriram
msg104972 - (view) Author: Sriram (sriram) Date: 2010-05-04 18:39
Hi,

On second thoughts, it made more sense to validate pdb directly instead of validating doctest's debugger.

I have also used few inputs, I got from irc chat at #python-dev room in writing the test case. Thanks to them.

I have attached the svn diff.

Please review them

Thanks
Sriram
msg112058 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2010-07-30 10:00
Fixed in r83271.  I think the first patch's approach is better, since it does not affect pdb, only doctest's adaption of pdb.  Consequently, I couldn't use the test case; thanks anyway for that!
History
Date User Action Args
2010-07-30 10:00:20georg.brandlsetstatus: open -> closed
resolution: fixed
messages: + msg112058
2010-05-04 20:39:39belopolskysetnosy: + belopolsky
2010-05-04 18:52:52exarkunsetstage: test needed -> patch review
2010-05-04 18:39:42sriramsetfiles: + pdb_doctest_readline.patch

messages: + msg104972
2010-04-27 17:52:35sriramsetfiles: + pdbreadline.patch

messages: + msg104340
2010-02-20 12:35:44ezio.melottisetpriority: normal
nosy: + ping, georg.brandl, ncoghlan

versions: + Python 2.7, Python 3.2
2009-12-27 18:33:37sriramsetfiles: + readline.patch

messages: + msg96916
2009-09-19 21:13:27sriramsetnosy: + sriram
messages: + msg92878
2009-06-05 14:59:08woscsetfiles: + doctest-readline.patch
keywords: + patch
messages: + msg88948
2009-04-22 04:12:48ajaksu2setstage: test needed
versions: + Python 2.6, Python 3.1, - Python 2.5
2009-04-09 11:25:03wosccreate