Title: Provide PDB hook to customize how to find source files
Type: enhancement Stage: needs patch
Components: Library (Lib) Versions: Python 3.7
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Pinku Surana, xdegaye
Priority: normal Keywords: patch

Created on 2016-11-04 16:18 by Pinku Surana, last changed 2016-11-08 20:33 by xdegaye.

File name Uploaded Description Edit xdegaye, 2016-11-04 20:18
debug_script.patch xdegaye, 2016-11-05 16:51 review
Messages (5)
msg280056 - (view) Author: Pinku Surana (Pinku Surana) Date: 2016-11-04 16:18
I am using Python as a hosted scripting runtime for a product. All the user scripts are stored in a database. I use "compile" and "exec" to run the scripts. I'd like to use PDB to debug scripts. Unfortunately, PDB makes a call to linecache, which calls assuming the code is in a file. I'd like to pass in a function that takes a filename and returns the source code in a string. 

At the very least, moving the call to "linecache.getline" into a separate method in PDB ("self.get_lines") would allow me to override that method with my own.
msg280077 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-11-04 20:18
The lazycache() function of the linecache module meets your request, I think. See the following debugging session using the attached script:

$ python -m pdb
> /path/to/cwd/<module>()
-> def debug_script(script):
(Pdb) n
> /path/to/cwd/<module>()
-> """
(Pdb) n
> /path/to/cwd/<module>()
-> code, globs = debug_script(s)
(Pdb) n
> /path/to/cwd/<module>()
-> exec(code, globs)
(Pdb) s
> /path/to/cwd/script_name(2)<module>()
-> x = 1
(Pdb) s
> /path/to/cwd/script_name(2)<module>()
-> x = 1
(Pdb) s
> /path/to/cwd/script_name(3)<module>()
-> y = 'blabla'
(Pdb) s
> /path/to/cwd/script_name(3)<module>()->None
-> y = 'blabla'
(Pdb) s
> /path/to/cwd/<module>()->None
-> exec(code, globs)
msg280117 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-11-05 16:51
This patch is an attempt at allowing the source debugging of scripts executed by the Python exec() function. It misses tests and documentation.

You may use it using the idiom given in the following example to avoid stepping into the pdb code on the first invocation of pdb.exec_script() (see the exec_script() doc string):

import sys

def main():
    foo = 123
    s = """if 1:
        x = foo
        x = 555

if __name__ == '__main__':
    if '--debug' in sys.argv[1:]:
        import pdb
        exec_script = pdb.exec_script
        exec_script = exec

msg280335 - (view) Author: Pinku Surana (Pinku Surana) Date: 2016-11-08 19:12
Thanks. This is clever. I've tried it out and it works. Would it be more appropriate to use "importlib" and "" to implement a custom loader for a string script? It looks like does the right thing.
msg280344 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-11-08 20:33
This is a simple code object compiled from a source (the string), a module is quite different and more complex. The patch uses a fake module Loader to use linecache, there is no gain in going any further and pulling from the importlib machinery, I think.
Date User Action Args
2016-11-08 20:33:34xdegayesetmessages: + msg280344
2016-11-08 19:12:02Pinku Suranasetmessages: + msg280335
2016-11-05 16:51:50xdegayesetfiles: + debug_script.patch
versions: + Python 3.7, - Python 3.5
messages: + msg280117

components: - Demos and Tools
keywords: + patch
stage: needs patch
2016-11-04 20:18:26xdegayesetfiles: +
nosy: + xdegaye
messages: + msg280077

2016-11-04 16:18:28Pinku Suranacreate