classification
Title: Provide PDB hook to customize how to find source files
Type: enhancement Stage: needs patch
Components: Library (Lib) Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: Pinku Surana, xdegaye
Priority: normal Keywords: patch

Created on 2016-11-04 16:18 by Pinku Surana, last changed 2016-11-08 20:33 by xdegaye.

Files
File name Uploaded Description Edit
debug_script.py xdegaye, 2016-11-04 20:18
debug_script.patch xdegaye, 2016-11-05 16:51 review
Messages (5)
msg280056 - (view) Author: Pinku Surana (Pinku Surana) Date: 2016-11-04 16:18
I am using Python as a hosted scripting runtime for a product. All the user scripts are stored in a database. I use "compile" and "exec" to run the scripts. I'd like to use PDB to debug scripts. Unfortunately, PDB makes a call to linecache, which calls tokenize.open assuming the code is in a file. I'd like to pass in a function that takes a filename and returns the source code in a string. 

At the very least, moving the call to "linecache.getline" into a separate method in PDB ("self.get_lines") would allow me to override that method with my own.
msg280077 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-11-04 20:18
The lazycache() function of the linecache module meets your request, I think. See the following debugging session using the attached script:

$ python -m pdb debug_script.py
> /path/to/cwd/debug_script.py(1)<module>()
-> def debug_script(script):
(Pdb) n
> /path/to/cwd/debug_script.py(19)<module>()
-> """
(Pdb) n
> /path/to/cwd/debug_script.py(20)<module>()
-> code, globs = debug_script(s)
(Pdb) n
> /path/to/cwd/debug_script.py(21)<module>()
-> exec(code, globs)
(Pdb) s
--Call--
> /path/to/cwd/script_name(2)<module>()
-> x = 1
(Pdb) s
> /path/to/cwd/script_name(2)<module>()
-> x = 1
(Pdb) s
> /path/to/cwd/script_name(3)<module>()
-> y = 'blabla'
(Pdb) s
--Return--
> /path/to/cwd/script_name(3)<module>()->None
-> y = 'blabla'
(Pdb) s
--Return--
> /path/to/cwd/debug_script.py(21)<module>()->None
-> exec(code, globs)
(Pdb)
msg280117 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-11-05 16:51
This patch is an attempt at allowing the source debugging of scripts executed by the Python exec() function. It misses tests and documentation.

You may use it using the idiom given in the following example to avoid stepping into the pdb code on the first invocation of pdb.exec_script() (see the exec_script() doc string):

import sys

def main():
    foo = 123
    s = """if 1:
        x = foo
        x = 555
    """
    exec_script(s)

if __name__ == '__main__':
    if '--debug' in sys.argv[1:]:
        import pdb
        exec_script = pdb.exec_script
        pdb.Pdb(skip=['pdb']).set_trace()
    else:
        exec_script = exec

    main()
msg280335 - (view) Author: Pinku Surana (Pinku Surana) Date: 2016-11-08 19:12
Thanks. This is clever. I've tried it out and it works. Would it be more appropriate to use "importlib" and "importlib.abc" to implement a custom loader for a string script? It looks like importlib.abc.InspectLoader does the right thing.
msg280344 - (view) Author: Xavier de Gaye (xdegaye) * (Python triager) Date: 2016-11-08 20:33
This is a simple code object compiled from a source (the string), a module is quite different and more complex. The patch uses a fake module Loader to use linecache, there is no gain in going any further and pulling from the importlib machinery, I think.
History
Date User Action Args
2016-11-08 20:33:34xdegayesetmessages: + msg280344
2016-11-08 19:12:02Pinku Suranasetmessages: + msg280335
2016-11-05 16:51:50xdegayesetfiles: + debug_script.patch
versions: + Python 3.7, - Python 3.5
messages: + msg280117

components: - Demos and Tools
keywords: + patch
stage: needs patch
2016-11-04 20:18:26xdegayesetfiles: + debug_script.py
nosy: + xdegaye
messages: + msg280077

2016-11-04 16:18:28Pinku Suranacreate