classification
Title: Debug (-d) mode not working in 3.9
Type: Stage: resolved
Components: Versions: Python 3.9
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: anthony shaw, lys.nikolaou, miss-islington, pablogsal, vstinner
Priority: normal Keywords: patch

Created on 2020-12-01 11:15 by anthony shaw, last changed 2020-12-02 22:01 by miss-islington. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 23607 merged pablogsal, 2020-12-02 03:04
PR 23615 merged miss-islington, 2020-12-02 17:57
Messages (9)
msg382236 - (view) Author: anthony shaw (anthony shaw) Date: 2020-12-01 11:15
I noticed since the new parser implementation, the debug mode in the tokeniser is no longer working.

This is the case for 3.9.0 and also 3.9.1rc1. Running python3.9 with a simple application like this:

 # Demo application
 def my_function():
   proceed

Does not output anything

> python3.9 -d ~/PycharmProjects/cpython-book-samples/13/test_tokens.py

> 
Produces no output

Whereas python3.10 (the latest alpha) outputs the expected results
msg382248 - (view) Author: Lysandros Nikolaou (lys.nikolaou) * (Python committer) Date: 2020-12-01 12:53
Hi Anthony,

could it be that your 3.9 is not a debug build, while the others are? We only enable parser debug ouput when Python has been build --with-pydebug.

Building the 3.9.0 tag --with-pydebug works fine for me.
msg382256 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-12-01 14:35
I cannot reproduce it either in 3.9. Building 3.9 in debug mode and using -d works as expected
msg382257 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-12-01 14:36
Also, note the -d flag activates the debug mode **in the parser**, not in the tokenizer.
msg382281 - (view) Author: anthony shaw (anthony shaw) Date: 2020-12-01 23:06
thanks, it would be great if this were documented in the --help section like the other options which require a debug build (-X showrefcount)
msg382282 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-12-01 23:15
In PyConfig, I chose to name the option "parser_debug" rather than the very generic "debug" name :-)

"""
Parser debug mode. If greater than 0, turn on parser debugging output (for expert only, depending on compilation options).

Incremented by the -d command line option. Set to the PYTHONDEBUG environment variable value.
"""

https://docs.python.org/dev/c-api/init_config.html#c.PyConfig.parser_debug
msg382283 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2020-12-01 23:16
> thanks, it would be great if this were documented in the --help section like the other options which require a debug build (-X showrefcount)

PyConfig.show_ref_count documentation explains that a debug build is required:

"Need a debug build of Python (Py_REF_DEBUG macro must be defined)."

https://docs.python.org/dev/c-api/init_config.html#c.PyConfig.show_ref_count

By the way, it would be nice to add a link on "debug build" to explain how to detect if Python is built in debug mode and how to build Python in debug mode.
msg382330 - (view) Author: Pablo Galindo Salgado (pablogsal) * (Python committer) Date: 2020-12-02 17:56
New changeset 99b594404d364b363c184f48338d6ee81bee26dd by Pablo Galindo in branch 'master':
bpo-42521: Add note about 'Python -d' only working on debug builds (GH-23607)
https://github.com/python/cpython/commit/99b594404d364b363c184f48338d6ee81bee26dd
msg382348 - (view) Author: miss-islington (miss-islington) Date: 2020-12-02 22:01
New changeset 9b34f34aa929941576a69a6f7de77f1fb0b96ed6 by Miss Islington (bot) in branch '3.9':
bpo-42521: Add note about 'Python -d' only working on debug builds (GH-23607)
https://github.com/python/cpython/commit/9b34f34aa929941576a69a6f7de77f1fb0b96ed6
History
Date User Action Args
2020-12-02 22:01:27miss-islingtonsetmessages: + msg382348
2020-12-02 17:59:23pablogsalsetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2020-12-02 17:57:08miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request22483
2020-12-02 17:56:45pablogsalsetmessages: + msg382330
2020-12-02 03:04:34pablogsalsetkeywords: + patch
stage: patch review
pull_requests: + pull_request22475
2020-12-01 23:16:58vstinnersetmessages: + msg382283
2020-12-01 23:15:19vstinnersetnosy: + vstinner
messages: + msg382282
2020-12-01 23:06:18anthony shawsetmessages: + msg382281
2020-12-01 14:36:33pablogsalsetmessages: + msg382257
2020-12-01 14:35:57pablogsalsetmessages: + msg382256
2020-12-01 12:53:58lys.nikolaousetmessages: + msg382248
2020-12-01 11:15:39anthony shawcreate