This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: ast - handling new line inside a string
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: Ilya Kamenshchikov, eric.smith
Priority: normal Keywords:

Created on 2019-06-06 19:41 by Ilya Kamenshchikov, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Messages (3)
msg344861 - (view) Author: Ilya Kamenshchikov (Ilya Kamenshchikov) * Date: 2019-06-06 19:41
parsing two different strings produces identical ast.Str nodes:

import ast

txt1 = '"""\\n"""'
txt2 = '"""\n"""'

tree1 = ast.parse(txt1)
tree2 = ast.parse(txt2)

print(tree1.body[0].value.s == tree2.body[0].value.s)
print(bytes(tree1.body[0].value.s, encoding='utf-8'))
print(bytes(tree2.body[0].value.s, encoding='utf-8'))

>>> True
>>> b'\n'
>>> b'\n'

Expected result: I should be able to distinguish between the nodes created from two different strings.
msg344867 - (view) Author: Ilya Kamenshchikov (Ilya Kamenshchikov) * Date: 2019-06-06 20:02
Same problem holds for tabs (\\t vs \t).

For \\r vs \r, it is even more fun: 
txt1 = '"""\\r"""'
txt2 = '"""\r"""'

>>> b'\r'
>>> b'\n'
msg344875 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2019-06-06 20:42
There was a recent discussion about this on the bug tracker, but of course now I can't find it. But for an earlier discussion, see issue 25885.

The decision was that your expectation that the nodes be distinguishable isn't a design goal of the ast. There's no expectation that code can round-trip through the ast and produce the original text, even for a string.
History
Date User Action Args
2022-04-11 14:59:16adminsetgithub: 81363
2019-06-06 20:42:20eric.smithsetstatus: open -> closed

nosy: + eric.smith
messages: + msg344875

resolution: not a bug
stage: resolved
2019-06-06 20:02:01Ilya Kamenshchikovsetmessages: + msg344867
2019-06-06 19:41:45Ilya Kamenshchikovcreate