classification
Title: Left bracket remains in format string result when '\' preceeds it
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: eric.smith Nosy List: Jim Fasarakis-Hilliard, eric.smith, haypo, refi64, serhiy.storchaka
Priority: normal Keywords: patch

Created on 2016-12-29 11:56 by Jim Fasarakis-Hilliard, last changed 2017-03-28 15:43 by serhiy.storchaka.

Files
File name Uploaded Description Edit
fstring_backslash.patch serhiy.storchaka, 2016-12-29 15:10 review
fstring_backslash_2.patch serhiy.storchaka, 2017-01-03 15:59 review
Pull Requests
URL Status Linked Edit
PR 490 open serhiy.storchaka, 2017-03-05 22:37
Messages (22)
msg284249 - (view) Author: Jim Fasarakis-Hilliard (Jim Fasarakis-Hilliard) * Date: 2016-12-29 11:56
In short:

    >>> f"\{10}"

yields:

    "\\{10"

This is reproducible only when `\` precedes the opening bracket, that is:

    >>> f"\ {10}"

results in: "\\ 10"
msg284256 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-12-29 12:45
I'm not sure this counts as an error. The backslash means to treat the next character literally, which this does. And since \{ is not a valid escape sequence, it keeps both characters, exactly like:

>>> '\c'
'\\c'

Furthermore, since unknown escape sequences are deprecated, this usage is going to be an error in the future. You can see this now with -Werror:

$ python.exe  -Werror
Python 3.6.0b1+ (3.6:87de1f12c41c+, Sep 16 2016, 07:05:57) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> f'\{10}'
DeprecationWarning: invalid escape sequence '\{'
>>> '\c'
DeprecationWarning: invalid escape sequence '\c'
>>>

Since this works as I expect it to now, and since it will become an error in the future, I don't think any change is warranted.
msg284257 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-12-29 12:48
But I admit that dropping the trailing } seems odd. I think this particular usage should be an error: no bare } allowed, similar to:

>>> f'10}'
  File "<stdin>", line 1
SyntaxError: f-string: single '}' is not allowed
msg284258 - (view) Author: Jim Fasarakis-Hilliard (Jim Fasarakis-Hilliard) * Date: 2016-12-29 12:57
I see, the original "complaint" about this behavior on stack overflow was made due to the discrepancy between the f-string the the format "equivalent": '\{}'.format(10)'.
msg284259 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-12-29 13:02
Do you have a link to the SO question?
msg284260 - (view) Author: Jim Fasarakis-Hilliard (Jim Fasarakis-Hilliard) * Date: 2016-12-29 13:07
Yes, should've attached in my previous message. See http://stackoverflow.com/questions/41330097/why-does-the-symbol-remain-when-f-10-is-evaluated-in-python-3-6
msg284261 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-12-29 13:37
The problem is not that the trailing } is dropped, but that the starting { starts an f-string expression.

>>> f'\{2*5}'
'\\{10'

I expected either '\\10' as in '\{}'.format(2*5), or at least '\\{2*5}'.

There is other f-string parsing error:

>>> f'\\N{2*5}'
'\\N{2*5}'

I expected '\\N10'. '\\N' doesn't start a unicode name escape. This is a legitimate expression.
msg284262 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-12-29 13:40
Yes:

>>> f'\{2*5}'
'\\{10'

I agree '\\10' would make sense.
msg284266 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2016-12-29 14:42
This problem was no doubt introduced after 3.6a1 when I changed the parsing to disallow backslashes inside {}.
msg284271 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-12-29 15:10
Proposed patch fixes parsing backslashes in f-strings.

P.S. Definitely we should find other name for this term. I hate misleading "f-string".
msg284558 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-01-03 13:12
The PEP 498 should be updated to define the expected behaviour of f'\{5}':
https://www.python.org/dev/peps/pep-0498/#escape-sequences

+        self.assertEqual(f'\{6*7}', '\\42')
+        self.assertEqual(f'\\{6*7}', '\\42')
+        self.assertEqual(fr'\{6*7}', '\\42')
+
+        AMPERSAND = 123
+        self.assertEqual(f'\N{AMPERSAND}', '&')
+        self.assertEqual(f'\\N{AMPERSAND}', '\\N123')
+        self.assertEqual(fr'\N{AMPERSAND}', '\\N123')
+        self.assertEqual(f'\\\N{AMPERSAND}', '\\&')

I'm not sure that I like this behaviour.

f'\\N{AMPERSAND}': reading a local variable looks like a typo or a security vulnerability, rather than a nice feature.

IMHO if you want an anti-slash (\) in the output string, it *must* be written "\\" since we *do* interpret other escape sequences:

* f'\n' returns '\n', the newline character U+000A
* f'\N{Snowman}' returns the nice snowman character U+2603 (☃)
* etc.

What is the issue with having to write "\\{10}" to get "\\10" string? It's less error prone.
msg284578 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-01-03 15:31
> f'\\N{AMPERSAND}': reading a local variable looks like a typo or a security vulnerability, rather than a nice feature.

This can look as a typo, but how would you write this if this is not a typo? f'\\N{AMPERSAND}' is legitimate syntax, and it would be weird to interpret this expression in any other way than as '\\' + 'N' + format(AMPERSAND).

> What is the issue with having to write "\\{10}" to get "\\10" string? It's less error prone.

There is no issue with f'\\{10}'. There is an issue with f'\{10}'.
msg284579 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2017-01-03 15:33
> There is no issue with f'\\{10}'. There is an issue with f'\{10}'.

Hum, I'm not sure that my previous comment is explicit: I suggest to raise a syntax error on f'\{10}' (or emit a warning, and raise an error in 3.7 if you insist ;-)).
msg284580 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-01-03 15:59
Updated patch adds a warning.
msg285949 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-01-21 11:22
Ping.
msg288511 - (view) Author: Jim Fasarakis-Hilliard (Jim Fasarakis-Hilliard) * Date: 2017-02-24 09:33
Serhiy, since review activity has dropped on b.p.o now that the move to github was made, would you like to make this into a pull request to get it reviewed and merged faster?
msg288979 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-04 15:15
I am not experienced with git and am waiting until new workflow be described in the devguide.
msg289645 - (view) Author: Eric V. Smith (eric.smith) * (Python committer) Date: 2017-03-15 07:19
Serhiy: I plan to find time in the next week or so to look at this. Sorry for the delay. Plus, the new workflow isn't helping me out: I need to get up to speed on it.
msg289689 - (view) Author: Ryan Gonzalez (refi64) * Date: 2017-03-15 19:07
Serhiy: if you want, you can give me your email and I'll submit a PR for you. If you want to do it yourself, just:

Download hub: https://hub.github.com/

git clone python/cpython cpython-git
cd cpython-git
git fork
curl https://raw.githubusercontent.com/mozilla/moz-git-tools/master/hg-patch-to-git-patch > hg-patch-to-git-patch
curl https://bugs.python.org/file46127/fstring_backslash_2.patch | python2 hg-patch-to-git-patch > fstring.patch
git checkout -b fstring-fix
git apply fstring.patch
git commit -am 'bpo-29104: Commit message here'
git push -u MY_GITHUB_USERNAME fstring-fix
git pull-request
msg289690 - (view) Author: Jim Fasarakis-Hilliard (Jim Fasarakis-Hilliard) * Date: 2017-03-15 19:09
A PR has been submitted, Ryan. See https://github.com/python/cpython/pull/490
msg289694 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-15 20:42
Thank you Ryan. I already created a PR, but I think your recipe can be helpful for me in future.
msg290733 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017-03-28 15:43
Ping again.
History
Date User Action Args
2017-03-28 15:43:30serhiy.storchakasetmessages: + msg290733
2017-03-15 20:42:09serhiy.storchakasetmessages: + msg289694
2017-03-15 19:09:29Jim Fasarakis-Hilliardsetmessages: + msg289690
2017-03-15 19:07:39refi64setnosy: + refi64
messages: + msg289689
2017-03-15 07:19:59eric.smithsetmessages: + msg289645
2017-03-15 07:03:43serhiy.storchakalinkissue29814 superseder
2017-03-05 22:37:23serhiy.storchakasetpull_requests: + pull_request403
2017-03-04 15:15:06serhiy.storchakasetmessages: + msg288979
2017-02-24 09:33:20Jim Fasarakis-Hilliardsetmessages: + msg288511
2017-01-21 11:22:28serhiy.storchakasetmessages: + msg285949
2017-01-03 15:59:07serhiy.storchakasetfiles: + fstring_backslash_2.patch

messages: + msg284580
2017-01-03 15:33:18hayposetmessages: + msg284579
2017-01-03 15:31:40serhiy.storchakasetmessages: + msg284578
2017-01-03 13:12:57hayposetnosy: + haypo
messages: + msg284558
2016-12-29 15:10:20serhiy.storchakasetfiles: + fstring_backslash.patch
keywords: + patch
messages: + msg284271
2016-12-29 14:42:55eric.smithsetmessages: + msg284266
2016-12-29 13:40:33eric.smithsetmessages: + msg284262
2016-12-29 13:37:39serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg284261
2016-12-29 13:07:25Jim Fasarakis-Hilliardsetmessages: + msg284260
2016-12-29 13:02:53eric.smithsetmessages: + msg284259
2016-12-29 12:57:38Jim Fasarakis-Hilliardsetmessages: + msg284258
2016-12-29 12:48:19eric.smithsetmessages: + msg284257
2016-12-29 12:45:48eric.smithsetassignee: eric.smith
messages: + msg284256
2016-12-29 12:05:18SilentGhostsetnosy: + eric.smith
type: behavior
2016-12-29 11:56:25Jim Fasarakis-Hilliardcreate