New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Crashes with lines of the form "async \" #76033
Comments
Hi. Python 3.6.2 crashes when interpreting lines with the text "async \" (future keyword 'async' and ending with a backslash). Tested in a docker environment (debian jessie). (see github.com/0xquad/docker-python36 if needed) Examples: $ docker run -ti --rm python36
root@4c09392f83c8:/# python3.6
Python 3.6.2 (default, Aug 4 2017, 14:35:04)
[GCC 6.4.0 20170724] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> async \
...
File "<stdin>", line 1
\ufffd\ufffdF\ufffd\ufffd
^
SyntaxError: invalid syntax
>>> async \
Segmentation fault
root@4c09392f83c8:/# Also, ----- file: test.py $ ./test.py
Segmentation fault
$ Haven't taken the time to produce a backtrace or investigate with latest the dev versions or any further. Let me know if I can assist in any way. |
This issue is fixed in the master branch (version 3.7.0 alpha 2). The issue was fixed in this PR: The cause is that async was not a proper keyword and the parser segfaults when checking for the new token and parsing the newline. In particular, this happens here: translate_newlines at Parser/tokenizer.c:713 This is the stack trace: #0 _PyObject_Alloc (ctx=<optimized out>, elsize=10, nelem=1, use_calloc=0) at Objects/obmalloc.c:806 After applying commit ac31770 the issue is fixed. Does it make sense to backport ac31770 to 3.6? |
No, async was not a keyword in Python 3.6 on purpose. Making it a keyword can break a lot of code. I confirm that Python 3.6 still crashs with a very high number of "async " prefixes: try attached async_parser_crash.py. Extract of the gdb traceback on a crash: (...) It looks like a stack overflow. The tokenizer may fail earlier on "async async ". |
Thank you Alexandre Hamelin for the bug report and Pablo Galindo for the fix ;-) |
Awesome work, thanks to you! Would it also be the case for 'await' ? |
"async" requires to maintain a "async_def" state. It seems like await doesn't need a state for itself, but rely on the "async_def" state which has been fixed. Extract of Parser/tokenizer.c: /* Current token length is 5. */
if (tok->async_def) {
/* We're inside an 'async def' function. */
if (memcmp(tok->start, "async", 5) == 0) {
return ASYNC;
}
if (memcmp(tok->start, "await", 5) == 0) {
return AWAIT;
}
} |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: