classification
Title: possible free statically allocated string in compiler when easter egg on
Type: Stage: patch review
Components: Versions: Python 3.8, Python 3.7, Python 3.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: barry, serhiy.storchaka, vstinner, xiang.zhang
Priority: normal Keywords: easy, patch

Created on 2018-07-10 11:37 by xiang.zhang, last changed 2018-07-12 08:03 by serhiy.storchaka.

Pull Requests
URL Status Linked Edit
PR 8242 closed vstinner, 2018-07-11 09:24
PR 8259 closed vstinner, 2018-07-11 21:22
PR 8262 open serhiy.storchaka, 2018-07-12 07:15
Messages (10)
msg321381 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2018-07-10 11:37
While reviewing PR8222, I found `err_ret->text` is assigned a not malloced string, but it will finally get freed in `err_input`.
msg321382 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-07-10 11:41
I expected it should be easy to reproduce a crash, but I failed to find an example.
msg321418 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-11 09:24
This issue is related to bpo-34080.
msg321420 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-11 09:33
> I expected it should be easy to reproduce a crash, but I failed to find an example.

I don't understand how parsetok.c is supposed to handle "from __future__ import barry_as_FLUFL":

* Parser/parsetok.c tests (ps->p_flags & CO_FUTURE_BARRY_AS_BDFL)
* Python/future.c uses "if (strcmp(feature, FUTURE_BARRY_AS_BDFL) == 0) { ff->ff_features |= CO_FUTURE_BARRY_AS_BDFL; }"
* Python/pythonrun.c: PARSER_FLAGS() copies CO_FUTURE_BARRY_AS_BDFL flag as PyPARSE_BARRY_AS_BDFL

The only way to use "<>" operator seems to pass explicitly CO_FUTURE_BARRY_AS_BDFL flag to compile() flags, which is not the convenient way to use "from __future__ import barry_as_FLUFL" :-(

Code:
---
import inspect
CO_FUTURE_BARRY_AS_BDFL = 0x40000
flags = CO_FUTURE_BARRY_AS_BDFL
code = "print(1 <> 2)"
try:
    compile(code, "filename", "exec", flags=0)
except SyntaxError:
    print("ok")
else:
    raise Exception("SyntaxError expected")
compile(code, "filename", "exec", flags=flags)
print("ok")
---

Maybe we need a test for the easter egg. Or maybe we should remove it? :-p
msg321421 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-11 09:43
Oh, this easter egg is tested by Lib/test/test_flufl.py.
msg321425 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-07-11 09:48
Could you add a test in test_flufl.py that will fail with not patched code?

This issue and issue34080 look unrelated to me. They can be fixed independently.
msg321429 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2018-07-11 10:13
> Could you add a test in test_flufl.py that will fail with not patched code?

It seems like PyObject_FREE() is never called with the static string "with Barry as BDFL, use '<>' instead of '!='".

When parsetok() goes to code path (1):

                err_ret->text = "with Barry as BDFL, use '<>' "
                                "instead of '!='";

Later, it goes to code path (2) as well:

        if (tok->buf != NULL) {
            ...
            err_ret->text = (char *) PyObject_MALLOC(len + 1);

Hum, I modified my PR to removed *dead code*:

                err_ret->text = "with Barry as BDFL, use '<>' "
                                "instead of '!='";

> This issue and issue34080 look unrelated to me. They can be fixed independently.

In practice, both issues are related and it seems easier to me to fix them both at the same time ;-)
msg321487 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-07-11 17:05
AFAICT, that future only works in the REPL.
msg321522 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-07-12 07:22
PR 8262 fixes the intended setting of error message for the "Barry as BDFL" easter egg.

>>> from __future__ import barry_as_FLUFL
>>> 1 != 2
  File "<stdin>", line 1
    1 != 2
       ^
SyntaxError: with Barry as BDFL, use '<>' instead of '!='

But there is other problem. As was exposed in the private communication with Victor and Barry, this easter egg works only in REPL (or if explicitly pass __future__.CO_FUTURE_BARRY_AS_BDFL to compile()). I'll try to fix this too.
msg321525 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2018-07-12 08:03
The problem is that 'from __future__ import' is parsed after converting the sources to the AST. But this flag affects the behavior of the tokenizer, before an AST is created.
History
Date User Action Args
2018-07-12 08:03:34serhiy.storchakasetmessages: + msg321525
2018-07-12 07:22:31serhiy.storchakasetmessages: + msg321522
2018-07-12 07:15:13serhiy.storchakasetpull_requests: + pull_request7795
2018-07-11 21:22:04vstinnersetpull_requests: + pull_request7792
2018-07-11 17:05:29barrysetmessages: + msg321487
2018-07-11 10:13:52vstinnersetmessages: + msg321429
2018-07-11 09:48:56serhiy.storchakasetmessages: + msg321425
2018-07-11 09:43:55vstinnersetmessages: + msg321421
2018-07-11 09:33:44vstinnersetmessages: + msg321420
2018-07-11 09:24:54vstinnersetmessages: + msg321418
2018-07-11 09:24:29vstinnersetkeywords: + patch
stage: patch review
pull_requests: + pull_request7775
2018-07-11 01:59:39xiang.zhangsetnosy: + vstinner
2018-07-10 11:41:30serhiy.storchakasetnosy: + barry
2018-07-10 11:41:14serhiy.storchakasetmessages: + msg321382
2018-07-10 11:38:12xiang.zhangsettitle: possible free statically allocated string -> possible free statically allocated string in compiler when easter egg on
2018-07-10 11:37:01xiang.zhangcreate