This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Long constant calculations stall compilation
Type: resource usage Stage: resolved
Components: Interpreter Core Versions: Python 3.6, Python 2.7
process
Status: closed Resolution: duplicate
Dependencies: Superseder: Too aggressive constant folding
View: 21074
Assigned To: Nosy List: SilentGhost, ammar2, fcostantini, serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2016-08-05 17:39 by fcostantini, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (5)
msg272046 - (view) Author: Franco Costantini (fcostantini) Date: 2016-08-05 17:39
Hi, we currently are fuzzing Python programs to find valid code that fails to compile. We found this program never finishes its compilation.

$ echo "raise ValueError ; 2 ** 12345678912345" > inf.py
$ python -m py_compile inf.py

We realize the computation is a large one and takes a very long time, in this case the compiler consumes more and more memory over time. Perhaps the compiler shouldn't try to calculate this indefinitely and just return an error.

Regards
msg272047 - (view) Author: SilentGhost (SilentGhost) * (Python triager) Date: 2016-08-05 17:47
On 3.6 it takes a very long time, but it does finish.

time ./python -c "raise ValueError ; 2 ** 12345678912345"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ValueError

real    1m35.673s
user    1m18.952s
sys     0m16.644s
msg272050 - (view) Author: Ammar Askar (ammar2) * (Python committer) Date: 2016-08-05 18:09
Just in case anyone is wondering why this happens. The compiler's peephole optimizer will fold constant expressions like 

x = 5 + 5

into

x = 10

More specifically, this bit here: https://github.com/python/cpython/blob/0f21fe6155227d11dc02bd3ef3b061de4ecea445/Python/peephole.c#L240


I'd say the current behavior of the compilation taking a long time is fine since at runtime it would run the exact same "2 ** 12345678912345" expression and spend a long time/run out of memory.

However I'd say the fact that this happens so silently can be a "gotcha" in some very rare cases.
msg272066 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2016-08-05 22:20
FYI my Python reimplementation of the bytecode peephole optimizer doesn't
have this issue. It estimates the size of a**b using exp() and ln()
functions.
http://bytecode.readthedocs.io/en/latest/
msg272105 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2016-08-06 19:54
See also issue21074.
History
Date User Action Args
2022-04-11 14:58:34adminsetgithub: 71882
2017-12-14 13:16:06serhiy.storchakasetstatus: open -> closed
superseder: Too aggressive constant folding
resolution: duplicate
stage: resolved
2016-08-06 19:54:30serhiy.storchakasetmessages: + msg272105
2016-08-06 17:49:14brett.cannonsettitle: Compilation doesnt' end -> Long constant calculations stall compilation
2016-08-05 22:20:29vstinnersetmessages: + msg272066
2016-08-05 18:09:59ammar2setnosy: + ammar2
messages: + msg272050
2016-08-05 17:47:23SilentGhostsetnosy: + SilentGhost
messages: + msg272047
2016-08-05 17:45:01SilentGhostsettype: compile error -> resource usage
versions: + Python 2.7
2016-08-05 17:43:26SilentGhostsetnosy: + vstinner, serhiy.storchaka

components: + Interpreter Core
versions: + Python 3.6, - Python 2.7
2016-08-05 17:39:36fcostantinicreate