This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Python3.8 optimizes away a "while" line
Type: Stage:
Components: Interpreter Core Versions: Python 3.8
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, iritkatriel, nedbat, serhiy.storchaka, vstinner, yselivanov
Priority: normal Keywords:

Created on 2018-10-04 00:00 by nedbat, last changed 2022-04-11 14:59 by admin.

Messages (7)
msg327019 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2018-10-04 00:00
Looks like the optimizer is getting more aggressive.  Line 2 (the constant while) no longer even appears in the bytecode:

$ cat -n whiletrue.py
     1	a = 1
     2	while 1:
     3	    print(a)
     4	b = 4

$ python3.7 -m dis < whiletrue.py
  1           0 LOAD_CONST               0 (1)
              2 STORE_NAME               0 (a)

  2           4 SETUP_LOOP              12 (to 18)

  3     >>    6 LOAD_NAME                1 (print)
              8 LOAD_NAME                0 (a)
             10 CALL_FUNCTION            1
             12 POP_TOP
             14 JUMP_ABSOLUTE            6
             16 POP_BLOCK

  4     >>   18 LOAD_CONST               1 (4)
             20 STORE_NAME               2 (b)
             22 LOAD_CONST               2 (None)
             24 RETURN_VALUE

$ python3.8 -m dis < whiletrue.py
  1           0 LOAD_CONST               0 (1)
              2 STORE_NAME               0 (a)

  3     >>    4 LOAD_NAME                1 (print)
              6 LOAD_NAME                0 (a)
              8 CALL_FUNCTION            1
             10 POP_TOP
             12 JUMP_ABSOLUTE            4

  4          14 LOAD_CONST               1 (4)
             16 STORE_NAME               2 (b)
             18 LOAD_CONST               2 (None)
             20 RETURN_VALUE


I understand why we want to make these optimizations.  It's good for those times when we run our programs.  But there are other times: when we are analyzing programs.

I'm begging you: please please please help me get https://bugs.python.org/issue2506 implemented (a way to disable optimizations).  It is becoming more and more difficult to write tools that analyze Python programs.

People are testing their libraries on Python 3.8-dev, and reporting problems using coverage.py.  I would like to support their efforts to test on the daily Python builds.  But it's difficult to keep coverage.py working under these conditions.

Anything you can do would be really appreciated.
msg327026 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-10-04 01:00
While I agree with you that we need a flag to disable optimizations, this particular change isn't related to AST or peephole optimizers.

See the bpo-17611 for more details (or https://github.com/python/cpython/commit/520b7ae27e39d1c77ea74ccd1b184d7cb43f9dcb)
msg327027 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2018-10-04 01:11
Yury, thanks. I haven't read the code in depth.  I assume there is some place that notices that the while condition is a constant, and therefore doesn't need to be explicitly evaluated?  That test can be disabled, yes?
msg327028 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-10-04 01:16
It's more complicated than that: there's no more SETUP_LOOP opcode anymore.  The ceval and compiler parts responsible for evaluating and compiling loops were rewritten.  FWIW the goal was more about simplifying the needlessly complicated implementation and long term maintenance than about performance.  So this is something that coverage will probably have to adapt to (peephole optimizer is another topic).
msg327029 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-10-04 01:18
> I assume there is some place that notices that the while condition is a constant, and therefore doesn't need to be explicitly evaluated?

Ah, I see what you're asking about.  I'll need to double check that.
msg327031 - (view) Author: Yury Selivanov (yselivanov) * (Python committer) Date: 2018-10-04 01:43
> I assume there is some place that notices that the while condition is a constant, and therefore doesn't need to be explicitly evaluated?

To answer your question: yes, and it's unrelated to both peephole optimizer and to the above mentioned grand refactoring of the ceval loop :)  But if we add "-X noopt" (or equivalent) we can disable those micro-optimizations too.

Let's track this in https://github.com/python/cpython/pull/9693
msg412026 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2022-01-28 19:12
Does PEP626 help with this problem?
History
Date User Action Args
2022-04-11 14:59:06adminsetgithub: 79069
2022-01-28 19:12:25iritkatrielsetnosy: + iritkatriel
messages: + msg412026
2018-10-04 01:43:11yselivanovsetmessages: + msg327031
2018-10-04 01:18:22yselivanovsetmessages: + msg327029
2018-10-04 01:16:57yselivanovsetmessages: + msg327028
2018-10-04 01:11:47nedbatsetmessages: + msg327027
2018-10-04 01:00:09yselivanovsetnosy: + yselivanov
messages: + msg327026
2018-10-04 00:17:00nedbatsetnosy: + vstinner, benjamin.peterson
2018-10-04 00:00:41nedbatcreate