This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Odd Bytecode Generation in 3.10
Type: Stage: patch review
Components: Interpreter Core Versions: Python 3.11, Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: JelleZijlstra, Mark.Shannon, saulshanabrook
Priority: normal Keywords: patch

Created on 2022-02-11 17:22 by saulshanabrook, last changed 2022-04-11 14:59 by admin.

Pull Requests
URL Status Linked Edit
PR 31285 merged saulshanabrook, 2022-02-11 20:39
PR 31326 merged Mark.Shannon, 2022-02-14 14:48
PR 31352 closed Mark.Shannon, 2022-02-15 09:43
PR 31354 merged Mark.Shannon, 2022-02-15 10:03
Messages (7)
msg413088 - (view) Author: Saul Shanabrook (saulshanabrook) * Date: 2022-02-11 17:22
I noticed that in Python 3.10, and also in main, a certain control flow construct produces some very odd bytecode (showing on main but same on python 3.10 tags):

```
./python.exe -c 'import dis; dis.dis("while not (a < b < c): pass")'
              0 RESUME                   0

  1           2 LOAD_NAME                0 (a)
              4 LOAD_NAME                1 (b)
              6 SWAP                     2
              8 COPY                     2
             10 COMPARE_OP               0 (<)
             12 POP_JUMP_IF_FALSE       11 (to 22)
             14 LOAD_NAME                2 (c)
             16 COMPARE_OP               0 (<)
             18 POP_JUMP_IF_TRUE        28 (to 56)
             20 JUMP_FORWARD             1 (to 24)
        >>   22 POP_TOP
        >>   24 LOAD_NAME                0 (a)
             26 LOAD_NAME                1 (b)
             28 SWAP                     2
             30 COPY                     2
             32 COMPARE_OP               0 (<)
             34 POP_JUMP_IF_FALSE       23 (to 46)
             36 LOAD_NAME                2 (c)
             38 COMPARE_OP               0 (<)
             40 POP_JUMP_IF_FALSE       12 (to 24)
             42 LOAD_CONST               0 (None)
             44 RETURN_VALUE
        >>   46 POP_TOP
             48 EXTENDED_ARG           255
             50 EXTENDED_ARG         65535
             52 EXTENDED_ARG         16777215
             54 JUMP_FORWARD         4294967280 (to 8589934616)
        >>   56 LOAD_CONST               0 (None)
             58 RETURN_VALUE
```

The last JUMP_FORWARD has a rather larger argument! This was the minimal example I could find to replicate this.

However, this is an example of some runnable code that also encounters it:

```
a = b = c = 1
while not (a < b < c):
    if c == 1:
        c = 3
    else:
        b = 2
    print(a, b, c)
```

This actually executes fine, but I notice that when it's executing it does execute that very large arg, but that the `oparg` to JUMP_FORWARD ends up being negative! By adding some tracing, I was able to see that the `oparg` variable in the `TARGET(JUMP_FORWARD)` case is `-32`.

I am not sure if this is a bug or intended behavior. It does seem a bit odd to have this unnecessarily large argument that ends up turning into a negative jump! But the behavior seems fine.

At the least, maybe `dis` should be modified so that it properly sees this as a negative jump and debugs it properly? I am happy to submit a PR to modify `dis` to handle this case, but I also wanted to flag that maybe it's a bug to being with.
msg413130 - (view) Author: Saul Shanabrook (saulshanabrook) * Date: 2022-02-12 13:56
I added a patch to make `dis` work with the overflow format.

I am looking into why the bytecode is emitted like this in the first place now. 

I see that currently, Python is trying to write a negative bytecode arg for the JUMP_ABSOLUTE and `instrsize` is saying this requires 4 args (which is not true, it only requires one). Is it intended behavior that the bytecode args will be negative? If so, then `instrsize` might have to be updated to handle this. It takes in an `unsigned int` but the oparg is actually a signed int.
msg413131 - (view) Author: Saul Shanabrook (saulshanabrook) * Date: 2022-02-12 14:24
Ah, I see that to represent a negative 4 byte signed integer as an arg, we actually do need the 4 instructions, since each is a byte, to get the most significant bit to a 1?

I wasn't able to find anything anywhere which says if the bytecode should or should not contain negative opargs, so I am not sure if this is a bug or intended behavior, to emit the three EXTENDED_ARGs in order to represent
msg413279 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2022-02-15 09:35
New changeset 3be1a443ca8e7d4ba85f95b78df5c4122cae9ede by Mark Shannon in branch 'main':
bpo-46724: Use `JUMP_ABSOLUTE` for all backward jumps. (GH-31326)
https://github.com/python/cpython/commit/3be1a443ca8e7d4ba85f95b78df5c4122cae9ede
msg413329 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2022-02-16 11:26
New changeset d4e4ef107a9fea257981d7701f023177b704a44f by Mark Shannon in branch '3.10':
[3.10] bpo-46724: Use `JUMP_ABSOLUTE` for all backward jumps. (GH-31326) (GH-31354)
https://github.com/python/cpython/commit/d4e4ef107a9fea257981d7701f023177b704a44f
msg413472 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2022-02-18 09:56
New changeset c3ce7781e3afe6f2dec5eef8e87fd5a664519ae9 by Saul Shanabrook in branch 'main':
bpo-46724: Fix dis support for overflow args (GH-31285)
https://github.com/python/cpython/commit/c3ce7781e3afe6f2dec5eef8e87fd5a664519ae9
msg415773 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2022-03-22 14:04
I think this is fixed (for 3.11 at least) by https://github.com/python/cpython/pull/31888
History
Date User Action Args
2022-04-11 14:59:56adminsetgithub: 90880
2022-03-22 14:04:22Mark.Shannonsetmessages: + msg415773
2022-02-18 09:56:31Mark.Shannonsetmessages: + msg413472
2022-02-16 11:26:17Mark.Shannonsetmessages: + msg413329
2022-02-15 10:03:47Mark.Shannonsetpull_requests: + pull_request29503
2022-02-15 09:43:18Mark.Shannonsetpull_requests: + pull_request29501
2022-02-15 09:35:28Mark.Shannonsetmessages: + msg413279
2022-02-14 14:48:26Mark.Shannonsetpull_requests: + pull_request29482
2022-02-12 14:24:13saulshanabrooksetmessages: + msg413131
2022-02-12 13:56:56saulshanabrooksetmessages: + msg413130
2022-02-11 20:39:41saulshanabrooksetkeywords: + patch
stage: patch review
pull_requests: + pull_request29445
2022-02-11 18:54:05JelleZijlstrasetnosy: + JelleZijlstra
2022-02-11 17:33:07mark.dickinsonsetnosy: + Mark.Shannon
2022-02-11 17:22:26saulshanabrookcreate