While working on pegen, we came across an inconsistency on how line number and column offset info is stored for (el)if nodes. When parsing a very simple if-elif construct like

if a:
    pass
elif b:
    pass

the following parse tree gets generated:

Module(
body=[
If(
test=Name(id="a", ctx=Load(), lineno=1, col_offset=3, end_lineno=1, end_col_offset=4),
body=[Pass(lineno=2, col_offset=4, end_lineno=2, end_col_offset=8)],
orelse=[
If(
test=Name(
id="b", ctx=Load(), lineno=3, col_offset=5, end_lineno=3, end_col_offset=6
),
body=[Pass(lineno=4, col_offset=4, end_lineno=4, end_col_offset=8)],
orelse=[],
lineno=3,
col_offset=5,
end_lineno=4,
end_col_offset=8,
)
],
lineno=1,
col_offset=0,
end_lineno=4,
end_col_offset=8,
)
],
type_ignores=[],
)

There is the inconsistency that the column offset for the if statement is 0, thus the if statement starts with the keyword if, whereas the column offset for elif if 5, which means that the elif keyword is skipped.

As Guido suggests over at we-like-parsers/pegen_experiments#107 (comment) we could very easily change Python/ast.c so that the elif statement start with the elif keyword as well.

I have a PR ready!

New changeset 025a602 by Miss Islington (bot) (Lysandros Nikolaou) in branch 'master':
bpo-39031: Include elif keyword when producing lineno/col-offset info for if_stmt (GH-17582)
025a602

Hmmm, there is some problem with the CI and the 3.7 branch. Seems like Travis CI is giving some problems again.... I will investigate or maybe ask Brett to make the check not required again (we still have Azure Pipelines, testing the same thing).

The same thing happens with 3.8.....

New changeset 0ed45d0 by Pablo Galindo in branch '3.7':
[3.7] bpo-39031: Include elif keyword when producing lineno/col-offset info for if_stmt (GH-17582) (bpo-17584)
0ed45d0

New changeset 3b18b17 by Pablo Galindo (Miss Islington (bot)) in branch '3.8':
bpo-39031: Include elif keyword when producing lineno/col-offset info for if_stmt (GH-17582) (GH-17589)
3b18b17

There was a bug in my last PR, hopefully I will get a fix some time later today.

The bug is as follows: I only updated the asdl_seq_SET call for an elif without an else, if an else is included then the behavior is like before.

After my last PR it looks like this, parsing

2:if a:
3: pass
4:elif b:
5: pass

outputs the following AST:

Module(
body=[
If(
test=Name(id="a", ctx=Load(), lineno=2, col_offset=3, end_lineno=2, end_col_offset=4),
body=[Pass(lineno=3, col_offset=4, end_lineno=3, end_col_offset=8)],
orelse=[
If(
test=Name(
id="b", ctx=Load(), lineno=4, col_offset=5, end_lineno=4, end_col_offset=6
),
body=[Pass(lineno=5, col_offset=4, end_lineno=5, end_col_offset=8)],
orelse=[],
lineno=4,
col_offset=0,
end_lineno=5,
end_col_offset=8,
)
],
lineno=2,
col_offset=0,
end_lineno=5,
end_col_offset=8,
)
],
type_ignores=[],
)

On the other hand parsing

2:if a:
3: pass
4:elif b:
5: pass
6:else:
7: pass

outputs

Module(
body=[
If(
test=Name(
id="a",
ctx=Load(),
lineno=2,
col_offset=3,
end_lineno=2,
end_col_offset=4,
),
body=[Pass(lineno=3, col_offset=4, end_lineno=3, end_col_offset=8)],
orelse=[
If(
test=Name(
id="b",
ctx=Load(),
lineno=4,
col_offset=5,
end_lineno=4,
end_col_offset=6,
),
body=[Pass(lineno=5, col_offset=4, end_lineno=5, end_col_offset=8)],
orelse=[
Pass(lineno=7, col_offset=4, end_lineno=7, end_col_offset=8)
],
lineno=4,
col_offset=5,
end_lineno=7,
end_col_offset=8,
)
],
lineno=2,
col_offset=0,
end_lineno=7,
end_col_offset=8,
)
],
type_ignores=[],
)

Thanks, Lysandros for the quick fix!