classification
Title: Generator expressions trace differently on Windows than on Mac
Type: Stage:
Components: Interpreter Core Versions: Python 3.10
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Mark.Shannon Nosy List: Mark.Shannon, nedbat
Priority: normal Keywords: 3.10regression

Created on 2021-06-18 11:50 by nedbat, last changed 2021-09-01 00:25 by nedbat.

Messages (9)
msg396050 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2021-06-18 11:50
Here is a trace involving generator expressions.  Using 3.10.0b3 on Mac, there are "line" events within the expression.  Those events are missing on Windows.

--- 8< -------------------------------
import linecache, sys

def trace(frame, event, arg):
    # The weird globals here is to avoid a NameError on shutdown...
    if frame.f_code.co_filename == globals().get("__file__"):
        lineno = frame.f_lineno
        print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, lineno).rstrip()))
    return trace

def doit():
    o = ((1,2), (3,4))
    o = (a for a in o)
    for tup in o:
        x = tup[0]
        y = tup[1]

print(sys.version)
sys.settrace(trace)
doit()
---------------------------------------

When run on Mac, it produces this output:


3.10.0b3 (default, Jun 18 2021, 06:43:38) [Clang 12.0.0 (clang-1200.0.32.29)]
call 10: def doit():
line 11:     o = ((1,2), (3,4))
line 12:     o = (a for a in o)
line 13:     for tup in o:
call 12:     o = (a for a in o)
line 12:     o = (a for a in o)
retu 12:     o = (a for a in o)
line 14:         x = tup[0]
line 15:         y = tup[1]
line 13:     for tup in o:
call 12:     o = (a for a in o)
line 12:     o = (a for a in o)
retu 12:     o = (a for a in o)
line 14:         x = tup[0]
line 15:         y = tup[1]
line 13:     for tup in o:
call 12:     o = (a for a in o)
retu 12:     o = (a for a in o)
retu 13:     for tup in o:


When run on Windows, it produces this output:

3.10.0b3 (tags/v3.10.0b3:865714a, Jun 17 2021, 20:39:25) [MSC v.1929 64 bit (AMD64)]
call 10: def doit():
line 11:     o = ((1,2), (3,4))
line 12:     o = (a for a in o)
line 13:     for tup in o:
call 12:     o = (a for a in o)
retu 12:     o = (a for a in o)
line 14:         x = tup[0]
line 15:         y = tup[1]
line 13:     for tup in o:
call 12:     o = (a for a in o)
retu 12:     o = (a for a in o)
line 14:         x = tup[0]
line 15:         y = tup[1]
line 13:     for tup in o:
call 12:     o = (a for a in o)
retu 12:     o = (a for a in o)
retu 13:     for tup in o:

On Windows, the "line 12" events are missing.
msg396126 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-06-19 11:01
Ned, is this a regression (does 3.9 do the right thing on Windows) or an inconsistency between Mac and Windows?

I suspect this might have something to do with the PREDICT macros. If that the were the case 3.9 should show the same inconsistency between Windows and the Mac.
msg396129 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2021-06-19 11:35
This happens with all the 3.10 betas, and does not happen in 3.9 and earlier:
https://github.com/nedbat/coveragepy/runs/2864611225
(Sorry, other failures with earlier 3.10 betas obscured the mac/win difference.)
msg396218 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-06-21 10:08
I think this is a combination of https://bugs.python.org/issue44297 and the PREDICT macros.

I don't have a  windows machine to confirm this on, but I suspect that if you rewrite `doit` as:

def doit():
    o = ((1,2), (3,4))
    o = (a for
         a in
         o)
    for tup in o:
        x = tup[0]
        y = tup[1]

then you should be able to observe a difference between Windows and Mac on 3.9 as well.
msg396227 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2021-06-21 11:14
I tried adding that rewritten doit as a new test, and it does not show a       mac/win difference on 3.9. In fact, it doesn't show a mac/win difference on 3.10!

https://github.com/nedbat/coveragepy/actions/runs/956791631
msg396232 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-06-21 11:31
Hmm, I'm a bit puzzled by that.

Did you test with 3.10b3 or the latest build from the 3.10 branch with the fix to https://bugs.python.org/issue44297 included?
msg396233 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2021-06-21 11:38
This was with 3.10.0b3.  I haven't got a way (yet) to build with my own-built versions of CPython.
msg399502 - (view) Author: Mark Shannon (Mark.Shannon) * (Python committer) Date: 2021-08-13 08:29
Ned, is this still an issue with the release candidate of 3.10?
msg400796 - (view) Author: Ned Batchelder (nedbat) * (Python triager) Date: 2021-09-01 00:25
Looks like this is fixed with 3.10.0rc1, thanks.
History
Date User Action Args
2021-09-01 00:25:32nedbatsetmessages: + msg400796
2021-08-13 08:29:24Mark.Shannonsetmessages: + msg399502
2021-06-21 11:38:32nedbatsetmessages: + msg396233
2021-06-21 11:31:57Mark.Shannonsetmessages: + msg396232
2021-06-21 11:14:28nedbatsetmessages: + msg396227
2021-06-21 10:08:29Mark.Shannonsetmessages: + msg396218
2021-06-19 15:32:11vstinnersetfiles: - ខ្មែរសប់លក់ទំនិញគ្រប់ប្រភេទ
2021-06-19 14:14:16epainesetnosy: - barry, terry.reedy, paul.moore, ronaldoussoren, vstinner, larry, tim.golden, ned.deily, ezio.melotti, eric.araujo, mrabarnett, r.david.murray, asvetlov, zach.ware, yselivanov, koobs, steve.dower, dstufft, Alex.Willmer, lys.nikolaou, pablogsal, habrecord22

type: behavior ->
components: - Build, Demos and Tools, Distutils, Documentation, Extension Modules, IDLE, Installation, Library (Lib), macOS, Regular Expressions, Tests, Tkinter, Unicode, Windows, XML, 2to3 (2.x to 3.x conversion tool), ctypes, IO, Cross-Build, email, asyncio, Argument Clinic, FreeBSD, SSL, C API, Subinterpreters, Parser
title: ខ្មែរសប់លក់ទំនិញ -> Generator expressions trace differently on Windows than on Mac
2021-06-19 11:46:26habrecord22setfiles: + ខ្មែរសប់លក់ទំនិញគ្រប់ប្រភេទ
title: Generator expressions trace differently on Windows than on Mac -> ខ្មែរសប់លក់ទំនិញ
nosy: + pablogsal, terry.reedy, larry, paul.moore, tim.golden, lys.nikolaou, asvetlov, ezio.melotti, koobs, r.david.murray, habrecord22, zach.ware, steve.dower, ned.deily, barry, Alex.Willmer, eric.araujo, dstufft, yselivanov, vstinner, ronaldoussoren, mrabarnett

components: + Build, Demos and Tools, Distutils, Documentation, Extension Modules, IDLE, Installation, Library (Lib), macOS, Regular Expressions, Tests, Tkinter, Unicode, Windows, XML, 2to3 (2.x to 3.x conversion tool), ctypes, IO, Cross-Build, email, asyncio, Argument Clinic, FreeBSD, SSL, C API, Subinterpreters, Parser
type: behavior
2021-06-19 11:35:56nedbatsetmessages: + msg396129
2021-06-19 11:01:10Mark.Shannonsetassignee: Mark.Shannon
messages: + msg396126
2021-06-18 11:50:04nedbatcreate