classification
Title: itertools.product not lazy
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.8
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: rhettinger Nosy List: miss-islington, nougmanoff, remi.lapeyre, rhettinger, tim.peters
Priority: normal Keywords: patch

Created on 2020-05-28 12:55 by nougmanoff, last changed 2020-05-28 16:58 by rhettinger. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 20492 merged nougmanoff, 2020-05-28 15:34
PR 20498 merged miss-islington, 2020-05-28 16:46
Messages (5)
msg370201 - (view) Author: Ramil Nugmanov (nougmanoff) * Date: 2020-05-28 12:55
def x(y):
    while True:
        yield y

p = product(x(1), x(2))

next(p)  # this string will never be reached.
msg370203 - (view) Author: RĂ©mi Lapeyre (remi.lapeyre) * Date: 2020-05-28 13:05
Hi Ramil, itertools.product() expect its argument to be finite iterables, it needs to keep all their elements in memory anyway at it "cycles around" to produce all possible pairs.

Basically, product(x(1), x(2)) is equivalent to product(tuple(x(1)), tuple(x(2))).


I see that the documentation does not mention that the arguments must be finite, could you open a PR to improve it?
msg370246 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-05-28 16:46
New changeset cfc6ce4d40f2f01314b7e283fb972a7bb3ed3faa by Ramil Nugmanov in branch 'master':
bpo-40806: Clarify that itertools.product immediately consumes its inpt (GH-20492)
https://github.com/python/cpython/commit/cfc6ce4d40f2f01314b7e283fb972a7bb3ed3faa
msg370249 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-05-28 16:50
Updated the docs to note that inputs are fully consumed before the iterator is run.

FYI, notes on infinite iterable inputs likely belong the FAQs because they aren't specific to product().  Similar effects would be seen with list(x(1)), sorted(x(1)), set(x(1)), tuple(x(1)), etc.
msg370250 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-05-28 16:58
New changeset e4cc3a7c1f5ba9ea2c3015a5bf09cb5b93db5d47 by Miss Islington (bot) in branch '3.9':
bpo-40806: itertools.product immediately consumes its inputs (GH-20492) (GH-20498)
https://github.com/python/cpython/commit/e4cc3a7c1f5ba9ea2c3015a5bf09cb5b93db5d47
History
Date User Action Args
2020-05-28 16:58:36rhettingersetmessages: + msg370250
2020-05-28 16:50:13rhettingersetstatus: open -> closed
resolution: fixed
messages: + msg370249

stage: patch review -> resolved
2020-05-28 16:46:36miss-islingtonsetnosy: + miss-islington
pull_requests: + pull_request19747
2020-05-28 16:46:26rhettingersetmessages: + msg370246
2020-05-28 15:43:27rhettingersetassignee: rhettinger
2020-05-28 15:34:32nougmanoffsetkeywords: + patch
stage: patch review
pull_requests: + pull_request19741
2020-05-28 13:06:10remi.lapeyresetnosy: + tim.peters, rhettinger
2020-05-28 13:05:53remi.lapeyresetnosy: + remi.lapeyre
messages: + msg370203
2020-05-28 12:55:46nougmanoffcreate