This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Possible performance improvement for itertools.product example on Python Docs
Type: enhancement Stage: resolved
Components: Documentation Versions:
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: docs@python Nosy List: ataher, docs@python, rhettinger
Priority: normal Keywords:

Created on 2020-06-25 23:11 by ataher, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
product example.py ataher, 2020-06-25 23:11 Contains the 3 versions and 3 outputs
Messages (3)
msg372392 - (view) Author: Abbas Taher (ataher) Date: 2020-06-25 23:11
In the documentation the following example is given:

def product(*args, repeat=1):
    # product('ABCD', 'xy') --> Ax Ay Bx By Cx Cy Dx Dy
    # product(range(2), repeat=3) --> 000 001 010 011 100 101 110 111
    pools = [tuple(pool) for pool in args] * repeat
    result = [[]]
    for pool in pools:
        result = [x+[y] for x in result for y in pool]
    for prod in result:
        yield tuple(prod)

The proposed enhancement uses a nested generator so no intermediate results are created.

def product2(*args, repeat=1):
    def concat(result, pool):
        yield from (x+[y] for x in result for y in pool)
        
    pools = [tuple(pool) for pool in args] * repeat
    result = [[]]
    for pool in pools:
        result = concat(result, pool)
    for prod in result:
        yield (tuple(prod))
msg372397 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2020-06-26 01:19
Thanks for the suggestion, but the purpose of the rough equivalent example is to give a better idea of what product() does.  It is not about performance — that job falls to the actual implementation.

Nested generators are an intermediate level Python skill, so using them in the example code makes the docs less accessible.
msg372418 - (view) Author: Abbas Taher (ataher) Date: 2020-06-26 09:39
Good morning Raymond,   Thank you for this quick response.
You are absolutely right on all your points if you only consider that the example given in the documentation as purely a suggestion of "equivalent" version to the one implemented in the library.
However, if you look at the examples given in the documentation as an illustration of an algorithm that can be customized and changed by the end-user if the existing implementation is not suitable then you might consider otherwise. 
This is exactly what happened when I wanted to do a Cartesian Product. Starting, I first looked at itertools and 
run my routine using itertools.product(). However, it turns out that I needed to flatten the result tuples which prompted me toadjust the given example accordingly and then add the nested generator.

As for your statement:  "Nested generators are an intermediate level Python skill, so using them in the example code makes the docs less accessible.". Let me point to the facts, that someone looking at itertools and trying to do a Cartesian Product 
is certainly an intermediate programmer and often likes to understand, learn or peek inside the algorithm used. Additionally, the suggested improvement is a generator similar to the implementation where as the existing example is not.

Thus, I hope I am able to convince you to reconsider your decision especially when you look at the given examples from a wider perspective and more so when the suggested improvement is minor and easy to understand.
Best Regards,
Abbas

Side note: one of the first tutorial videos I used to learn Python were the ones you prepared for Oreilly. Thank you and thank you for all the work you did and still doing for free for the Python language and the Python community over so many many many years.

    On Thursday, June 25, 2020, 9:19:55 PM EDT, Raymond Hettinger <report@bugs.python.org> wrote:  

Raymond Hettinger <raymond.hettinger@gmail.com> added the comment:

Thanks for the suggestion, but the purpose of the rough equivalent example is to give a better idea of what product() does.  It is not about performance — that job falls to the actual implementation.

Nested generators are an intermediate level Python skill, so using them in the example code makes the docs less accessible.

----------
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41120>
_______________________________________
History
Date User Action Args
2022-04-11 14:59:32adminsetgithub: 85292
2020-06-26 09:39:22atahersetmessages: + msg372418
2020-06-26 01:19:47rhettingersetstatus: open -> closed
resolution: rejected
messages: + msg372397

stage: resolved
2020-06-26 00:36:13xtreaksetnosy: + rhettinger
2020-06-25 23:11:22atahercreate