This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author exhuma
Recipients exhuma
Date 2018-06-11.07:16:12
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1528701373.03.0.592728768989.issue33827@psf.upfronthosting.co.za>
In-reply-to
Content
Consider the following code:

    # filename: foo.py

    from functools import lru_cache


    @lru_cache(10)
    def bar():
        yield 10
        yield 20
        yield 30


    # This loop will work as expected
    for row in bar():
        print(row)

    # This loop will not loop over anything.
    # The cache will return an already consumed generator.
    for row in bar():
        print(row)


This behaviour is natural, but it is almost invisible to the caller of "foo".

The main issue is one of "surprise". When inspecting the output of "foo" it is clear that the output is a generator:

    >>> import foo
    >>> foo.bar()
    <generator object bar at 0x7fbfecb66a40>

**Very** careful inspection will reveal that each call will return the same generator instance.

So to an observant user the following is an expected behaviour:

    >>> result = foo.bar()
    >>> for row in result:
    ...    print(row)
    ...
    10
    20
    30
    >>> for row in result:
    ...     print(row)
    ...
    >>>

However, the following is not:

    >>> import foo
    >>> result = foo.bar()
    >>> for row in result:
    ...     print(row)
    ...
    10
    20
    30
    >>> result = foo.bar()
    >>> for row in result:
    ...     print(row)
    ...
    >>>


Would it make sense to emit a warning (or even raise an exception) in `lru_cache` if the return value of the cached function is a generator?

I can think of situation where it makes sense to combine the two. For example the situation I am currently in:

I have a piece of code which loops several times over the same SNMP table. Having a generator makes the application far more responsive. And having the cache makes it even faster on subsequent calls. But the gain I get from the cache is bigger than the gain from the generator. So I would be okay with converting the result to a list before storing it in the cache.

What is your opinion on this issue? Would it make sense to add a warning?
History
Date User Action Args
2018-06-11 07:16:13exhumasetrecipients: + exhuma
2018-06-11 07:16:13exhumasetmessageid: <1528701373.03.0.592728768989.issue33827@psf.upfronthosting.co.za>
2018-06-11 07:16:13exhumalinkissue33827 messages
2018-06-11 07:16:12exhumacreate