Author serhiy.storchaka
Recipients gvanrossum, serhiy.storchaka
Date 2021-06-22.05:40:17
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1624340418.29.0.31085247702.issue44482@roundup.psfhosted.org>
In-reply-to
Content
There is possible resource leak in glob on alternate Python implementation witch do not use reference counting to immediate releasing resources.

It is in the code

    names = list(_iterdir(dirname, dir_fd, dironly))

where _iterdir() is a generator function which calls os.scandir() and yields entry names after some filtering. If an exception happens inside _iterdir(), the scandir iterator will be immediately closed, because of using the with statement. But an exception can happens outside of _iterdir(), in the list constructor (MemoryError). In this case the generator will be closed immediately in CPython because of reference counting (newly created generator has only one reference), but on other implementation it can be deferred on arbitrary time. And even in CPython we rely on garbage collector if there reference loops in the generator frame. This issue has very small chance to occur but still.

The right way is to close the generator explicitly:

    it = _iterdir(dirname, dir_fd, dironly)
    try:
        names = list(it)
    finally:
        it.close()

or

    with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
        names = list(it)

We should analyze all other generator functions which acquire some resources and ensure that they are always properly closed.
History
Date User Action Args
2021-06-22 05:40:18serhiy.storchakasetrecipients: + serhiy.storchaka, gvanrossum
2021-06-22 05:40:18serhiy.storchakasetmessageid: <1624340418.29.0.31085247702.issue44482@roundup.psfhosted.org>
2021-06-22 05:40:18serhiy.storchakalinkissue44482 messages
2021-06-22 05:40:17serhiy.storchakacreate