This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author philipspencer
Recipients philipspencer
Date 2008-06-14.22:42:50
SpamBayes Score 2.980572e-05
Marked as misclassified No
Message-id <1213483378.87.0.0889155831871.issue3115@psf.upfronthosting.co.za>
In-reply-to
Content
Python's os.listdir function has the following code in
Modules/posixmodule.c:

   errno = 0
   ...
    for (;;) {
         Py_BEGIN_ALLOW_THREADS
         ep = readdir(dirp);
         Py_END_ALLOW_THREADS
         if (ep == NULL)
              break;
         ...
         a bunch of other stuff, including PyString_FromStringAndSize
         which calls malloc
         ...
    }
    if (errno != 0 && d != NULL) {

The assumption is that errno will be nonzero only if readdir failed.
However, this is not the case. GLibc's malloc will, in some rare cases,
set errno even when it succeeds. So, during one pass through the loop
errno gets set to ENOMEM and then it is still set to ENOMEM when the
final readdir returns null at the end of the directory listing.

The fix is to move the line "errno = 0" from outside the loop to
right before the readdir inside the loop. That is the only way to
ensure that, when the loop is exited with readdir returning null,
the condition "errno != 0" is equivalent to "readdir actually failed."

The attached patch does this. Without this patch, we experience
frequent failures with the python tools "creatrepo" and "repomanage"
when run on very large directories (> 5000 packages) on Fedora 8
systems; see RedHat bugzilla entry #451494. With the patch, the commands
work as expected.

The patch is against 2.5.1 but from what I can see of the posixmodule.c
code from SVN it should apply cleanly there too.
History
Date User Action Args
2008-06-14 22:42:59philipspencersetspambayes_score: 2.98057e-05 -> 2.980572e-05
recipients: + philipspencer
2008-06-14 22:42:58philipspencersetspambayes_score: 2.98057e-05 -> 2.98057e-05
messageid: <1213483378.87.0.0889155831871.issue3115@psf.upfronthosting.co.za>
2008-06-14 22:42:56philipspencerlinkissue3115 messages
2008-06-14 22:42:54philipspencercreate