This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author loewis
Recipients Trundle, brian.curtin, giampaolo.rodola, loewis, neologix, nvetoshkin, pitrou, socketpair, terry.reedy
Date 2011-03-11.09:43:00
SpamBayes Score 0.0
Marked as misclassified No
Message-id <4D79EEA2.40904@v.loewis.de>
In-reply-to <1299821146.39.0.151638795504.issue11406@psf.upfronthosting.co.za>
Content
> http://lwn.net/Articles/216948/
> Why kernel.org is slow

That's all independent of the issue at hand. Whether or not getdents is
slow doesn't really matter here because we need to call getdents,
anyway: it's the only API to get at the directory contents, iterative
or not.

The issue at hand is whether xlistdir actually provides any advantages
to a real application, and that cannot be answered by looking at
benchmarking results that benchmarked the kernel. The *only* way to
determine whether xlistdir will help is to measure it in a real
application.

I stand by my claim that
a) in cases where you use listdir, you typically have to consider
    all file names in the directory eventually. The total number
    of getdents calls to be made doesn't change when you switch from
    listdir to xlistdir (except in non-realistic micro-benchmarks).

    The cases that you don't need to look at all file names are
    typically dealt with by knowing the file name in advance
    (or perhaps a few alternative spellings it may have), and
    hence you don't use listdir at all (but stat/open).

b) If there is some real-world processing of the files (e.g.
    at least to look at the file modification time), this processing
    (and the IO that goes along with it) by far outweigh the cost
    of reading the directory. So even if you could speed up listdir
    by making it iterative, the overall gain would be very small.

There are also good reasons *not* to add xlistdir, primarily to
avoid user confusion. If xlistdir was added, all peope would run
off and change all applications of listdir "because it is faster",
and have then to deal with backwards compatibility, even though
in most applications, a single getdents call will fetch the entire
directory contents just fine (and hence there is *no* change
in xlistdir, except that the list is not created which uses
a few dozen bytes).
History
Date User Action Args
2011-03-11 09:43:02loewissetrecipients: + loewis, terry.reedy, pitrou, giampaolo.rodola, Trundle, brian.curtin, nvetoshkin, neologix, socketpair
2011-03-11 09:43:01loewislinkissue11406 messages
2011-03-11 09:43:00loewiscreate