I agree with you in regards to the statement, "Apache's redirect can be turned off, whereas this can't."  That is why my first thought would be to try and control this response to the client a bit better by providing a variable max-age.

self.send_header("Cache-Control", "max-age=xxx")

Removing the condition completely would be good idea as well, but would be my second course of action due to this functionality being in the Python code base for awhile now.

if os.path.isdir(path):
    for index in "index.html", "index.htm":
        index = os.path.join(path, index)
        if os.path.exists(index):
            path = index
            break


Those are my thoughts on the matter at least, possibly someone else in the community could weigh in as well?