This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

Author syeberman
Recipients syeberman
Date 2012-11-01.18:56:18
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1351796179.35.0.165892579115.issue16384@psf.upfronthosting.co.za>
In-reply-to
Content
The PyMarshal_Read* functions raise EOFError when the end of the file is unexpectedly met.  The current import.c functions propagate this error when reading .pyc or .pyo files.  One consequence of this is that Python will abort on startup if, say, encodings.utf_8's .pyc file is truncated.

I have encountered a scenario where this truncation is common.  If a second Python process is launched while the first is writing the "core" .pyc files, the second process may open the files before they are completely written and will thus see truncated files and abort.  This is a race condition that I was able to reproduce consistently on several Windows Server 2008 RC2 Standard SP1 machines running 32-bit Python 3.2.3 from GNU make with "-j 16" (Intel Xeon E5405 2GHz 2 processors 8GB 64-bit OS).  (Of course, I had to clean the __pycache__ directories between tests.)

This can be fixed in load_source_module by making read_compiled_module failures non-fatal:
    if (cpathname != NULL &&
        (fpc = check_compiled_module(pathname, st.st_mtime, cpathname))) {
        co = read_compiled_module(cpathname, fpc);
        if (co == NULL) PyErr_Clear();
        fclose(fpc);
    }
    if (co != NULL) {
        // etc...
    }
    else {
        co = parse_source_module(pathname, fp);
        // etc...
                write_compiled_module(co, cpathname, &st);
    }
This is similar to how write_compiled_module ignores failures in writing the .pyc files.  It ensures that if the .pyc file is corrupt for _any_ reason, it will get rewritten; this could be made specific to EOFError, but I don't recommed that.  Mostly, it ensures that corrupt .pyc files do not prevent Python from loading if the .py files are valid.
History
Date User Action Args
2012-11-01 18:56:19syebermansetrecipients: + syeberman
2012-11-01 18:56:19syebermansetmessageid: <1351796179.35.0.165892579115.issue16384@psf.upfronthosting.co.za>
2012-11-01 18:56:19syebermanlinkissue16384 messages
2012-11-01 18:56:18syebermancreate