-
-
Notifications
You must be signed in to change notification settings - Fork 29.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ModuleFinder.load_module skips incorrect number of bytes in pyc files #64977
Comments
ModuleFinder.load_module currently only skips 8 bytes before trying to marshal.load the rest of the file, but it should skip 12 since 3.3 (magic, date, file size). I'm attaching a patch with test case. BTW this was very painful to find out, since I couldn't find any reference to written pyc files - am I searching wrong or is this really not documented anywhere? (Note, that this was originally reported at https://bugzilla.redhat.com/show_bug.cgi?id=1060338). |
It's not documented because the format of .pyc files is considered an internal implementation detail. |
Probably want to make sure that modulefinder uses importlib._bootstrap._validate_bytecode_header() to do the parsing. |
Since modulefinder is used by freeze tools (notably cx_Freeze, which seems to be the most popular currently), should this be considered a release blocker? |
It's been broken since Python 3.3 so this is not a 3.3 regression. |
Right. I'm asking if it should be a release blocker for the next 3.3, too :) I'm not saying it should be, just raising the question. It's in my mind because I'm currently using cx_Freeze in a project for a client. It could have affected me, since I was going to release using 3.3, but as it turns out I have to use 2.7 because of a non-ported dependency that I don't have time to port myself. |
Oops, didn't mean to remove the keyword. |
Don't know why this is any more special of a bug because it influences cx_freeze compared to any other bug that influences a popular project. I mean I'm not going to stop you from making it a blocker but I'm also not going to rush to fix it myself either. |
Well, because of the fact that freeze tools are used to distribute programs on the Windows platform. But, given that it hasn't been reported before and has been a problem since 3.3, it seems like there is no rush. |
New changeset 432cb56db05d by Brett Cannon in branch '3.3': New changeset b6e999c8907c by Brett Cannon in branch 'default': |
Fixed in Python 3.3.6 and 3.4.1. Bohuslav, could you sign the contributor agreement at http://www.python.org/psf/contrib/contrib-form/ ? While I didn't directly use your patch this time I noticed you are already in Misc/ACKS so it would be helpful if you could sign the document. |
Brett, I signed it just yesterday, it probably wasn't processed yet. Hopefully it will be in few days. Thanks for patching this! |
Whoops, reopened by accident. Closing again. |
For future reference, cx_Freeze ships its own copy of ModuleFinder, so it doesn't depend on the stdlib copy. This issue was fixed there some time around the release of Python 3.3. I realised recently that this is based on code in the stdlib, and I've been meaning to work out whether cx_Freeze can use any of the stdlib code and lose parts of its own implementation. I guess it's been diverging for quite some time, though. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: