This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: import on Windows will recompile a pyc file created on Unix
Type: behavior Stage:
Components: Interpreter Core Versions: Python 3.1, Python 3.2, Python 3.3, Python 2.7, Python 2.6
process
Status: closed Resolution: duplicate
Dependencies: Superseder: import.c sometimes generates incorrect timestamps on Windows + NTFS
View: 13863
Assigned To: Nosy List: mackeith, mark.dickinson, pitrou
Priority: normal Keywords:

Created on 2012-07-12 15:32 by mackeith, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
testImportRe.py mackeith, 2012-07-12 15:32 The attached file tests the symptom described above.
Messages (3)
msg165309 - (view) Author: Andrew MacKeith (mackeith) Date: 2012-07-12 15:32
In certain cases, a compiled Python file (.pyc) created on Unix will be recompiled when imported on Windows, despite the original code file (.py) being the same.
The cause is the use of the c fstat function in import.c.

This behavior is contrary to the stated Python behavior that a precompiled Python file is portable between platforms.

The fix to this bug would be to use the posixmodule stat implementation in place of fstat in import.c.

An example is the following, in Python 2.6.2:

This was tested on July 11th, 2012, USA EDT 
(i.e. DST is in effect, and local time is GMT - 6 hours). 
Lib/re.py has a mtime of 01/01/2009  11:46 AM according to Windows DIR.

Windows fstat st_mtime returns 1230828370 (0x495cf352) 16:46 GMT
Linux   fstat st_mtime returns 1230824770 (0x495ce542) 15:46 GMT

The effect of this is that when import opens the .pyc file and reads the mtime (the 2nd 4-byte field of the .pyc file) it sees it as "bad" and therefore recompiles the file.

The Python os.stat function works correctly on both Windows and Unix. There is a note in posixmodule.c that warns of a problem with Windows fstat:
"""
#ifdef MS_WINDOWS
/* The CRT of Windows has a number of flaws wrt. its stat() implementation:
   - time stamps are restricted to second resolution
   - file modification times suffer from forth-and-back conversions between
     UTC and local time
   Therefore, we implement our own stat, based on the Win32 API directly.
*/
"""

A Python file can also be precompiled using py_compile.compile; this uses os.stat to get the mtime of the .py file, and writes that to the .pyc file. 
This has the Unix behavior, and is "correct".

Output from time functions on Linux:
(The results are the same on Windows)

>>> print time.gmtime(0x495ce542)
time.struct_time(tm_year=2009, tm_mon=1, tm_mday=1, tm_hour=15, tm_min=46, tm_sec=10, tm_wday=3, tm_yday=1, tm_isdst=0)

>>> print time.localtime(0x495ce542)
time.struct_time(tm_year=2009, tm_mon=1, tm_mday=1, tm_hour=10, tm_min=46, tm_sec=10, tm_wday=3, tm_yday=1, tm_isdst=0)

>>> print time.gmtime(0x495cf352)
time.struct_time(tm_year=2009, tm_mon=1, tm_mday=1, tm_hour=16, tm_min=46, tm_sec=10, tm_wday=3, tm_yday=1, tm_isdst=0)

>>> print time.localtime(0x495cf352)
time.struct_time(tm_year=2009, tm_mon=1, tm_mday=1, tm_hour=11, tm_min=46, tm_sec=10, tm_wday=3, tm_yday=1, tm_isdst=0)

>>>

There has been no change in the relevant import.c code between Python-2.6 and Python-3.3.

X:\work\testimportmtime>c:\python26\python
Python 2.6.5 (r265:79096, Mar 19 2010, 18:02:59) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> print platform.platform()
Windows-post2008Server-6.1.7601-SP1
>>> print platform.architecture()
('64bit', 'WindowsPE')
>>> print platform.python_compiler()
MSC v.1500 64 bit (AMD64)
msg165310 - (view) Author: Mark Dickinson (mark.dickinson) * (Python committer) Date: 2012-07-12 15:37
Is this the same issue as issue #13863?
msg165312 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2012-07-12 15:40
> Is this the same issue as issue #13863?

Judging by Andrew's diagnosis, I think so.
History
Date User Action Args
2022-04-11 14:57:32adminsetgithub: 59538
2012-07-12 15:40:55pitrousetstatus: open -> closed

nosy: + pitrou
messages: + msg165312

superseder: import.c sometimes generates incorrect timestamps on Windows + NTFS
resolution: duplicate
2012-07-12 15:37:00mark.dickinsonsetnosy: + mark.dickinson
messages: + msg165310
2012-07-12 15:32:20mackeithcreate