classification
Title: Importlib.reload has different behaviour in script and interactive mode
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.5
process
Status: closed Resolution: rejected
Dependencies: Superseder:
Assigned To: Nosy List: brett.cannon, orpembery
Priority: normal Keywords:

Created on 2018-11-28 22:03 by orpembery, last changed 2018-11-29 18:55 by brett.cannon. This issue is now closed.

Messages (2)
msg330631 - (view) Author: Owen Pembery (orpembery) Date: 2018-11-28 22:03
This may be a bug, or may be a hole in my understanding of how importlib.reload works.

In short, when running importlib.reload in a script, it seems that a module is not reloaded before it is used, whereas running the same script a line at a time in interactive mode seems to reload the script.

Relevant scripts are in the github repository: https://github.com/orpembery/python-reload-bug.

I find the bug using python 3.5.2 on Ubuntu 16.04.

The expected output from running

python3 master.py

is:
1
2

However, I instead find that the output is:
1
1

If I paste the commands in master.py into the interpreter one at a time, I get the expected behaviour:

Python 3.5.2 (default, Nov 12 2018, 13:43:14) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from shutil import copy
>>> 
>>> from importlib import reload
>>> # Move a file
... copy('fn_1.py','fn.py')
'fn.py'
>>> # Import it
... import fn
>>> # Run it
... fn.fn()
1
>>> # Move a different file to the same location.
... copy('fn_2.py','fn.py')
'fn.py'
>>> # Reload it
... reload(fn)
<module 'fn' from '/home/owen/code/python-reload-bug/fn.py'>
>>> # Run it
... fn.fn()
2


However, if the commands in master.py are pasted in all at one (i.e., with a single CTRL-V keystroke), I obtain the unexpected behaviour:
Python 3.5.2 (default, Nov 12 2018, 13:43:14) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from shutil import copy
>>> from importlib import reload
>>> 
>>> # Move a file
... copy('fn_1.py','fn.py')
'fn.py'
>>> 
>>> # Import it
... import fn
>>> 
>>> # Run it
... fn.fn()
1
>>> 
>>> # Move a different file to the same location.
... copy('fn_2.py','fn.py')
'fn.py'
>>> 
>>> # Reload it
... reload(fn)
<module 'fn' from '/home/owen/code/python-reload-bug/fn.py'>
>>> 
>>> # Run it
... fn.fn()
1


Possible causes:
I'm very far from an expert, but it seems to be something to do with caching; running

python3 -B master.py

gives the expected behaviour.
msg330711 - (view) Author: Brett Cannon (brett.cannon) * (Python committer) Date: 2018-11-29 18:55
It will because your file change happens so quickly by script that your file system isn't leading to a different mtime on the source, and so the bytecode isn't being regenerated. When you do it by hand you're physically slow enough to have the file copies take long enough for your file system to record the new mtime.

You can try using importlib.invalidate_caches() to help with this. Otherwise realize that importlib.reload() is really meant for interactive use and not script use anyway because it doesn't update old references to the module.
History
Date User Action Args
2018-11-29 18:55:28brett.cannonsetstatus: open -> closed
resolution: rejected
messages: + msg330711

stage: resolved
2018-11-28 22:04:25orpemberysettype: behavior
2018-11-28 22:03:48orpemberycreate