This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: inspect.getsource doesn't work on functions imported from a zipfile
Type: behavior Stage:
Components: Library (Lib) Versions: Python 2.6, Python 2.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ncoghlan Nosy List: belopolsky, exarkun, gpolo, loewis, ncoghlan
Priority: normal Keywords:

Created on 2008-10-28 17:19 by exarkun, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (12)
msg75292 - (view) Author: Jean-Paul Calderone (exarkun) * (Python committer) Date: 2008-10-28 17:19
It'd be better if it did.
msg75295 - (view) Author: Guilherme Polo (gpolo) * (Python committer) Date: 2008-10-28 17:27
doesn't it ? it works here
msg75296 - (view) Author: Guilherme Polo (gpolo) * (Python committer) Date: 2008-10-28 17:43
Well anyway, some sample code:

import zipfile

z = zipfile.ZipFile('aaa.zip', mode='w')
z.writestr('aa.py', 'def x(): print "hi there"\n\ndef y(): print "hi"')
z.close()


and then:


import sys
import inspect

sys.path.append('aaa.zip')
import aa

inspect.getsource(aa.x)
inspect.getsource(aa.y)


Doesn't that work for you ?
msg75298 - (view) Author: Jean-Paul Calderone (exarkun) * (Python committer) Date: 2008-10-28 17:51
It seems to depend on working directory:

exarkun@charm:/tmp$ python
Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) 
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append('/home/exarkun/foobar.zip')
>>> import foobar, inspect
>>> inspect.getsource(foobar.foo)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/inspect.py", line 629, in getsource
    lines, lnum = getsourcelines(object)
  File "/usr/lib/python2.5/inspect.py", line 618, in getsourcelines
    lines, lnum = findsource(object)
  File "/usr/lib/python2.5/inspect.py", line 468, in findsource
    raise IOError('could not get source code')
IOError: could not get source code
>>> 

versus:

exarkun@charm:~$ python
Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) 
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append('/home/exarkun/foobar.zip')
>>> import foobar, inspect
>>> inspect.getsource(foobar.foo)
'def foo():\n    pass\n'
>>>
msg75301 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-10-28 18:51
This must be very similar to #4197 and #4201 that I reported a few days 
ago.  I'll see if a similar patch would work in this case.
msg75302 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-10-28 19:24
Hmm, apparently inspect was made to work with zipped modules back in r45248 .

I cannot reproduce the problem either.

Jean-Paul, can you attach your foobar.zip?  What else do you have in your 
/tmp directory?
msg75303 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-10-28 21:48
In any case, it's clearly not a candidate for 2.5.3, due to the lack of
a clear problem description, and a working patch. That essentially means
that associating the version 2.5 is also fairly pointless, but I'll
leave that in, anyway.
msg75306 - (view) Author: Jean-Paul Calderone (exarkun) * (Python committer) Date: 2008-10-28 21:59
Here is a transcript for a complete, self-contained, minimal reproduction:

exarkun@charm:~$ cd /tmp
exarkun@charm:/tmp$ mkdir zipimportbugexample
exarkun@charm:/tmp$ cd zipimportbugexample/
exarkun@charm:/tmp/zipimportbugexample$ mkdir foobar
exarkun@charm:/tmp/zipimportbugexample$ echo "def foo(): pass" >
foobar/__init__.py
exarkun@charm:/tmp/zipimportbugexample$ zip foobar.zip foobar
  adding: foobar/ (stored 0%)
exarkun@charm:/tmp/zipimportbugexample$ zip foobar.zip foobar/__init__.py 
  adding: foobar/__init__.py (stored 0%)
exarkun@charm:/tmp/zipimportbugexample$ rm -r foobar
exarkun@charm:/tmp/zipimportbugexample$ mkdir workingdirectory
exarkun@charm:/tmp/zipimportbugexample$ cd workingdirectory/
exarkun@charm:/tmp/zipimportbugexample/workingdirectory$ python
Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) 
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append('/tmp/zipimportbugexample/foobar.zip')
>>> import foobar, inspect
>>> inspect.getsource(foobar.foo)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/inspect.py", line 629, in getsource
    lines, lnum = getsourcelines(object)
  File "/usr/lib/python2.5/inspect.py", line 618, in getsourcelines
    lines, lnum = findsource(object)
  File "/usr/lib/python2.5/inspect.py", line 468, in findsource
    raise IOError('could not get source code')
IOError: could not get source code
>>>
msg75308 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-10-28 22:46
Thanks, Jean-Paul, I can now reproduce your problem and I think I found 
the culprit.

I believe the file name check in linecache.updatecache is unnecessary 
and is responsible for this problem.  With the following patch:

--- Lib/linecache.py	(revision 67040)
+++ Lib/linecache.py	(working copy)
@@ -88,7 +88,7 @@
             get_source = getattr(loader, 'get_source', None)
 
             if name and get_source:
-                if basename.startswith(name.split('.')[-1]+'.'):
+#                if basename.startswith(name.split('.')[-1]+'.'):
                     try:
                         data = get_source(name)
                     except (ImportError, IOError):


and Jean-Paul's foobar:

$ PYTHONPATH=/tmp/foobar.zip ./python.exe  -c "import inspect,foobar; 
print inspect.getsource(foobar)"
def foo(): pass


Martin, is there a chance for this issue and  #4197, #4201  to make it 
to 2.5.3?  If so, I'll prepare a combined patch with tests for your 
consideration shortly.
msg77772 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-12-14 01:31
After looking into this, I think Alexander is correct. There is no
standard mapping between __file__ and __name__ and linecache is mistaken
in assuming such a mapping exists for all importers (and is the same as
the standard filesystem to name mapping to boot).

In this particular case, it was the differences between the way the two
relate for a package vs a normal module that was confusing linecache,
but for more exotic cases the filesystem based rules that linecache was
attempting to enforce may be completely irrelevant.
msg77788 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-12-14 10:59
Fixed for 2.7 in r67750.

Will be ported to 2.6, 3.0 and 3.1.
msg77795 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-12-14 12:12
Final revisions for fix:
2.7 = r67751 (there was a new test file missing from the initial checkin)
2.6 = r67752
3.1 = r67753
3.0 = r67754
History
Date User Action Args
2022-04-11 14:56:40adminsetgithub: 48473
2008-12-14 12:12:41ncoghlansetstatus: open -> closed
resolution: fixed
messages: + msg77795
2008-12-14 10:59:13ncoghlansetmessages: + msg77788
2008-12-14 01:31:52ncoghlansetmessages: + msg77772
2008-12-03 20:29:22ncoghlansetassignee: ncoghlan
nosy: + ncoghlan
2008-10-28 22:46:59belopolskysetmessages: + msg75308
2008-10-28 21:59:39exarkunsetmessages: + msg75306
2008-10-28 21:48:40loewissetnosy: + loewis
messages: + msg75303
versions: - Python 2.5.3
2008-10-28 19:24:13belopolskysetmessages: + msg75302
2008-10-28 18:51:49belopolskysetnosy: + belopolsky
messages: + msg75301
2008-10-28 17:51:24exarkunsetmessages: + msg75298
2008-10-28 17:43:54gpolosetmessages: + msg75296
2008-10-28 17:27:30gpolosetnosy: + gpolo
messages: + msg75295
2008-10-28 17:19:45exarkuncreate