Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

inspect.getsource doesn't work on functions imported from a zipfile #48473

Closed
exarkun mannequin opened this issue Oct 28, 2008 · 12 comments
Closed

inspect.getsource doesn't work on functions imported from a zipfile #48473

exarkun mannequin opened this issue Oct 28, 2008 · 12 comments
Assignees
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@exarkun
Copy link
Mannequin

exarkun mannequin commented Oct 28, 2008

BPO 4223
Nosy @loewis, @ncoghlan, @abalkin

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = 'https://github.com/ncoghlan'
closed_at = <Date 2008-12-14.12:12:41.067>
created_at = <Date 2008-10-28.17:19:45.958>
labels = ['type-bug', 'library']
title = "inspect.getsource doesn't work on functions imported from a zipfile"
updated_at = <Date 2008-12-14.12:12:41.066>
user = 'https://bugs.python.org/exarkun'

bugs.python.org fields:

activity = <Date 2008-12-14.12:12:41.066>
actor = 'ncoghlan'
assignee = 'ncoghlan'
closed = True
closed_date = <Date 2008-12-14.12:12:41.067>
closer = 'ncoghlan'
components = ['Library (Lib)']
creation = <Date 2008-10-28.17:19:45.958>
creator = 'exarkun'
dependencies = []
files = []
hgrepos = []
issue_num = 4223
keywords = []
message_count = 12.0
messages = ['75292', '75295', '75296', '75298', '75301', '75302', '75303', '75306', '75308', '77772', '77788', '77795']
nosy_count = 5.0
nosy_names = ['loewis', 'exarkun', 'ncoghlan', 'belopolsky', 'gpolo']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue4223'
versions = ['Python 2.6', 'Python 2.5']

@exarkun
Copy link
Mannequin Author

exarkun mannequin commented Oct 28, 2008

It'd be better if it did.

@exarkun exarkun mannequin added stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Oct 28, 2008
@gpolo
Copy link
Mannequin

gpolo mannequin commented Oct 28, 2008

doesn't it ? it works here

@gpolo
Copy link
Mannequin

gpolo mannequin commented Oct 28, 2008

Well anyway, some sample code:

import zipfile

z = zipfile.ZipFile('aaa.zip', mode='w')
z.writestr('aa.py', 'def x(): print "hi there"\n\ndef y(): print "hi"')
z.close()

and then:

import sys
import inspect

sys.path.append('aaa.zip')
import aa

inspect.getsource(aa.x)
inspect.getsource(aa.y)

Doesn't that work for you ?

@exarkun
Copy link
Mannequin Author

exarkun mannequin commented Oct 28, 2008

It seems to depend on working directory:

exarkun@charm:/tmp$ python
Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) 
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append('/home/exarkun/foobar.zip')
>>> import foobar, inspect
>>> inspect.getsource(foobar.foo)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/inspect.py", line 629, in getsource
    lines, lnum = getsourcelines(object)
  File "/usr/lib/python2.5/inspect.py", line 618, in getsourcelines
    lines, lnum = findsource(object)
  File "/usr/lib/python2.5/inspect.py", line 468, in findsource
    raise IOError('could not get source code')
IOError: could not get source code
>>> 

versus:

exarkun@charm:~$ python
Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) 
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append('/home/exarkun/foobar.zip')
>>> import foobar, inspect
>>> inspect.getsource(foobar.foo)
'def foo():\n    pass\n'
>>>

@abalkin
Copy link
Member

abalkin commented Oct 28, 2008

This must be very similar to bpo-4197 and bpo-4201 that I reported a few days
ago. I'll see if a similar patch would work in this case.

@abalkin
Copy link
Member

abalkin commented Oct 28, 2008

Hmm, apparently inspect was made to work with zipped modules back in r45248 .

I cannot reproduce the problem either.

Jean-Paul, can you attach your foobar.zip? What else do you have in your
/tmp directory?

@loewis
Copy link
Mannequin

loewis mannequin commented Oct 28, 2008

In any case, it's clearly not a candidate for 2.5.3, due to the lack of
a clear problem description, and a working patch. That essentially means
that associating the version 2.5 is also fairly pointless, but I'll
leave that in, anyway.

@exarkun
Copy link
Mannequin Author

exarkun mannequin commented Oct 28, 2008

Here is a transcript for a complete, self-contained, minimal reproduction:

exarkun@charm:~$ cd /tmp
exarkun@charm:/tmp$ mkdir zipimportbugexample
exarkun@charm:/tmp$ cd zipimportbugexample/
exarkun@charm:/tmp/zipimportbugexample$ mkdir foobar
exarkun@charm:/tmp/zipimportbugexample$ echo "def foo(): pass" >
foobar/__init__.py
exarkun@charm:/tmp/zipimportbugexample$ zip foobar.zip foobar
  adding: foobar/ (stored 0%)
exarkun@charm:/tmp/zipimportbugexample$ zip foobar.zip foobar/__init__.py 
  adding: foobar/__init__.py (stored 0%)
exarkun@charm:/tmp/zipimportbugexample$ rm -r foobar
exarkun@charm:/tmp/zipimportbugexample$ mkdir workingdirectory
exarkun@charm:/tmp/zipimportbugexample$ cd workingdirectory/
exarkun@charm:/tmp/zipimportbugexample/workingdirectory$ python
Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) 
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append('/tmp/zipimportbugexample/foobar.zip')
>>> import foobar, inspect
>>> inspect.getsource(foobar.foo)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/inspect.py", line 629, in getsource
    lines, lnum = getsourcelines(object)
  File "/usr/lib/python2.5/inspect.py", line 618, in getsourcelines
    lines, lnum = findsource(object)
  File "/usr/lib/python2.5/inspect.py", line 468, in findsource
    raise IOError('could not get source code')
IOError: could not get source code
>>>

@abalkin
Copy link
Member

abalkin commented Oct 28, 2008

Thanks, Jean-Paul, I can now reproduce your problem and I think I found
the culprit.

I believe the file name check in linecache.updatecache is unnecessary
and is responsible for this problem. With the following patch:

--- Lib/linecache.py	(revision 67040)
+++ Lib/linecache.py	(working copy)
@@ -88,7 +88,7 @@
             get_source = getattr(loader, 'get_source', None)
 
             if name and get_source:
-                if basename.startswith(name.split('.')[-1]+'.'):
+#                if basename.startswith(name.split('.')[-1]+'.'):
                     try:
                         data = get_source(name)
                     except (ImportError, IOError):

and Jean-Paul's foobar:

$ PYTHONPATH=/tmp/foobar.zip ./python.exe  -c "import inspect,foobar; 
print inspect.getsource(foobar)"
def foo(): pass

Martin, is there a chance for this issue and bpo-4197, bpo-4201 to make it
to 2.5.3? If so, I'll prepare a combined patch with tests for your
consideration shortly.

@ncoghlan ncoghlan self-assigned this Dec 3, 2008
@ncoghlan
Copy link
Contributor

After looking into this, I think Alexander is correct. There is no
standard mapping between __file__ and __name__ and linecache is mistaken
in assuming such a mapping exists for all importers (and is the same as
the standard filesystem to name mapping to boot).

In this particular case, it was the differences between the way the two
relate for a package vs a normal module that was confusing linecache,
but for more exotic cases the filesystem based rules that linecache was
attempting to enforce may be completely irrelevant.

@ncoghlan
Copy link
Contributor

Fixed for 2.7 in r67750.

Will be ported to 2.6, 3.0 and 3.1.

@ncoghlan
Copy link
Contributor

Final revisions for fix:
2.7 = r67751 (there was a new test file missing from the initial checkin)
2.6 = r67752
3.1 = r67753
3.0 = r67754

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

2 participants