classification
Title: inspect.findsource() returns binary data for shared library modules
Type: behavior Stage: commit review
Components: Library (Lib) Versions: Python 3.1, Python 2.7
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: r.david.murray Nosy List: ajaksu2, brodie, pitrou, r.david.murray, vstinner
Priority: normal Keywords: patch

Created on 2008-10-06 03:20 by brodie, last changed 2009-05-13 18:02 by r.david.murray. This issue is now closed.

Files
File name Uploaded Description Edit
inspect-findsource-binary.diff brodie, 2008-10-06 03:20
binary_source_test.diff ajaksu2, 2009-04-25 21:10 Check that findsource(C-module) raises IO error instead of returning garbage
Messages (6)
msg74358 - (view) Author: Brodie Rao (brodie) Date: 2008-10-06 03:20
Calling inspect.findsource() on a module whose __file__ attribute points 
to a shared library causes findsource() to return the binary's data:

>>> import time
>>> time.__file__
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/
lib-dynload/time.so'
>>> import inspect
>>> inspect.findsource(time)
(['\xca\xfe\xba\xbe\x00\x00\x00\x04\x00\x00\x00\x12\x00\x00\x00\n', 
'\x00\x00\x10\x00\x00\x00Jt\x00...

It should raise an error saying it can't retrieve the source. It doesn't 
do this because when it tries to find the source file it calls both 
getsourcefile() - and getfile() when that fails. It should call only 
getsourcefile() - which checks to see if __file__ points to a binary - 
and if the result is None, raise an error.

I'm attaching a patch against trunk that fixes this. The issue affects 
3.0, 2.6, 2.5, and probably previous versions.
msg86553 - (view) Author: Daniel Diniz (ajaksu2) (Python triager) Date: 2009-04-25 21:10
Test for the patch.
msg87640 - (view) Author: Daniel Diniz (ajaksu2) (Python triager) Date: 2009-05-12 16:14
This bug seems to break help(C-module) in py3k after rev 70587:

>>> import pickle
>>> help(pickle)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "~/py3k/Lib/site.py", line 429, in __call__
    return pydoc.help(*args, **kwds)
  File "~/py3k/Lib/pydoc.py", line 1709, in __call__
    self.help(request)
  File "~/py3k/Lib/pydoc.py", line 1755, in help
    else: doc(request, 'Help on %s:')
  File "~/py3k/Lib/pydoc.py", line 1505, in doc
    pager(render_doc(thing, title, forceload))
  File "~/py3k/Lib/pydoc.py", line 1500, in render_doc
    return title % desc + '\n\n' + text.document(object, name)
  File "~/py3k/Lib/pydoc.py", line 320, in document
    if inspect.ismodule(object): return self.docmodule(*args)
  File "~/py3k/Lib/pydoc.py", line 1086, in docmodule
    contents.append(self.document(value, key, name))
  File "~/py3k/Lib/pydoc.py", line 321, in document
    if inspect.isclass(object): return self.docclass(*args)
  File "~/py3k/Lib/pydoc.py", line 1131, in docclass
    doc = getdoc(object)
  File "~/py3k/Lib/pydoc.py", line 81, in getdoc
    result = inspect.getdoc(object) or inspect.getcomments(object)
  File "~/py3k/Lib/inspect.py", line 581, in getcomments
    lines, lnum = findsource(object)
  File "~/py3k/Lib/inspect.py", line 524, in findsource
    lines = linecache.getlines(file, module.__dict__)
  File "~/py3k/Lib/linecache.py", line 41, in getlines
    return updatecache(filename, module_globals)
  File "~/py3k/Lib/linecache.py", line 130, in updatecache
    lines = fp.readlines()
  File "~/py3k/Lib/codecs.py", line 300, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x80 in position 24:
unexpected code byte


brodierao's patch fixes it.
msg87691 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2009-05-13 16:03
It's true inspect.findsource() behaves likes this, but help() has only
been broken very recently (and only in py3k), so I suggest these are
separate bugs.
msg87696 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-05-13 17:45
Patches applied to 2.7 in r72605 and 3.1 in r72607, after discussion on
#python-dev.  Blocked on 2.6 since it represents a behavior change and
doesn't actually break anything in the 2 series.

The change that introduced the pydoc regression was not backported to
3.0, which opens the question of whether or not this fix should be
backported.
msg87698 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2009-05-13 18:02
Decision is not to backport.
History
Date User Action Args
2009-05-13 18:02:52r.david.murraysetstatus: open -> closed

messages: + msg87698
versions: + Python 3.1, Python 2.7, - Python 3.0
2009-05-13 17:45:29r.david.murraysetpriority: release blocker -> normal
versions: - Python 3.1, Python 2.7
messages: + msg87696

resolution: fixed
stage: patch review -> commit review
2009-05-13 16:28:07r.david.murraysetassignee: r.david.murray

nosy: + r.david.murray
versions: + Python 2.7, - Python 2.6
2009-05-13 16:03:56pitrousetnosy: + pitrou
messages: + msg87691
2009-05-12 16:14:34ajaksu2setpriority: normal -> release blocker
versions: + Python 3.1
nosy: + vstinner

messages: + msg87640
2009-04-25 21:10:02ajaksu2setfiles: + binary_source_test.diff
priority: normal

versions: - Python 2.5
nosy: + ajaksu2

messages: + msg86553
stage: patch review
2009-04-25 20:51:20ajaksu2linkissue5742 superseder
2008-10-06 03:20:45brodiecreate