This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Doctest module does not work with zipped packages
Type: crash Stage:
Components: Library (Lib) Versions: Python 2.7, Python 2.6
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: ncoghlan Nosy List: belopolsky, loewis, ncoghlan
Priority: normal Keywords: patch

Created on 2008-10-24 21:59 by belopolsky, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
testmodule.zip belopolsky, 2008-10-24 21:59 Zip file with a module containing doctest inside
doctest.patch belopolsky, 2008-10-24 22:07
doctest-1.patch belopolsky, 2008-10-24 22:12
Messages (14)
msg75186 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-10-24 21:59
With attached teestmodule.zip and svn revision 67006,

$ ./python.exe testmodule.zip 
Traceback (most recent call last):
  File "/Users/sasha/Work/python-svn/trunk/Lib/runpy.py", line 121, in 
_run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/Users/sasha/Work/python-svn/trunk/Lib/runpy.py", line 34, in 
_run_code
    exec code in run_globals
  File "testmodule.zip/__main__.py", line 7, in <module>
  File "/Users/sasha/Work/python-svn/trunk/Lib/doctest.py", line 1817, 
in testmod
    for test in finder.find(m, name, globs=globs, 
extraglobs=extraglobs):
  File "/Users/sasha/Work/python-svn/trunk/Lib/doctest.py", line 842, in 
find
    self._find(tests, obj, name, module, source_lines, globs, {})
  File "/Users/sasha/Work/python-svn/trunk/Lib/doctest.py", line 884, in 
_find
    test = self._get_test(obj, name, module, globs, source_lines)
  File "/Users/sasha/Work/python-svn/trunk/Lib/doctest.py", line 965, in 
_get_test
    if filename[-4:] in (".pyc", ".pyo"):
TypeError: 'NoneType' object is unsubscriptable
msg75188 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-10-24 22:07
Attached patch addresses the crash, but two issues remain:

$ ./python.exe testmodule.zip 
**********************************************************************
File "__main__", line ?, in __main__.c
Failed example:
    'line 2'
Expected nothing
Got:
    'line 2'
**********************************************************************
1 items had failures:
   1 of   1 in __main__.c
***Test Failed*** 1 failures.


The line number for the failed example cannot be found and file name is 
reported as __main__.c instead of __main__.py
msg75189 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-10-24 22:12
The next patch, doctest-1.patch addresses the line number issue, but the 
file name is still wrong.

$ ./python.exe testmodule.zip 
**********************************************************************
File "__main__", line 2, in __main__.c
Failed example:
    'line 2'
Expected nothing
Got:
    'line 2'
**********************************************************************
1 items had failures:
   1 of   1 in __main__.c
***Test Failed*** 1 failures.
msg75191 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-10-24 22:22
I don't think there is an easy way to fix source file name reporting 
without injecting __file__ = '<path to archive>/<filename>' into the 
module namespace is zipimporter, but I don't know if any code relies on 
__file__ set to None.

Setting __file__ to (in the test case) testmodule.zip/__main__.py will 
eliminate the need for the first patch and the second patch will only need 
to pass globs to linecache.getlines() to fix the line number issue.
msg76837 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-12-03 20:49
runpy needs a non-standard PEP 302 extension to set __file__ correctly
in the modules it runs. The pkgutil stuff it uses to find pure Python
modules in the filesystem supports that extension, but zipimport doesn't
(yet).
msg76860 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-12-04 00:44
I created issue4512 to implement get_filename extension in zipimport.  
Will post a patch shortly.
msg76872 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-12-04 05:54
While the patch at issue4512 fixes the crash, it does not fix the line 
number issue.  I would prefer to start with doctest-1.patch like solution 
because changing the PEP 302 loader protocol by adding get_filename needs 
additional discussion and certainly is not a candidate for 2.5.x or 2.6.x 
bug fix releases.
msg77511 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2008-12-10 08:55
The precise solution still seems to be debated, so this is out of scope
for 2.5.3.
msg77783 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-12-14 10:56
Fixed for 2.7 in r67750.

Will be ported to 2.6, 3.0 and 3.1.
msg77791 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-12-14 12:11
Final revisions for fix:
2.7 = r67751 (there was a new test file missing from the initial checkin)
2.6 = r67752
3.1 = r67753
3.0 = r67754
msg77805 - (view) Author: Alexander Belopolsky (belopolsky) * (Python committer) Date: 2008-12-14 16:32
Nick,

Your commit does not fix the line number reporting from doctest failures:

$ ./python.exe testmodule.zip
**********************************************************************
File "testmodule.zip/__main__.py", line ?, in __main__.c
...

Note the '?' in the output.  Did you miss the doctest.py changes in
doctest-1.patch, or should I open a separate issue for the line number
bug?

There is also a problem of __main__.c which my patch does not address,
but it is less important than the line number.

On Sun, Dec 14, 2008 at 7:11 AM, Nick Coghlan <report@bugs.python.org> wrote:
>
> Nick Coghlan <ncoghlan@gmail.com> added the comment:
>
> Final revisions for fix:
> 2.7 = r67751 (there was a new test file missing from the initial checkin)
> 2.6 = r67752
> 3.1 = r67753
> 3.0 = r67754
>
> ----------
> resolution:  -> fixed
> status: open -> closed
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue4197>
> _______________________________________
>
msg77857 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-12-15 10:29
Re-opening, pending development of a fully passing test specifically for
the __main__.py in zipfile case.
msg77858 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-12-15 11:05
I have a test case locally that demonstrates both the missing line
number in the __main__.py from a zipfile case, as well as the display of
main.c as the filename (the latter is not specific to the zipfile case -
it also happens for a normal main module).

It still seems to me that there is something going wrong at a deeper
level for doctest to be getting the "in main.c" part wrong. I want to
get to the bottom of that before working around the problem at the line
number lookup stage.
msg77863 - (view) Author: Nick Coghlan (ncoghlan) * (Python committer) Date: 2008-12-15 12:09
It turns out you were pretty close to pinpointing the problem in
doctest, but didn't quite manage to identify which step was going wrong.
The problem was actually that even after __file__ was being set
correctly, the call to linecache.getlines wasn't being given the
__loader__ access it needed in order to retrieve the source code from
inside the zipfile.

The __main__.c thing turned out just to be a red herring caused by the
choice of class name in the test code (I changed it to be __main__.Test
for the checked in unit test).

Checkin versions:
2.7 = r67790
2.6 = r67791
3.1 = r67792
3.0 = r67793
History
Date User Action Args
2022-04-11 14:56:40adminsetgithub: 48447
2008-12-15 12:09:35ncoghlansetstatus: open -> closed
resolution: fixed
messages: + msg77863
2008-12-15 11:05:52ncoghlansetmessages: + msg77858
2008-12-15 10:29:12ncoghlansetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg77857
2008-12-14 16:32:45belopolskysetmessages: + msg77805
2008-12-14 12:11:16ncoghlansetstatus: open -> closed
resolution: fixed
messages: + msg77791
2008-12-14 10:56:41ncoghlansetmessages: + msg77783
2008-12-10 08:55:55loewissetnosy: + loewis
messages: + msg77511
versions: - Python 2.5.3
2008-12-04 05:54:53belopolskysetmessages: + msg76872
2008-12-04 00:44:56belopolskysetmessages: + msg76860
2008-12-03 20:49:29ncoghlansetmessages: + msg76837
2008-12-03 20:29:01ncoghlansetassignee: ncoghlan
nosy: + ncoghlan
2008-10-25 01:53:49belopolskysetversions: + Python 2.6, Python 2.5.3
2008-10-24 22:22:14belopolskysetmessages: + msg75191
2008-10-24 22:12:44belopolskysetfiles: + doctest-1.patch
messages: + msg75189
2008-10-24 22:07:44belopolskysetfiles: + doctest.patch
keywords: + patch
2008-10-24 22:07:07belopolskysetmessages: + msg75188
2008-10-24 21:59:36belopolskycreate