This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: AttributeError in doctest.DocTestFinder.find
Type: Stage: resolved
Components: Interpreter Core, Library (Lib) Versions: Python 3.8, Python 3.7
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: barry, jaraco
Priority: normal Keywords: 3.7regression, patch

Created on 2018-03-04 04:47 by jaraco, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Pull Requests
URL Status Linked Edit
PR 5980 merged jaraco, 2018-03-05 01:49
PR 5997 merged miss-islington, 2018-03-05 23:29
Messages (8)
msg313197 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2018-03-04 04:47
In Python 3.6, one could find doctests on a namespace package:

```
$ mkdir foo
$ python3.6
Python 3.6.4 (v3.6.4:d48ecebad5, Dec 18 2017, 21:07:28)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import foo
>>> foo.__file__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'foo' has no attribute '__file__'
>>> import doctest
>>> doctest.DocTestFinder().find(foo)
[]
```

In recent builds of Python 3.7, these namespace packages inherited a `__file__` attribute whose value is `None`, which causes DocTestFinder.find to fail:

```
$ python
Python 3.7.0b2 (tags/v3.7.0b2:b0ef5c979b, Feb 27 2018, 20:38:21)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import doctest
>>> import foo
>>> foo.__file__
>>> doctest.DocTestFinder().find(foo)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/doctest.py", line 893, in find
    file = inspect.getsourcefile(obj)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/inspect.py", line 687, in getsourcefile
    if any(filename.endswith(s) for s in all_bytecode_suffixes):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/inspect.py", line 687, in <genexpr>
    if any(filename.endswith(s) for s in all_bytecode_suffixes):
AttributeError: 'NoneType' object has no attribute 'endswith'
```

Scanning through the recent changes, issue32305 seems to be related, but when I look at the code ancestry, I can't see the related commits on the 3.7 branch, so I couldn't immediately confirm if it is indeed implicated.

I encountered this issue when testing jaraco.functools on Python 3.7.0b2 on macOS, but did not encounter it on Python 3.7.0a4+ as found on the Travis nightly builds. More details are logged in https://github.com/pytest-dev/pytest/issues/3276.

I'm not sure yet whether inspect.getfile should be adapted to raise a TypeError in this case, or if doctest.DocTestFinder.find should account for getfile returning None.

If we choose to update inspect.getfile, I should caution there's a bit of copy/paste there, so two branches of code will need to be updated.

Barry, I'd love to hear what your thoughts are on this and what you'd like to do. And definitely let me know if I can help.
msg313215 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2018-03-04 16:41
The main questions I have before proceeding with creating tests relate to what interfaces Python wishes to support. Here's the decision tree I have in my head.

- Retain the change of adding a __file__ attribute to namespace packages for Python 3.7?
  yes:
  - Should inspect.getfile still raise a TypeError on a namespace package?
    no:
    - The NEWS file should reflect this change also.
    - doctest.DocTestFinder.find should account for this change.
    yes:
    - Capture with a test and and restore that expectation.
  no:
  - The change should be reverted.
- Regardless, the test suite should capture and assert that doctest.DocTestFinder.find should succeed on a namespace package.
msg313216 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2018-03-04 17:05
I tried creating a test, but I'm struggling. I added [this patch](https://gist.github.com/e795a9a34594d202711aedf22c484af9), and tried to run it, but it's not being run.

```
$ ./python.exe Tools/scripts/run_tests.py 'test_doctest'
/Users/jaraco/Dropbox/code/public/cpython/python.exe -u -W default -bb -E -m test -r -w -j 0 -u all,-largefile,-audio,-gui test_doctest
Using random seed 3885464
Run tests in parallel using 6 child processes
0:00:00 load avg: 2.02 [1/1] test_doctest passed
1 test OK.

Total duration: 828 ms
Tests result: SUCCESS
```

I'm pretty sure doctest doesn't have any unit tests, but only functional tests exercising actual doctests.

I don't understand Python test framework enough to know how to create a discoverable unit test. I could use some help with how one could wire up the test_doctest module to run a unit test.
msg313224 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2018-03-04 22:58
Okay. I've wired up some unittests in test_doctest, and with [this patch](https://gist.github.com/jaraco/ea992719ac931fa761a6e9ef7a354542), it now captures the failed expectation of this ticket.
msg313225 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2018-03-04 23:06
And [this patch](https://gist.github.com/7184fa32670f2c63333377ddeb710676) corrects the failure such that the test passes. It does so by restoring the expectation that inspect.getfile will once again raise a TypeError for these namespace packages (the yes/yes path in the decision tree). That's my recommendation going forward.

Feel free to review the work and apply the patches. As time permits, I may revisit the Developers Guide and learn the new process for submitting pull requests.
msg313276 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2018-03-05 18:15
Good catch Jason.  Your fix is exactly right.  I approved your PR, which is against master, so it should definitely be backported to 3.7.  No need to backport to 3.6; we reverted the change for that release.
msg313299 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2018-03-05 23:29
New changeset b9650a04a81355c8a7dcd0464c28febfb4bfc0a9 by Jason R. Coombs in branch 'master':
bpo-32991: Restore expectation that inspect.getfile raises TypeError on namespace package (GH-5980)
https://github.com/python/cpython/commit/b9650a04a81355c8a7dcd0464c28febfb4bfc0a9
msg313332 - (view) Author: Jason R. Coombs (jaraco) * (Python committer) Date: 2018-03-06 15:16
New changeset 5a0c3987abd6a71b4fadeb525477eb5f560e8514 by Jason R. Coombs (Miss Islington (bot)) in branch '3.7':
bpo-32991: Restore expectation that inspect.getfile raises TypeError on namespace package (GH-5980) (GH-5997)
https://github.com/python/cpython/commit/5a0c3987abd6a71b4fadeb525477eb5f560e8514
History
Date User Action Args
2022-04-11 14:58:58adminsetgithub: 77172
2018-03-06 15:53:24jaracosetstatus: open -> closed
stage: patch review -> resolved
2018-03-06 15:16:13jaracosetmessages: + msg313332
2018-03-05 23:29:19miss-islingtonsetpull_requests: + pull_request5763
2018-03-05 23:29:11jaracosetmessages: + msg313299
2018-03-05 18:15:55barrysetmessages: + msg313276
versions: + Python 3.8
2018-03-05 01:49:37jaracosetkeywords: + patch
stage: patch review
pull_requests: + pull_request5746
2018-03-04 23:06:33jaracosetmessages: + msg313225
2018-03-04 22:58:31jaracosetmessages: + msg313224
2018-03-04 17:05:10jaracosetmessages: + msg313216
2018-03-04 16:41:53jaracosetmessages: + msg313215
2018-03-04 04:47:12jaracocreate