Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Suspicious operation in doctest.py: None - 1 #90464

Closed
sobolevn opened this issue Jan 8, 2022 · 2 comments
Closed

Suspicious operation in doctest.py: None - 1 #90464

sobolevn opened this issue Jan 8, 2022 · 2 comments
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@sobolevn
Copy link
Member

sobolevn commented Jan 8, 2022

BPO 46306
Nosy @ericvsmith, @sobolevn
PRs
  • bpo-46306: simplify CodeType attribute access in doctest.py #30481
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2022-01-08.20:14:43.017>
created_at = <Date 2022-01-08.10:39:04.759>
labels = ['type-bug', 'library', '3.9', '3.10', '3.11']
title = 'Suspicious operation in `doctest.py`: `None - 1`'
updated_at = <Date 2022-01-08.20:14:43.017>
user = 'https://github.com/sobolevn'

    bugs.python.org fields:

    activity = <Date 2022-01-08.20:14:43.017>
actor = 'eric.smith'
assignee = 'none'
closed = True
closed_date = <Date 2022-01-08.20:14:43.017>
closer = 'eric.smith'
components = ['Library (Lib)']
creation = <Date 2022-01-08.10:39:04.759>
creator = 'sobolevn'
dependencies = []
files = []
hgrepos = []
issue_num = 46306
keywords = ['patch']
message_count = 2.0
messages = ['410092', '410113']
nosy_count = 2.0
nosy_names = ['eric.smith', 'sobolevn']
pr_nums = ['30481']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue46306'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']
    @sobolevn
    Copy link
    Member Author

    sobolevn commented Jan 8, 2022

    This line lineno = getattr(obj, 'co_firstlineno', None)-1 does not look good. Link: https://github.com/python/cpython/blame/45d44b950f1dab0ef90d0a8f4fa75ffaae71500b/Lib/doctest.py#L1116

    Why?

    1. CodeType is guaranteed to have co_firstlineno. Docs:

      cpython/Lib/inspect.py

      Line 487 in 45d44b9

      co_firstlineno number of first line in Python source code
    2. Even if it does not have it for some reason, this getattr does not help. It raises unhandled TypeError. We can simulate it by replacing lineno = getattr(obj, 'co_firstlineno', None)-1 with lineno = None-1

    Here's what happens in this case:

    » ./python.exe -m test -v test_doctest
== CPython 3.11.0a3+ (heads/issue-26767:45d44b950f, Jan 8 2022, 12:23:45) [Clang 11.0.0 (clang-1100.0.33.16)]
== Darwin-18.7.0-x86_64-i386-64bit little-endian
== cwd: /Users/sobolev/Desktop/cpython/build/test_python_78065æ
== CPU count: 4
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 3.51 Run tests sequentially
0:00:00 load avg: 3.51 [1/1] test_doctest
Failed to call load_tests:
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/unittest/loader.py", line 108, in loadTestsFromModule
    return load_tests(self, tests, pattern)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_doctest.py", line 3134, in load_tests
    tests.addTest(doctest.DocTestSuite(doctest))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 2391, in DocTestSuite
    tests = test_finder.find(module, globs=globs, extraglobs=extraglobs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 940, in find
    self._find(tests, obj, name, module, source_lines, globs, {})
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 1013, in _find
    self._find(tests, val, valname, module, source_lines,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 1001, in _find
    test = self._get_test(obj, name, module, globs, source_lines)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 1069, in _get_test
    lineno = self._find_lineno(obj, source_lines)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/doctest.py", line 1117, in _find_lineno
    None - 1
    ~~~~~^~~
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

test test_doctest crashed -- Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py", line 352, in _runtest_inner
    refleak = _runtest_inner2(ns, test_name)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py", line 309, in _runtest_inner2
    test_runner()
    ^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/test/libregrtest/runtest.py", line 272, in _test_module
    raise Exception("errors while loading tests")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Exception: errors while loading tests

test_doctest failed (uncaught exception)

== Tests result: FAILURE ==

1 test failed:
    test_doctest

Total duration: 1.0 sec
Tests result: FAILURE

    So, we have two options:

    1. Think of a case when CodeType does not have co_firstlineno and handle it properly, like using 0 or None as lineno
    2. Simplify this line to remove potential TypeError

    I think that this line should be rewritten as lineno = obj.co_firstlineno - 1

    I will send a PR for others to judge.

    @sobolevn sobolevn added 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error labels Jan 8, 2022
    @AlexWaygood AlexWaygood changed the title Suspicios operation in doctest.py: None - 1 Suspicious operation in doctest.py: None - 1 Jan 8, 2022
    @AlexWaygood AlexWaygood changed the title Suspicios operation in doctest.py: None - 1 Suspicious operation in doctest.py: None - 1 Jan 8, 2022
    @ericvsmith
    Copy link
    Member

    New changeset 0fc58c1 by Nikita Sobolev in branch 'main':
    bpo-46306: simplify CodeType attribute access in doctest.py (GH-30481)
    0fc58c1

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    2 participants
    @ericvsmith @sobolevn