Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Using appropriate checks in tests #60714

Closed
serhiy-storchaka opened this issue Nov 19, 2012 · 12 comments
Closed

Using appropriate checks in tests #60714

serhiy-storchaka opened this issue Nov 19, 2012 · 12 comments
Labels
tests Tests in the Lib/test dir type-feature A feature request or enhancement

Comments

@serhiy-storchaka
Copy link
Member

BPO 16510
Nosy @terryjreedy, @jcea, @jaraco, @pitrou, @ezio-melotti, @bitdancer, @voidspace, @serhiy-storchaka
PRs
  • bpo-16510: Using more specific assertions in miscellaneous tests #792
    		•
    Dependencies
  • bpo-19589: Use specific asserts in test_asyncio
  • bpo-19590: Use specific asserts in test_email
  • bpo-19591: Use specific asserts in ctype tests
  • bpo-19592: Use specific asserts in lib2to3 tests
  • bpo-19593: Use specific asserts in importlib tests
  • bpo-19594: Use specific asserts in unittest tests
  • bpo-19600: Use specific asserts in distutils tests
  • bpo-19601: Use specific asserts in sqlite3 tests
  • bpo-19602: Use specific asserts in tkinter tests
  • bpo-19603: Use specific asserts in test_decr
  • bpo-19604: Use specific asserts in array tests
  • bpo-19605: Use specific asserts in datetime tests
  • bpo-19606: Use specific asserts in http.cookiejar tests
  • bpo-19607: Use specific asserts in weakref tests
  • bpo-20544: Use specific asserts in operator tests
  • bpo-20545: Use specific asserts in unicode tests
  • bpo-20546: Use specific asserts in int tests
  • bpo-20547: Use specific asserts in bigmem tests
  • bpo-20548: Use specific asserts in warnings and exceptions tests
  • bpo-20549: Use specific asserts in mailbox, smtplib and poplib tests
  • bpo-20550: Use specific asserts in collections tests
  • bpo-20551: Use specific asserts in decimal tests
  • bpo-20552: Use specific asserts in bytes tests
  • bpo-20553: Use specific asserts in ipaddress tests
  • bpo-20554: Use specific asserts in optparse test
  • bpo-20555: Use specific asserts in urllib related tests
  • bpo-20556: Use specific asserts in threading tests
  • bpo-20557: Use specific asserts in io tests
    		•
    Files
  • tests_asserts.patch
  • tests_asserts_2.patch
  • tests_asserts_3.patch
  • tests_asserts_4.patch
  • tests_asserts_7.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2017-04-05.05:44:23.736>
created_at = <Date 2012-11-19.10:44:18.561>
labels = ['type-feature', 'tests']
title = 'Using appropriate checks in tests'
updated_at = <Date 2017-04-05.05:44:23.734>
user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2017-04-05.05:44:23.734>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2017-04-05.05:44:23.736>
closer = 'serhiy.storchaka'
components = ['Tests']
creation = <Date 2012-11-19.10:44:18.561>
creator = 'serhiy.storchaka'
dependencies = ['19589', '19590', '19591', '19592', '19593', '19594', '19600', '19601', '19602', '19603', '19604', '19605', '19606', '19607', '20544', '20545', '20546', '20547', '20548', '20549', '20550', '20551', '20552', '20553', '20554', '20555', '20556', '20557']
files = ['28041', '31505', '32650', '33984', '46779']
hgrepos = []
issue_num = 16510
keywords = ['patch']
message_count = 12.0
messages = ['175953', '175956', '175960', '186713', '186743', '196459', '202909', '203053', '203057', '203070', '210566', '221904']
nosy_count = 9.0
nosy_names = ['terry.reedy', 'jcea', 'jaraco', 'pitrou', 'ezio.melotti', 'r.david.murray', 'michael.foord', 'tshepang', 'serhiy.storchaka']
pr_nums = ['792']
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue16510'
versions = ['Python 3.4']
    @serhiy-storchaka
    Copy link
    Member Author

    The proposed patch upgrades tests to use specialized checks added in 3.1 and 3.2 (assertIsNone(x) instead assertTrue(x is None), assertLess(a, b) instead assertTrue(a < b), etc). This modern checks provide a more useful error message in case of a fail.

    Replaced only those checks that are not related to the tested operators. For example, assertTrue(a < b) preserved if the operator "<" is tested.

    @serhiy-storchaka serhiy-storchaka added tests Tests in the Lib/test dir type-feature A feature request or enhancement labels Nov 19, 2012
    @bitdancer
    Copy link
    Member

    Thanks for the work, but we don't generally make bulk changes like this. It generates churn in the codebase, and has the risk of inadvertently changing the meaning of the tests, to little actual benefit. Instead we modernize tests when we touch them for other reasons and are in a position to confirm that the changes do not change the meaning of the tests. (I realize that for most of your changes the meaning is trivially preserved...but when you make a lot of changes you are almost certain to make some mistakes...thus the resistance to doing bulk updates.)

    @serhiy-storchaka
    Copy link
    Member Author

    I understand this. I checked the patch few times, with long (more than a month) intervals between inspections. If someone wants to modernize some tests, he can turn to this patch for reference.

    @tshepang
    Copy link
    Mannequin

    tshepang mannequin commented Apr 13, 2013

    @david is that policy documented in the devguide? If not, should it be?

    @bitdancer
    Copy link
    Member

    If it isn't documented it should be, and you could open an issue for it.

    @serhiy-storchaka
    Copy link
    Member Author

    Patch updated to sync with tip. It is 5% less than first version because some changes already applied in other commits.

    @terryjreedy
    Copy link
    Member

    I think there are two counterargument to leaving things alone.

    1. The reason for the specialized checks is to change behavior when the test fails, to give more informative error messages. So the change is not purely cosmetic or stylistic. I believe there was a recent case where assertTrue(a == b), a and b strings, had to be changed to assertEqual(a, b) to find out why the assert failed. As I remember, nothing else in the test file obviously needed changing and it would not have been touched otherwise.

    2. People adding tests may not review existing tests for modernization.

    That said, it *is* possible to ignore 2. and not worry about 1. until a test fails. And I agree that the original patch is too much to review.

    @jaraco
    Copy link
    Member

    jaraco commented Nov 16, 2013

    A third counterargument is:

    1. Newer developers (and even seasoned ones) adding new tests may use existing tests as a model for best practices. If the existing tests model sub-optimal practices, then those practices will be perpetuated both in the codebase and in the minds of contributors.

    Given that Serhiy has so diligently prepared and updated the patch, I'm inclined to say the codebase would be better off accepting the patch.

    David, can you imagine a process by which a patch like this could be acceptable?

    @serhiy-storchaka
    Copy link
    Member Author

    Here is synchronized to tip the patch which doesn't include patches for already separated children issues. It touches 129 files and modifies over 600 lines.

    @bitdancer
    Copy link
    Member

    Jason: yes, if it is broken down into small enough pieces for the module maintainer to be comfortable with the review :). I approved Serhiy's separate patch for test_email. I still wasn't sure about backporting, but I really don't like the 'unless' construct in test_email, so I decided to say OK.

    @serhiy-storchaka
    Copy link
    Member Author

    After chipping off yet some small separate issues, there are only 276 changed lines left in 80 files.

    @BreamoreBoy
    Copy link
    Mannequin

    BreamoreBoy mannequin commented Jun 29, 2014

    Can we follow up on this please as it's referenced from bpo-9554 as well.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    tests Tests in the Lib/test dir type-feature A feature request or enhancement
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    4 participants
    @jaraco @bitdancer @serhiy-storchaka @terryjreedy