Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

idle test fails at least on the 3.6 branch #77007

Closed
doko42 opened this issue Feb 12, 2018 · 10 comments
Closed

idle test fails at least on the 3.6 branch #77007

doko42 opened this issue Feb 12, 2018 · 10 comments
Assignees
Labels
3.7 (EOL) end of life 3.8 only security fixes easy topic-IDLE type-bug An unexpected behavior, bug, or error

Comments

@doko42
Copy link
Member

doko42 commented Feb 12, 2018

BPO 32826
Nosy @terryjreedy, @doko42, @serhiy-storchaka, @csabella, @miss-islington
PRs
  • bpo-32826: Add "encoding=utf-8" to open() in idle_test/test_help_about. #5639
  • [3.7] bpo-32826: Add "encoding=utf-8" to open() in idle_test/test_help_about. (GH-5639) #5642
  • [3.6] bpo-32826: Add "encoding=utf-8" to open() in idle_test/test_help_about. (GH-5639) #5643
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/terryjreedy'
    closed_at = <Date 2018-02-12.22:28:44.194>
    created_at = <Date 2018-02-12.10:43:22.992>
    labels = ['easy', '3.8', 'expert-IDLE', 'type-bug', '3.7']
    title = 'idle test fails at least on the 3.6 branch'
    updated_at = <Date 2018-02-12.22:28:44.192>
    user = 'https://github.com/doko42'

    bugs.python.org fields:

    activity = <Date 2018-02-12.22:28:44.192>
    actor = 'terry.reedy'
    assignee = 'terry.reedy'
    closed = True
    closed_date = <Date 2018-02-12.22:28:44.194>
    closer = 'terry.reedy'
    components = ['IDLE']
    creation = <Date 2018-02-12.10:43:22.992>
    creator = 'doko'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 32826
    keywords = ['patch', 'easy']
    message_count = 10.0
    messages = ['312037', '312059', '312060', '312064', '312065', '312066', '312073', '312075', '312077', '312086']
    nosy_count = 5.0
    nosy_names = ['terry.reedy', 'doko', 'serhiy.storchaka', 'cheryl.sabella', 'miss-islington']
    pr_nums = ['5639', '5642', '5643']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue32826'
    versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

    @doko42
    Copy link
    Member Author

    doko42 commented Feb 12, 2018

    seen with the 3.6 branch 20180212, last known succeeding test is the 3.6.4 release.

    test test_idle failed -- Traceback (most recent call last):
      File "/home/packages/python/3.6/python3.6-3.6.4/Lib/idlelib/idle_test/test_help_about.py", line 78, in test_file_buttons
        self.assertEqual(f.readline().strip(), get('1.0', '1.end'))
      File "/home/packages/python/3.6/python3.6-3.6.4/build-debug/../Lib/encodings/ascii.py", line 26, in decode
        return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1540: ordinal not in range(128)

    @terryjreedy
    Copy link
    Member

    What system? test_idle passes on Windows. Unfortunately, none of the *nix or Mac buildbots run tkinter and hence no tests that require a live tk window, and this one does. Cheryl, what do you see on your linux?

    Byte 0xc3 in position 1540 is the first byte of the utf-8 encoding of the 'ö' in Löwis on line 27 of CREDITS.txt. So on your system, readline must be reading *and decoding* more of the file.

    I am puzzled that the test fails now if it passed with the 3.6.4 release, as it and the textfile have not been touched since last summer.
    The files are the same on all 3 python versions.

    In any case, I will add encoding='utf-8'to open. I will also use subtest to identify failing cases. I will need you to verify the fix on your system.

    @terryjreedy terryjreedy added 3.7 (EOL) end of life 3.8 only security fixes type-bug An unexpected behavior, bug, or error labels Feb 12, 2018
    @serhiy-storchaka
    Copy link
    Member

    The failure is reproduced on Posix locale. Files are open with bare open() which uses the locale encoding by default. Which is ASCII in this case.

    @serhiy-storchaka serhiy-storchaka added easy tests Tests in the Lib/test dir labels Feb 12, 2018
    @terryjreedy
    Copy link
    Member

    I just discovered that the default for me is some version of latin1, not ascii, so that utf-8 'ö' is read as 'ö'. Hence, no UnicodeDecodeError even if 'ö' is being read and decoded by the initial readline call. I presume the same is true on current Windows buildbots. Hence, it was a bug to not include an encoding in the test file open calls. (The widget itself *does* use explicit 'ascii' or 'utf-8' encodings for open().)

    @terryjreedy terryjreedy removed the tests Tests in the Lib/test dir label Feb 12, 2018
    @csabella
    Copy link
    Contributor

    I've been able to run the IDLE test suite on Linux.

    As far as this issue, there's also a readline in test_textview that you may want to add the encoding to, even though it's only reading __file__.

    @terryjreedy
    Copy link
    Member

    I reproduced the exact failure by temporarily replacing 'utf-8' with 'ascii' in my clone. So I don't need anyone else to verify the fix.

    @terryjreedy
    Copy link
    Member

    New changeset f34e03e by Terry Jan Reedy in branch 'master':
    bpo-32826: Add "encoding=utf-8" to open() in idle_test/test_help_about. (GH-5639)
    f34e03e

    @miss-islington
    Copy link
    Contributor

    New changeset 38b4dd7 by Miss Islington (bot) in branch '3.7':
    bpo-32826: Add "encoding=utf-8" to open() in idle_test/test_help_about. (GH-5639)
    38b4dd7

    @miss-islington
    Copy link
    Contributor

    New changeset 46daf39 by Miss Islington (bot) in branch '3.6':
    bpo-32826: Add "encoding=utf-8" to open() in idle_test/test_help_about. (GH-5639)
    46daf39

    @terryjreedy
    Copy link
    Member

    Serhiy, thanks for comment and review.
    Cheryl, bpo-32837 expands upon your comment.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes easy topic-IDLE type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    5 participants