Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IDLE fails to display the README file #70093

Closed
serhiy-storchaka opened this issue Dec 18, 2015 · 6 comments
Closed

IDLE fails to display the README file #70093

serhiy-storchaka opened this issue Dec 18, 2015 · 6 comments
Assignees
Labels
topic-IDLE type-bug An unexpected behavior, bug, or error

Comments

@serhiy-storchaka
Copy link
Member

BPO 25905
Nosy @terryjreedy, @kbkaiser, @larryhastings, @serwy, @Rosuav, @serhiy-storchaka

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2015-12-18.21:36:55.327>
created_at = <Date 2015-12-18.12:29:52.353>
labels = ['expert-IDLE', 'type-bug']
title = 'IDLE fails to display the README file'
updated_at = <Date 2016-01-17.04:54:24.392>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2016-01-17.04:54:24.392>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2015-12-18.21:36:55.327>
closer = 'terry.reedy'
components = ['IDLE']
creation = <Date 2015-12-18.12:29:52.353>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 25905
keywords = []
message_count = 6.0
messages = ['256679', '256706', '256707', '256710', '258449', '258451']
nosy_count = 7.0
nosy_names = ['terry.reedy', 'kbk', 'larry', 'roger.serwy', 'python-dev', 'Rosuav', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue25905'
versions = ['Python 2.7', 'Python 3.5', 'Python 3.6']

@serhiy-storchaka
Copy link
Member Author

When open the About IDLE dialog and press the README button on bottom line, I get the UnicodeDecodeError:

Exception in Tkinter callback
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tkinter/__init__.py", line 1548, in __call__
    return self.func(*args)
  File "/home/serhiy/py/cpython/Lib/idlelib/aboutDialog.py", line 127, in ShowIDLEAbout
    self.display_file_text('About - Readme', 'README.txt')
  File "/home/serhiy/py/cpython/Lib/idlelib/aboutDialog.py", line 139, in display_file_text
    textView.view_file(self, title, fn, encoding)
  File "/home/serhiy/py/cpython/Lib/idlelib/textView.py", line 74, in view_file
    contents = file.read()
  File "/home/serhiy/py/cpython/Lib/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 72: invalid start byte

That is because IDLE opens the Lib/idlelib/README.txt file with default (locale) encoding, while it contains the RIGHT SINGLE QUOTATION MARK character encoded with the CP1252 encoding and non-decodable with UTF-8.

I think IDLE should open all distributed files with UTF-8 encoding. Lib/idlelib/CREDITS.txt and Lib/idlelib/README.txt should be recoded to UTF-8.

@serhiy-storchaka serhiy-storchaka added topic-IDLE type-bug An unexpected behavior, bug, or error labels Dec 18, 2015
@python-dev
Copy link
Mannequin

python-dev mannequin commented Dec 18, 2015

New changeset 11c789c034fe by Terry Jan Reedy in branch '2.7':
Issue bpo-25905: Revert unwanted conversion of ' to ’ RIGHT SINGLE QUOTATION MARK.
https://hg.python.org/cpython/rev/11c789c034fe

New changeset 42963dd81600 by Terry Jan Reedy in branch '3.4':
Issue bpo-25905: Revert unwanted conversion of ' to ’ RIGHT SINGLE QUOTATION MARK.
https://hg.python.org/cpython/rev/42963dd81600

@terryjreedy
Copy link
Member

Strange and accidental. This is the one line I kept from the old version, before deleting the old and adding the new. I believe I edited either with IDLE or Notepad++, and I would be surprised if the latter, an editor for programmers, would turn ' into ’. It does not for .py and .rst files. README.txt should be Ascii-only.

Larry, I applied the README.txt reverse replacement to 3.4 in case you feel like cherry picking it into 3.4.4, but it is not a blocker. No NEWS entries would be needed for 3.4 as this would fix an unreleased reversion.

@terryjreedy terryjreedy self-assigned this Dec 18, 2015
@terryjreedy
Copy link
Member

For CREDITS.txt, the only issue is 'Löwis' (ö has latin-1 code 246), which was changed from Loewis in 2003. If not changed back to the ascii form, what matters is that the encoding used for decoding from bytes to unicode matches the actual encoding. AFAIK, it currently does

def ShowIDLECredits(self):  # aboutDialog.py, line 130
    self.display_file_text('About - Credits', 'CREDITS.txt', 'iso-8859-1')

No encoding is given for README.txt, line 133, because none should be needed. I am going to leave this alone for now.

@python-dev
Copy link
Mannequin

python-dev mannequin commented Jan 17, 2016

New changeset 21356a5b8a5e by Terry Jan Reedy in branch '2.7':
Issue bpo-25905: Specify 'ascii' encoding for README.txt and NEWS.txt.
https://hg.python.org/cpython/rev/21356a5b8a5e

New changeset 59852a79b508 by Terry Jan Reedy in branch '3.5':
Issue bpo-25905: Specify 'ascii' encoding for README.txt and NEWS.txt.
https://hg.python.org/cpython/rev/59852a79b508

@terryjreedy
Copy link
Member

I re-encoded CREDITS to utf-8 and specified 'ascii' for the other so *I* will see an error if a non-ascii chars gets in the file again. Serhiy, I am confident that this will work on all OSes, but feel free to test AboutIDLE again sometime.

(Note to myself) To automate a test that the files will open, change
from idlelib import textView # to
from idlelib.textView import view_text, view_file
and change display... and show.. methods to module functions so view_file can be replaced and the show functions called without involving tkinter. This can be part of future refactoring in 3.5+.

@ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic-IDLE type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

2 participants