This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: IDLE: Integrate external code analysis tools
Type: enhancement Stage: patch review
Components: IDLE Versions: Python 3.4
process
Status: closed Resolution: rejected
Dependencies: Superseder: IDLE: Ability to run 3rd party code checkers
View: 21880
Assigned To: Nosy List: Al.Sweigart, JayKrish, Ramchandra Apte, Todd.Rovito, ezio.melotti, philwebster, rhettinger, terry.reedy
Priority: normal Keywords: patch

Created on 2013-08-10 22:17 by JayKrish, last changed 2022-04-11 14:57 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
PEP8Extension_1.patch JayKrish, 2013-08-10 22:17 review
Messages (13)
msg194841 - (view) Author: R. Jayakrishnan (JayKrish) * Date: 2013-08-10 22:17
This is to create an IDLE extension that would integrate PEP8 (Syntax Style Checker https://pypi.python.org/pypi/pep8 ) into the IDLE editor window,so the developer who is using the IDLE editor can quickly and frequently check the PEP8 standards of the code which is being developed in an IDLE editor window. Many modern IDE's have not only syntax highlighting but on the fly syntax checking.

Python ensures the quality of code with its own standards implemented as PEP8. The tool pep8 checks Python code against some of the style conventions in PEP8. Also the PEP8 standardization should not be a ‘must emphasizing’ for a python developer, therefore configuring pep8 to enable/disable errors and warning in various levels should be implemented.

The implementation work is started by Todd at https://bitbucket.org/rovitotv/pythonpatches/src/fae1fdab936043ec35db541808f5789325762fb3/PEP8CheckExtension.patch?at=default

The submitted patch is an effort started from Todd's patch as an example. 
The IDLE with this patch will show PEP8 style check toggle menu in Options.
Once PEP8 check enabled the current module is checked by pep8 tool and the defects will be shown in a extended left pane. 
Defect codes will be listed as a Buttons, button click leads to highlighting the line and column of the defect and scroll editor window if needed. 
It is expected that the developer will correct the defect with the help of the message and eventually press the Resolve button,which refresh the pane.

This is an initial work and I am looking forward for suggestions on improvements and mistakes of this patch.
msg194874 - (view) Author: Ramchandra Apte (Ramchandra Apte) * Date: 2013-08-11 08:45
I like the idea of having a PEP8-checker for IDLE.
msg195048 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2013-08-13 07:08
-1 on giving PEP 8 any more weight than it already has.

Automated PEP 8 checker tools do away the human commen sense component (the foolish consistency quote is in PEP 8 for a reason).

Another downside is that rigid adherence to PEP 8 tends to focus beginners on the least important aspects of code quality.  We would be much better-off with integration of PyFlakes or a similar tool that focus on actual semantic errors in the code.
msg195383 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-08-16 18:07
Can't this be developed as an external extension?
I don't see a reason to include it in the stdlib, especially if it requires other dependencies (like the pep8 module).
msg195676 - (view) Author: Todd Rovito (Todd.Rovito) * Date: 2013-08-20 02:22
Ezio,
   I think modern editors are expected to have this sort of functionality built into them [1].  XCode is simply amazing where it will pop up errors and quote the C99 standard [2].  We don't expect IDLE to have all that functionality but it seemed to us like it would be great to add pep8 or PyFlakes to IDLE.  It is possible we could create an extension but that would void Python's motto "batteries included".  In addition tools like pep8 and PyFlakes could be useful for other purposes as well.  JayKrish is checking with the authors of pep8 about license issues.  As always we will defer to the judgement of Core Developers and could pursue either option.  


[1] Wikipeida Microsoft Visual Studio, http://en.wikipedia.org/wiki/Microsoft_Visual_Studio#Code_editor
[2] XCode, http://en.wikipedia.org/wiki/XCode
msg195677 - (view) Author: Todd Rovito (Todd.Rovito) * Date: 2013-08-20 02:30
Raymond,
   Would you prefer PyFlakes instead?  Try to consider IDLE being for beginners so they need all the help they can get.  Advanced users can always turn the extension off.  Thanks for your input.
msg195700 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2013-08-20 15:40
> I think modern editors are expected to have this sort of functionality
> built into them [1].

Mine doesn't :)

> XCode is simply amazing where it will pop up errors and quote the C99
> standard [2].

That's a standard though, the ones in PEP8 are just conventions/guidelines/suggestions.

> We don't expect IDLE to have all that functionality but it seemed to
> us like it would be great to add pep8 or PyFlakes to IDLE.

AFAIU PyFlakes should be able to catch actual problems with the code -- not just simply suggesting a particular style -- so it would be a better candidate.

> It is possible we could create an extension but that would void
> Python's motto "batteries included".

Often editors are able to download extensions automatically from a central place, without having to ship them all by default.  Doing this is probably more work though, but on the other hand it would be more flexible.

> In addition tools like pep8 and PyFlakes could be useful for other purposes as well.

Are you suggesting to include it directly in the stdlib, and not as part of IDLE?

> JayKrish is checking with the authors of pep8 about license issues.

Note that that's not the only issue.  Even if the license is OK, before inclusion we would have to make sure that the module meets the quality standards for the repo and that the author is still active and willing to maintain it for at least a few years in the CPython repo.  A PEP would likely be required, especially if the module gets included directly in the stdlib (as opposed as including it just in IDLE).  This also means that external development of the module should stop and move to the stdlib, and follow the release schedule of Python instead of release whenever they want (in theory they could maintain both, but it's more work).
msg195711 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2013-08-20 19:11
Summary:
I am in favor of making more Python code analysis available from within Idle. I have an alternate, more generalized proposal. I am opposed to this specific proposal and think this issue should be closed.

Alternatives: when and how to analyze.

1. Local analysis as the user types ('on the fly'). One example is smart indentation, which includes conversion of \t to spaces (this is PEP 8 enforcement!) and backspace to multiple deletes. Others are calltips and code completion.  We might consider doing a bit more syntax checking, but would have to be very careful about style enforcement.

2. Block or whole file analysis within the text widget upon user request. Examples include search, reformatting, and whitespace stripping. The result is shown within the text widget (except for the bug on Windows that search still does not highlight found results while the dialog is active).

3. Analysis of one or more external files with Idle code. 'Find in Files' applies one pattern to file names and another to lines within found files and presents hits in an output (editor) window. The output lines contain file name, line number, and code line formatted so that right click, 'Go to file/line' opens the file if necessary and jumps to the indicated line.

4. Analysis of the saved editor file with external code. 'Run' (normally) does this (and more), with the complication that a socket rather than stdio streams are used for communication. Results appear in the shell window. Although tracebacks put file name and line number on one line and code on the next, 'Go to file/line' works when either is right-clicked. (Although Run now uses the same interpreter that is running Idle, this does not necessarily have to be so.)


Proposal: Run with ... 

What I would like to see is a generic option to save and run the current window with appropriate external programs. This proposal combines ideas from 3. and 4. above. The interface would be the addition of 'Run with ...' to the Run menu. Selecting that would bring up a submenu.

I would use subprocess to start the program (as is done with Run now). A command-line template would be needed for each supported external program. Unlike Run, subprocess would be used to capture stdout and stderr, as needed. The filtered output would be presented in a new Output Window, as done with Find-in-Files.

To me, the main value-added of doing this from Idle is being able to jump from output chunks to targets in an editor window for possible editing. I would only officially support external programs for which this is sensible, whose output includes line numbers and messages for possible action. Each supported program would need an output filter function to add the filename (if not present) to the line number and message and otherwise format as needed.

This proposal needs more details. how are external programs located if installed? (Ezio suggested the possibility of easy download and installation if not. I would start with instructions in the docs.) How is the submenu populated. How might users add templates and filter functions for programs not supported?  Such program might include locally written programs. There should be a PEP or at least PEP-like specification. Since the only stdlib addition would be to idlelib, backporting could be considered.


What I think is wrong with the current proposal.

1. There is no specification.

2. It is limited to one category of analysis. Within that category, it 'plays favorites' by picking a winner. I think users should be able to use whatever style analysis program they have already chosen, with the configuration options they have already selected. Not yet mentioned here are PyLint and PyChecker.

3. Rather than pick the 'best of breed' (if there is one), it picks what some would consider to be the 'worst of breed'. The PEP8 module, or at least the way it has been used, violates PEP 8 by treating guidelines as rigid rules, even though PEP 8 says not to do that. Some of the recent edits to PEP 8 were intended to counteract the negative effect of this module.

4. It proposes to add an external program to the stdlib without the needed PEP. I believe the idea of adding one of PyChecker, PyFlakes, and PyLint has already been rejected for multiple reasons (there is no winner, these are apps or tools, not library 'batteries', they are better off maintained where they are, we cannot maintain what we already have, etc.). The fact that these programs have (or should have) persistent configuration and customization files shows that they are apps, like Idle, and not library modules.

5. It proposes to add the external program in the wrong place. Idlelib should only contain Idle-specific code. Need I add that Idle is also under-maintained?

My proposal would start with a specification, would allow user choice, would generalize to other categories and programs (spell checking?), would not expand the stdlib, and would only add idle-specific glue code to idlelib.
msg195743 - (view) Author: Ramchandra Apte (Ramchandra Apte) * Date: 2013-08-21 03:09
Agree with Terry Reedy.
msg197778 - (view) Author: R. Jayakrishnan (JayKrish) * Date: 2013-09-15 15:43
Thanks for showing great interest on this idea.
As usual, it seems this will going to be another nice project with IDLE for me. Better to close this issue then, and create an appropriate initiation with the suggested points(preferring a commencement by core developers).
 
Anyway to keep in touch with the Python analyzing and related tools, I have initiated a separate project with porting the works done on the patch here https://github.com/Jay-Krish/IDLEAnalyzer.
I Hope this will help further while working on this project.
msg197785 - (view) Author: Raymond Hettinger (rhettinger) * (Python committer) Date: 2013-09-15 16:42
[TJR]
> I am opposed to this specific proposal and 
> think this issue should be closed.

[Ramchandra Apte]
> Agree with Terry Reedy.

[R. Jayakrishnan]
> Better to close this issue then ...

I agree as well.
msg197901 - (view) Author: R. Jayakrishnan (JayKrish) * Date: 2013-09-16 14:21
I would prefer this issue concludes with a navigation to the new proposed enhancement.
msg240526 - (view) Author: Al Sweigart (Al.Sweigart) * Date: 2015-04-12 02:59
-1 on adding PEP8 checks. PEP8 is a guideline, and having IDLE present them as warnings or errors elevates their importance.

Also, this would involve adding the pep8 dependency to the standard library.

A linter might be a good idea, but not a style checker.
History
Date User Action Args
2022-04-11 14:57:49adminsetgithub: 62904
2015-04-12 02:59:59Al.Sweigartsetnosy: + Al.Sweigart
messages: + msg240526
2015-04-05 21:39:49terry.reedysetsuperseder: IDLE: Ability to run 3rd party code checkers
2014-02-26 01:10:25terry.reedysettitle: IDLE: PEP8 Style Check Integration -> IDLE: Integrate external code analysis tools
2013-09-16 14:21:58JayKrishsetmessages: + msg197901
2013-09-15 16:42:14rhettingersetstatus: open -> closed
resolution: rejected
messages: + msg197785
2013-09-15 15:43:44JayKrishsetmessages: + msg197778
2013-08-21 03:09:48Ramchandra Aptesetmessages: + msg195743
2013-08-20 19:11:14terry.reedysetmessages: + msg195711
stage: patch review
2013-08-20 15:40:45ezio.melottisetmessages: + msg195700
2013-08-20 02:49:04Todd.Rovitosetnosy: + terry.reedy
2013-08-20 02:30:51Todd.Rovitosetmessages: + msg195677
2013-08-20 02:22:06Todd.Rovitosetstatus: pending -> open

messages: + msg195676
2013-08-16 18:07:49ezio.melottisetstatus: open -> pending
nosy: + ezio.melotti
messages: + msg195383

2013-08-13 07:08:40rhettingersetversions: - Python 2.7, Python 3.3
2013-08-13 07:08:31rhettingersetnosy: + rhettinger
messages: + msg195048
2013-08-13 02:08:10JayKrishsetnosy: + philwebster
2013-08-11 08:45:42Ramchandra Aptesetnosy: + Ramchandra Apte
messages: + msg194874
2013-08-10 22:20:53JayKrishsetnosy: + Todd.Rovito
2013-08-10 22:17:20JayKrishcreate