This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: lib2to3 log_error method behavior is inconsitent with documentation
Type: behavior Stage: resolved
Components: 2to3 (2.x to 3.x conversion tool) Versions: Python 3.8, Python 3.7, Python 3.6, Python 2.7
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: benjamin.peterson, iritkatriel, soupytwist, xtreak
Priority: normal Keywords:

Created on 2018-02-02 18:37 by soupytwist, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (3)
msg311507 - (view) Author: Nick Smith (soupytwist) Date: 2018-02-02 18:37
The log_error method in refactor.RefactoringTool raises the exception:

    def log_error(self, msg, *args, **kwds):
        """Called when an error occurs."""
        raise

but every usage of it implies that it does not, e.g:

    def refactor_string(self, data, name):
        """Refactor a given input string.
        Args:
            data: a string holding the code to be refactored.
            name: a human-readable name for use in error/log messages.
        Returns:
            An AST corresponding to the refactored input stream; None if
            there were errors during the parse.
        """
        # [..]
        try:
            tree = self.driver.parse_string(data)
        except Exception as err:
            self.log_error("Can't parse %s: %s: %s",
                           name, err.__class__.__name__, err)
            return
        finally:
        # [..]

This is the only explicit conflict I found in the documentation. From looking at the refactor_string function, it seems it should never raise on parse errors. Other uses of it are followed immediately by a return.

I'd like to see log_error only log the exception and continue.
msg326389 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python committer) Date: 2018-09-25 18:47
Thanks Nick for the details. I think 2to3 uses StdoutRefactoringTool [1] which inherits from RefactoringTool in the chain and implements log_error [2] that logs the error. 

StdoutRefactoringTool inherits from MultiprocessRefactoringTool [3] which in turn inherits from RefactoringTool [4] that raises the exception on log_error. But StdoutRefactoringTool re-implemented it to log and not to raise. I couldn't see any errors on scripts that have parsing errors. I think the docstring is right from the context of StdoutRefactoringTool but they were essentially present in RefactoringTool adding to the confusion I hope. Suggestions welcome on improving this. Feel free to correct me if I am misunderstanding the issue.

Example I have used to verify : 

$ cat ../backups/bpo32750.py # parse error file should be logged and skipped
foo +- # Invalid file
$ cat ../backups/bpo32750.py # valid file
print "hello"
$ 2to3 ../backups/bpo32750.py ../backups/bpo32750_valid.py
RefactoringTool: Skipping optional fixer: buffer
RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: set_literal
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: Can't parse ../backups/bpo32750.py: ParseError: bad input: type=4, value='\n', context=('', (1, 6))
RefactoringTool: Refactored ../backups/bpo32750_valid.py
--- ../backups/bpo32750_valid.py	(original)
+++ ../backups/bpo32750_valid.py	(refactored)
@@ -1 +1 @@
-print "hello"
+print("hello")
RefactoringTool: Files that need to be modified:
RefactoringTool: ../backups/bpo32750_valid.py
RefactoringTool: There was 1 error:
RefactoringTool: Can't parse ../backups/bpo32750.py: ParseError: bad input: type=4, value='\n', context=('', (1, 6))

Hope this helps.

Thanks again

[1] StdoutRefactoringTool - https://github.com/tirkarthi/cpython/blob/f6c8007a29b95b3ea3ca687a9b4924769a696328/Lib/lib2to3/main.py#L245

[2] StdoutRefactoringTool.log_error - https://github.com/tirkarthi/cpython/blob/f6c8007a29b95b3ea3ca687a9b4924769a696328/Lib/lib2to3/main.py#L65

[3] MultiprocessRefactoringTool - https://github.com/tirkarthi/cpython/blob/f6c8007a29b95b3ea3ca687a9b4924769a696328/Lib/lib2to3/refactor.py#L676

[4] RefactoringTool - https://github.com/tirkarthi/cpython/blob/f6c8007a29b95b3ea3ca687a9b4924769a696328/Lib/lib2to3/refactor.py#L155
msg399313 - (view) Author: Irit Katriel (iritkatriel) * (Python committer) Date: 2021-08-10 10:24
If I understand correctly, this is not about an observed bug but about a confusion about which log_error function is being called. Can we close this or is there something to do here?
History
Date User Action Args
2022-04-11 14:58:57adminsetgithub: 76931
2021-09-06 19:51:01iritkatrielsetstatus: pending -> closed
stage: resolved
2021-08-10 10:24:26iritkatrielsetstatus: open -> pending

nosy: + iritkatriel
messages: + msg399313

resolution: not a bug
2018-09-25 18:47:37xtreaksetmessages: + msg326389
2018-09-25 12:21:07xtreaksetnosy: + xtreak
2018-02-09 21:16:15terry.reedysetnosy: + benjamin.peterson

versions: - Python 3.4, Python 3.5
2018-02-02 18:37:47soupytwistcreate