classification
Title: compiler.transformer dict key bug d[1,] = 1
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 2.7
process
Status: closed Resolution: wont fix
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, ChuckRhode, amaury.forgeotdarc, benjamin.peterson, kees, meador.inge, terry.reedy
Priority: normal Keywords: patch

Created on 2009-09-23 13:35 by kees, last changed 2014-09-29 18:11 by terry.reedy. This issue is now closed.

Files
File name Uploaded Description Edit
transformer.py.patch kees, 2009-10-07 12:30 patch for transformer.py
compiler-bug-issue6978.patch kees, 2010-08-25 11:05 Fix for python 2.7
Messages (17)
msg93034 - (view) Author: Kees Bos (kees) * Date: 2009-09-23 13:35
compiler.parse("d[1] = 1") should have a single tuple as subs


>>> compiler.parse("d[1] = 1")
Module(None, Stmt([Assign([Subscript(Name('d'), 'OP_ASSIGN',
[Const(1)])], Const(1))]))
>>> compiler.parse("d[1,] = 2")
Module(None, Stmt([Assign([Subscript(Name('d'), 'OP_ASSIGN',
[Const(1)])], Const(2))]))
>>> compiler.parse("d[1,2] = 3")
Module(None, Stmt([Assign([Subscript(Name('d'), 'OP_ASSIGN', [Const(1),
Const(2)])], Const(3))]))
>>> compiler.parse("d[(1,)] = 2")
Module(None, Stmt([Assign([Subscript(Name('d'), 'OP_ASSIGN',
[Tuple([Const(1)])])], Const(2))]))
msg93036 - (view) Author: Kees Bos (kees) * Date: 2009-09-23 14:11
I just see that my patch is not correct, since the following is
supported by the language:

O[1:2:3, 4:5:6]

Where O[1:2:3, 4:5:6] == O[slice(1,2,3), slice(4,5,6)] ==
O.__getitem__((slice(1,2,3), slice(4,5,6)))
msg93037 - (view) Author: Kees Bos (kees) * Date: 2009-09-23 14:24
patch which honors O[1:2:3, 4:5:6]
msg93690 - (view) Author: Amaury Forgeot d'Arc (amaury.forgeotdarc) * (Python committer) Date: 2009-10-07 11:55
your patch is no more a diff file (the 'previous' file is empty)
msg93693 - (view) Author: Kees Bos (kees) * Date: 2009-10-07 12:30
Sorry. Renamed .bak to .orig ...
Here's the patch compiler/transformer.py (against python 2.5)
msg94320 - (view) Author: Chuck Rhode (ChuckRhode) Date: 2009-10-21 19:48
PythonTidy encounters this problem.

o http://lacusveris.com/PythonTidy/PythonTidy.python

It is unable correctly to render line 694 of test_grammar.py in the
Python Test Suite:

    d[1,] = 2

becomes:

    d[1] = 2

because the *compiler* module does not return an abstract syntax tree
containing a tuple for the subscript.
msg94332 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2009-10-22 01:47
The patch should have a test.
msg98550 - (view) Author: Meador Inge (meador.inge) * (Python committer) Date: 2010-01-30 04:36
I think this should be closed out, since the compiler package was deprecated in 2.6.
msg98715 - (view) Author: Chuck Rhode (ChuckRhode) Date: 2010-02-02 04:23
Here are four ways to generate things called in the documentation Abstract Syntax Trees (ASTs).  Obviously they are not all the same kind of object:

#!/usr/local/bin/python2.6

import sys
import compiler
import parser
import ast

STATEMENT = 'd[1,] = 2'

print 'level %s' % sys.version
print compiler.parse(STATEMENT)
print parser.suite(STATEMENT).tolist()
print ast.dump(
    compile(STATEMENT, '<string>', 'exec', ast.PyCF_ONLY_AST),
    annotate_fields=False,
    include_attributes=False,
    )
print ast.dump(
    ast.parse(STATEMENT),
    annotate_fields=False,
    include_attributes=False,
    )

# Fin

Here are the results:

>>> level 2.6.2 (r262:71600, Jun 29 2009, 08:08:18) 
[GCC 4.3.2]
>>> Module(None, Stmt([Assign([Subscript(Name('d'), 'OP_ASSIGN', [Const(1)])], Const(2))]))
>>> [257, [267, [268, [269, [270, [327, [304, [305, [306, [307, [308, [310, [311, [312, [313, [314, [315, [316, [317, [318, [1, 'd']], [322, [9, '['], [323, [324, [304, [305, [306, [307, [308, [310, [311, [312, [313, [314, [315, [316, [317, [318, [2, '1']]]]]]]]]]]]]]]], [12, ',']], [10, ']']]]]]]]]]]]]]]]], [22, '='], [327, [304, [305, [306, [307, [308, [310, [311, [312, [313, [314, [315, [316, [317, [318, [2, '2']]]]]]]]]]]]]]]]]], [4, '']]], [0, '']]
>>> Module([Assign([Subscript(Name('d', Load()), Index(Tuple([Num(1)], Load())), Store())], Num(2))])
>>> Module([Assign([Subscript(Name('d', Load()), Index(Tuple([Num(1)], Load())), Store())], Num(2))])

To me the *compiler* module has vestigial utility.  It would be nice if it returned correct results.
msg98716 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2010-02-02 04:55
I don't see why the compiler module is any better than any of the other ways that produce reasonable AST.
msg98718 - (view) Author: Chuck Rhode (ChuckRhode) Date: 2010-02-02 05:08
> I don't see why the compiler module is any better than any of the other 
> ways that produce reasonable AST.

It is available on Python releases older than 2.6?
msg98721 - (view) Author: Kees Bos (kees) * Date: 2010-02-02 06:21
It's available at least since 2.4.

We're using it in Pyjamas (pyjs.org) to generate javascript code from python code.

If there are better ways to produce python asts, I'd be happy to know that.
msg98770 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2010-02-03 02:00
As I said before, the patch needs a test and should remove those ugly \ from the list comprehension.
msg112746 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2010-08-04 03:38
This can only be fixed in 2.7, where compiler is deprecated in favor of ast. Compiler is gone in 3.x. Kees, are you planning to update the patch (with no guarantee anyone will apply) or should we close this?
msg114892 - (view) Author: Kees Bos (kees) * Date: 2010-08-25 11:05
Added fix for python 2.7, which includes a test (testDictWithTupleKey) for the compiler test (Lib/test/test_compiler.py).
msg227789 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-09-29 09:57
Can somebody set this to "patch review" and do the honours please.  FWIW I don't like "tulplesub" in the patch.
msg227814 - (view) Author: Terry J. Reedy (terry.reedy) * (Python committer) Date: 2014-09-29 18:11
I am closing this because a) Meador is correct that we do not normally patch deprecated features and b) the current emphasis on 2.7-only patches is security and keeping 2.7 working on current systems.  The fix would only benefit 2.7.9+ code or 2.7.9+ and 3.x code, neither of which should be using compiler module.

Benjamin, if you disagree, review and apply, as no one else will.
History
Date User Action Args
2014-09-29 18:11:49terry.reedysetstatus: open -> closed
resolution: wont fix
messages: + msg227814

stage: resolved
2014-09-29 09:57:48BreamoreBoysetnosy: + BreamoreBoy
messages: + msg227789
2010-08-25 11:05:49keessetfiles: + compiler-bug-issue6978.patch
status: pending -> open
messages: + msg114892
2010-08-04 03:38:39terry.reedysetstatus: open -> pending
versions: + Python 2.7, - Python 2.6, Python 2.5
nosy: + terry.reedy

messages: + msg112746
2010-08-04 03:25:09terry.reedysetfiles: - compiler.transformer.patch
2010-02-03 02:00:50benjamin.petersonsetmessages: + msg98770
2010-02-02 06:21:25keessetmessages: + msg98721
2010-02-02 05:08:18ChuckRhodesetmessages: + msg98718
2010-02-02 04:55:27benjamin.petersonsetmessages: + msg98716
2010-02-02 04:23:48ChuckRhodesetmessages: + msg98715
2010-01-30 04:36:19meador.ingesetnosy: + meador.inge
messages: + msg98550
2009-10-22 01:47:10benjamin.petersonsetnosy: + benjamin.peterson
messages: + msg94332
2009-10-21 19:48:19ChuckRhodesetnosy: + ChuckRhode
messages: + msg94320
2009-10-07 12:30:53keessetfiles: + transformer.py.patch

messages: + msg93693
2009-10-07 11:55:33amaury.forgeotdarcsetnosy: + amaury.forgeotdarc
messages: + msg93690
2009-09-23 14:24:54keessetfiles: - compiler.transformer.patch
2009-09-23 14:24:23keessetfiles: + compiler.transformer.patch

messages: + msg93037
2009-09-23 14:11:26keessetmessages: + msg93036
2009-09-23 13:35:34keescreate