Author jiwon
Date 2004-01-07.12:10:43
SpamBayes Score
Marked as misclassified
Since I was interested in pep 289(generator
expression), I dabbled with it, and implemented a
working version of it. I'm not sure if I did it right,
but maybe someone who knows better can fix it right.

1. Grammar has two changes, which is
 a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ...

     changes to 

    atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ...

     where testgenexpr defines like this.
    testgenexpr: test ( gen_for | (',' test)* [','] )

 b. argument: [test '='] test
     changes to

    argument: [test '='] test [gen_for]

 (gen_for, gen_if, gen_iter is similar to list_for,
list_if, list_iter respectively.)

2. Instead of changing rule of arglist in Grammar to
accept generator expression, I changed argument rule
like 1.b. This means Grammar accepts generator
expression without parenthesis in function call even
there are several arguments, like

reduce(operator.add, (x for x in range(10))) 

This is against what pep requires, so I made
parsermodule.c and compile.c to catch that and throw
error message when there's more than one argument in a
function. The reason I did it that way is to go around
a problem of ambiguity in the grammar by adding
generator expression to arglist.

3. I implemented generator expression as equivalent to
a call to a function which has variable capturing code
and generator code. For example,

x = 1
g = (x for i in range(10))

is equivalent to

x = 1
def __gen_wrapper():
    _[x] = x # variable capture here
    def __gen():
        for i in range(10):
            yield _[x]
    return __gen()

g = __gen_wrapper()

4. Since I implemented generator expression equivalent
to a function call, symbol table should enter new scope
when it meets generator expression. So I ended up with
adding following code to symtable.c

PyObject *
PySymtableEntry_New(struct symtable *st, char *name,
int type, int lineno)

        switch (type) {
        case funcdef:
        case lambdef:
!       case testgenexpr: /* generator expression */
!       case argument:    /* generator expression */
                ste->ste_type = TYPE_FUNCTION;

so that generator expression can be dealt as function
type. This is kind of stupid, but I couldn't find other
easy for it, so I just left it like that.

5. this patch does not include diff of
src/Lib/, so you must run python Lib/
to get it right.
Date User Action Args
2007-08-23 15:31:35adminlinkissue872326 messages
2007-08-23 15:31:35admincreate