this patch touches only Python/ceval.c.

1. the only existing thing it modifies is PyEval_EvalFrameEx (adds 7
extra cases for the new 3.0 opcodes, doesn't mess w/ any of the existing
ones, or anything else as a matter of fact)

2. that, plus it defines (the new opcodes)
#define SET_ADD 17
#define STORE_LOCALS 69
#define LOAD_BUILD_CLASS 34 // 71 in py3k
#define MAKE_BYTES 35 // unused in py3k
#define POP_EXCEPT 36 // 89 in py3k
#define UNPACK_EX 94
#define BUILD_SET 109 // 104 in py3k
and some backported vars & helper functions

only 2 contiguous areas of ceval.c is patched (1. & 2. - areas are
delimited by the comments '// py3k')

this simple patch seems sufficient in theory to allow the interpreter to
natively execute most of 3.0's language syntax (nonlocal, print,
extended unpacking, ... have been tested to work)
*the one exception being pep3102 - keyword-only arguments

i wrote 2 small scripts which gives an interactive function 'compile_py3k'
similar to __builtin__.compile.  its a wrapper function which queries
the byte-compiling task to a python 3.0 server via pipe io.

example demonstrating pep3132 extended unpacking syntax:
a,b,*c = 1,2,3,4
(note the backported 3.0 opcode used in line2 of the disassembly)



Python 2.5.2 (r252:60911, Jun 27 2008, 21:19:51)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-9)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> import py3to2
py3to2 server restarting with io: (4, 5)
py3to2 server: Python 3.0b1 (r30b1:64395, Jun 24 2008, 21:53:55)
py3to2 server: [GCC 3.4.6 20060404 (Red Hat 3.4.6-9)] on linux2
py3to2 server: Type "help", "copyright", "credits" or "license" for more
information.
py3to2 server:

>>> src = "a,b,*c = 1,2,3,4"
>>> codeobject = py3to2.compile_py3k(src,"","exec")
 1           0 LOAD_CONST               5 ((1, 2, 3, 4))
             3 UNPACK_EX_py3k           2
             6 STORE_NAME               0 (a)
             9 STORE_NAME               1 (b)
            12 STORE_NAME               2 (c)
            15 LOAD_CONST               4 (None)
            18 RETURN_VALUE

>>> exec(codeobject)
>>> print a,b,c
1, 2, [3, 4]
>>>

u can go c
http://pypi.python.org/pypi?name=py3to2&version=20080628&:action=display
for more info on the script

anyway, i think it would b a great boost for python 3.0 (which i think
is very cool) if developers can test/develop it under the 2.x
environment.  this patch plus some scripts to emulated 3k extensions 
(bytes, bytearray, ...) can go a long way in making 3.0 available for
the masses.

ps. i've also attempted the reverse (forward-port 2.x opcodes to 3.0),
& its a bit more complicated (namely the PRINTxxx opcodes).  if there's
significant interest in that, i could work on it a bit more, but my
current interest is in extending 3.0 functionality to the vast 2.x
software base.

The first beta release for 2.6 has been made, so no new features can be
accepted here. Deferring this to 2.7.

ok

Some remarks:

- I don't think doing a bulk backport of ceval.c is the right approach.
Instead, each functionality should be considered and backported
independently, if desired.

- Guido already said that 3.0 should be mostly clean from
compatibility-related code, so forward-porting 2.x opcodes is out of the
question. There's a reason they removed in the first place :-)

- Some functionalities shouldn't be backported, e.g. introducing
SETUP_EXCEPT was motivated by the different semantics of exception
cleanup in py3k, backporting it would probably break some 2.x code.

- Compatibility between the two eval loops (2.x and py3k) is probably
the tip of the iceberg.

ideally that may be true.

but its quite frustrating testing/developing new py3k software when many
modules/extensions we take for granted in 2.x isn't available.  in the
meantime, this patch serves as a very convenient stopgap for developers
(even w/ its bugs), for writing py3k migration code in 2.x (& access to
all its extensions)

for example, its a difficult task to port a big project like numpy to
py3k all @ once.  but this patch could allow u to piece-wise transform
it, one script @ a time, to py3k language syntax compliance.

I don't know why this is assigned to me.

update: these 3k language features have been tested to work in python
2.5.2 w/ the backported opcodes

   pep3104  Access to Names in Outer Scopes
   pep3105  Make print a function
   pep3111  Simple input built-in in Python 3000
   pep3113  Removal of Tuple Parameter Unpacking
   pep3114  Renaming iterator.next() to .__next__()
   pep3115  Metaclasses in Python 3000
   pep3127  Integer Literal Support and Syntax
   pep3129  Class Decorators
   pep3132  Extended Iterable Unpacking

had to backport __build_class__ in bltinmodule.c to get metaclasses
working.  u can go to http://www-rcf.usc.edu/~kaizhu/work/py3to2/ for
more info

install/usage summary:
  1. build python 2.5.2 w/ patched ceval.c & bltinmodule.c
  2. download scripts: py3to2.py & _py3to2.py
  3. ln -s python3.0 python3k
  4. run python2.5 & type 'import py3to2' # it will automatically run
the pep tests

the script provides 3 functions similar to those in __builtin__:
compile_py3k, exec_py3k, eval_py3k

right now, working on backporting pep 3102 & 3107 - annotations &
keyword-only arguments.  apparently appending the co_kwonlyargcount attr
to codeobject in 2.5.2 doesn't seem to affect the build

import/reload now works.
accomplished by adding 5 lines in parse_source_module (import.c) to 1st
check for the hook __builtins__.parse_source_module_py3k.
the hook will automatically compile in py3k format if it finds the magic
comment: "# import as py3k\n"

* below is a working python 3.0 script integrated w/ numpy.

also added:
  pep3102  Keyword-Only Arguments
  pep3112  Bytes literals in Python 3000

download: http://www-rcf.usc.edu/~kaizhu/work/py3to2/current/

patched files:
  ceval.c (unchanged from last)
  bltinmodule.c (unchanged from last)
  import.c (added 5 continuous lines to parse_source_module)

there r 7 unimplemented pep's remaining: any suggested solutions?
  pep3107  Function Annotations (easy, just haven't gotten around yet)
  pep3109/3110  Exceptions in Python 3000 (hard?)
  pep3120  Using UTF-8 as the default source encoding
  pep3123  Making PyObject_HEAD conform to C (hard/who cares for 2.x?)
  pep3131  Supporting Non-ASCII Identifiers (looks emulable)
  pep3138  String representation in Python 3000

@ any rate, i think its feature complete enough to b useful in certain
areas (for me its scientific computing).

################################################################################
"""
numpy_py3k.py
this is a py3to2 demo showing a python3.0 script being run under python
2.5 &
utilizing numpy, a python2.5 extension module.

add the magic comment '# import as py3k\n' to import / reload a script
in py3k format

interactive usage:
>>> import py3to2
>>> import numpy_py3k
>>> reload(numpy_py3k)

commandline usage: python -c 'import py3to2; import numpy_py3k'
"""

# import as py3k
import numpy

print('pep3102  Keyword-Only Arguments')
# nth order polynomial fit of multiple y data
def polyfits(nth, x, *ys, rcond = None, full = False):
  return [numpy.polyfit(x, y, nth, rcond, full) for y in ys]

fits = polyfits(2, # 2nd order fit
                numpy.arange(16), # x data
                numpy.random.rand(16), numpy.random.rand(16), # multiple
y data
                rcond = numpy.MachAr().eps, # precision
                full = False, # return only coeffs
                )
print('fits', fits); print('#'*64)

print('pep3112  Bytes literals in Python 3000')
x = bytes( numpy.arange(256, dtype = numpy.int8).tostring() )
print('bytes', x); print('#'*64)

print('pep3114  Renaming iterator.next() to .__next__()')
x = (x for x in numpy.arange(16))
print('x.__next__()', x.__next__(), x.__next__(), x.__next__());
print('#'*64)

print('pep3132  Extended Iterable Unpacking')
a,b,*c = numpy.random.rand(4)
print('a = %s, b = %s, c = %s'%(a,b,c)); print('#'*64)
################################################################################

This isn't dealing with the 2to3 conversion tool. Please do not add it.

why not?  it allows developers to migrate 2.x scripts one-by-one to
working 3.0 conformant ones while maintaining backwards-compatibility w/
existing 2.x scripts & extension modules (eg. numpy, PIL, zope, ...)

py3to2 can transparently import & mix & match 2.x & 3.0 scripts (but
builtins/extensions must b 2.x - hence its a 2to3 migration tool).

@ the moment, every script compilable by py3to2 should b 3.0 language
conformant, or otherwise it would fail the syntax check & byte-compile
stage performed by the python 3.0 slave interpreter (see Mechanism for
details).

On Sun, Jul 13, 2008 at 2:03 PM, kai zhu <report@bugs.python.org> wrote:
>
> kai zhu <davidbranniganz@gmail.com> added the comment:
>
> why not?  it allows developers to migrate 2.x scripts one-by-one to
> working 3.0 conformant ones while maintaining backwards-compatibility w/
> existing 2.x scripts & extension modules (eg. numpy, PIL, zope, ...)
>
> py3to2 can transparently import & mix & match 2.x & 3.0 scripts (but
> builtins/extensions must b 2.x - hence its a 2to3 migration tool).

Yes, I realize that, but there is another tool called 2to3 that
preforms syntax transformations on 2.x code. That's what the "2to3
conversion" tool component is for.

then the 2 can complement each other well ;) i don't c any competition
between them as they have completely different objectives

2to3 -> convert 2x scripts to 3k
py3to2 <- use the newly created 3k scripts in existing 2x environ

u have to admit migrating to 3k is quite painful & slow
w/ its lack of 2x's vast extension base. given py3to2's potential to
alleviate this problem for migrating developers,
don't u think that possibly merits its mention in the 2to3 category?

btw, py3to2's patches minimally affects 2.x compatibility:
  ceval.c - backports missing 3.0 opcodes (8 in python 2.5.2)
  bltinmodule.c - backports missing 3.0 function __build_class__
  import.c - adds a 5 line hook to function parse_source_module
             allowing automatic checking for and compiling of 3k scripts
             during import/reload

i just ran 'make test' w/ a patched build of python 2.5.2 w/ following
results on redhat, x86_64:
285 tests OK.
37 tests skipped:
    test_aepack test_al test_applesingle test_bsddb185 test_bsddb3
    test_cd test_cl test_codecmaps_cn test_codecmaps_hk
    test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
    test_dl test_gdbm test_gl test_imageop test_imgfile
    test_linuxaudiodev test_macfs test_macostools test_normalization
    test_ossaudiodev test_pep277 test_plistlib test_rgbimg
    test_scriptpackages test_socket_ssl test_socketserver
    test_startfile test_sunaudiodev test_timeout test_urllib2net
    test_urllibnet test_winreg test_winsound test_zipfile64
1 skip unexpected on linux2:
    test_gdbm

not bad considering it now supports 11 of the 18 (soon to b 12 & likely
more...) 3000 series pep's ^_^

> don't u think that possibly merits its mention in the 2to3 category?

Please trust our judgment on how to use the bug tracker. This isn't a
competition for publicity; it's a tool to help organizing work among
committers and other contributors.

ported to python-2.6 & is a bit more stable (written a few py3k programs
w/ it).  the patches have been simplified & consolidated to a single
file: ceval.c

u can also test out scripts generated by 2to3 by adding to them the
magic line:
"from __future__ import py3k_syntax"

an example "python-2.6" script using "python-3.0" language syntax (&
numpy if its installed) is provided @:
http://www-rcf.usc.edu/~kaizhu/work/py3to2/current/example_py3k.py

new features added by patch to python-2.6 (under py3k_syntax mode):
  pep3102  Keyword-Only Arguments
  pep3104  Access to Names in Outer Scopes
  pep3107  Function Annotations
  pep3111  Simple input built-in in Python 3000
  pep3113  Removal of Tuple Parameter Unpacking
  pep3114  Renaming iterator.next() to iterator.__next__()
  pep3115  Metaclasses in Python 3000
  pep3127  Integer Literal Support and Syntax
           - oct() functions differently in py3k_syntax mode
  pep3129  Class Decorators
  pep3132  Extended Iterable Unpacking
  pep3135  New Super
  misc     exec becomes a function

sans unicode & py3k's builtins module, the above (plus __future__)
implements pretty much all of py3k's language syntax

py3k_syntax mode:
  a special bitflag is embedded in codeobj->co_argcount, which triggers
PyEval_EvalFrameEx to enable the backported opcodes (which r disabled by
default) & use an alternate py3k __builtin__.

  codeobj->co_argcount now limited to range 0-0xffff with upper bits
reserved for py3k flags

  codeobj->co_kwonlyargcount (limited to range 0-0xff) is embedded in
upper bits of codeobj->co_argcount

most of the trouble have been gettin kwonlyarg, annotations, & classes
to work.

there is one unrelated opcode (LOAD_ATTR_py3k) added to ceval.c for my
personal use (curries attributes as a universal function), but can b
safely disabled by commenting its case out of PyEvalFrameEx in ceval.c &
deleting it from dict opnew_py2x in py3to2.py

Recommend rejecting this.  I believe that it wasn't backported for a
reason.  Guido, care to pronounce?

hi, i'm happy i got a response on this :)

anyway, can u elaborate on the "reason" y something like this was never
done?

yes, i kno the patch is rough right now (& will never get accepted in
its current state), but its now proven to work, & gives migrating
developers *something* to test their py3k_syntax code while *still*
maintaining compatibility w/ python2.x extension modules.

isn't that the spirit of python 2.7?  in the meantime, i'm in no hurry
to have anything committed.

This doesn't sound like it will ever be reliable.

Kai Zhu, a better strategy would be for you to maintain your own
experimental port of Python 2.6+3.0 based upon these patches.

k

post fyi:

here's moderately complex python-3.0 script (plus curry extension) i
wrote using numpy, PIL, & scipy.weave 2.6 extensions.

it takes a jpeg, gif... image file & outputs it in colorized ascii art.
the actual purpose is for colorized 3-d scientific plottin in text
terminal (screenshots of image conversion & 3d plots in putty terminal
included)

usage:
python -c "import py3to2; import img2txt; img2txt.img2txt.test()"
python -c "import py3to2; import img2txt; img2txt.tplot3d.test()"

> Added file: http://bugs.python.org/file11875/mario.jpg

Please don't use the bug tracker to upload unrelated code
(such as Numpy code - Numpy is not part of Python core).
I'm tempted to mark all these images as spam.