################################################################################ REVISION DATE: 20081019 AUTHOR: kai zhu - kaizhu@hpc.usc.edu HOMEPAGE: http://www-rcf.usc.edu/~kaizhu/work/py3to2 ABSTRACT: py3to2 is a patch / module that backports py3k opcode instructions to python 2.x (py2x). this allows py2x to run py3k code natively @ the bytecode-level. the motivation is that py3k lacks many useful 2.x modules. the lack of 3rd-party modules in particular (eg scipy, PIL, ...) is a hindrance to py3k development/testing. py3to2 addresses these issues by allowing py3k code to b developed/integrated/run natively under the robust, software-rich, py2x environment. MECHANISM: 1. upon init, py3to2 starts up a py3k server w/ pipe io. 2. -> received via pipe io from py3to2, the py3k server 1st natively compiles the src code into a py3k codeobj, then converts it to py2x format w/ the addition of the backported opcodes. 3. <- the serialized py2x codeobj is piped back to py3to2, which unserializes it & eval/exec it as normal. in theory, this mechanism should transparently implement any bytecode-level py3k language feature. performance and robustness should b minimally impacted, since the compiled code is directly run as native 2.x++ bytecode. BACKWARD COMPATIBILITY: make test (patched python 2.6 on redhat opteron x86_64) ... 324 tests OK. 36 tests skipped: test_aepack test_al test_applesingle test_bsddb185 test_bsddb3 test_cd test_cl test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses test_dl test_gdbm test_gl test_imageop test_imgfile test_kqueue test_linuxaudiodev test_macos test_macostools test_normalization test_ossaudiodev test_pep277 test_py3kwarn test_scriptpackages test_socketserver test_startfile test_sunaudiodev test_timeout test_urllib2net test_urllibnet test_winreg test_winsound test_zipfile64 1 skip unexpected on linux2: test_gdbm ################################################################################ PY3K FEATURES TESTED TO WORK: pep3102 Keyword-Only Arguments pep3104 Access to Names in Outer Scopes pep3105 Make print a function pep3107 Function Annotations pep3111 Simple input built-in in Python 3000 pep3112 Bytes literals in Python 3000 pep3113 Removal of Tuple Parameter Unpacking pep3114 Renaming iterator.next() to iterator.__next__() pep3115 Metaclasses in Python 3000 pep3127 Integer Literal Support and Syntax pep3129 Class Decorators pep3132 Extended Iterable Unpacking pep3135 New Super PY3K FEATURES NOT WORKING: pep3101 Advanced String Formatting pep3106 Revamping dict.keys(), .values() & .items() pep3108 Standard Library Reorganization pep3109/3110 Exceptions in Python 3000 pep3116 New I/O pep3118 Revising the buffer protocol pep3119 Introducing Abstract Base Classes pep3120 Using UTF-8 as the default source encoding pep3121 Extension Module Initialization & Finalization pep3123 Making PyObject_HEAD conform to standard C pep3131 Supporting Non-ASCII Identifiers pep3134 Exception Chaining and Embedded Tracebacks pep3137 Immutable Bytes and Mutable Buffer pep3141 A Type Hierarchy for Numbers pep3138 String representation in Python 3000 REQUIREMENTS: * linux / unix (windows has io problems w/ py3k server) * python-2.6 src (to b patched) * python-3.0rc1 executable (as py3k server) py3to2 files: (download from http://www-rcf.usc.edu/~kaizhu/work/py3to2) * ceval.c - patched file * py3to2.py - python script BUILD / INSTALL: * get python-2.6 src. backup & replace python-2.6/Python/ceval.c * build & install python-2.6 (eg "./configure --prefix=/installdir; make clean; make; make install") * copy py3to2.py to /installdir/.../site-packages/ * install vanilla python-3.0 & make sure $PATH env contains "python3.0" executable USAGE: simply add the magic line "from __future__ import py3k_syntax" to ur script EXAMPLE: pep3132 Extended Iterable Unpacking ################################################################################ # save to file pep3132.py from __future__ import py3k_syntax a, b, *c = 1, 2, 3, 4 print(a, b, c) ################################################################################ Python 2.6 (r26:66714, Oct 19 2008, 02:49:13) [GCC 3.4.6 20060404 (Red Hat 3.4.6-10)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> >>> import py3to2 created read/write pipes: (4, 5) py3k server starting with pipes in/out/err: 7 5 -2 py3k server: Python 3.0rc1 (r30rc1:66499, Oct 15 2008, 15:43:09) py3k server: [GCC 3.4.6 20060404 (Red Hat 3.4.6-10)] on linux2 py3k server: Type "help", "copyright", "credits" or "license" for more information. py3k server: >>> '' >>> >>> import pep3132 1 2 [3, 4] ################################################################################ CHANGELOG: 20081019 ported to python-2.6 consolidate & simplify patches to 1 file: ceval.c created extension module builtins_py3k revamped import hook again removed unicode support & restrict source code to ascii-only 20080727 revampled import hook 20080911 consolidate patches to 2 files: bltinmodule.c & ceval.c 20080828 add kwonlyargcount 'attr' to codeobj add __annotations__ & __kwdefaults__ attr to funcobj add __pseudomethod__ feature to baseobj 20080819 pure python import hook - removed magic comment & use magic path instead revamped str & bytes handling revamped py3k .pyc file handling 20080802 pep3135 New Super 20080717 pep3107 Function Annotations pep3120 Using UTF-8 as the default source encoding pep3131 Supporting Non-ASCII Identifiers 20080713 import / reload works transparently on py3k scripts using a magic comment added pep3102 Keyword-Only Arguments 20080709 added a py3k preparser 20080702 rewrote py3k server's pipe io. implemented partial bytearray & bytes class. wrote a few simple tests 20080630 __build_class__ function to bltmodule.c. tested class decorators to b working. ################################################################################