Index: Doc/lib/libdis.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libdis.tex,v retrieving revision 1.38 diff -c -r1.38 libdis.tex *** Doc/lib/libdis.tex 5 Aug 2002 23:33:54 -0000 1.38 --- Doc/lib/libdis.tex 15 Aug 2002 10:27:42 -0000 *************** *** 23,39 **** \begin{verbatim} >>> dis.dis(myfunc) ! 0 SET_LINENO 1 ! ! 3 SET_LINENO 2 ! 6 LOAD_GLOBAL 0 (len) ! 9 LOAD_FAST 0 (alist) ! 12 CALL_FUNCTION 1 ! 15 RETURN_VALUE ! 16 LOAD_CONST 0 (None) ! 19 RETURN_VALUE \end{verbatim} The \module{dis} module defines the following functions and constants: \begin{funcdesc}{dis}{\optional{bytesource}} --- 23,37 ---- \begin{verbatim} >>> dis.dis(myfunc) ! 2 0 LOAD_GLOBAL 0 (len) ! 3 LOAD_FAST 0 (alist) ! 6 CALL_FUNCTION 1 ! 9 RETURN_VALUE ! 10 RETURN_NONE \end{verbatim} + (The ``2'' is a line number). + The \module{dis} module defines the following functions and constants: \begin{funcdesc}{dis}{\optional{bytesource}} *************** *** 56,61 **** --- 54,60 ---- was provided. The output is divided in the following columns: \begin{enumerate} + \item the line number, for the first instruction of each line \item the current instruction, indicated as \samp{-->}, \item a labelled instruction, indicated with \samp{>\code{>}}, \item the address of the instruction, *************** *** 402,407 **** --- 401,414 ---- Returns with TOS to the caller of the function. \end{opcodedesc} + \begin{opcodedesc}{RETURN_NONE}{} + Returns \constant{None} to the caller of the function. This opcode is + generated as the last opcode of every function and only then, for + reasons to do with tracing support. See the comments in the function + \cfunction{maybe_call_line_trace} in \file{Python/ceval.c} for the + gory details. \versionadded{2.3}. + \end{opcodedesc} + \begin{opcodedesc}{YIELD_VALUE}{} Pops \code{TOS} and yields it from a generator. \end{opcodedesc} *************** *** 621,627 **** \end{opcodedesc} \begin{opcodedesc}{SET_LINENO}{lineno} ! Sets the current line number to \var{lineno}. \end{opcodedesc} \begin{opcodedesc}{RAISE_VARARGS}{argc} --- 628,634 ---- \end{opcodedesc} \begin{opcodedesc}{SET_LINENO}{lineno} ! This opcode is obsolete. \end{opcodedesc} \begin{opcodedesc}{RAISE_VARARGS}{argc} Index: Doc/lib/libtraceback.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libtraceback.tex,v retrieving revision 1.15 diff -c -r1.15 libtraceback.tex *** Doc/lib/libtraceback.tex 25 Jul 2002 21:11:23 -0000 1.15 --- Doc/lib/libtraceback.tex 15 Aug 2002 10:27:43 -0000 *************** *** 118,127 **** \begin{funcdesc}{tb_lineno}{tb} This function returns the current line number set in the traceback ! object. This is normally the same as the \code{\var{tb}.tb_lineno} ! field of the object, but when optimization is used (the -O flag) this ! field is not updated correctly; this function calculates the correct ! value. \end{funcdesc} --- 118,127 ---- \begin{funcdesc}{tb_lineno}{tb} This function returns the current line number set in the traceback ! object. This function was necessary because in versions of Python ! prior to 2.3 when the \programopt{O} flag was passed to Python the ! \code{\var{tb}.tb_lineno} was not updated correctly. This function ! has no use in versions past 2.3. \end{funcdesc} Index: Doc/tut/tut.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/tut/tut.tex,v retrieving revision 1.169 diff -c -r1.169 tut.tex *** Doc/tut/tut.tex 14 Aug 2002 15:26:18 -0000 1.169 --- Doc/tut/tut.tex 15 Aug 2002 10:27:43 -0000 *************** *** 2340,2351 **** \item When the Python interpreter is invoked with the \programopt{-O} flag, ! optimized code is generated and stored in \file{.pyo} files. ! The optimizer currently doesn't help much; it only removes ! \keyword{assert} statements and \code{SET_LINENO} instructions. ! When \programopt{-O} is used, \emph{all} bytecode is optimized; ! \code{.pyc} files are ignored and \code{.py} files are compiled to ! optimized bytecode. \item Passing two \programopt{-O} flags to the Python interpreter --- 2340,2350 ---- \item When the Python interpreter is invoked with the \programopt{-O} flag, ! optimized code is generated and stored in \file{.pyo} files. The ! optimizer currently doesn't help much; it only removes ! \keyword{assert} statements. When \programopt{-O} is used, \emph{all} ! bytecode is optimized; \code{.pyc} files are ignored and \code{.py} ! files are compiled to optimized bytecode. \item Passing two \programopt{-O} flags to the Python interpreter Index: Doc/whatsnew/whatsnew23.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/whatsnew/whatsnew23.tex,v retrieving revision 1.44 diff -c -r1.44 whatsnew23.tex *** Doc/whatsnew/whatsnew23.tex 15 Aug 2002 00:40:21 -0000 1.44 --- Doc/whatsnew/whatsnew23.tex 15 Aug 2002 10:27:43 -0000 *************** *** 656,663 **** \end{verbatim} - \end{itemize} - %====================================================================== \section{New and Improved Modules} --- 656,661 ---- *************** *** 987,995 **** when running Python's \file{configure} script. (Contributed by Ondrej Palkovsky.) ! \item The \csimplemacro{DL_EXPORT} and \csimplemacro{DL_IMPORT} macros are now ! deprecated. Initialization functions for Python extension modules ! should now be declared using the new macro \csimplemacro{PyMODINIT_FUNC}, while the Python core will generally use the \csimplemacro{PyAPI_FUNC} and \csimplemacro{PyAPI_DATA} macros. --- 985,993 ---- when running Python's \file{configure} script. (Contributed by Ondrej Palkovsky.) ! \item The \csimplemacro{DL_EXPORT} and \csimplemacro{DL_IMPORT} macros ! are now deprecated. Initialization functions for Python extension ! modules should now be declared using the new macro \csimplemacro{PyMODINIT_FUNC}, while the Python core will generally use the \csimplemacro{PyAPI_FUNC} and \csimplemacro{PyAPI_DATA} macros. *************** *** 1075,1080 **** --- 1073,1102 ---- \item The tools used to build the documentation now work under Cygwin as well as \UNIX. + + \item The \code{SET_LINENO} opcode has been removed. Back in the + mists of time, this opcode was needed to produce line numbers in + tracebacks and support trace functions (for, e.g., \module{pdb}). + Since Python 1.5, the line numbers in tracebacks have been computed + using a different mechanism that works with ``python -O''. For Python + 2.3 Michael Hudson implemented a similar scheme to determine when to + call the trace function, removing the need for \code{SET_LINENO} + entirely. + + Python code will be hard pushed to notice a difference from this + change, apart from a slight speed up when python is run without + \programopt{-O}. + + C extensions that access the \member{f_lineno} field of frame objects + should instead call \code{PyCode_Addr2Line(f->f_code, f->f_lasti)}. + This will have the added effect of making the code work as desired + under ``python -O'' in earlier versions of Python. + + To make tracing work as expected, it was found necessary to add a new + opcode, \cdata{RETURN_NONE}, to the VM. If you want to know why, read + the comments in the function \cfunction{maybe_call_line_trace} in + \file{Python/ceval.c}. + \end{itemize} \end{itemize} Index: Include/opcode.h =================================================================== RCS file: /cvsroot/python/python/dist/src/Include/opcode.h,v retrieving revision 2.39 diff -c -r2.39 opcode.h *** Include/opcode.h 13 Jun 2002 17:59:51 -0000 2.39 --- Include/opcode.h 15 Aug 2002 10:27:43 -0000 *************** *** 71,76 **** --- 71,79 ---- #define INPLACE_OR 79 #define BREAK_LOOP 80 + #define RETURN_NONE 81 /* *only* for function epilogues + -- see comments in + ceval.c:maybe_call_line_trace for why */ #define LOAD_LOCALS 82 #define RETURN_VALUE 83 #define IMPORT_STAR 84 *************** *** 118,125 **** #define LOAD_FAST 124 /* Local variable number */ #define STORE_FAST 125 /* Local variable number */ #define DELETE_FAST 126 /* Local variable number */ - - #define SET_LINENO 127 /* Current line number */ #define RAISE_VARARGS 130 /* Number of raise arguments (1, 2 or 3) */ /* CALL_FUNCTION_XXX opcodes defined below depend on this definition */ --- 121,126 ---- Index: Lib/dis.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/dis.py,v retrieving revision 1.41 diff -c -r1.41 dis.py *** Lib/dis.py 13 Jun 2002 17:59:51 -0000 1.41 --- Lib/dis.py 15 Aug 2002 10:27:43 -0000 *************** *** 55,60 **** --- 55,74 ---- def disassemble(co, lasti=-1): """Disassemble a code object.""" code = co.co_code + + byte_increments = [ord(c) for c in co.co_lnotab[0::2]] + line_increments = [ord(c) for c in co.co_lnotab[1::2]] + table_length = len(byte_increments) # == len(line_increments) + + lineno = co.co_firstlineno + table_index = 0 + while (table_index < table_length + and byte_increments[table_index] == 0): + lineno += line_increments[table_index] + table_index += 1 + addr = 0 + line_incr = 0 + labels = findlabels(code) n = len(code) i = 0 *************** *** 63,69 **** while i < n: c = code[i] op = ord(c) ! if op == SET_LINENO and i > 0: print # Extra blank line if i == lasti: print '-->', else: print ' ', if i in labels: print '>>', --- 77,99 ---- while i < n: c = code[i] op = ord(c) ! ! if i >= addr: ! lineno += line_incr ! while table_index < table_length: ! addr += byte_increments[table_index] ! line_incr = line_increments[table_index] ! table_index += 1 ! if line_incr: ! break ! else: ! addr = sys.maxint ! if i > 0: ! print ! print "%3d"%lineno, ! else: ! print ' ', ! if i == lasti: print '-->', else: print ' ', if i in labels: print '>>', *************** *** 224,229 **** --- 254,260 ---- def_op('INPLACE_OR', 79) def_op('BREAK_LOOP', 80) + def_op('RETURN_NONE', 81) def_op('LOAD_LOCALS', 82) def_op('RETURN_VALUE', 83) def_op('IMPORT_STAR', 84) *************** *** 276,284 **** haslocal.append(125) def_op('DELETE_FAST', 126) # Local variable number haslocal.append(126) - - def_op('SET_LINENO', 127) # Current line number - SET_LINENO = 127 def_op('RAISE_VARARGS', 130) # Number of raise arguments (1, 2, or 3) def_op('CALL_FUNCTION', 131) # #args + (#kwargs << 8) --- 307,312 ---- Index: Lib/inspect.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/inspect.py,v retrieving revision 1.37 diff -c -r1.37 inspect.py *** Lib/inspect.py 4 Aug 2002 17:22:59 -0000 1.37 --- Lib/inspect.py 15 Aug 2002 10:27:43 -0000 *************** *** 711,717 **** raise TypeError, 'arg is not a frame or traceback object' filename = getsourcefile(frame) or getfile(frame) ! lineno = getlineno(frame) if context > 0: start = lineno - 1 - context//2 try: --- 711,717 ---- raise TypeError, 'arg is not a frame or traceback object' filename = getsourcefile(frame) or getfile(frame) ! lineno = frame.f_lineno if context > 0: start = lineno - 1 - context//2 try: *************** *** 730,747 **** def getlineno(frame): """Get the line number from a frame object, allowing for optimization.""" ! # Written by Marc-André Lemburg; revised by Jim Hugunin and Fredrik Lundh. ! lineno = frame.f_lineno ! code = frame.f_code ! if hasattr(code, 'co_lnotab'): ! table = code.co_lnotab ! lineno = code.co_firstlineno ! addr = 0 ! for i in range(0, len(table), 2): ! addr = addr + ord(table[i]) ! if addr > frame.f_lasti: break ! lineno = lineno + ord(table[i+1]) ! return lineno def getouterframes(frame, context=1): """Get a list of records for a frame and all higher (calling) frames. --- 730,737 ---- def getlineno(frame): """Get the line number from a frame object, allowing for optimization.""" ! # FrameType.f_lineno is now a descriptor that grovels co_lnotab ! return frame.f_lineno def getouterframes(frame, context=1): """Get a list of records for a frame and all higher (calling) frames. Index: Lib/pdb.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/pdb.py,v retrieving revision 1.54 diff -c -r1.54 pdb.py *** Lib/pdb.py 12 Jul 2002 13:10:53 -0000 1.54 --- Lib/pdb.py 15 Aug 2002 10:27:43 -0000 *************** *** 105,111 **** if len(line) > 0 and line[0] != '#': self.onecmd(line) ! # Override Bdb methods (except user_call, for now) def user_line(self, frame): """This function is called when we stop or break at this line.""" --- 105,117 ---- if len(line) > 0 and line[0] != '#': self.onecmd(line) ! # Override Bdb methods ! ! def user_call(self, frame, argument_list): ! """This method is called when there is the remote possibility ! that we ever need to stop in this function.""" ! print '--Call--' ! self.interaction(frame, None) def user_line(self, frame): """This function is called when we stop or break at this line.""" Index: Lib/traceback.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/traceback.py,v retrieving revision 1.27 diff -c -r1.27 traceback.py *** Lib/traceback.py 2 Jun 2002 03:04:51 -0000 1.27 --- Lib/traceback.py 15 Aug 2002 10:27:43 -0000 *************** *** 59,65 **** n = 0 while tb is not None and (limit is None or n < limit): f = tb.tb_frame ! lineno = tb_lineno(tb) co = f.f_code filename = co.co_filename name = co.co_name --- 59,65 ---- n = 0 while tb is not None and (limit is None or n < limit): f = tb.tb_frame ! lineno = tb.tb_lineno co = f.f_code filename = co.co_filename name = co.co_name *************** *** 92,98 **** n = 0 while tb is not None and (limit is None or n < limit): f = tb.tb_frame ! lineno = tb_lineno(tb) co = f.f_code filename = co.co_filename name = co.co_name --- 92,98 ---- n = 0 while tb is not None and (limit is None or n < limit): f = tb.tb_frame ! lineno = tb.tb_lineno co = f.f_code filename = co.co_filename name = co.co_name *************** *** 263,269 **** list = [] n = 0 while f is not None and (limit is None or n < limit): ! lineno = f.f_lineno # XXX Too bad if -O is used co = f.f_code filename = co.co_filename name = co.co_name --- 263,269 ---- list = [] n = 0 while f is not None and (limit is None or n < limit): ! lineno = f.f_lineno co = f.f_code filename = co.co_filename name = co.co_name *************** *** 279,301 **** def tb_lineno(tb): """Calculate correct line number of traceback given in tb. ! Even works with -O on. """ ! # Coded by Marc-Andre Lemburg from the example of PyCode_Addr2Line() ! # in compile.c. ! # Revised version by Jim Hugunin to work with JPython too. ! ! c = tb.tb_frame.f_code ! if not hasattr(c, 'co_lnotab'): ! return tb.tb_lineno ! ! tab = c.co_lnotab ! line = c.co_firstlineno ! stopat = tb.tb_lasti ! addr = 0 ! for i in range(0, len(tab), 2): ! addr = addr + ord(tab[i]) ! if addr > stopat: ! break ! line = line + ord(tab[i+1]) ! return line --- 279,284 ---- def tb_lineno(tb): """Calculate correct line number of traceback given in tb. ! Obsolete in 2.3. """ ! return tb.tb_lineno Index: Lib/test/test_hotshot.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_hotshot.py,v retrieving revision 1.11 diff -c -r1.11 test_hotshot.py *** Lib/test/test_hotshot.py 5 Aug 2002 22:16:40 -0000 1.11 --- Lib/test/test_hotshot.py 15 Aug 2002 10:27:43 -0000 *************** *** 91,100 **** f_lineno = f.func_code.co_firstlineno g_lineno = g.func_code.co_firstlineno events = [(ENTER, ("test_hotshot", g_lineno, "g")), - (LINE, ("test_hotshot", g_lineno, "g")), (LINE, ("test_hotshot", g_lineno+1, "g")), (ENTER, ("test_hotshot", f_lineno, "f")), - (LINE, ("test_hotshot", f_lineno, "f")), (LINE, ("test_hotshot", f_lineno+1, "f")), (LINE, ("test_hotshot", f_lineno+2, "f")), (EXIT, ("test_hotshot", f_lineno, "f")), --- 91,98 ---- Index: Misc/NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.468 diff -c -r1.468 NEWS *** Misc/NEWS 14 Aug 2002 21:20:32 -0000 1.468 --- Misc/NEWS 15 Aug 2002 10:27:44 -0000 *************** *** 57,62 **** --- 57,67 ---- Core and builtins + - SET_LINENO is gone. co_lnotab is now consulted to determine when to + call the trace function. C code that accessed f_lineno should call + PyCode_Addr2Line instead (f_lineno is still there, but not kept up + to date). + - There's a new warning category, FutureWarning. This is used to warn about a number of situations where the value or sign of an integer result will change in Python 2.4 as a result of PEP 237 (integer Index: Modules/_hotshot.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/_hotshot.c,v retrieving revision 1.25 diff -c -r1.25 _hotshot.c *** Modules/_hotshot.c 8 Aug 2002 19:46:15 -0000 1.25 --- Modules/_hotshot.c 15 Aug 2002 10:27:44 -0000 *************** *** 152,158 **** * Low bits: Opcode: Meaning: * 0x00 ENTER enter a frame * 0x01 EXIT exit a frame ! * 0x02 LINENO SET_LINENO instruction was executed * 0x03 OTHER more bits are needed to deecode * * If the type is OTHER, the record is not packed so tightly, and the --- 152,158 ---- * Low bits: Opcode: Meaning: * 0x00 ENTER enter a frame * 0x01 EXIT exit a frame ! * 0x02 LINENO execution moved onto a different line * 0x03 OTHER more bits are needed to deecode * * If the type is OTHER, the record is not packed so tightly, and the *************** *** 888,896 **** case PyTrace_LINE: if (self->linetimings) ! return pack_lineno_tdelta(self, frame->f_lineno, get_tdelta(self)); else ! return pack_lineno(self, frame->f_lineno); default: /* ignore PyTrace_EXCEPTION */ --- 888,899 ---- case PyTrace_LINE: if (self->linetimings) ! return pack_lineno_tdelta(self, PyCode_Addr2Line(frame->f_code, ! frame->f_lasti), ! get_tdelta(self)); else ! return pack_lineno(self, PyCode_Addr2Line(frame->f_code, ! frame->f_lasti)); default: /* ignore PyTrace_EXCEPTION */ *************** *** 1227,1234 **** "\n" "closed: True if the profiler has already been closed.\n" "frametimings: True if ENTER/EXIT events collect timing information.\n" ! "lineevents: True if SET_LINENO events are reported to the profiler.\n" ! "linetimings: True if SET_LINENO events collect timing information."); static PyTypeObject ProfilerType = { PyObject_HEAD_INIT(NULL) --- 1230,1237 ---- "\n" "closed: True if the profiler has already been closed.\n" "frametimings: True if ENTER/EXIT events collect timing information.\n" ! "lineevents: True if line events are reported to the profiler.\n" ! "linetimings: True if line events collect timing information."); static PyTypeObject ProfilerType = { PyObject_HEAD_INIT(NULL) Index: Objects/frameobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/frameobject.c,v retrieving revision 2.63 diff -c -r2.63 frameobject.c *** Objects/frameobject.c 1 Aug 2002 18:50:33 -0000 2.63 --- Objects/frameobject.c 15 Aug 2002 10:27:44 -0000 *************** *** 16,22 **** {"f_builtins", T_OBJECT, OFF(f_builtins),RO}, {"f_globals", T_OBJECT, OFF(f_globals), RO}, {"f_lasti", T_INT, OFF(f_lasti), RO}, - {"f_lineno", T_INT, OFF(f_lineno), RO}, {"f_restricted",T_INT, OFF(f_restricted),RO}, {"f_trace", T_OBJECT, OFF(f_trace)}, {"f_exc_type", T_OBJECT, OFF(f_exc_type)}, --- 16,21 ---- *************** *** 33,40 **** --- 32,50 ---- return f->f_locals; } + static PyObject * + frame_getlineno(PyFrameObject *f, void *closure) + { + int lineno; + + lineno = PyCode_Addr2Line(f->f_code, f->f_lasti); + + return PyInt_FromLong(lineno); + } + static PyGetSetDef frame_getsetlist[] = { {"f_locals", (getter)frame_getlocals, NULL, NULL}, + {"f_lineno", (getter)frame_getlineno, NULL, NULL}, {0} }; *************** *** 306,312 **** f->f_exc_type = f->f_exc_value = f->f_exc_traceback = NULL; f->f_tstate = tstate; ! f->f_lasti = 0; f->f_lineno = code->co_firstlineno; f->f_restricted = (builtins != tstate->interp->builtins); f->f_iblock = 0; --- 316,322 ---- f->f_exc_type = f->f_exc_value = f->f_exc_traceback = NULL; f->f_tstate = tstate; ! f->f_lasti = -1; f->f_lineno = code->co_firstlineno; f->f_restricted = (builtins != tstate->interp->builtins); f->f_iblock = 0; Index: Python/ceval.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/ceval.c,v retrieving revision 2.323 diff -c -r2.323 ceval.c *** Python/ceval.c 9 Aug 2002 18:35:52 -0000 2.323 --- Python/ceval.c 15 Aug 2002 10:27:45 -0000 *************** *** 51,56 **** --- 51,59 ---- static void call_trace_protected(Py_tracefunc, PyObject *, PyFrameObject *, int); static void call_exc_trace(Py_tracefunc, PyObject *, PyFrameObject *); + static void maybe_call_line_trace(int, Py_tracefunc, PyObject *, + PyFrameObject *, int *, int *); + static PyObject *apply_slice(PyObject *, PyObject *, PyObject *); static int assign_slice(PyObject *, PyObject *, PyObject *, PyObject *); *************** *** 499,504 **** --- 502,517 ---- PyObject *retval = NULL; /* Return value */ PyThreadState *tstate = PyThreadState_GET(); PyCodeObject *co; + + /* when tracing we set things up so that + + not (instr_lb <= current_bytecode_offset < instr_ub) + + is true when the line being executed has changed. The + initial values are such as to make this false the first + time it is tested. */ + int instr_ub = -1, instr_lb = 0; + unsigned char *first_instr; PyObject *names; PyObject *consts; *************** *** 586,592 **** fastlocals = f->f_localsplus; freevars = f->f_localsplus + f->f_nlocals; _PyCode_GETCODEPTR(co, &first_instr); ! next_instr = first_instr + f->f_lasti; stack_pointer = f->f_stacktop; assert(stack_pointer != NULL); f->f_stacktop = NULL; /* remains NULL unless yield suspends frame */ --- 599,610 ---- fastlocals = f->f_localsplus; freevars = f->f_localsplus + f->f_nlocals; _PyCode_GETCODEPTR(co, &first_instr); ! if (f->f_lasti < 0) { ! next_instr = first_instr; ! } ! else { ! next_instr = first_instr + f->f_lasti; ! } stack_pointer = f->f_stacktop; assert(stack_pointer != NULL); f->f_stacktop = NULL; /* remains NULL unless yield suspends frame */ *************** *** 637,644 **** w = NULL; for (;;) { ! assert(stack_pointer >= f->f_valuestack); /* else underflow */ ! assert(STACK_LEVEL() <= f->f_stacksize); /* else overflow */ /* Do periodic things. Doing this every time through the loop would add too much overhead, so we do it only every Nth instruction. We also do it if --- 655,663 ---- w = NULL; for (;;) { ! assert(stack_pointer >= f->f_valuestack); /* else underflow */ ! assert(STACK_LEVEL() <= f->f_stacksize); /* else overflow */ ! /* Do periodic things. Doing this every time through the loop would add too much overhead, so we do it only every Nth instruction. We also do it if *************** *** 658,665 **** #if !defined(HAVE_SIGNAL_H) || defined(macintosh) /* If we have true signals, the signal handler will call Py_AddPendingCall() so we don't ! have to call sigcheck(). On the Mac and ! DOS, alas, we have to call it. */ if (PyErr_CheckSignals()) { why = WHY_EXCEPTION; goto on_error; --- 677,684 ---- #if !defined(HAVE_SIGNAL_H) || defined(macintosh) /* If we have true signals, the signal handler will call Py_AddPendingCall() so we don't ! have to call PyErr_CheckSignals(). On the ! Mac and DOS, alas, we have to call it. */ if (PyErr_CheckSignals()) { why = WHY_EXCEPTION; goto on_error; *************** *** 686,694 **** fast_next_opcode: /* Extract opcode and argument */ - #if defined(Py_DEBUG) || defined(LLTRACE) f->f_lasti = INSTR_OFFSET(); - #endif opcode = NEXTOP(); if (HAS_ARG(opcode)) --- 705,711 ---- *************** *** 708,722 **** if (lltrace) { if (HAS_ARG(opcode)) { printf("%d: %d, %d\n", ! (int) (INSTR_OFFSET() - 3), ! opcode, oparg); } else { printf("%d: %d\n", ! (int) (INSTR_OFFSET() - 1), opcode); } } #endif /* Main switch on opcode */ switch (opcode) { --- 725,750 ---- if (lltrace) { if (HAS_ARG(opcode)) { printf("%d: %d, %d\n", ! f->f_lasti, opcode, oparg); } else { printf("%d: %d\n", ! f->f_lasti, opcode); } } #endif + + /* line-by-line tracing support */ + + if (tstate->c_tracefunc != NULL && !tstate->tracing) { + /* see maybe_call_line_trace + for expository comments */ + maybe_call_line_trace(opcode, + tstate->c_tracefunc, + tstate->c_traceobj, + f, &instr_lb, &instr_ub); + } + /* Main switch on opcode */ switch (opcode) { *************** *** 728,753 **** /* case STOP_CODE: this is an error! */ - case SET_LINENO: - #ifdef LLTRACE - if (lltrace) - printf("--- %s:%d \n", filename, oparg); - #endif - f->f_lineno = oparg; - if (tstate->c_tracefunc == NULL || tstate->tracing) - goto fast_next_opcode; - /* Trace each line of code reached */ - f->f_lasti = INSTR_OFFSET(); - /* Inline call_trace() for performance: */ - tstate->tracing++; - tstate->use_tracing = 0; - err = (tstate->c_tracefunc)(tstate->c_traceobj, f, - PyTrace_LINE, Py_None); - tstate->use_tracing = (tstate->c_tracefunc - || tstate->c_profilefunc); - tstate->tracing--; - break; - case LOAD_FAST: x = GETLOCAL(oparg); if (x != NULL) { --- 756,761 ---- *************** *** 1504,1512 **** --- 1512,1528 ---- why = WHY_RETURN; break; + case RETURN_NONE: + retval = Py_None; + Py_INCREF(retval); + why = WHY_RETURN; + break; + case YIELD_VALUE: retval = POP(); f->f_stacktop = stack_pointer; + /* abuse the lasti field: here it points to + the *next* instruction */ f->f_lasti = INSTR_OFFSET(); why = WHY_YIELD; break; *************** *** 1954,1960 **** int n = na + 2 * nk; PyObject **pfunc = stack_pointer - n - 1; PyObject *func = *pfunc; - f->f_lasti = INSTR_OFFSET() - 3; /* For tracing */ /* Always dispatch PyCFunction first, because these are presumed to be the most frequent --- 1970,1975 ---- *************** *** 2022,2028 **** n++; pfunc = stack_pointer - n - 1; func = *pfunc; - f->f_lasti = INSTR_OFFSET() - 3; /* For tracing */ if (PyMethod_Check(func) && PyMethod_GET_SELF(func) != NULL) { --- 2037,2042 ---- *************** *** 2134,2140 **** default: fprintf(stderr, "XXX lineno: %d, opcode: %d\n", ! f->f_lineno, opcode); PyErr_SetString(PyExc_SystemError, "unknown opcode"); why = WHY_EXCEPTION; break; --- 2148,2155 ---- default: fprintf(stderr, "XXX lineno: %d, opcode: %d\n", ! PyCode_Addr2Line(f->f_code, f->f_lasti), ! opcode); PyErr_SetString(PyExc_SystemError, "unknown opcode"); why = WHY_EXCEPTION; break; *************** *** 2189,2197 **** /* Log traceback info if this is a real exception */ if (why == WHY_EXCEPTION) { - f->f_lasti = INSTR_OFFSET() - 1; - if (HAS_ARG(opcode)) - f->f_lasti -= 2; PyTraceBack_Here(f); if (tstate->c_tracefunc != NULL) --- 2204,2209 ---- *************** *** 2873,2878 **** --- 2885,3010 ---- || (tstate->c_profilefunc != NULL)); tstate->tracing--; return result; + } + + static void + maybe_call_line_trace(int opcode, Py_tracefunc func, PyObject *obj, + PyFrameObject *frame, int *instr_lb, int *instr_ub) + { + /* The theory of SET_LINENO-less tracing. + + In a nutshell, we use the co_lnotab field of the code object + to tell when execution has moved onto a different line. + + As mentioned above, the basic idea is so set things up so + that + + *instr_lb <= frame->f_lasti < *instr_ub + + is true so long as execution does not change lines. + + This is all fairly simple. Digging the information out of + co_lnotab takes some work, but is conceptually clear. + + Somewhat harder to explain is why we don't call the line + trace function when executing a POP_TOP or RETURN_NONE + opcodes. An example probably serves best. + + Consider this code: + + 1: def f(a): + 2: if a: + 3: print 1 + 4: else: + 5: print 2 + + which compiles to this: + + 2 0 LOAD_FAST 0 (a) + 3 JUMP_IF_FALSE 9 (to 15) + 6 POP_TOP + + 3 7 LOAD_CONST 1 (1) + 10 PRINT_ITEM + 11 PRINT_NEWLINE + 12 JUMP_FORWARD 6 (to 21) + >> 15 POP_TOP + + 5 16 LOAD_CONST 2 (2) + 19 PRINT_ITEM + 20 PRINT_NEWLINE + >> 21 RETURN_NONE + + If a is false, execution will jump to instruction at offset + 15 and the co_lnotab will claim that execution has moved to + line 3. This is at best misleading. In this case we could + associate the POP_TOP with line 4, but that doesn't make + sense in all cases (I think). + + On the other hand, if a is true, execution will jump from + instruction offset 12 to offset 21. Then the co_lnotab would + imply that execution has moved to line 5, which is again + misleading. + + This is why it is important that RETURN_NONE is *only* used + for the "falling off the end of the function" form of + returning None -- using it for code like + + 1: def f(): + 2: return + + would, once again, lead to misleading tracing behaviour. + + It is also worth mentioning that getting tracing behaviour + right is the *entire* motivation for adding the RETURN_NONE + opcode. + */ + + if (opcode != POP_TOP && opcode != RETURN_NONE && + (frame->f_lasti < *instr_lb || frame->f_lasti > *instr_ub)) { + PyCodeObject* co = frame->f_code; + int size, addr; + unsigned char* p; + + call_trace(func, obj, frame, PyTrace_LINE, Py_None); + + size = PyString_Size(co->co_lnotab) / 2; + p = (unsigned char*)PyString_AsString( + co->co_lnotab); + + /* possible optimization: if f->f_lasti == instr_ub (likely + to be a common case) then we already know instr_lb -- if + we stored the matching value of p somwhere we could skip + the first while loop. */ + + addr = 0; + + /* see comments in compile.c for the description of + co_lnotab. A point to remember: increments to p should + come in pairs -- although we don't care about the line + increments here, treating them as byte increments gets + confusing, to say the least. */ + + while (size >= 0) { + if (addr + *p > frame->f_lasti) + break; + addr += *p++; + p++; + --size; + } + *instr_lb = addr; + if (size > 0) { + while (--size >= 0) { + addr += *p++; + if (*p++) + break; + } + *instr_ub = addr; + } + else { + *instr_ub = INT_MAX; + } + } } void Index: Python/compile.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/compile.c,v retrieving revision 2.257 diff -c -r2.257 compile.c *** Python/compile.c 14 Aug 2002 15:51:28 -0000 2.257 --- Python/compile.c 15 Aug 2002 10:27:45 -0000 *************** *** 407,415 **** /* All about c_lnotab. ! c_lnotab is an array of unsigned bytes disguised as a Python string. In -O ! mode, SET_LINENO opcodes aren't generated, and bytecode offsets are mapped ! to source code line #s (when needed for tracebacks) via c_lnotab instead. The array is conceptually a list of (bytecode offset increment, line number increment) pairs. The details are important and delicate, best illustrated by example: --- 407,416 ---- /* All about c_lnotab. ! c_lnotab is an array of unsigned bytes disguised as a Python string. Since ! version 2.3, SET_LINENO opcodes are never generated and bytecode offsets are ! mapped to source code line #s via c_lnotab instead. ! The array is conceptually a list of (bytecode offset increment, line number increment) pairs. The details are important and delicate, best illustrated by example: *************** *** 830,840 **** com_addoparg(struct compiling *c, int op, int arg) { int extended_arg = arg >> 16; - if (op == SET_LINENO) { - com_set_lineno(c, arg); - if (Py_OptimizeFlag) - return; - } if (extended_arg){ com_addbyte(c, EXTENDED_ARG); com_addint(c, extended_arg); --- 831,836 ---- *************** *** 1738,1744 **** break; if (ch->n_lineno != lineno) { lineno = ch->n_lineno; ! com_addoparg(c, SET_LINENO, lineno); } com_argument(c, ch, &keywords); if (keywords == NULL) --- 1734,1740 ---- break; if (ch->n_lineno != lineno) { lineno = ch->n_lineno; ! com_set_lineno(c, lineno); } com_argument(c, ch, &keywords); if (keywords == NULL) *************** *** 3168,3174 **** continue; } if (i > 0) ! com_addoparg(c, SET_LINENO, ch->n_lineno); com_node(c, ch); com_addfwref(c, JUMP_IF_FALSE, &a); com_addbyte(c, POP_TOP); --- 3164,3170 ---- continue; } if (i > 0) ! com_set_lineno(c, ch->n_lineno); com_node(c, ch); com_addfwref(c, JUMP_IF_FALSE, &a); com_addbyte(c, POP_TOP); *************** *** 3195,3201 **** com_addfwref(c, SETUP_LOOP, &break_anchor); block_push(c, SETUP_LOOP); c->c_begin = c->c_nexti; ! com_addoparg(c, SET_LINENO, n->n_lineno); com_node(c, CHILD(n, 1)); com_addfwref(c, JUMP_IF_FALSE, &anchor); com_addbyte(c, POP_TOP); --- 3191,3197 ---- com_addfwref(c, SETUP_LOOP, &break_anchor); block_push(c, SETUP_LOOP); c->c_begin = c->c_nexti; ! com_set_lineno(c, n->n_lineno); com_node(c, CHILD(n, 1)); com_addfwref(c, JUMP_IF_FALSE, &anchor); com_addbyte(c, POP_TOP); *************** *** 3228,3234 **** com_node(c, CHILD(n, 3)); com_addbyte(c, GET_ITER); c->c_begin = c->c_nexti; ! com_addoparg(c, SET_LINENO, n->n_lineno); com_addfwref(c, FOR_ITER, &anchor); com_push(c, 1); com_assign(c, CHILD(n, 1), OP_ASSIGN, NULL); --- 3224,3230 ---- com_node(c, CHILD(n, 3)); com_addbyte(c, GET_ITER); c->c_begin = c->c_nexti; ! com_set_lineno(c, n->n_lineno); com_addfwref(c, FOR_ITER, &anchor); com_push(c, 1); com_assign(c, CHILD(n, 1), OP_ASSIGN, NULL); *************** *** 3339,3345 **** } except_anchor = 0; com_push(c, 3); /* tb, val, exc pushed by exception */ ! com_addoparg(c, SET_LINENO, ch->n_lineno); if (NCH(ch) > 1) { com_addbyte(c, DUP_TOP); com_push(c, 1); --- 3335,3341 ---- } except_anchor = 0; com_push(c, 3); /* tb, val, exc pushed by exception */ ! com_set_lineno(c, ch->n_lineno); if (NCH(ch) > 1) { com_addbyte(c, DUP_TOP); com_push(c, 1); *************** *** 3401,3407 **** com_push(c, 3); com_backpatch(c, finally_anchor); ch = CHILD(n, NCH(n)-1); ! com_addoparg(c, SET_LINENO, ch->n_lineno); com_node(c, ch); com_addbyte(c, END_FINALLY); block_pop(c, END_FINALLY); --- 3397,3403 ---- com_push(c, 3); com_backpatch(c, finally_anchor); ch = CHILD(n, NCH(n)-1); ! com_set_lineno(c, ch->n_lineno); com_node(c, ch); com_addbyte(c, END_FINALLY); block_pop(c, END_FINALLY); *************** *** 3727,3733 **** case simple_stmt: /* small_stmt (';' small_stmt)* [';'] NEWLINE */ ! com_addoparg(c, SET_LINENO, n->n_lineno); { int i; for (i = 0; i < NCH(n)-1; i += 2) --- 3723,3729 ---- case simple_stmt: /* small_stmt (';' small_stmt)* [';'] NEWLINE */ ! com_set_lineno(c, n->n_lineno); { int i; for (i = 0; i < NCH(n)-1; i += 2) *************** *** 3736,3742 **** break; case compound_stmt: ! com_addoparg(c, SET_LINENO, n->n_lineno); n = CHILD(n, 0); goto loop; --- 3732,3738 ---- break; case compound_stmt: ! com_set_lineno(c, n->n_lineno); n = CHILD(n, 0); goto loop; *************** *** 3990,3999 **** c->c_infunction = 1; com_node(c, CHILD(n, 4)); c->c_infunction = 0; ! com_addoparg(c, LOAD_CONST, com_addconst(c, Py_None)); ! com_push(c, 1); ! com_addbyte(c, RETURN_VALUE); ! com_pop(c, 1); } static void --- 3986,3992 ---- c->c_infunction = 1; com_node(c, CHILD(n, 4)); c->c_infunction = 0; ! com_addbyte(c, RETURN_NONE); } static void *************** *** 4050,4056 **** static void compile_node(struct compiling *c, node *n) { ! com_addoparg(c, SET_LINENO, n->n_lineno); switch (TYPE(n)) { --- 4043,4049 ---- static void compile_node(struct compiling *c, node *n) { ! com_set_lineno(c, n->n_lineno); switch (TYPE(n)) { *************** *** 4060,4078 **** n = CHILD(n, 0); if (TYPE(n) != NEWLINE) com_node(c, n); ! com_addoparg(c, LOAD_CONST, com_addconst(c, Py_None)); ! com_push(c, 1); ! com_addbyte(c, RETURN_VALUE); ! com_pop(c, 1); c->c_interactive--; break; case file_input: /* A whole file, or built-in function exec() */ com_file_input(c, n); ! com_addoparg(c, LOAD_CONST, com_addconst(c, Py_None)); ! com_push(c, 1); ! com_addbyte(c, RETURN_VALUE); ! com_pop(c, 1); break; case eval_input: /* Built-in function input() */ --- 4053,4065 ---- n = CHILD(n, 0); if (TYPE(n) != NEWLINE) com_node(c, n); ! com_addbyte(c, RETURN_NONE); c->c_interactive--; break; case file_input: /* A whole file, or built-in function exec() */ com_file_input(c, n); ! com_addbyte(c, RETURN_NONE); break; case eval_input: /* Built-in function input() */ Index: Python/frozen.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/frozen.c,v retrieving revision 1.12 diff -c -r1.12 frozen.c *** Python/frozen.c 14 Jun 2002 01:11:57 -0000 1.12 --- Python/frozen.c 15 Aug 2002 10:27:45 -0000 *************** *** 13,24 **** static unsigned char M___hello__[] = { 99,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0, ! 0,115,15,0,0,0,127,0,0,127,1,0,100,0,0,71, ! 72,100,1,0,83,40,2,0,0,0,115,14,0,0,0,72, ! 101,108,108,111,32,119,111,114,108,100,46,46,46,78,40,0, ! 0,0,0,40,0,0,0,0,40,0,0,0,0,40,0,0, ! 0,0,115,8,0,0,0,104,101,108,108,111,46,112,121,115, ! 1,0,0,0,63,1,0,0,0,115,0,0,0,0, }; #define SIZE (int)sizeof(M___hello__) --- 13,24 ---- static unsigned char M___hello__[] = { 99,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0, ! 0,115,9,0,0,0,100,0,0,71,72,100,1,0,83,40, ! 2,0,0,0,115,14,0,0,0,72,101,108,108,111,32,119, ! 111,114,108,100,46,46,46,78,40,0,0,0,0,40,0,0, ! 0,0,40,0,0,0,0,40,0,0,0,0,115,8,0,0, ! 0,104,101,108,108,111,46,112,121,115,1,0,0,0,63,1, ! 0,0,0,115,0,0,0,0, }; #define SIZE (int)sizeof(M___hello__) Index: Python/import.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/import.c,v retrieving revision 2.208 diff -c -r2.208 import.c *** Python/import.c 30 Jun 2002 15:26:10 -0000 2.208 --- Python/import.c 15 Aug 2002 10:27:45 -0000 *************** *** 49,54 **** --- 49,57 ---- algorithm relying on the above scheme. Perhaps we should simply start counting in increments of 10 from now on ?! + MWH, 2002-08-03: Removed SET_LINENO. Couldn't be bothered figuring + out the MAGIC schemes, so just incremented it by 10. + Known values: Python 1.5: 20121 Python 1.5.1: 20121 *************** *** 60,67 **** Python 2.1.2: 60202 Python 2.2: 60717 Python 2.3a0: 62011 */ ! #define MAGIC (62011 | ((long)'\r'<<16) | ((long)'\n'<<24)) /* Magic word as global; note that _PyImport_Init() can change the value of this global to accommodate for alterations of how the --- 63,71 ---- Python 2.1.2: 60202 Python 2.2: 60717 Python 2.3a0: 62011 + Python 2.3a0: 62021 */ ! #define MAGIC (62021 | ((long)'\r'<<16) | ((long)'\n'<<24)) /* Magic word as global; note that _PyImport_Init() can change the value of this global to accommodate for alterations of how the Index: Python/traceback.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/traceback.c,v retrieving revision 2.38 diff -c -r2.38 traceback.c *** Python/traceback.c 14 Apr 2002 20:12:41 -0000 2.38 --- Python/traceback.c 15 Aug 2002 10:27:45 -0000 *************** *** 103,110 **** }; static tracebackobject * ! newtracebackobject(tracebackobject *next, PyFrameObject *frame, int lasti, ! int lineno) { tracebackobject *tb; if ((next != NULL && !PyTraceBack_Check(next)) || --- 103,109 ---- }; static tracebackobject * ! newtracebackobject(tracebackobject *next, PyFrameObject *frame) { tracebackobject *tb; if ((next != NULL && !PyTraceBack_Check(next)) || *************** *** 118,125 **** tb->tb_next = next; Py_XINCREF(frame); tb->tb_frame = frame; ! tb->tb_lasti = lasti; ! tb->tb_lineno = lineno; PyObject_GC_Track(tb); } return tb; --- 117,125 ---- tb->tb_next = next; Py_XINCREF(frame); tb->tb_frame = frame; ! tb->tb_lasti = frame->f_lasti; ! tb->tb_lineno = PyCode_Addr2Line(frame->f_code, ! frame->f_lasti); PyObject_GC_Track(tb); } return tb; *************** *** 130,137 **** { PyThreadState *tstate = frame->f_tstate; tracebackobject *oldtb = (tracebackobject *) tstate->curexc_traceback; ! tracebackobject *tb = newtracebackobject(oldtb, ! frame, frame->f_lasti, frame->f_lineno); if (tb == NULL) return -1; tstate->curexc_traceback = (PyObject *)tb; --- 130,136 ---- { PyThreadState *tstate = frame->f_tstate; tracebackobject *oldtb = (tracebackobject *) tstate->curexc_traceback; ! tracebackobject *tb = newtracebackobject(oldtb, frame); if (tb == NULL) return -1; tstate->curexc_traceback = (PyObject *)tb; Index: Tools/scripts/trace.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/scripts/trace.py,v retrieving revision 1.7 diff -c -r1.7 trace.py *** Tools/scripts/trace.py 25 Jul 2002 16:09:35 -0000 1.7 --- Tools/scripts/trace.py 15 Aug 2002 10:27:45 -0000 *************** *** 370,410 **** except IOError, err: sys.stderr.write("cannot save counts files because %s" % err) ! # Given a code string, return the SET_LINENO information ! def _find_LINENO_from_string(co_code): ! """return all of the SET_LINENO information from a code string""" ! import dis linenos = {} ! # This code was filched from the `dis' module then modified ! n = len(co_code) ! i = 0 ! prev_op = None ! prev_lineno = 0 ! while i < n: ! c = co_code[i] ! op = ord(c) ! if op == dis.SET_LINENO: ! if prev_op == op: ! # two SET_LINENO in a row, so the previous didn't ! # indicate anything. This occurs with triple ! # quoted strings (?). Remove the old one. ! del linenos[prev_lineno] ! prev_lineno = ord(co_code[i+1]) + ord(co_code[i+2])*256 ! linenos[prev_lineno] = 1 ! if op >= dis.HAVE_ARGUMENT: ! i = i + 3 ! else: ! i = i + 1 ! prev_op = op return linenos def _find_LINENO(code): ! """return all of the SET_LINENO information from a code object""" import types # get all of the lineno information from the code of this scope level ! linenos = _find_LINENO_from_string(code.co_code) # and check the constants for references to other code objects for c in code.co_consts: --- 370,398 ---- except IOError, err: sys.stderr.write("cannot save counts files because %s" % err) ! def _find_LINENO_from_code(code): ! """return the numbers of the lines containing the source code that ! was compiled into code""" linenos = {} ! line_increments = [ord(c) for c in code.co_lnotab[1::2]] ! table_length = len(line_increments) ! ! lineno = code.co_first_lineno ! ! for li in line_increments: ! linenos[lineno] = 1 ! lineno += li ! linenos[lineno] = 1 ! return linenos def _find_LINENO(code): ! """return all of the lineno information from a code object""" import types # get all of the lineno information from the code of this scope level ! linenos = _find_LINENO_from_code(code) # and check the constants for references to other code objects for c in code.co_consts: *************** *** 416,424 **** def find_executable_linenos(filename): """return a dict of the line numbers from executable statements in a file - Works by finding all of the code-like objects in the module then searching - the byte code for 'SET_LINENO' terms (so this won't work one -O files). - """ import parser --- 404,409 ---- *************** *** 427,436 **** prog = open(filename).read() ast = parser.suite(prog) code = parser.compileast(ast, filename) - - # The only way I know to find line numbers is to look for the - # SET_LINENO instructions. Isn't there some way to get it from - # the AST? return _find_LINENO(code) --- 412,417 ----