classification
Title: "-m pdb" SyntaxError for "\r\n" formatted py files
Type: behavior Stage: test needed
Components: Library (Lib) Versions: Python 3.1
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: georg.brandl, jason.coombs, mnewman, r.david.murray
Priority: normal Keywords: patch

Created on 2010-02-19 12:52 by mnewman, last changed 2010-07-30 14:16 by georg.brandl. This issue is now closed.

Files
File name Uploaded Description Edit
version_check.py mnewman, 2010-02-19 12:52 Cross-Version Compatible Version Checker
issue 7964 fix with test.patch jason.coombs, 2010-02-21 22:13
Messages (8)
msg99570 - (view) Author: Michael Newman (mnewman) Date: 2010-02-19 12:52
Attached is a version checking script. When you run it normally, it produces output such as:

E:\notes\Programming\python3>c:\Python26\python.exe version_check.py
2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]

E:\notes\Programming\python3>c:\Python31\python.exe version_check.py
3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)]

I wanted to test using "-m pdb" on this script. It works okay for Python 2.6:

E:\notes\Programming\python3>c:\Python26\python.exe -m pdb version_check.py
> e:\notes\programming\python3\version_check.py(4)<module>()
-> Last Updated: 2008-12-21"""
(Pdb) c
2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit (Intel)]
The program finished and will be restarted
> e:\notes\programming\python3\version_check.py(4)<module>()
-> Last Updated: 2008-12-21"""
(Pdb) q

However, if I try it on Python 3.1 I get a SyntaxError:

# --- Begin Python 3.1 Example with "\r\n" source code --- #

E:\notes\Programming\python3>c:\Python31\python.exe -m pdb version_check.py
SyntaxError: ('invalid syntax', ('e:\\notes\\programming\\python3\\version_check
.py', 4, 132, '"""Check what version of python is running.\r\n\r\nWritten by: Mi
chael Newman <michael.b.newman@gmail.com>\r\nLast Updated: 2008-12-21"""\r\n'))
> <string>(1)<module>()
(Pdb) c
Traceback (most recent call last):
  File "c:\Python31\lib\pdb.py", line 1297, in main
    pdb._runscript(mainpyfile)
  File "c:\Python31\lib\pdb.py", line 1216, in _runscript
    self.run(statement)
  File "c:\Python31\lib\bdb.py", line 378, in run
    exec(cmd, globals, locals)
  File "<string>", line 1, in <module>
  File "e:\notes\programming\python3\version_check.py", line 4
    """Check what version of python is running.

Written by: Michael Newman <michael.b.newman@gmail.com>
Last Updated: 2008-12-21"""



^
SyntaxError: invalid syntax
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> <string>(1)<module>()
(Pdb) q
Post mortem debugger finished. The version_check.py will be restarted
SyntaxError: ('invalid syntax', ('e:\\notes\\programming\\python3\\version_check
.py', 4, 132, '"""Check what version of python is running.\r\n\r\nWritten by: Mi
chael Newman <michael.b.newman@gmail.com>\r\nLast Updated: 2008-12-21"""\r\n'))
> <string>(1)<module>()
(Pdb) q

# --- End Python 3.1 Example with "\r\n" source code --- #

As a hunch, my program is has Windows style line endings "\r\n", so I saved the file to another file name and converted it to Unix style "\n" line endings... and this version does work:

# --- Begin Python 3.1 Example using "\n" source code --- #

E:\notes\Programming\python3>copy version_check.py version_check_unix.py
        1 file(s) copied.

E:\notes\Programming\python3>which d2u
C:\Program Files\GnuWin32\bin\d2u.EXE

E:\notes\Programming\python3>d2u version_check_unix.py
version_check_unix.py: done.

E:\notes\Programming\python3>py31 -m pdb version_check_unix.py
> e:\notes\programming\python3\version_check_unix.py(4)<module>()
-> Last Updated: 2008-12-21"""
(Pdb) c
3.1.1 (r311:74483, Aug 17 2009, 17:02:12) [MSC v.1500 32 bit (Intel)]
The program finished and will be restarted
> e:\notes\programming\python3\version_check_unix.py(4)<module>()
-> Last Updated: 2008-12-21"""
(Pdb) q

# --- End Python 3.1 Example using "\n" source code --- #

Is "\r\n" not officially supported by "-m pdb"? Or is this a bug?
msg99600 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-02-19 23:13
It's a bug, and the bug is fixed (indirectly) in py3k trunk via improvements to the 'compile' function.  That fix can't be backported because it is a behavior change.

On the other hand, it appears as though the fix is to change the open statement from using 'rb' to using 'r'.  I'm not at all sure why it is opened in 'rb' mode, and it's possible it should be changed in trunk as well.
msg99687 - (view) Author: Jason R. Coombs (jason.coombs) * (Python committer) Date: 2010-02-21 21:33
I'm interested in finding a workaround for this issue in the next 24 hours. I can also help contribute a test case. I'll investigate further.
msg99690 - (view) Author: Jason R. Coombs (jason.coombs) * (Python committer) Date: 2010-02-21 22:13
Attached is a patch against the py3k branch that fixes the issue by changing the mode used to open the target script. It includes a unittest that elicits the issue and validates the fix.

The patch should also probably be applied to the 31maint branch.
msg102376 - (view) Author: Jason R. Coombs (jason.coombs) * (Python committer) Date: 2010-04-05 15:20
Is there a reason this didn't get reviewed for the 3.1.2 release? What steps need to be taken to see that it makes it into a 3.1.3 release?
msg112049 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2010-07-30 08:47
This is fixed in py3k trunk by other means; exec() now accepts bytestrings with "unusual" newlines.
msg112075 - (view) Author: Jason R. Coombs (jason.coombs) * (Python committer) Date: 2010-07-30 13:19
Is it still worthwhile to add the test to the suite to catch a regression?
msg112081 - (view) Author: Georg Brandl (georg.brandl) * (Python committer) Date: 2010-07-30 14:16
Yes, you're right. Committed in r83283.
History
Date User Action Args
2010-07-30 14:16:50georg.brandlsetmessages: + msg112081
2010-07-30 13:19:51jason.coombssetmessages: + msg112075
2010-07-30 08:47:38georg.brandlsetstatus: open -> closed

nosy: + georg.brandl
messages: + msg112049

resolution: out of date
2010-04-05 15:20:40jason.coombssetmessages: + msg102376
2010-02-21 22:13:20jason.coombssetfiles: + issue 7964 fix with test.patch
keywords: + patch
messages: + msg99690
2010-02-21 21:33:11jason.coombssetnosy: + jason.coombs
messages: + msg99687
2010-02-19 23:13:00r.david.murraysetpriority: normal
versions: - Python 3.2
nosy: + r.david.murray

messages: + msg99600

stage: test needed
2010-02-19 12:52:14mnewmancreate