Author pdonis
Recipients pdonis
Date 2008-01-12.05:57:31
SpamBayes Score 0.0527754
Marked as misclassified No
Message-id <1200117455.4.0.563963262174.issue1812@psf.upfronthosting.co.za>
In-reply-to
Content
When running doctest.testfile on a Linux machine, testing a txt file 
saved on a Windows machine, doctest raised a SyntaxError exception for 
each Windows newline in the txt file. On examining the code in the 
_load_testfile function, it looks to me like there are actually two 
issues:

(1) When there is no package keyword argument given, or the package 
argument points to a package without a __loader__.get_data method, the 
txt file data is retrieved by calling open(filename).read(); this is 
the code path that my Windows-saved file triggered. However, since the 
default file mode for open is 'r', not 'rU', there is no universal 
newline conversion done on the file data. This was the issue I 
initially observed.

(2) When there is a package.__loader__.get_data method found, that 
method reads the data (using file mode 'rb'), and then newline 
conversion is performed by this line:

return file_contents.replace(os.linesep, '\n')

This does not seem to match what universal newline conversion does; 
that is supposed to convert both '\r\n' and '\r' to '\n', but running 
on Linux, os.linesep is '\n', so the replace operation in the current 
code is a no-op, even if the txt file was saved with Windows or Mac 
newlines. It seems to me that the correct operation would be:

for linesep in ('\r\n', '\r'):
   file_contents = file_contents.replace(linesep, '\n')

I have attached a diff against the current svn trunk showing my 
suggested fix for both of the above issues.

Peter Donis
History
Date User Action Args
2008-01-12 05:57:35pdonissetspambayes_score: 0.0527754 -> 0.0527754
recipients: + pdonis
2008-01-12 05:57:35pdonissetspambayes_score: 0.0527754 -> 0.0527754
messageid: <1200117455.4.0.563963262174.issue1812@psf.upfronthosting.co.za>
2008-01-12 05:57:33pdonislinkissue1812 messages
2008-01-12 05:57:32pdoniscreate