Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(17)

Side by Side Diff: Lib/test/test_tokenize.py

Issue 10639: reindent.py converts newlines to platform default
Patch Set: Created 8 years, 8 months ago
Left:
Right:
Use n/p to move between diff chunks; N/P to move between comments. Please Sign in to add in-line comments.
Jump to:
View unified diff | Download patch
« no previous file with comments | « Lib/test/test_tk.py ('k') | Lib/test/test_ttk_guionly.py » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 doctests = """ 1 doctests = """
2 Tests for the tokenize module. 2 Tests for the tokenize module.
3 3
4 The tests can be really simple. Given a small fragment of source 4 The tests can be really simple. Given a small fragment of source
5 code, print out a table with tokens. The ENDMARK is omitted for 5 code, print out a table with tokens. The ENDMARK is omitted for
6 brevity. 6 brevity.
7 7
8 >>> dump_tokens("1 + 1") 8 >>> dump_tokens("1 + 1")
9 ENCODING 'utf-8' (0, 0) (0, 0) 9 ENCODING 'utf-8' (0, 0) (0, 0)
10 NUMBER '1' (1, 0) (1, 1) 10 NUMBER '1' (1, 0) (1, 1)
(...skipping 631 matching lines...) Expand 10 before | Expand all | Expand 10 after
642 class TestTokenizerAdheresToPep0263(TestCase): 642 class TestTokenizerAdheresToPep0263(TestCase):
643 """ 643 """
644 Test that tokenizer adheres to the coding behaviour stipulated in PEP 0263. 644 Test that tokenizer adheres to the coding behaviour stipulated in PEP 0263.
645 """ 645 """
646 646
647 def _testFile(self, filename): 647 def _testFile(self, filename):
648 path = os.path.join(os.path.dirname(__file__), filename) 648 path = os.path.join(os.path.dirname(__file__), filename)
649 return roundtrip(open(path, 'rb')) 649 return roundtrip(open(path, 'rb'))
650 650
651 def test_utf8_coding_cookie_and_no_utf8_bom(self): 651 def test_utf8_coding_cookie_and_no_utf8_bom(self):
652 f = 'tokenize_tests-utf8-coding-cookie-and-no-utf8-bom-sig.txt' 652 f = 'tokenize_tests-utf8-coding-cookie-and-utf8-bom-sig.txt'
653 self.assertTrue(self._testFile(f)) 653 self.assertTrue(self._testFile(f))
654 654
655 def test_latin1_coding_cookie_and_utf8_bom(self): 655 def test_latin1_coding_cookie_and_utf8_bom(self):
656 """ 656 """
657 As per PEP 0263, if a file starts with a utf-8 BOM signature, the only 657 As per PEP 0263, if a file starts with a utf-8 BOM signature, the only
658 allowed encoding for the comment is 'utf-8'. The text file used in 658 allowed encoding for the comment is 'utf-8'. The text file used in
659 this test starts with a BOM signature, but specifies latin1 as the 659 this test starts with a BOM signature, but specifies latin1 as the
660 coding, so verify that a SyntaxError is raised, which matches the 660 coding, so verify that a SyntaxError is raised, which matches the
661 behaviour of the interpreter when it encounters a similar condition. 661 behaviour of the interpreter when it encounters a similar condition.
662 """ 662 """
(...skipping 262 matching lines...) Expand 10 before | Expand all | Expand 10 after
925 def test_main(): 925 def test_main():
926 from test import test_tokenize 926 from test import test_tokenize
927 support.run_doctest(test_tokenize, True) 927 support.run_doctest(test_tokenize, True)
928 support.run_unittest(TestTokenizerAdheresToPep0263) 928 support.run_unittest(TestTokenizerAdheresToPep0263)
929 support.run_unittest(Test_Tokenize) 929 support.run_unittest(Test_Tokenize)
930 support.run_unittest(TestDetectEncoding) 930 support.run_unittest(TestDetectEncoding)
931 support.run_unittest(TestTokenize) 931 support.run_unittest(TestTokenize)
932 932
933 if __name__ == "__main__": 933 if __name__ == "__main__":
934 test_main() 934 test_main()
OLDNEW
« no previous file with comments | « Lib/test/test_tk.py ('k') | Lib/test/test_ttk_guionly.py » ('j') | no next file with comments »

RSS Feeds Recent Issues | This issue
This is Rietveld 894c83f36cb7+