Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(11)

Side by Side Diff: Tools/scripts/reindent.py

Issue 10639: reindent.py converts newlines to platform default
Patch Set: Created 8 years, 8 months ago
Left:
Right:
Use n/p to move between diff chunks; N/P to move between comments. Please Sign in to add in-line comments.
Jump to:
View unified diff | Download patch
« no previous file with comments | « Tools/scripts/patchcheck.py ('k') | no next file » | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
1 #! /usr/bin/env python3 1 #! /usr/bin/env python3
2 2
3 # Released to the public domain, by Tim Peters, 03 October 2000. 3 # Released to the public domain, by Tim Peters, 03 October 2000.
4 4
5 """reindent [-d][-r][-v] [ path ... ] 5 """reindent [-d][-r][-v] [ path ... ]
6 6
7 -d (--dryrun) Dry run. Analyze, but don't make any changes to, files. 7 -d (--dryrun) Dry run. Analyze, but don't make any changes to, files.
8 -r (--recurse) Recurse. Search for all .py files in subdirectories too. 8 -r (--recurse) Recurse. Search for all .py files in subdirectories too.
9 -n (--nobackup) No backup. Does not make a ".bak" file before reindenting. 9 -n (--nobackup) No backup. Does not make a ".bak" file before reindenting.
10 -v (--verbose) Verbose. Print informative msgs; else no output. 10 -v (--verbose) Verbose. Print informative msgs; else no output.
11 (--newline) Newline. Specify the newline character to use (CRLF, LF).
12 Default is the same as the original file.
11 -h (--help) Help. Print this usage information and exit. 13 -h (--help) Help. Print this usage information and exit.
12 14
13 Change Python (.py) files to use 4-space indents and no hard tab characters. 15 Change Python (.py) files to use 4-space indents and no hard tab characters.
14 Also trim excess spaces and tabs from ends of lines, and remove empty lines 16 Also trim excess spaces and tabs from ends of lines, and remove empty lines
15 at the end of files. Also ensure the last line ends with a newline. 17 at the end of files. Also ensure the last line ends with a newline.
16 18
17 If no paths are given on the command line, reindent operates as a filter, 19 If no paths are given on the command line, reindent operates as a filter,
18 reading a single source file from standard input and writing the transformed 20 reading a single source file from standard input and writing the transformed
19 source to standard output. In this case, the -d, -r and -v flags are 21 source to standard output. In this case, the -d, -r and -v flags are
20 ignored. 22 ignored.
21 23
22 You can pass one or more file and/or directory paths. When a directory 24 You can pass one or more file and/or directory paths. When a directory
23 path, all .py files within the directory will be examined, and, if the -r 25 path, all .py files within the directory will be examined, and, if the -r
24 option is given, likewise recursively for subdirectories. 26 option is given, likewise recursively for subdirectories.
25 27
26 If output is not to standard output, reindent overwrites files in place, 28 If output is not to standard output, reindent overwrites files in place,
27 renaming the originals with a .bak extension. If it finds nothing to 29 renaming the originals with a .bak extension. If it finds nothing to
28 change, the file is left alone. If reindent does change a file, the changed 30 change, the file is left alone. If reindent does change a file, the changed
29 file is a fixed-point for future runs (i.e., running reindent on the 31 file is a fixed-point for future runs (i.e., running reindent on the
30 resulting .py file won't change it again). 32 resulting .py file won't change it again).
31 33
32 The hard part of reindenting is figuring out what to do with comment 34 The hard part of reindenting is figuring out what to do with comment
33 lines. So long as the input files get a clean bill of health from 35 lines. So long as the input files get a clean bill of health from
34 tabnanny.py, reindent should do a good job. 36 tabnanny.py, reindent should do a good job.
35 37
36 The backup file is a copy of the one that is being reindented. The ".bak" 38 The backup file is a copy of the one that is being reindented. The ".bak"
37 file is generated with shutil.copy(), but some corner cases regarding 39 file is generated with shutil.copy(), but some corner cases regarding
38 user/group and permissions could leave the backup file more readable that 40 user/group and permissions could leave the backup file more readable than
39 you'd prefer. You can always use the --nobackup option to prevent this. 41 you'd prefer. You can always use the --nobackup option to prevent this.
40 """ 42 """
41 43
42 __version__ = "1" 44 __version__ = "1"
43 45
44 import tokenize 46 import tokenize
45 import os 47 import os
46 import shutil 48 import shutil
47 import sys 49 import sys
48 50
49 verbose = False 51 verbose = False
50 recurse = False 52 recurse = False
51 dryrun = False 53 dryrun = False
52 makebackup = True 54 makebackup = True
53 55
54 56
55 def usage(msg=None): 57 def usage(msg=None):
56 if msg is None: 58 if msg is None:
57 msg = __doc__ 59 msg = __doc__
58 print(msg, file=sys.stderr) 60 print(msg, file=sys.stderr)
59 61
60 62
61 def errprint(*args): 63 def errprint(*args):
62 sys.stderr.write(" ".join(str(arg) for arg in args)) 64 sys.stderr.write(" ".join(str(arg) for arg in args))
63 sys.stderr.write("\n") 65 sys.stderr.write("\n")
64 66
65 67
66 def main(): 68 def main():
67 import getopt 69 import getopt
68 global verbose, recurse, dryrun, makebackup 70 global verbose, recurse, dryrun, makebackup, spec_newline
71 spec_newline = None
69 try: 72 try:
70 opts, args = getopt.getopt(sys.argv[1:], "drnvh", 73 opts, args = getopt.getopt(sys.argv[1:], "drnvh",
71 ["dryrun", "recurse", "nobackup", "verbose", "help"]) 74 ["dryrun", "recurse", "nobackup", "verbose", "newline=", "help"])
72 except getopt.error as msg: 75 except getopt.error as msg:
73 usage(msg) 76 usage(msg)
74 return 77 return
75 for o, a in opts: 78 for o, a in opts:
76 if o in ('-d', '--dryrun'): 79 if o in ('-d', '--dryrun'):
77 dryrun = True 80 dryrun = True
78 elif o in ('-r', '--recurse'): 81 elif o in ('-r', '--recurse'):
79 recurse = True 82 recurse = True
80 elif o in ('-n', '--nobackup'): 83 elif o in ('-n', '--nobackup'):
81 makebackup = False 84 makebackup = False
82 elif o in ('-v', '--verbose'): 85 elif o in ('-v', '--verbose'):
83 verbose = True 86 verbose = True
87 elif o in ('--newline',):
88 if not a.upper() in ('CRLF', 'LF'):
89 usage()
90 return
91 spec_newline = dict(CRLF='\r\n', LF='\n')[a.upper()]
84 elif o in ('-h', '--help'): 92 elif o in ('-h', '--help'):
85 usage() 93 usage()
86 return 94 return
87 if not args: 95 if not args:
88 r = Reindenter(sys.stdin) 96 r = Reindenter(sys.stdin)
89 r.run() 97 r.run()
90 r.write(sys.stdout) 98 r.write(sys.stdout)
91 return 99 return
92 for arg in args: 100 for arg in args:
93 check(arg) 101 check(arg)
94 102
95 103
96 def check(file): 104 def check(file):
97 if os.path.isdir(file) and not os.path.islink(file): 105 if os.path.isdir(file) and not os.path.islink(file):
98 if verbose: 106 if verbose:
99 print("listing directory", file) 107 print("listing directory", file)
100 names = os.listdir(file) 108 names = os.listdir(file)
101 for name in names: 109 for name in names:
102 fullname = os.path.join(file, name) 110 fullname = os.path.join(file, name)
103 if ((recurse and os.path.isdir(fullname) and 111 if ((recurse and os.path.isdir(fullname) and
104 not os.path.islink(fullname) and 112 not os.path.islink(fullname) and
105 not os.path.split(fullname)[1].startswith(".")) 113 not os.path.split(fullname)[1].startswith("."))
106 or name.lower().endswith(".py")): 114 or name.lower().endswith(".py")):
107 check(fullname) 115 check(fullname)
108 return 116 return
109 117
110 if verbose: 118 if verbose:
111 print("checking", file, "...", end=' ') 119 print("checking", file, "...", end=' ')
112 with open(file, 'rb') as f: 120 with open(file, 'rb') as f:
113 encoding, _ = tokenize.detect_encoding(f.readline) 121 encoding, _ = tokenize.detect_encoding(f.readline)
114 try: 122 try:
115 with open(file, encoding=encoding) as f: 123 with open(file, encoding=encoding) as f:
116 r = Reindenter(f) 124 r = Reindenter(f)
117 except IOError as msg: 125 except IOError as msg:
118 errprint("%s: I/O Error: %s" % (file, str(msg))) 126 errprint("%s: I/O Error: %s" % (file, str(msg)))
127 return
128
129 newline = spec_newline if spec_newline else r.newlines
130 if isinstance(newline, tuple):
131 errprint("%s: mixed newlines detected; cannot continue without --newline " % file)
119 return 132 return
120 133
121 if r.run(): 134 if r.run():
122 if verbose: 135 if verbose:
123 print("changed.") 136 print("changed.")
124 if dryrun: 137 if dryrun:
125 print("But this is a dry run, so leaving it alone.") 138 print("But this is a dry run, so leaving it alone.")
126 if not dryrun: 139 if not dryrun:
127 bak = file + ".bak" 140 bak = file + ".bak"
128 if makebackup: 141 if makebackup:
129 shutil.copyfile(file, bak) 142 shutil.copyfile(file, bak)
130 if verbose: 143 if verbose:
131 print("backed up", file, "to", bak) 144 print("backed up", file, "to", bak)
132 with open(file, "w", encoding=encoding) as f: 145 with open(file, "w", encoding=encoding, newline=newline) as f:
133 r.write(f) 146 r.write(f)
134 if verbose: 147 if verbose:
135 print("wrote new", file) 148 print("wrote new", file)
136 return True 149 return True
137 else: 150 else:
138 if verbose: 151 if verbose:
139 print("unchanged.") 152 print("unchanged.")
140 return False 153 return False
141 154
142 155
(...skipping 26 matching lines...) Expand all
169 self.lines = [_rstrip(line).expandtabs() + "\n" 182 self.lines = [_rstrip(line).expandtabs() + "\n"
170 for line in self.raw] 183 for line in self.raw]
171 self.lines.insert(0, None) 184 self.lines.insert(0, None)
172 self.index = 1 # index into self.lines of next line 185 self.index = 1 # index into self.lines of next line
173 186
174 # List of (lineno, indentlevel) pairs, one for each stmt and 187 # List of (lineno, indentlevel) pairs, one for each stmt and
175 # comment line. indentlevel is -1 for comment lines, as a 188 # comment line. indentlevel is -1 for comment lines, as a
176 # signal that tokenize doesn't know what to do about them; 189 # signal that tokenize doesn't know what to do about them;
177 # indeed, they're our headache! 190 # indeed, they're our headache!
178 self.stats = [] 191 self.stats = []
192
193 # Save the newlines found in the file so they can be used to
194 # create output without mutating the newlines.
195 self.newlines = f.newlines
179 196
180 def run(self): 197 def run(self):
181 tokens = tokenize.generate_tokens(self.getline) 198 tokens = tokenize.generate_tokens(self.getline)
182 for _token in tokens: 199 for _token in tokens:
183 self.tokeneater(*_token) 200 self.tokeneater(*_token)
184 # Remove trailing empty lines. 201 # Remove trailing empty lines.
185 lines = self.lines 202 lines = self.lines
186 while lines and lines[-1] == "\n": 203 while lines and lines[-1] == "\n":
187 lines.pop() 204 lines.pop()
188 # Sentinel. 205 # Sentinel.
(...skipping 114 matching lines...) Expand 10 before | Expand all | Expand 10 after
303 # Count number of leading blanks. 320 # Count number of leading blanks.
304 def getlspace(line): 321 def getlspace(line):
305 i, n = 0, len(line) 322 i, n = 0, len(line)
306 while i < n and line[i] == " ": 323 while i < n and line[i] == " ":
307 i += 1 324 i += 1
308 return i 325 return i
309 326
310 327
311 if __name__ == '__main__': 328 if __name__ == '__main__':
312 main() 329 main()
OLDNEW
« no previous file with comments | « Tools/scripts/patchcheck.py ('k') | no next file » | no next file with comments »

RSS Feeds Recent Issues | This issue
This is Rietveld 894c83f36cb7+