annotate contrib/pep8/DeleteTrailingSpaces.py @ 1757:22c5d8b59163

add .styl to list of file suffixes checked; use unicode strings for messages
author Roger Haase <crosseyedpenguin@yahoo.com>
date Tue, 28 Aug 2012 14:06:56 -0700
parents f7604105f64a
children 384555088cab
rev   line source
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
1 #!/usr/bin/env python
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
2 # Copyright: 2012 by MoinMoin:RogerHaase
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
3 # License: GNU GPL v2 (or any later version), see LICENSE.txt for details.
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
4
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
5 """
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
6 Detect and correct violations of the moin2 coding standards:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
7 - no trailing blanks
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
8 - exactly one linefeed at file end, see PEP8
1757
22c5d8b59163 add .styl to list of file suffixes checked; use unicode strings for messages
Roger Haase <crosseyedpenguin@yahoo.com>
parents: 1398
diff changeset
9 - DOS line endings on .bat and .cmd files, unix line endings everywhere else
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
10
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
11 Execute this script from the root directory of the moin2 repository or
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
12 from anywhere within the contrib path.
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
13 """
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
14
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
15 import sys, os
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
16 import warnings
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
17 warnings.simplefilter("once")
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
18
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
19 # file types to be processed
1757
22c5d8b59163 add .styl to list of file suffixes checked; use unicode strings for messages
Roger Haase <crosseyedpenguin@yahoo.com>
parents: 1398
diff changeset
20 SELECTED_SUFFIXES = set("py bat cmd html css js styl".split())
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
21
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
22 # stuff considered DOS/WIN
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
23 WIN_SUFFIXES = set("bat cmd".split())
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
24
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
25
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
26 def directories_to_ignore(starting_dir):
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
27 """Return a list of directories that will not be processed."""
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
28 # list format: [(fully qualified directory name, sub-directory name), ... ]
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
29 ignore_dirs = []
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
30 level2_dirs = ".hg contrib dlc docs env moin.egg-info wiki".split()
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
31 for dir in level2_dirs:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
32 ignore_dirs.append((starting_dir, dir))
1396
86e998d808e7 pep8 whitespace fixes
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1366
diff changeset
33 ignore_dirs.append((starting_dir + os.sep + "MoinMoin", "translations"))
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
34 return ignore_dirs
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
35
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
36
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
37 def check_files(filename, suffix):
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
38 """Delete trailing blanks,
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
39 force a single linefeed at file end,
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
40 force line ending to be \r\n for bat files and \n for all others."""
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
41 suffix = suffix.lower()
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
42 if suffix in WIN_SUFFIXES:
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
43 line_end = "\r\n"
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
44 else:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
45 line_end = "\n"
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
46
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
47 with open(filename, "rb") as f:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
48 lines = f.readlines()
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
49
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
50 # now look at file end and get rid of all whitespace-only lines there:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
51 while lines:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
52 if not lines[-1].strip():
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
53 del lines[-1]
1757
22c5d8b59163 add .styl to list of file suffixes checked; use unicode strings for messages
Roger Haase <crosseyedpenguin@yahoo.com>
parents: 1398
diff changeset
54 warnings.warn(u"%s was changed to remove empty lines at eof" % filename)
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
55 else:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
56 break
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
57
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
58 with open(filename, "wb") as f:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
59 for line in lines:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
60 pep8_line = line.rstrip() + line_end
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
61 f.write(pep8_line)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
62 # if line was changed, issue warning once for each type of change
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
63 if suffix in WIN_SUFFIXES and not line.endswith("\r\n"):
1757
22c5d8b59163 add .styl to list of file suffixes checked; use unicode strings for messages
Roger Haase <crosseyedpenguin@yahoo.com>
parents: 1398
diff changeset
64 warnings.warn(u"%s was changed to DOS line endings" % filename)
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
65 elif suffix not in WIN_SUFFIXES and line.endswith("\r\n"):
1757
22c5d8b59163 add .styl to list of file suffixes checked; use unicode strings for messages
Roger Haase <crosseyedpenguin@yahoo.com>
parents: 1398
diff changeset
66 warnings.warn(u"%s was changed to Unix line endings" % filename)
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
67 elif pep8_line != line:
1757
22c5d8b59163 add .styl to list of file suffixes checked; use unicode strings for messages
Roger Haase <crosseyedpenguin@yahoo.com>
parents: 1398
diff changeset
68 warnings.warn(u"%s was changed to remove trailing blanks" % filename)
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
69
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
70
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
71 def file_picker(starting_dir):
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
72 """Select target files and pass each to file checker."""
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
73 ignore_dirs = directories_to_ignore(starting_dir)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
74
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
75 for root, dirs, files in os.walk(starting_dir):
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
76 # delete directories in ignore list
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
77 for mama_dir, baby_dir in ignore_dirs:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
78 if mama_dir == root and baby_dir in dirs:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
79 dirs.remove(baby_dir)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
80 # check files with selected suffixes
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
81 for file in files:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
82 suffix = file.split(".")[-1]
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
83 if suffix in SELECTED_SUFFIXES:
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
84 filename = os.path.join(root, file)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
85 check_files(filename, suffix)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
86
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
87
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
88 if __name__ == "__main__":
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
89 if len(sys.argv) > 1:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
90 starting_dir = os.path.abspath(sys.argv[1])
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
91 else:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
92 starting_dir = os.path.abspath(os.path.dirname(__file__))
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
93 starting_dir = starting_dir.split(os.sep + 'contrib')[0]
1757
22c5d8b59163 add .styl to list of file suffixes checked; use unicode strings for messages
Roger Haase <crosseyedpenguin@yahoo.com>
parents: 1398
diff changeset
94 warnings.warn(u"%s is starting directory" % starting_dir)
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
95 file_picker(starting_dir)