annotate contrib/pep8/coding_std.py @ 2303:8e8e19a86cfb

simplified output status messages, corrected one message
author RogerHaase
date Sun, 10 Nov 2013 13:49:14 -0700
parents 9269c5cf2931
children 96742f62f1a6
rev   line source
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
1 #!/usr/bin/env python
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
2 # Copyright: 2012 by MoinMoin:RogerHaase
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
3 # License: GNU GPL v2 (or any later version), see LICENSE.txt for details.
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
4
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
5 """
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
6 Detect and correct violations of the moin2 coding standards:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
7 - no trailing blanks
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
8 - exactly one linefeed at file end, see PEP8
1757
22c5d8b59163 add .styl to list of file suffixes checked; use unicode strings for messages
Roger Haase <crosseyedpenguin@yahoo.com>
parents: 1398
diff changeset
9 - DOS line endings on .bat and .cmd files, unix line endings everywhere else
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
10
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
11 Execute this script from the root directory of the moin2 repository or
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
12 from anywhere within the contrib path.
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
13 """
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
14
1975
384555088cab lots of pep8 fixes
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1757
diff changeset
15 import sys
384555088cab lots of pep8 fixes
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1757
diff changeset
16 import os
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
17
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
18
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
19 # file types to be processed
1757
22c5d8b59163 add .styl to list of file suffixes checked; use unicode strings for messages
Roger Haase <crosseyedpenguin@yahoo.com>
parents: 1398
diff changeset
20 SELECTED_SUFFIXES = set("py bat cmd html css js styl".split())
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
21
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
22 # stuff considered DOS/WIN
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
23 WIN_SUFFIXES = set("bat cmd".split())
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
24
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
25
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
26 class NoDupsLogger(object):
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
27 """Suppress duplicate messages."""
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
28 def __init__(self):
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
29 self.seen = set()
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
30
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
31 def log(self, msg):
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
32 if msg not in self.seen:
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
33 print msg
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
34 self.seen.add(msg)
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
35
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
36
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
37 def directories_to_ignore(starting_dir):
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
38 """Return a list of directories that will not be processed."""
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
39 # list format: [(fully qualified directory name, sub-directory name), ... ]
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
40 ignore_dirs = []
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
41 level2_dirs = ".hg contrib dlc docs env moin.egg-info wiki".split()
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
42 for dir in level2_dirs:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
43 ignore_dirs.append((starting_dir, dir))
1396
86e998d808e7 pep8 whitespace fixes
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1366
diff changeset
44 ignore_dirs.append((starting_dir + os.sep + "MoinMoin", "translations"))
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
45 return ignore_dirs
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
46
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
47
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
48 def check_files(filename, suffix):
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
49 """Delete trailing blanks,
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
50 force a single linefeed at file end,
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
51 force line ending to be \r\n for bat files and \n for all others."""
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
52 suffix = suffix.lower()
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
53 if suffix in WIN_SUFFIXES:
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
54 line_end = "\r\n"
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
55 else:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
56 line_end = "\n"
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
57 logger = NoDupsLogger()
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
58
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
59 with open(filename, "rb") as f:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
60 lines = f.readlines()
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
61
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
62 # now look at file end and get rid of all whitespace-only lines there:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
63 while lines:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
64 if not lines[-1].strip():
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
65 del lines[-1]
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
66 logger.log(u"%s was changed to remove empty lines at eof" % filename)
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
67 else:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
68 break
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
69
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
70 with open(filename, "wb") as f:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
71 for line in lines:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
72 pep8_line = line.rstrip() + line_end
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
73 f.write(pep8_line)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
74 # if line was changed, issue warning once for each type of change
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
75 if suffix in WIN_SUFFIXES and not line.endswith("\r\n"):
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
76 logger.log(u"%s was changed to DOS line endings" % filename)
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
77 elif suffix not in WIN_SUFFIXES and line.endswith("\r\n"):
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
78 logger.log(u"%s was changed to Unix line endings" % filename)
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
79 elif pep8_line != line:
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
80 if len(pep8_line) < len(line):
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
81 logger.log(u"%s was changed to remove trailing blanks" % filename)
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
82 else:
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
83 logger.log(u"%s was changed to add end of line character at end of file" % filename)
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
84
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
85
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
86 def file_picker(starting_dir):
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
87 """Select target files and pass each to file checker."""
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
88 ignore_dirs = directories_to_ignore(starting_dir)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
89
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
90 for root, dirs, files in os.walk(starting_dir):
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
91 # delete directories in ignore list
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
92 for mama_dir, baby_dir in ignore_dirs:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
93 if mama_dir == root and baby_dir in dirs:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
94 dirs.remove(baby_dir)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
95 # check files with selected suffixes
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
96 for file in files:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
97 suffix = file.split(".")[-1]
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
98 if suffix in SELECTED_SUFFIXES:
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
99 filename = os.path.join(root, file)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
100 check_files(filename, suffix)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
101
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
102
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
103 if __name__ == "__main__":
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
104 if len(sys.argv) > 1:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
105 starting_dir = os.path.abspath(sys.argv[1])
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
106 else:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
107 starting_dir = os.path.abspath(os.path.dirname(__file__))
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
108 starting_dir = starting_dir.split(os.sep + 'contrib')[0]
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
109 NoDupsLogger().log(u"Starting directory is %s" % starting_dir)
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
110 file_picker(starting_dir)