annotate contrib/pep8/coding_std.py @ 2790:6af6e61dc25f

change coding-std.py to inspect docs/*.rst; mass changes to rst files to remove trailing blanks, tabs, etc
author RogerHaase <haaserd@gmail.com>
date Sun, 21 Sep 2014 08:53:23 -0700
parents f8e28b0f4c36
children 5ab6c46a4b75
rev   line source
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
1 #!/usr/bin/env python
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
2 # Copyright: 2012-2014 by MoinMoin:RogerHaase
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
3 # License: GNU GPL v2 (or any later version), see LICENSE.txt for details.
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
4
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
5 """
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
6 Detect and correct violations of the moin2 coding standards:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
7 - no trailing blanks
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
8 - exactly one linefeed at file end, see PEP8
1757
22c5d8b59163 add .styl to list of file suffixes checked; use unicode strings for messages
Roger Haase <crosseyedpenguin@yahoo.com>
parents: 1398
diff changeset
9 - DOS line endings on .bat and .cmd files, unix line endings everywhere else
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
10
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
11 Detect and write informative message:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
12 - improper indentation of template files ending with .html suffix
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
13
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
14 Execute this script from the root directory of the moin2 repository or
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
15 from anywhere within the contrib path to process all files in the repo.
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
16
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
17 Or, pass a directory path on the command line.
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
18 """
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
19
1975
384555088cab lots of pep8 fixes
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1757
diff changeset
20 import sys
384555088cab lots of pep8 fixes
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1757
diff changeset
21 import os
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
22
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
23
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
24 # file types to be processed
2790
6af6e61dc25f change coding-std.py to inspect docs/*.rst; mass changes to rst files to remove trailing blanks, tabs, etc
RogerHaase <haaserd@gmail.com>
parents: 2762
diff changeset
25 SELECTED_SUFFIXES = set("py bat cmd html css js styl rst".split())
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
26
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
27 # stuff considered DOS/WIN
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
28 WIN_SUFFIXES = set("bat cmd".split())
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
29
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
30
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
31 class NoDupsLogger(object):
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
32 """
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
33 A simple report logger that suppresses duplicate headings and messages.
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
34 """
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
35 def __init__(self):
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
36 self.messages = set()
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
37 self.headings = set()
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
38
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
39 def log(self, heading, message):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
40 if heading and heading not in self.headings:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
41 print u"\n%s" % heading
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
42 self.headings.add(heading)
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
43
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
44 if message and message not in self.messages:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
45 print u" ", message
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
46 self.messages.add(message)
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
47
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
48
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
49 def directories_to_ignore(starting_dir):
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
50 """Return a list of directories that will not be processed."""
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
51 # list format: [(fully qualified directory name, sub-directory name), ... ]
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
52 ignore_dirs = []
2790
6af6e61dc25f change coding-std.py to inspect docs/*.rst; mass changes to rst files to remove trailing blanks, tabs, etc
RogerHaase <haaserd@gmail.com>
parents: 2762
diff changeset
53 level2_dirs = ".hg contrib dlc env moin.egg-info wiki".split()
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
54 for dir in level2_dirs:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
55 ignore_dirs.append((starting_dir, dir))
1396
86e998d808e7 pep8 whitespace fixes
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1366
diff changeset
56 ignore_dirs.append((starting_dir + os.sep + "MoinMoin", "translations"))
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
57 return ignore_dirs
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
58
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
59
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
60 def calc_indentation(line):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
61 """
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
62 Return tuple (length of indentation, line stripped of leading blanks).
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
63 """
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
64 stripped = line.lstrip(' ')
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
65 indentation = len(line) - len(stripped)
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
66 return indentation, stripped
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
67
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
68
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
69 def check_template_indentation(lines, filename, logger):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
70 """
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
71 Identify non-standard indentation, print messages to assist user in manual correction.
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
72
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
73 In simple cases, non-standard indent, non-standard dedent messages tells users which lines to indent.
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
74 """
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
75 indent_after = ('{% block ', '{% if ', '{% elif ', '{% else ', '{% for ', '{% macro ',
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
76 '{%- block ', '{%- if ', '{%- elif ', '{%- else ', '{%- for ', '{%- macro ', '{{ gen.form.open', )
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
77 indent_before = ('{% endblock ', '{% endif ', '{% endfor ', '{% endmacro', '</',
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
78 '{%- endblock ', '{%- endif ', '{%- endfor ', '{%- endmacro', '{{ gen.form.close', )
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
79 block_endings = {'{% block': ('{% endblock %}', '{%- endblock %}', '{%- endblock -%}', '{% endblock -%}', ),
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
80 '{% if': ('{% endif %}', '{%- endif %}', '{%- endif -%}', '{% endif -%}', ),
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
81 '{% elif': ('{% endif %}', '{%- endif %}', '{%- endif -%}', '{% endif -%}', ),
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
82 '{% else': ('{% endif %}', '{%- endif %}', '{%- endif -%}', '{% endif -%}', ),
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
83 '{% for': ('{% endfor %}', '{%- endfor %}', '{%- endfor -%}', '{% endfor -%}', ),
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
84 '{% macro': ('{% endmacro %}', '{%- endmacro %}', '{%- endmacro -%}', '{% endmacro -%}', ),
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
85 '{{ gen.form.open': ('{{ gen.form.close }}', ),
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
86 }
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
87 ends = ('{% end', '{%- end')
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
88
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
89 for idx, line in enumerate(lines):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
90 indentation, stripped = calc_indentation(line)
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
91
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
92 if stripped.startswith(indent_after):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
93 # we have found the beginning of a block
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
94 incre = 1
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
95 try:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
96 while lines[idx + incre].strip() == '':
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
97 incre += 1
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
98 next_indentation, next_line = calc_indentation(lines[idx + incre])
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
99 if next_indentation <= indentation:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
100 # next non-blank line does not have expected indentation
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
101 # truncate "{{ gen.form.open(form, ..." to "{{ gen.form.open"; "{%- if ..." to "{% if"
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
102 block_start = stripped.replace('-', '').split('(')[0].split(' ')
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
103 block_start = ' '.join(block_start[:2])
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
104 block_end = block_endings.get(block_start)
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
105 if not block_end:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
106 # should never get here, mismatched indent_after and block_endings
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
107 logger.log(filename, u"Unexpected block type '%s' discovered at line %d!" % (block_start, idx + 1))
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
108 continue
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
109 if any(x in stripped for x in block_end):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
110 # found line similar to: {% block ... %}...{% endblock %}
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
111 continue
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
112 if any(x in lines[idx + incre] for x in block_end):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
113 # found 2 consecutive lines similar to: {% block....\n{% endblock %}
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
114 continue
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
115 logger.log(filename, u"Non-standard indent after line %d -- not fixed!" % (idx + 1))
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
116 except IndexError:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
117 # should never get here, there is an unclosed block near end of template
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
118 logger.log(filename, u"End of file reached with open block element at line %d!" % (idx + 1))
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
119
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
120 elif stripped.startswith(indent_before):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
121 # we have found the end of a block
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
122 decre = -1
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
123 while idx + decre >= 0 and lines[idx + decre].strip() == '':
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
124 decre -= 1
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
125 if idx + decre < 0:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
126 # should never get here; file begins with something like {% endblock %} or </div>
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
127 logger.log(filename, u"Beginning of file reached searching for block content at line %d!" % (idx + 1))
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
128 continue
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
129 prior_indentation, prior_line = calc_indentation(lines[idx + decre])
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
130 if prior_indentation <= indentation:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
131 # prior non-blank line does not have expected indentation
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
132 if stripped.startswith('</'):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
133 tag_open = stripped.split('>')[0].replace('/', '') # convert </div> to <div, etc.
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
134 if prior_line.startswith(tag_open):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
135 # found lines similar to: <td>...\n</td>
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
136 continue
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
137 if stripped.startswith(ends):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
138 # convert {% endif %} to tuple ('{% if', '{%- if') or similar
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
139 block_open = (stripped.split(' %}')[0].replace('end', '').replace('-', ''),)
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
140 block_open += ((block_open[0].replace('{%', '{%-')), )
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
141 if prior_line.startswith(block_open):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
142 # found lines similar to: {% block...\n{% endblock %}
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
143 continue
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
144 logger.log(filename, u"Non-standard dedent before line %d -- not fixed!" % (idx + 1))
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
145
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
146
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
147 def check_files(filename, suffix):
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
148 """
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
149 Delete trailing blanks, single linefeed at file end, line ending to be \r\n for bat files and \n for all others.
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
150 """
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
151 suffix = suffix.lower()
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
152 if suffix in WIN_SUFFIXES:
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
153 line_end = "\r\n"
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
154 else:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
155 line_end = "\n"
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
156 logger = NoDupsLogger()
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
157
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
158 with open(filename, "rb") as f:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
159 lines = f.readlines()
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
160
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
161 if filename.endswith('.html'):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
162 check_template_indentation(lines, filename, logger)
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
163
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
164 # now look at file end and get rid of all whitespace-only lines there:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
165 while lines:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
166 if not lines[-1].strip():
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
167 del lines[-1]
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
168 logger.log(filename, u"Empty lines at eof removed.")
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
169 else:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
170 break
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
171
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
172 with open(filename, "wb") as f:
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
173 for idx, line in enumerate(lines):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
174 line_length = len(line)
2566
96742f62f1a6 improve coding_std.py by replacing tabs with 4 spaces
RogerHaase <haaserd@gmail.com>
parents: 2303
diff changeset
175 line = line.replace('\t', ' ')
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
176 if len(line) != line_length:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
177 logger.log(filename, u"Tab characters replaced with 4 spaces.")
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
178 pep8_line = line.rstrip() + line_end
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
179 f.write(pep8_line)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
180 # if line was changed, issue warning once for each type of change
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
181 if suffix in WIN_SUFFIXES and not line.endswith("\r\n"):
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
182 logger.log(filename, u"Line endings changed to DOS style.")
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
183 elif suffix not in WIN_SUFFIXES and line.endswith("\r\n"):
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
184 logger.log(filename, u"Line endings changed to Unix style.")
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
185 elif pep8_line != line:
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
186 if len(pep8_line) < len(line):
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
187 logger.log(filename, u"Trailing blanks removed.")
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
188 else:
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
189 logger.log(filename, u"End of line character added at end of file.")
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
190
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
191
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
192 def file_picker(starting_dir):
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
193 """Select target files and pass each to file checker."""
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
194 ignore_dirs = directories_to_ignore(starting_dir)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
195
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
196 for root, dirs, files in os.walk(starting_dir):
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
197 # delete directories in ignore list
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
198 for mama_dir, baby_dir in ignore_dirs:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
199 if mama_dir == root and baby_dir in dirs:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
200 dirs.remove(baby_dir)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
201 # check files with selected suffixes
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
202 for file in files:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
203 suffix = file.split(".")[-1]
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
204 if suffix in SELECTED_SUFFIXES:
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
205 filename = os.path.join(root, file)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
206 check_files(filename, suffix)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
207
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
208
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
209 if __name__ == "__main__":
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
210 if len(sys.argv) > 1:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
211 starting_dir = os.path.abspath(sys.argv[1])
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
212 else:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
213 starting_dir = os.path.abspath(os.path.dirname(__file__))
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
214 starting_dir = starting_dir.split(os.sep + 'contrib')[0]
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
215 NoDupsLogger().log(u"Starting directory is %s" % starting_dir, None)
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
216 file_picker(starting_dir)