annotate contrib/pep8/coding_std.py @ 2793:5ab6c46a4b75

change coding-std to check .less files and ignore /docs/_build directory
author RogerHaase <haaserd@gmail.com>
date Tue, 23 Sep 2014 09:22:54 -0700
parents 6af6e61dc25f
children 5a49eeb1f307
rev   line source
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
1 #!/usr/bin/env python
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
2 # Copyright: 2012-2014 by MoinMoin:RogerHaase
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
3 # License: GNU GPL v2 (or any later version), see LICENSE.txt for details.
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
4
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
5 """
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
6 Detect and correct violations of the moin2 coding standards:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
7 - no trailing blanks
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
8 - exactly one linefeed at file end, see PEP8
1757
22c5d8b59163 add .styl to list of file suffixes checked; use unicode strings for messages
Roger Haase <crosseyedpenguin@yahoo.com>
parents: 1398
diff changeset
9 - DOS line endings on .bat and .cmd files, unix line endings everywhere else
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
10
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
11 Detect and write informative message:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
12 - improper indentation of template files ending with .html suffix
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
13
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
14 Execute this script from the root directory of the moin2 repository or
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
15 from anywhere within the contrib path to process all files in the repo.
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
16
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
17 Or, pass a directory path on the command line.
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
18 """
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
19
1975
384555088cab lots of pep8 fixes
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1757
diff changeset
20 import sys
384555088cab lots of pep8 fixes
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1757
diff changeset
21 import os
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
22
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
23
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
24 # file types to be processed
2793
5ab6c46a4b75 change coding-std to check .less files and ignore /docs/_build directory
RogerHaase <haaserd@gmail.com>
parents: 2790
diff changeset
25 SELECTED_SUFFIXES = set("py bat cmd html css js styl less rst".split())
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
26
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
27 # stuff considered DOS/WIN
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
28 WIN_SUFFIXES = set("bat cmd".split())
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
29
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
30
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
31 class NoDupsLogger(object):
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
32 """
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
33 A simple report logger that suppresses duplicate headings and messages.
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
34 """
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
35 def __init__(self):
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
36 self.messages = set()
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
37 self.headings = set()
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
38
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
39 def log(self, heading, message):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
40 if heading and heading not in self.headings:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
41 print u"\n%s" % heading
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
42 self.headings.add(heading)
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
43
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
44 if message and message not in self.messages:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
45 print u" ", message
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
46 self.messages.add(message)
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
47
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
48
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
49 def directories_to_ignore(starting_dir):
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
50 """Return a list of directories that will not be processed."""
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
51 # list format: [(fully qualified directory name, sub-directory name), ... ]
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
52 ignore_dirs = []
2790
6af6e61dc25f change coding-std.py to inspect docs/*.rst; mass changes to rst files to remove trailing blanks, tabs, etc
RogerHaase <haaserd@gmail.com>
parents: 2762
diff changeset
53 level2_dirs = ".hg contrib dlc env moin.egg-info wiki".split()
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
54 for dir in level2_dirs:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
55 ignore_dirs.append((starting_dir, dir))
1396
86e998d808e7 pep8 whitespace fixes
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1366
diff changeset
56 ignore_dirs.append((starting_dir + os.sep + "MoinMoin", "translations"))
2793
5ab6c46a4b75 change coding-std to check .less files and ignore /docs/_build directory
RogerHaase <haaserd@gmail.com>
parents: 2790
diff changeset
57 ignore_dirs.append((starting_dir + os.sep + "docs", "_build"))
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
58 return ignore_dirs
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
59
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
60
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
61 def calc_indentation(line):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
62 """
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
63 Return tuple (length of indentation, line stripped of leading blanks).
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
64 """
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
65 stripped = line.lstrip(' ')
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
66 indentation = len(line) - len(stripped)
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
67 return indentation, stripped
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
68
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
69
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
70 def check_template_indentation(lines, filename, logger):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
71 """
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
72 Identify non-standard indentation, print messages to assist user in manual correction.
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
73
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
74 In simple cases, non-standard indent, non-standard dedent messages tells users which lines to indent.
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
75 """
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
76 indent_after = ('{% block ', '{% if ', '{% elif ', '{% else ', '{% for ', '{% macro ',
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
77 '{%- block ', '{%- if ', '{%- elif ', '{%- else ', '{%- for ', '{%- macro ', '{{ gen.form.open', )
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
78 indent_before = ('{% endblock ', '{% endif ', '{% endfor ', '{% endmacro', '</',
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
79 '{%- endblock ', '{%- endif ', '{%- endfor ', '{%- endmacro', '{{ gen.form.close', )
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
80 block_endings = {'{% block': ('{% endblock %}', '{%- endblock %}', '{%- endblock -%}', '{% endblock -%}', ),
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
81 '{% if': ('{% endif %}', '{%- endif %}', '{%- endif -%}', '{% endif -%}', ),
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
82 '{% elif': ('{% endif %}', '{%- endif %}', '{%- endif -%}', '{% endif -%}', ),
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
83 '{% else': ('{% endif %}', '{%- endif %}', '{%- endif -%}', '{% endif -%}', ),
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
84 '{% for': ('{% endfor %}', '{%- endfor %}', '{%- endfor -%}', '{% endfor -%}', ),
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
85 '{% macro': ('{% endmacro %}', '{%- endmacro %}', '{%- endmacro -%}', '{% endmacro -%}', ),
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
86 '{{ gen.form.open': ('{{ gen.form.close }}', ),
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
87 }
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
88 ends = ('{% end', '{%- end')
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
89
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
90 for idx, line in enumerate(lines):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
91 indentation, stripped = calc_indentation(line)
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
92
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
93 if stripped.startswith(indent_after):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
94 # we have found the beginning of a block
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
95 incre = 1
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
96 try:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
97 while lines[idx + incre].strip() == '':
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
98 incre += 1
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
99 next_indentation, next_line = calc_indentation(lines[idx + incre])
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
100 if next_indentation <= indentation:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
101 # next non-blank line does not have expected indentation
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
102 # truncate "{{ gen.form.open(form, ..." to "{{ gen.form.open"; "{%- if ..." to "{% if"
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
103 block_start = stripped.replace('-', '').split('(')[0].split(' ')
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
104 block_start = ' '.join(block_start[:2])
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
105 block_end = block_endings.get(block_start)
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
106 if not block_end:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
107 # should never get here, mismatched indent_after and block_endings
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
108 logger.log(filename, u"Unexpected block type '%s' discovered at line %d!" % (block_start, idx + 1))
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
109 continue
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
110 if any(x in stripped for x in block_end):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
111 # found line similar to: {% block ... %}...{% endblock %}
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
112 continue
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
113 if any(x in lines[idx + incre] for x in block_end):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
114 # found 2 consecutive lines similar to: {% block....\n{% endblock %}
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
115 continue
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
116 logger.log(filename, u"Non-standard indent after line %d -- not fixed!" % (idx + 1))
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
117 except IndexError:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
118 # should never get here, there is an unclosed block near end of template
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
119 logger.log(filename, u"End of file reached with open block element at line %d!" % (idx + 1))
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
120
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
121 elif stripped.startswith(indent_before):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
122 # we have found the end of a block
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
123 decre = -1
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
124 while idx + decre >= 0 and lines[idx + decre].strip() == '':
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
125 decre -= 1
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
126 if idx + decre < 0:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
127 # should never get here; file begins with something like {% endblock %} or </div>
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
128 logger.log(filename, u"Beginning of file reached searching for block content at line %d!" % (idx + 1))
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
129 continue
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
130 prior_indentation, prior_line = calc_indentation(lines[idx + decre])
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
131 if prior_indentation <= indentation:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
132 # prior non-blank line does not have expected indentation
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
133 if stripped.startswith('</'):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
134 tag_open = stripped.split('>')[0].replace('/', '') # convert </div> to <div, etc.
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
135 if prior_line.startswith(tag_open):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
136 # found lines similar to: <td>...\n</td>
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
137 continue
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
138 if stripped.startswith(ends):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
139 # convert {% endif %} to tuple ('{% if', '{%- if') or similar
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
140 block_open = (stripped.split(' %}')[0].replace('end', '').replace('-', ''),)
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
141 block_open += ((block_open[0].replace('{%', '{%-')), )
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
142 if prior_line.startswith(block_open):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
143 # found lines similar to: {% block...\n{% endblock %}
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
144 continue
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
145 logger.log(filename, u"Non-standard dedent before line %d -- not fixed!" % (idx + 1))
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
146
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
147
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
148 def check_files(filename, suffix):
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
149 """
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
150 Delete trailing blanks, single linefeed at file end, line ending to be \r\n for bat files and \n for all others.
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
151 """
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
152 suffix = suffix.lower()
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
153 if suffix in WIN_SUFFIXES:
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
154 line_end = "\r\n"
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
155 else:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
156 line_end = "\n"
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
157 logger = NoDupsLogger()
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
158
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
159 with open(filename, "rb") as f:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
160 lines = f.readlines()
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
161
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
162 if filename.endswith('.html'):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
163 check_template_indentation(lines, filename, logger)
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
164
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
165 # now look at file end and get rid of all whitespace-only lines there:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
166 while lines:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
167 if not lines[-1].strip():
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
168 del lines[-1]
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
169 logger.log(filename, u"Empty lines at eof removed.")
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
170 else:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
171 break
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
172
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
173 with open(filename, "wb") as f:
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
174 for idx, line in enumerate(lines):
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
175 line_length = len(line)
2566
96742f62f1a6 improve coding_std.py by replacing tabs with 4 spaces
RogerHaase <haaserd@gmail.com>
parents: 2303
diff changeset
176 line = line.replace('\t', ' ')
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
177 if len(line) != line_length:
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
178 logger.log(filename, u"Tab characters replaced with 4 spaces.")
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
179 pep8_line = line.rstrip() + line_end
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
180 f.write(pep8_line)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
181 # if line was changed, issue warning once for each type of change
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
182 if suffix in WIN_SUFFIXES and not line.endswith("\r\n"):
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
183 logger.log(filename, u"Line endings changed to DOS style.")
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
184 elif suffix not in WIN_SUFFIXES and line.endswith("\r\n"):
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
185 logger.log(filename, u"Line endings changed to Unix style.")
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
186 elif pep8_line != line:
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
187 if len(pep8_line) < len(line):
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
188 logger.log(filename, u"Trailing blanks removed.")
2303
8e8e19a86cfb simplified output status messages, corrected one message
RogerHaase
parents: 2302
diff changeset
189 else:
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
190 logger.log(filename, u"End of line character added at end of file.")
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
191
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
192
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
193 def file_picker(starting_dir):
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
194 """Select target files and pass each to file checker."""
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
195 ignore_dirs = directories_to_ignore(starting_dir)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
196
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
197 for root, dirs, files in os.walk(starting_dir):
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
198 # delete directories in ignore list
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
199 for mama_dir, baby_dir in ignore_dirs:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
200 if mama_dir == root and baby_dir in dirs:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
201 dirs.remove(baby_dir)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
202 # check files with selected suffixes
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
203 for file in files:
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
204 suffix = file.split(".")[-1]
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
205 if suffix in SELECTED_SUFFIXES:
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
206 filename = os.path.join(root, file)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
207 check_files(filename, suffix)
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
208
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
209
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
210 if __name__ == "__main__":
1398
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
211 if len(sys.argv) > 1:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
212 starting_dir = os.path.abspath(sys.argv[1])
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
213 else:
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
214 starting_dir = os.path.abspath(os.path.dirname(__file__))
f7604105f64a DeleteTrailingSpaces.py: work conforming to pep8, add starting_dir commandline param
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1396
diff changeset
215 starting_dir = starting_dir.split(os.sep + 'contrib')[0]
2762
f8e28b0f4c36 add template indentation checks to coding-std.py;
RogerHaase <haaserd@gmail.com>
parents: 2566
diff changeset
216 NoDupsLogger().log(u"Starting directory is %s" % starting_dir, None)
1366
6f785bab3b97 mass update part 3 - add script to delete trailing spaces to contrib
Roger Haase <crosseyedpenguin@yahoo.com>
parents:
diff changeset
217 file_picker(starting_dir)