annotate MoinMoin/parser/_ParserBase.py @ 5829:a744f573fe30

advancedsearch: fix problem with non-ascii chars breaking up words for OR and NOT terms
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Sat, 04 Feb 2012 16:52:24 +0100
parents ff39884957af
children 500f68d3e2fd
rev   line source
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
1 # -*- coding: iso-8859-1 -*-
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
2 """
2274
01f6172dabd0 remove tabs from source files, remove excludes from test_tab
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1920
diff changeset
3 MoinMoin - Base Source Parser
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
4
3911
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
5 @copyright: 2002 by Taesu Pyo <bigflood@hitel.net>,
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
6 2005 by Oliver Graf <ograf@bitart.de>,
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
7 2005-2008 MoinMoin:ThomasWaldmann
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
8
249
de46a69b9865 changed wrong license to intended one in Taesu Pyo's parser code contribution
Thomas Waldmann <tw@waldmann-edv.de>
parents: 0
diff changeset
9 @license: GNU GPL, see COPYING for details.
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
10
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
11
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
12 basic css:
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
13
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
14 pre.codearea { font-style: sans-serif; color: #000000; }
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
15
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
16 pre.codearea span.ID { color: #000000; }
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
17 pre.codearea span.Char { color: #004080; }
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
18 pre.codearea span.Comment { color: #808080; }
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
19 pre.codearea span.Number { color: #008080; font-weight: bold; }
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
20 pre.codearea span.String { color: #004080; }
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
21 pre.codearea span.SPChar { color: #0000C0; }
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
22 pre.codearea span.ResWord { color: #4040ff; font-weight: bold; }
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
23 pre.codearea span.ConsWord { color: #008080; font-weight: bold; }
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
24
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
25 """
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
26
4363
817d99d715fe remove direct usage of deprecated sha module - use hashlib, if possible
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3915
diff changeset
27 import re
3911
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
28
4384
09328e4aab45 deprecate ParserBase syntax highlighter
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4363
diff changeset
29 from MoinMoin import log
09328e4aab45 deprecate ParserBase syntax highlighter
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4363
diff changeset
30 logging = log.getLogger(__name__)
09328e4aab45 deprecate ParserBase syntax highlighter
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4363
diff changeset
31
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
32 from MoinMoin import config, wikiutil
4363
817d99d715fe remove direct usage of deprecated sha module - use hashlib, if possible
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3915
diff changeset
33 from MoinMoin.support.python_compatibility import hash_new
5254
fc47ff90b96f parser: move parse_start_stop to package init so that ParserBase deprecation warning is not accidentally triggered
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4384
diff changeset
34 from MoinMoin.parser import parse_start_step
fc47ff90b96f parser: move parse_start_stop to package init so that ParserBase deprecation warning is not accidentally triggered
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4384
diff changeset
35
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
36
3911
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
37 class FormatTextBase:
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
38 pass
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
39
3915
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
40 class FormatBeginLine(FormatTextBase):
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
41 def formatString(self, formatter, word):
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
42 return formatter.code_line(1)
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
43
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
44 class FormatEndLine(FormatTextBase):
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
45 def formatString(self, formatter, word):
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
46 return formatter.code_line(0)
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
47
3911
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
48 class FormatText(FormatTextBase):
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
49
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
50 def __init__(self, fmt):
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
51 self.fmt = fmt
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
52
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
53 def formatString(self, formatter, word):
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
54 return (formatter.code_token(1, self.fmt) +
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
55 formatter.text(word) +
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
56 formatter.code_token(0, self.fmt))
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
57
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
58 class FormatTextID(FormatTextBase):
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
59
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
60 def __init__(self, fmt, icase=False):
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
61 if not isinstance(fmt, FormatText):
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
62 fmt = FormatText(fmt)
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
63 self.setDefaultFormat(fmt)
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
64 self._ignore_case = icase
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
65 self.fmt = {}
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
66
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
67 def setDefaultFormat(self, fmt):
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
68 self._def_fmt = fmt
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
69
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
70 def addFormat(self, word, fmt):
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
71 if self._ignore_case:
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
72 word = word.lower()
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
73 self.fmt[word] = fmt
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
74
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
75 def formatString(self, formatter, word):
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
76 if self._ignore_case:
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
77 sword = word.lower()
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
78 else:
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
79 sword = word
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
80 return self.fmt.get(sword, self._def_fmt).formatString(formatter, word)
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
81
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
82
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
83 class FormattingRuleSingle:
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
84
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
85 def __init__(self, name, str_re, icase=False):
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
86 self.name = name
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
87 self.str_re = str_re
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
88
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
89 def getStartRe(self):
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
90 return self.str_re
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
91
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
92 def getText(self, parser, hit):
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
93 return hit
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
94
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
95
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
96 class FormattingRulePair:
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
97
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
98 def __init__(self, name, str_begin, str_end, icase=False):
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
99 self.name = name
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
100 self.str_begin = str_begin
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
101 self.str_end = str_end
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
102 re_flags = re.M
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
103 if icase:
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
104 re_flags |= re.I
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
105 self.end_re = re.compile(str_end, re_flags)
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
106
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
107 def getStartRe(self):
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
108 return self.str_begin
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
109
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
110 def getText(self, parser, hit):
3913
ab09415de20b ParserBase: use less confusing names, avoid calculating len(text) repeatedly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3912
diff changeset
111 match = self.end_re.search(parser.text, parser.lastpos)
3911
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
112 if not match:
3913
ab09415de20b ParserBase: use less confusing names, avoid calculating len(text) repeatedly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3912
diff changeset
113 next_lastpos = parser.text_len
3911
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
114 else:
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
115 next_lastpos = match.end() + (match.end() == parser.lastpos)
3913
ab09415de20b ParserBase: use less confusing names, avoid calculating len(text) repeatedly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3912
diff changeset
116 r = parser.text[parser.lastpos:next_lastpos]
3911
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
117 parser.lastpos = next_lastpos
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
118 return hit + r
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
119
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
120
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
121 class ParserBase:
4384
09328e4aab45 deprecate ParserBase syntax highlighter
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4363
diff changeset
122 """ DEPRECATED highlighting parser - please use/extend pygments library """
09328e4aab45 deprecate ParserBase syntax highlighter
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4363
diff changeset
123 logging.warning('Using ParserBase is deprecated - please use/extend pygments syntax highlighting library.')
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
124
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
125 parsername = 'ParserBase'
3911
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
126 tabwidth = 4
2286
01f05e74aa9c Big PEP8 and whitespace cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2274
diff changeset
127
3915
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
128 # for dirty tricks, see comment in format():
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
129 STARTL, STARTL_RE = u"^\n", ur"\^\n"
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
130 ENDL, ENDL_RE = u"\n$", ur"\n\$"
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
131 LINESEP = ENDL + STARTL
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
132
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
133 def __init__(self, raw, request, **kw):
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
134 self.raw = raw
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
135 self.request = request
2286
01f05e74aa9c Big PEP8 and whitespace cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2274
diff changeset
136 self.show_nums, self.num_start, self.num_step, attrs = parse_start_step(request, kw.get('format_args', ''))
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
137
3911
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
138 self._ignore_case = False
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
139 self._formatting_rules = []
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
140 self._formatting_rules_n2r = {}
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
141 self._formatting_rule_index = 0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
142 self.rule_fmt = {}
3912
01ba9b2e96f6 ParserBase: refactoring: reduce code duplication, remove unused code, rearrange
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3911
diff changeset
143 #self.line_count = len(raw.split('\n')) + 1
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
144
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
145 def setupRules(self):
3915
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
146 self.addRuleFormat("BEGINLINE", FormatBeginLine())
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
147 self.addRuleFormat("ENDLINE", FormatEndLine())
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
148 # we need a little dirty trick here, see comment in format():
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
149 self.addRule("BEGINLINE", self.STARTL_RE)
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
150 self.addRule("ENDLINE", self.ENDL_RE)
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
151
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
152 self.def_format = FormatText('Default')
3912
01ba9b2e96f6 ParserBase: refactoring: reduce code duplication, remove unused code, rearrange
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3911
diff changeset
153 self.reserved_word_format = FormatText('ResWord')
01ba9b2e96f6 ParserBase: refactoring: reduce code duplication, remove unused code, rearrange
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3911
diff changeset
154 self.constant_word_format = FormatText('ConsWord')
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
155 self.ID_format = FormatTextID('ID', self._ignore_case)
1920
b06ef2a53efa 'make pylint', fixed lots of minor stuff found by pylint (and there is still lots left to do)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1791
diff changeset
156 self.addRuleFormat("ID", self.ID_format)
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
157 self.addRuleFormat("Operator")
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
158 self.addRuleFormat("Char")
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
159 self.addRuleFormat("Comment")
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
160 self.addRuleFormat("Number")
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
161 self.addRuleFormat("String")
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
162 self.addRuleFormat("SPChar")
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
163 self.addRuleFormat("ResWord")
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
164 self.addRuleFormat("ResWord2")
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
165 self.addRuleFormat("ConsWord")
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
166 self.addRuleFormat("Special")
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
167 self.addRuleFormat("Preprc")
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
168 self.addRuleFormat("Error")
3912
01ba9b2e96f6 ParserBase: refactoring: reduce code duplication, remove unused code, rearrange
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3911
diff changeset
169
01ba9b2e96f6 ParserBase: refactoring: reduce code duplication, remove unused code, rearrange
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3911
diff changeset
170 def _addRule(self, name, fmt):
01ba9b2e96f6 ParserBase: refactoring: reduce code duplication, remove unused code, rearrange
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3911
diff changeset
171 self._formatting_rule_index += 1
01ba9b2e96f6 ParserBase: refactoring: reduce code duplication, remove unused code, rearrange
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3911
diff changeset
172 name = "%s_%s" % (name, self._formatting_rule_index) # create unique name
01ba9b2e96f6 ParserBase: refactoring: reduce code duplication, remove unused code, rearrange
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3911
diff changeset
173 self._formatting_rules.append((name, fmt))
01ba9b2e96f6 ParserBase: refactoring: reduce code duplication, remove unused code, rearrange
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3911
diff changeset
174 self._formatting_rules_n2r[name] = fmt
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
175
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
176 def addRule(self, name, str_re):
3912
01ba9b2e96f6 ParserBase: refactoring: reduce code duplication, remove unused code, rearrange
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3911
diff changeset
177 self._addRule(name, FormattingRuleSingle(name, str_re, self._ignore_case))
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
178
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
179 def addRulePair(self, name, start_re, end_re):
3912
01ba9b2e96f6 ParserBase: refactoring: reduce code duplication, remove unused code, rearrange
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3911
diff changeset
180 self._addRule(name, FormattingRulePair(name, start_re, end_re, self._ignore_case))
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
181
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
182 def addWords(self, words, fmt):
1920
b06ef2a53efa 'make pylint', fixed lots of minor stuff found by pylint (and there is still lots left to do)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1791
diff changeset
183 if not isinstance(fmt, FormatTextBase):
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
184 fmt = FormatText(fmt)
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
185 for w in words:
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
186 self.ID_format.addFormat(w, fmt)
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
187
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
188 def addReserved(self, words):
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
189 self.addWords(words, self.reserved_word_format)
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
190
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
191 def addConstant(self, words):
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
192 self.addWords(words, self.constant_word_format)
2286
01f05e74aa9c Big PEP8 and whitespace cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2274
diff changeset
193
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
194 def addRuleFormat(self, name, fmt=None):
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
195 if fmt is None:
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
196 fmt = FormatText(name)
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
197 self.rule_fmt[name] = fmt
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
198
5820
ff39884957af parsers: avoid crash when Parser.format() call gets inhibit_p kw arg
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5254
diff changeset
199 def format(self, formatter, form=None, **kw):
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
200 """ Send the text.
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
201 """
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
202
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
203 self.setupRules()
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
204
3911
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
205 formatting_regexes = ["(?P<%s>%s)" % (n, f.getStartRe())
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
206 for n, f in self._formatting_rules]
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
207 re_flags = re.M
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
208 if self._ignore_case:
3911
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
209 re_flags |= re.I
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
210 scan_re = re.compile("|".join(formatting_regexes), re_flags)
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
211
3913
ab09415de20b ParserBase: use less confusing names, avoid calculating len(text) repeatedly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3912
diff changeset
212 self.text = self.raw
3915
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
213
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
214 # dirty little trick to work around re lib's limitations (it can't have
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
215 # zero length matches at line beginning for ^ and at the same time match
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
216 # something else at the beginning of the line):
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
217 self.text = self.LINESEP.join([line.replace('\r', '') for line in self.text.splitlines()])
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
218 self.text = self.STARTL + self.text + self.ENDL
3913
ab09415de20b ParserBase: use less confusing names, avoid calculating len(text) repeatedly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3912
diff changeset
219 self.text_len = len(self.text)
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
220
3914
bd0aa9e93ed6 ParserBase / text_python parser: only use a single request.write
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3913
diff changeset
221 result = [] # collects output
bd0aa9e93ed6 ParserBase / text_python parser: only use a single request.write
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3913
diff changeset
222
4363
817d99d715fe remove direct usage of deprecated sha module - use hashlib, if possible
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3915
diff changeset
223 self._code_id = hash_new('sha1', self.raw.encode(config.charset)).hexdigest()
3914
bd0aa9e93ed6 ParserBase / text_python parser: only use a single request.write
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3913
diff changeset
224 result.append(formatter.code_area(1, self._code_id, self.parsername, self.show_nums, self.num_start, self.num_step))
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
225
3914
bd0aa9e93ed6 ParserBase / text_python parser: only use a single request.write
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3913
diff changeset
226 self.lastpos = 0
3913
ab09415de20b ParserBase: use less confusing names, avoid calculating len(text) repeatedly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3912
diff changeset
227 match = scan_re.search(self.text)
ab09415de20b ParserBase: use less confusing names, avoid calculating len(text) repeatedly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3912
diff changeset
228 while match and self.lastpos < self.text_len:
3915
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
229 # add the rendering of the text left of the match we found
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
230 text = self.text[self.lastpos:match.start()]
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
231 if text:
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
232 result.extend(self.format_normal_text(formatter, text))
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
233 self.lastpos = match.end() + (match.end() == self.lastpos)
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
234
3915
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
235 # add the rendering of the match we found
3914
bd0aa9e93ed6 ParserBase / text_python parser: only use a single request.write
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3913
diff changeset
236 result.extend(self.format_match(formatter, match))
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
237
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
238 # search for the next one
3913
ab09415de20b ParserBase: use less confusing names, avoid calculating len(text) repeatedly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3912
diff changeset
239 match = scan_re.search(self.text, self.lastpos)
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
240
3915
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
241 # add the rendering of the text right of the last match we found
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
242 text = self.text[self.lastpos:]
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
243 if text:
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
244 result.extend(self.format_normal_text(formatter, text))
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
245
3914
bd0aa9e93ed6 ParserBase / text_python parser: only use a single request.write
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3913
diff changeset
246 result.append(formatter.code_area(0, self._code_id))
bd0aa9e93ed6 ParserBase / text_python parser: only use a single request.write
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3913
diff changeset
247 self.request.write(''.join(result))
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
248
3914
bd0aa9e93ed6 ParserBase / text_python parser: only use a single request.write
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3913
diff changeset
249 def format_normal_text(self, formatter, text):
3915
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
250 return [formatter.text(text.expandtabs(self.tabwidth))]
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
251
3914
bd0aa9e93ed6 ParserBase / text_python parser: only use a single request.write
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3913
diff changeset
252 def format_match(self, formatter, match):
bd0aa9e93ed6 ParserBase / text_python parser: only use a single request.write
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3913
diff changeset
253 result = []
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
254 for n, hit in match.groupdict().items():
3915
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
255 if hit is None:
2274
01f6172dabd0 remove tabs from source files, remove excludes from test_tab
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1920
diff changeset
256 continue
0
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
257 r = self._formatting_rules_n2r[n]
77665d8e2254 tag of nonpublic@localhost--archive/moin--enterprise--1.5--base-0
Thomas Waldmann <tw-public@gmx.de>
parents:
diff changeset
258 s = r.getText(self, hit)
1920
b06ef2a53efa 'make pylint', fixed lots of minor stuff found by pylint (and there is still lots left to do)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1791
diff changeset
259 c = self.rule_fmt.get(r.name, None)
b06ef2a53efa 'make pylint', fixed lots of minor stuff found by pylint (and there is still lots left to do)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1791
diff changeset
260 if not c:
2274
01f6172dabd0 remove tabs from source files, remove excludes from test_tab
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1920
diff changeset
261 c = self.def_format
3915
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
262 if s:
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
263 lines = s.expandtabs(self.tabwidth).split(self.LINESEP)
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
264 for line in lines[:-1]:
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
265 result.append(c.formatString(formatter, line))
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
266 result.append(FormatEndLine().formatString(formatter, ''))
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
267 result.append(FormatBeginLine().formatString(formatter, ''))
b4bdfaaa79b0 ParserBase: do code_line() open and close calls correctly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3914
diff changeset
268 result.append(c.formatString(formatter, lines[-1]))
3914
bd0aa9e93ed6 ParserBase / text_python parser: only use a single request.write
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3913
diff changeset
269 return result
3911
348c9b6e0033 ParserBase (and other parsers deriving from it): cleanup / refactorings:
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2286
diff changeset
270