annotate MoinMoin/support/pygments/filter.py @ 5184:0970ed47d2cd

updated pygments to 1.1.1+ ( http://dev.pocoo.org/hg/pygments-main/rev/948f8885af16 )
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Sun, 04 Oct 2009 17:47:54 +0200
parents 00b1307bd9c2
children 74fa6f714526
rev   line source
4377
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
1 # -*- coding: utf-8 -*-
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
2 """
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
3 pygments.filter
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
4 ~~~~~~~~~~~~~~~
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
5
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
6 Module that implements the default filter.
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
7
5184
0970ed47d2cd updated pygments to 1.1.1+ ( http://dev.pocoo.org/hg/pygments-main/rev/948f8885af16 )
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4377
diff changeset
8 :copyright: Copyright 2006-2009 by the Pygments team, see AUTHORS.
0970ed47d2cd updated pygments to 1.1.1+ ( http://dev.pocoo.org/hg/pygments-main/rev/948f8885af16 )
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4377
diff changeset
9 :license: BSD, see LICENSE for details.
4377
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
10 """
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
11
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
12
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
13 def apply_filters(stream, filters, lexer=None):
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
14 """
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
15 Use this method to apply an iterable of filters to
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
16 a stream. If lexer is given it's forwarded to the
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
17 filter, otherwise the filter receives `None`.
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
18 """
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
19 def _apply(filter_, stream):
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
20 for token in filter_.filter(lexer, stream):
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
21 yield token
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
22 for filter_ in filters:
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
23 stream = _apply(filter_, stream)
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
24 return stream
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
25
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
26
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
27 def simplefilter(f):
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
28 """
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
29 Decorator that converts a function into a filter::
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
30
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
31 @simplefilter
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
32 def lowercase(lexer, stream, options):
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
33 for ttype, value in stream:
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
34 yield ttype, value.lower()
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
35 """
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
36 return type(f.__name__, (FunctionFilter,), {
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
37 'function': f,
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
38 '__module__': getattr(f, '__module__'),
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
39 '__doc__': f.__doc__
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
40 })
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
41
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
42
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
43 class Filter(object):
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
44 """
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
45 Default filter. Subclass this class or use the `simplefilter`
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
46 decorator to create own filters.
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
47 """
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
48
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
49 def __init__(self, **options):
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
50 self.options = options
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
51
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
52 def filter(self, lexer, stream):
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
53 raise NotImplementedError()
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
54
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
55
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
56 class FunctionFilter(Filter):
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
57 """
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
58 Abstract class used by `simplefilter` to create simple
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
59 function filters on the fly. The `simplefilter` decorator
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
60 automatically creates subclasses of this class for
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
61 functions passed to it.
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
62 """
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
63 function = None
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
64
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
65 def __init__(self, **options):
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
66 if not hasattr(self, 'function'):
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
67 raise TypeError('%r used without bound function' %
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
68 self.__class__.__name__)
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
69 Filter.__init__(self, **options)
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
70
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
71 def filter(self, lexer, stream):
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
72 # pylint: disable-msg=E1102
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
73 for ttype, value in self.function(lexer, stream, self.options):
00b1307bd9c2 Adding Pygments to MoinMoin/support
Radomir Dopieralski <moindev@sheep.art.pl>
parents:
diff changeset
74 yield ttype, value