annotate MoinMoin/filter/__init__.py @ 6133:a6283e189869 tip

fixup: remove nonexisting passlib.utils._blowfish this was removed by the passlib 1.7.1 upgrade.
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Thu, 01 Jun 2017 18:10:19 +0200
parents 5204e8eb9737
children
rev   line source
438
8bb545d58e6f lupy: filter plugins for attachment search
Thomas Waldmann <tw@waldmann-edv.de>
parents:
diff changeset
1 # -*- coding: iso-8859-1 -*-
8bb545d58e6f lupy: filter plugins for attachment search
Thomas Waldmann <tw@waldmann-edv.de>
parents:
diff changeset
2 """
8bb545d58e6f lupy: filter plugins for attachment search
Thomas Waldmann <tw@waldmann-edv.de>
parents:
diff changeset
3 MoinMoin - Filter Package
8bb545d58e6f lupy: filter plugins for attachment search
Thomas Waldmann <tw@waldmann-edv.de>
parents:
diff changeset
4
5150
b424a3d66cfc indexing filters: use subprocess (on py >= 2.4), kill filter process after 300s
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4450
diff changeset
5 @copyright: 2006-2009 MoinMoin:ThomasWaldmann
438
8bb545d58e6f lupy: filter plugins for attachment search
Thomas Waldmann <tw@waldmann-edv.de>
parents:
diff changeset
6 @license: GNU GPL, see COPYING for details.
8bb545d58e6f lupy: filter plugins for attachment search
Thomas Waldmann <tw@waldmann-edv.de>
parents:
diff changeset
7 """
8bb545d58e6f lupy: filter plugins for attachment search
Thomas Waldmann <tw@waldmann-edv.de>
parents:
diff changeset
8
4450
b77726872b1f execfilter: add missing import
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3939
diff changeset
9 import sys, os
5150
b424a3d66cfc indexing filters: use subprocess (on py >= 2.4), kill filter process after 300s
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4450
diff changeset
10 import time
b424a3d66cfc indexing filters: use subprocess (on py >= 2.4), kill filter process after 300s
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4450
diff changeset
11
3675
f048227cabc3 search index filters: redirect stderr to logging system
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1918
diff changeset
12 from MoinMoin import log
f048227cabc3 search index filters: redirect stderr to logging system
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1918
diff changeset
13 logging = log.getLogger(__name__)
f048227cabc3 search index filters: redirect stderr to logging system
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1918
diff changeset
14
438
8bb545d58e6f lupy: filter plugins for attachment search
Thomas Waldmann <tw@waldmann-edv.de>
parents:
diff changeset
15 from MoinMoin.util import pysupport
8bb545d58e6f lupy: filter plugins for attachment search
Thomas Waldmann <tw@waldmann-edv.de>
parents:
diff changeset
16
1133
db56bd53fc32 cleanup plugin related code / attributes, remove unused stuff
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 999
diff changeset
17 modules = pysupport.getPackageModules(__file__)
438
8bb545d58e6f lupy: filter plugins for attachment search
Thomas Waldmann <tw@waldmann-edv.de>
parents:
diff changeset
18
999
97ce857bdebf whitespace-only cleanup and minor style changes
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 470
diff changeset
19 standard_codings = ['utf-8', 'iso-8859-15', 'iso-8859-1', ]
470
957d482215ac lupy: some filters, update mode
Thomas Waldmann <tw@waldmann-edv.de>
parents: 438
diff changeset
20
5213
5204e8eb9737 merged moin/1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5212
diff changeset
21 from MoinMoin.util.SubProcess import exec_cmd
5212
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5181
diff changeset
22
3939
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
23
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
24 def quote_filename(filename):
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
25 """ quote a filename (could contain blanks or other special chars) in a
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
26 way suitable for the platform we run on.
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
27 """
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
28 # XXX Use os.name AND/OR sys.platform?
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
29 if os.name == 'posix':
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
30 filename = "'%s'" % filename
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
31 elif sys.platform == 'win32':
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
32 filename = '"%s"' % filename
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
33 else:
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
34 raise ValueError("MoinMoin.filter.quote_filename: os/platform not supported")
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
35 return filename
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
36
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
37
470
957d482215ac lupy: some filters, update mode
Thomas Waldmann <tw@waldmann-edv.de>
parents: 438
diff changeset
38 def execfilter(cmd, filename, codings=standard_codings):
957d482215ac lupy: some filters, update mode
Thomas Waldmann <tw@waldmann-edv.de>
parents: 438
diff changeset
39 """ use cmd to get plaintext content of filename
957d482215ac lupy: some filters, update mode
Thomas Waldmann <tw@waldmann-edv.de>
parents: 438
diff changeset
40 to decode to unicode, we use the first coding of codings list that
957d482215ac lupy: some filters, update mode
Thomas Waldmann <tw@waldmann-edv.de>
parents: 438
diff changeset
41 does not throw an exception or force ascii
957d482215ac lupy: some filters, update mode
Thomas Waldmann <tw@waldmann-edv.de>
parents: 438
diff changeset
42 """
3939
4e904da48c4c filters: make filename quoting platform-dependant and move it to execfilter, do antiword HOME env workaround only for posix platform
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3675
diff changeset
43 filter_cmd = cmd % quote_filename(filename)
5212
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5181
diff changeset
44 data, errors, rc = exec_cmd(filter_cmd, timeout=300)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5181
diff changeset
45 logging.debug("Command '%s', rc: %d, stdout: %d bytes, stderr: %s" % (filter_cmd, rc, len(data), errors))
470
957d482215ac lupy: some filters, update mode
Thomas Waldmann <tw@waldmann-edv.de>
parents: 438
diff changeset
46 for c in codings:
957d482215ac lupy: some filters, update mode
Thomas Waldmann <tw@waldmann-edv.de>
parents: 438
diff changeset
47 try:
957d482215ac lupy: some filters, update mode
Thomas Waldmann <tw@waldmann-edv.de>
parents: 438
diff changeset
48 return data.decode(c)
957d482215ac lupy: some filters, update mode
Thomas Waldmann <tw@waldmann-edv.de>
parents: 438
diff changeset
49 except UnicodeError:
957d482215ac lupy: some filters, update mode
Thomas Waldmann <tw@waldmann-edv.de>
parents: 438
diff changeset
50 pass
957d482215ac lupy: some filters, update mode
Thomas Waldmann <tw@waldmann-edv.de>
parents: 438
diff changeset
51 return data.decode('ascii', 'replace')
957d482215ac lupy: some filters, update mode
Thomas Waldmann <tw@waldmann-edv.de>
parents: 438
diff changeset
52