annotate MoinMoin/converter/archive_in.py @ 1414:614005b5edb3

fixed some .format() unicode issues, partially fixing bug #124
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Sat, 07 Jul 2012 20:22:21 +0200
parents 4ac437141bbe
children bff62fea804a
rev   line source
26
4cd8edde9e94 automatically transform module docstrings, moving legal stuff to comments
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 24
diff changeset
1 # Copyright: 2010 MoinMoin:ThomasWaldmann
4cd8edde9e94 automatically transform module docstrings, moving legal stuff to comments
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 24
diff changeset
2 # License: GNU GPL v2 (or any later version), see LICENSE.txt for details.
4cd8edde9e94 automatically transform module docstrings, moving legal stuff to comments
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 24
diff changeset
3
0
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
4 """
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
5 MoinMoin - Archives converter (e.g. zip, tar)
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
6
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
7 Make a DOM Tree representation of an archive (== list contents of it in a table).
26
4cd8edde9e94 automatically transform module docstrings, moving legal stuff to comments
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 24
diff changeset
8 """
0
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
9
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
10
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
11 from datetime import datetime
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
12 import tarfile
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
13 import zipfile
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
14
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
15 from ._table import TableMixin
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
16
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
17 from MoinMoin import log
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
18 logging = log.getLogger(__name__)
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
19
1118
505ad860856a Added table header for archive content listing.
Julian Brost <julian.brost@googlemail.com>
parents: 1114
diff changeset
20 from MoinMoin.i18n import _, L_, N_
0
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
21 from MoinMoin.util.iri import Iri
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
22 from MoinMoin.util.tree import moin_page, xlink
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
23
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
24
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
25 class ArchiveException(Exception):
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
26 """
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
27 exception class used in case of trouble with opening/listing an archive
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
28 """
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
29
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
30 class ArchiveConverter(TableMixin):
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
31 """
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
32 Base class for archive converters, convert an archive to a DOM table
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
33 with an archive listing.
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
34 """
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
35 @classmethod
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
36 def _factory(cls, input, output, **kw):
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
37 return cls()
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
38
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
39 def process_name(self, member_name):
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
40 attrib = {
1414
614005b5edb3 fixed some .format() unicode issues, partially fixing bug #124
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1397
diff changeset
41 xlink.href: Iri(scheme='wiki', authority='', path='/'+self.item_name, query=u'do=get&member={0}'.format(member_name)),
0
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
42 }
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
43 return moin_page.a(attrib=attrib, children=[member_name, ])
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
44
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
45 def process_datetime(self, dt):
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
46 return dt.isoformat()
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
47
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
48 def process_size(self, size):
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
49 return unicode(size)
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
50
1114
91f6360bba7a archive converter: we get a revision object, use rev.data for the file
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 969
diff changeset
51 def __call__(self, rev, contenttype=None, arguments=None):
91f6360bba7a archive converter: we get a revision object, use rev.data for the file
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 969
diff changeset
52 self.item_name = rev.item.name
0
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
53 try:
1114
91f6360bba7a archive converter: we get a revision object, use rev.data for the file
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 969
diff changeset
54 contents = self.list_contents(rev.data)
0
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
55 contents = [(self.process_size(size),
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
56 self.process_datetime(dt),
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
57 self.process_name(name),
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
58 ) for size, dt, name in contents]
1118
505ad860856a Added table header for archive content listing.
Julian Brost <julian.brost@googlemail.com>
parents: 1114
diff changeset
59 return self.build_dom_table(contents, head=[_("Size"), _("Date"), _("Name")], cls='zebra')
135
75312c9b94e3 use except...as syntax, use raise Exception(...) syntax, fix some imports
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 26
diff changeset
60 except ArchiveException as err:
0
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
61 logging.exception("An exception within archive file handling occurred:")
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
62 # XXX we also use a table for error reporting, could be
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
63 # something more adequate, though:
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
64 return self.build_dom_table([[str(err)]])
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
65
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
66 def list_contents(self, fileobj):
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
67 """
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
68 analyze archive we get as fileobj and return data for table rendering.
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
69
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
70 We return a list of rows, each row is a list of cells.
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
71
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
72 Usually each row is [size, datetime, name] for each archive member.
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
73
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
74 In case of problems, it shall raise ArchiveException(error_msg).
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
75 """
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
76 raise NotImplementedError
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
77
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
78
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
79 class TarConverter(ArchiveConverter):
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
80 """
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
81 Support listing tar files.
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
82 """
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
83 def list_contents(self, fileobj):
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
84 try:
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
85 rows = []
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
86 tf = tarfile.open(fileobj=fileobj, mode='r')
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
87 for tinfo in tf.getmembers():
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
88 rows.append((
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
89 tinfo.size,
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
90 datetime.utcfromtimestamp(tinfo.mtime),
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
91 tinfo.name,
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
92 ))
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
93 return rows
135
75312c9b94e3 use except...as syntax, use raise Exception(...) syntax, fix some imports
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 26
diff changeset
94 except tarfile.TarError as err:
0
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
95 raise ArchiveException(str(err))
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
96
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
97
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
98 class ZipConverter(ArchiveConverter):
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
99 """
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
100 Support listing zip files.
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
101 """
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
102 def list_contents(self, fileobj):
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
103 try:
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
104 rows = []
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
105 zf = zipfile.ZipFile(fileobj, mode='r')
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
106 for zinfo in zf.filelist:
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
107 rows.append((
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
108 zinfo.file_size,
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
109 datetime(*zinfo.date_time), # y,m,d,h,m,s
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
110 zinfo.filename,
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
111 ))
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
112 return rows
135
75312c9b94e3 use except...as syntax, use raise Exception(...) syntax, fix some imports
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 26
diff changeset
113 except (RuntimeError, zipfile.BadZipfile) as err:
0
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
114 # RuntimeError is raised by zipfile stdlib module in case of
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
115 # problems (like inconsistent slash and backslash usage in the
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
116 # archive or a defective zip file).
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
117 raise ArchiveException(str(err))
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
118
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
119
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
120 from . import default_registry
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
121 from MoinMoin.util.mime import Type, type_moin_document
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
122 default_registry.register(TarConverter._factory, Type('application/x-tar'), type_moin_document)
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
123 default_registry.register(TarConverter._factory, Type('application/x-gtar'), type_moin_document)
5568cf133caf create moin/2.0 repo, drop all history (see notes below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
124 default_registry.register(ZipConverter._factory, Type('application/zip'), type_moin_document)