annotate MoinMoin/script/migration/_conv160.py @ 2599:13f0331f3a42

1.6 converter: add content conversion (unfinished), cleanup
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Sat, 04 Aug 2007 16:54:35 +0200
parents b609934451e6
children b601db2e4d34
rev   line source
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
1 # -*- coding: iso-8859-1 -*-
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
2 """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
3 MoinMoin - migration from base rev 105xxyy
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
4
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
5 What it should do when it is ready:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
6
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
7 a) reverse underscore == blank stuff in pagenames (introducing this was a fault)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
8
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
9 pagename quoted pagename
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
10 -----------------------------------------------------
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
11 old MainPage/Sub_Page MainPage(2f)Sub_Page
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
12 new MainPage/Sub Page MainPage(2f)Sub(20)Page or
2586
a838bdc4e40a 1.6 converter: quicklinks/subscribed pages (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2584
diff changeset
13 new MainPage/Sub_Page MainPage(2f)Sub_Page (user has to decide by editing rename1.txt)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
14
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
15
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
16 markup
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
17 ----------------------------------------------------
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
18 old MoinMoin:MainPage/Sub_Page ../Sub_Page2
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
19 new MoinMoin:"MainPage/Sub Page" "../Sub Page2"???? (TODO check if this works)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
20
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
21
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
22 b) decode url encoded chars in attachment names (and quote the whole fname):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
23
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
24 markup
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
25 ----------------------------------------------------
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
26 old attachment:file%20with%20blanks.txt
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
27 new attachment:"file with blanks.txt"
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
28
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
29 c) users: move bookmarks from separate files into user profile
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
30 d) users: generate new name[] for lists and name{} for dicts
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
31
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
32 TODO:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
33 * process page content / convert markup
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
34
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
35 DONE:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
36 pass 1
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
37 * creating the rename.txt works
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
38 pass 2
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
39 * renaming of pages works
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
40 * renamed pagedirs
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
41 * renamed page names in global edit-log
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
42 * renamed page names in local edit-log
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
43 * renamed page names in event-log
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
44 * renamed pages in user subscribed pages
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
45 * renamed pages in user quicklinks
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
46 * renaming of attachments works
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
47 * renamed attachment files
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
48 * renamed attachment names in global edit-log
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
49 * renamed attachment names in local edit-log
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
50 * migrate separate user bookmark files into user profiles
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
51 * support new dict/list syntax in user profiles
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
52
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
53 @copyright: 2007 by Thomas Waldmann
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
54 @license: GNU GPL, see COPYING for details.
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
55 """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
56
2599
13f0331f3a42 1.6 converter: add content conversion (unfinished), cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2593
diff changeset
57 import os.path
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
58 import codecs, urllib, glob
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
59
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
60 from MoinMoin import config, wikiutil
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
61 from MoinMoin.script.migration.migutil import opj, listdir, copy_file, move_file, copy_dir
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
62
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
63 import mimetypes # this MUST be after wikiutil import!
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
64
2599
13f0331f3a42 1.6 converter: add content conversion (unfinished), cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2593
diff changeset
65 from _conv160_wiki import convert_wiki
13f0331f3a42 1.6 converter: add content conversion (unfinished), cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2593
diff changeset
66
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
67 def markup_converter(text, renames):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
68 """ Convert the <text> content of some Page, using <renames> dict to rename
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
69 links correctly. Additionally, convert some changed markup.
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
70 """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
71 if "#format wiki" not in text and "#format" in text:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
72 return text # this is not a wiki page, leave it as is
2599
13f0331f3a42 1.6 converter: add content conversion (unfinished), cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2593
diff changeset
73
13f0331f3a42 1.6 converter: add content conversion (unfinished), cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2593
diff changeset
74 text = convert_wiki(text, renames)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
75 return text
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
76
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
77
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
78 class EventLog:
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
79 def __init__(self, request, fname):
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
80 self.request = request
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
81 self.fname = fname
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
82 self.data = None
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
83 self.renames = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
84
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
85 def read(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
86 """ read complete event-log from disk """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
87 data = []
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
88 f = file(self.fname, 'r')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
89 for line in f:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
90 line = line.replace('\r', '').replace('\n', '')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
91 if not line.strip(): # skip empty lines
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
92 continue
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
93 fields = line.split('\t')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
94 timestamp, action, kvpairs = fields
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
95 timestamp = int(timestamp)
2593
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
96 kvdict = wikiutil.parseQueryString(kvpairs)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
97 data.append((timestamp, action, kvdict))
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
98 self.data = data
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
99
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
100 def write(self, fname):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
101 """ write complete event-log to disk """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
102 f = file(fname, 'w')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
103 for timestamp, action, kvdict in self.data:
2593
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
104 pagename = kvdict.get('pagename')
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
105 if pagename and ('PAGE', pagename) in self.renames:
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
106 kvdict['pagename'] = self.renames[('PAGE', pagename)]
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
107 kvpairs = wikiutil.makeQueryString(kvdict, want_unicode=False)
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
108 fields = str(timestamp), action, kvpairs
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
109 line = '\t'.join(fields) + '\n'
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
110 f.write(line)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
111 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
112
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
113 def copy(self, destfname, renames):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
114 self.renames = renames
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
115 self.read()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
116 self.write(destfname)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
117
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
118
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
119 class EditLog:
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
120 def __init__(self, request, fname):
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
121 self.request = request
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
122 self.fname = fname
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
123 self.data = None
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
124 self.renames = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
125
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
126 def read(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
127 """ read complete edit-log from disk """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
128 data = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
129 f = file(self.fname, 'r')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
130 for line in f:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
131 line = line.replace('\r', '').replace('\n', '')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
132 if not line.strip(): # skip empty lines
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
133 continue
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
134 fields = line.split('\t') + [''] * 9
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
135 timestamp, rev, action, pagename, ip, hostname, userid, extra, comment = fields[:9]
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
136 timestamp = int(timestamp)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
137 rev = int(rev)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
138 pagename = wikiutil.unquoteWikiname(pagename)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
139 data[(timestamp, rev, pagename)] = (timestamp, rev, action, pagename, ip, hostname, userid, extra, comment)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
140 self.data = data
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
141
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
142 def write(self, fname):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
143 """ write complete edit-log to disk """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
144 editlog = self.data.items()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
145 editlog.sort()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
146 f = file(fname, "w")
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
147 for key, fields in editlog:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
148 timestamp, rev, action, pagename, ip, hostname, userid, extra, comment = fields
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
149 if action.startswith('ATT'):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
150 try:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
151 fname = urllib.unquote(extra).decode('utf-8')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
152 except UnicodeDecodeError:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
153 fname = urllib.unquote(extra).decode('iso-8859-1')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
154 if ('FILE', pagename, fname) in self.renames:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
155 fname = self.renames[('FILE', pagename, fname)]
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
156 extra = urllib.quote(fname.encode('utf-8'))
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
157 if ('PAGE', pagename) in self.renames:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
158 pagename = self.renames[('PAGE', pagename)]
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
159 timestamp = str(timestamp)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
160 rev = '%08d' % rev
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
161 pagename = wikiutil.quoteWikinameFS(pagename)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
162 fields = timestamp, rev, action, pagename, ip, hostname, userid, extra, comment
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
163 log_str = '\t'.join(fields) + '\n'
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
164 f.write(log_str)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
165 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
166
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
167 def copy(self, destfname, renames):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
168 self.renames = renames
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
169 self.read()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
170 self.write(destfname)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
171
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
172
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
173 class PageRev:
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
174 """ a single revision of a page """
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
175 def __init__(self, request, rev_dir, rev):
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
176 self.request = request
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
177 self.rev_dir = rev_dir
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
178 self.rev = rev
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
179
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
180 def read(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
181 fname = opj(self.rev_dir, '%08d' % self.rev)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
182 f = file(fname, "rb")
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
183 data = f.read()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
184 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
185 data = data.decode(config.charset)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
186 return data
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
187
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
188 def write(self, data, rev_dir, rev=None):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
189 if rev is None:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
190 rev = self.rev
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
191 data = markup_converter(data, self.renames)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
192 fname = opj(rev_dir, '%08d' % rev)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
193 data = data.encode(config.charset)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
194 f = file(fname, "wb")
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
195 f.write(data)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
196 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
197
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
198 def copy(self, rev_dir, renames):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
199 self.renames = renames
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
200 data = self.read()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
201 self.write(data, rev_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
202
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
203
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
204 class Attachment:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
205 """ a single attachment """
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
206 def __init__(self, request, attach_dir, attfile):
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
207 self.request = request
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
208 self.path = opj(attach_dir, attfile)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
209 self.name = attfile.decode('utf-8')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
210
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
211 def copy(self, attach_dir):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
212 """ copy attachment file from orig path to new destination """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
213 attfile = self.name.encode('utf-8')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
214 dest = opj(attach_dir, attfile)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
215 copy_file(self.path, dest)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
216
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
217
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
218 class Page:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
219 """ represents a page with all related data """
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
220 def __init__(self, request, pages_dir, qpagename):
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
221 self.request = request
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
222 self.name = wikiutil.unquoteWikiname(qpagename)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
223 self.name_old = self.name # renaming: still original name when self.name has the new name
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
224 self.page_dir = opj(pages_dir, qpagename)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
225 self.current = None # int current
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
226 self.editlog = None # dict (see read_editlog)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
227 self.revlist = None # list of ints (page text revisions)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
228 self.revisions = None # dict int: pagerev obj
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
229 self.attachments = None # dict of unicode fname: full path
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
230 self.renames = {} # info for renaming pages/attachments
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
231
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
232 def read(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
233 """ read a page, including revisions, log, attachments from disk """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
234 page_dir = self.page_dir
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
235 # read current file
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
236 current_fname = opj(page_dir, 'current')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
237 if os.path.exists(current_fname):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
238 current_file = file(current_fname, "r")
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
239 current_rev = current_file.read()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
240 current_file.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
241 self.current = int(current_rev)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
242 # read edit-log
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
243 editlog_fname = opj(page_dir, 'edit-log')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
244 if os.path.exists(editlog_fname):
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
245 self.editlog = EditLog(self.request, editlog_fname)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
246 # read page revisions
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
247 rev_dir = opj(page_dir, 'revisions')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
248 if os.path.exists(rev_dir):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
249 revlist = listdir(rev_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
250 revlist = [int(rev) for rev in revlist]
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
251 revlist.sort()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
252 self.revlist = revlist
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
253 self.revisions = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
254 for rev in revlist:
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
255 self.revisions[rev] = PageRev(self.request, rev_dir, rev)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
256 # read attachment filenames
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
257 attach_dir = opj(page_dir, 'attachments')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
258 if os.path.exists(attach_dir):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
259 self.attachments = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
260 attlist = listdir(attach_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
261 for attfile in attlist:
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
262 a = Attachment(self.request, attach_dir, attfile)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
263 self.attachments[a.name] = a
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
264
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
265 def write(self, pages_dir):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
266 """ write a page, including revisions, log, attachments to disk """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
267 if ('PAGE', self.name) in self.renames:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
268 name_new = self.renames[('PAGE', self.name)]
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
269 if name_new != self.name:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
270 print "Renaming page %r -> %r" % (self.name, name_new)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
271 self.name_old = self.name
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
272 self.name = name_new
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
273 qpagename = wikiutil.quoteWikinameFS(self.name)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
274 page_dir = opj(pages_dir, qpagename)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
275 os.makedirs(page_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
276 # write current file
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
277 if self.current is not None:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
278 current_fname = opj(page_dir, 'current')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
279 current_file = file(current_fname, "w")
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
280 current_str = '%08d\n' % self.current
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
281 current_file.write(current_str)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
282 current_file.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
283 # copy edit-log
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
284 if self.editlog is not None:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
285 editlog_fname = opj(page_dir, 'edit-log')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
286 self.editlog.copy(editlog_fname, self.renames)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
287 # copy page revisions
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
288 if self.revisions is not None:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
289 rev_dir = opj(page_dir, 'revisions')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
290 os.makedirs(rev_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
291 for rev in self.revlist:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
292 self.revisions[rev].copy(rev_dir, self.renames)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
293 # copy attachments
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
294 if self.attachments is not None:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
295 attach_dir = opj(page_dir, 'attachments')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
296 os.makedirs(attach_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
297 for fn, att in self.attachments.items():
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
298 # we have to check for renames here because we need the (old) pagename, too:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
299 if ('FILE', self.name_old, fn) in self.renames:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
300 fn_new = self.renames[('FILE', self.name_old, fn)]
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
301 if fn_new != fn:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
302 print "Renaming file %r %r -> %r" % (self.name_old, fn, fn_new)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
303 att.name = fn_new
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
304 att.copy(attach_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
305
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
306 def copy(self, pages_dir, renames):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
307 self.renames = renames
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
308 self.read()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
309 self.write(pages_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
310
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
311
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
312 class User:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
313 """ represents a user with all related data """
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
314 def __init__(self, request, users_dir, uid):
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
315 self.request = request
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
316 self.uid = uid
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
317 self.users_dir = users_dir
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
318 self.profile = None
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
319 self.bookmarks = None
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
320
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
321 def read(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
322 """ read profile and bookmarks data from disk """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
323 self.profile = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
324 fname = opj(self.users_dir, self.uid)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
325 # read user profile
2593
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
326 f = codecs.open(fname, 'r', config.charset)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
327 for line in f:
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
328 line = line.replace(u'\r', '').replace(u'\n', '')
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
329 if not line.strip() or line.startswith(u'#'): # skip empty or comment lines
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
330 continue
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
331 key, value = line.split(u'=', 1)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
332 self.profile[key] = value
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
333 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
334 # read bookmarks
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
335 self.bookmarks = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
336 fname_pattern = opj(self.users_dir, "%s.*.bookmark" % self.uid)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
337 for fname in glob.glob(fname_pattern):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
338 f = file(fname, "r")
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
339 bookmark = f.read()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
340 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
341 wiki = fname.replace('.bookmark', '').replace(opj(self.users_dir, self.uid+'.'), '')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
342 self.bookmarks[wiki] = int(bookmark)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
343 # don't care about trail
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
344
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
345 def write(self, users_dir):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
346 """ write profile and bookmarks data to disk """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
347 fname = opj(users_dir, self.uid)
2593
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
348 f = codecs.open(fname, 'w', config.charset)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
349 for key, value in self.profile.items():
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
350 if key in (u'subscribed_pages', u'quicklinks'):
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
351 pages = value.split(u'\t')
2586
a838bdc4e40a 1.6 converter: quicklinks/subscribed pages (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2584
diff changeset
352 for i in range(len(pages)):
a838bdc4e40a 1.6 converter: quicklinks/subscribed pages (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2584
diff changeset
353 pagename = pages[i]
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
354 try:
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
355 interwiki, pagename = pagename.split(u':', 1)
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
356 except:
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
357 interwiki, pagename = u'Self', pagename
2593
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
358 if interwiki == u'Self' or interwiki == self.request.cfg.interwikiname:
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
359 if ('PAGE', pagename) in self.renames:
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
360 pagename = self.renames[('PAGE', pagename)]
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
361 pages[i] = u'%s:%s' % (interwiki, pagename)
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
362 key += '[]' # we have lists here
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
363 value = u'\t'.join(pages)
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
364 f.write(u"%s=%s\n" % (key, value))
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
365 else:
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
366 f.write(u"%s=%s\n" % (key, value))
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
367 bookmark_entries = [u'%s:%s' % item for item in self.bookmarks.items()]
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
368 key = u"bookmarks{}"
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
369 value = u'\t'.join(bookmark_entries)
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
370 f.write(u"%s=%s\n" % (key, value))
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
371 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
372 # don't care about trail
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
373
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
374 def copy(self, users_dir, renames):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
375 self.renames = renames
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
376 self.read()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
377 self.write(users_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
378
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
379
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
380 class DataConverter(object):
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
381 def __init__(self, request, src_data_dir, dest_data_dir):
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
382 self.request = request
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
383 self.sdata = src_data_dir
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
384 self.ddata = dest_data_dir
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
385 self.pages = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
386 self.users = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
387 self.renames = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
388 self.rename_fname1 = opj(self.sdata, 'rename1.txt')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
389 self.rename_fname2 = opj(self.sdata, 'rename2.txt')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
390
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
391 def pass1(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
392 """ First create the rename list - the user has to review/edit it as
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
393 we can't decide about page/attachment names automatically.
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
394 """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
395 self.read_src()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
396 # pages
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
397 for pn, p in self.pages.items():
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
398 p.read()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
399 if not p.revisions:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
400 continue # we don't care for pages with no revisions (trash)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
401 if "_" in pn:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
402 # log all pagenames with underscores
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
403 self.renames[('PAGE', pn)] = None
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
404 if p.attachments is not None:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
405 for fn in p.attachments:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
406 try:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
407 fn_str = fn.encode('ascii')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
408 log = False # pure ascii filenames are no problem
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
409 except UnicodeEncodeError:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
410 log = True # this file maybe has a strange representation in wiki markup
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
411 else:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
412 if ' ' in fn_str or '%' in fn_str: # files with blanks need quoting
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
413 log = True
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
414 if log:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
415 # log all strange attachment filenames
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
416 fn_str = fn.encode('utf-8')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
417 self.renames[('FILE', pn, fn)] = None
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
418 self.save_renames()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
419
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
420 def save_renames(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
421 f = codecs.open(self.rename_fname1, 'w', 'utf-8')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
422 for k in self.renames:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
423 rtype, pn, fn = (k + (None, ))[:3]
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
424 if rtype == 'PAGE':
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
425 line = u"%s\t%s\t%s\r\n" % (rtype, pn, pn)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
426 elif rtype == 'FILE':
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
427 line = u"%s\t%s\t%s\t%s\r\n" % (rtype, pn, fn, fn)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
428 f.write(line)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
429 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
430
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
431 def load_renames(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
432 f = codecs.open(self.rename_fname2, 'r', 'utf-8')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
433 for line in f:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
434 line = line.rstrip()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
435 if not line:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
436 continue
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
437 t = line.split(u'\t')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
438 rtype, p1, p2, p3 = (t + [None]*3)[:4]
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
439 if rtype == u'PAGE':
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
440 self.renames[(str(rtype), p1)] = p2
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
441 elif rtype == u'FILE':
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
442 self.renames[(str(rtype), p1, p2)] = p3
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
443 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
444
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
445 def pass2(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
446 """ Second, read the (user edited) rename list and do the renamings everywhere. """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
447 self.read_src()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
448 self.load_renames()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
449 self.write_dest()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
450
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
451 def read_src(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
452 # create Page objects in memory
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
453 pages_dir = opj(self.sdata, 'pages')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
454 pagelist = listdir(pages_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
455 for qpagename in pagelist:
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
456 p = Page(self.request, pages_dir, qpagename)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
457 self.pages[p.name] = p
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
458
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
459 # create User objects in memory
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
460 users_dir = opj(self.sdata, 'user')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
461 userlist = listdir(users_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
462 userlist = [fn for fn in userlist if not fn.endswith(".trail") and not fn.endswith(".bookmark")]
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
463 for userid in userlist:
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
464 u = User(self.request, users_dir, userid)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
465 self.users[u.uid] = u
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
466
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
467 # create log objects in memory
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
468 self.editlog = EditLog(self.request, opj(self.sdata, 'edit-log'))
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
469 self.eventlog = EventLog(self.request, opj(self.sdata, 'event-log'))
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
470
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
471 def write_dest(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
472 self.init_dest()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
473 # copy pages
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
474 pages_dir = opj(self.ddata, 'pages')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
475 for page in self.pages.values():
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
476 page.copy(pages_dir, self.renames)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
477
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
478 # copy users
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
479 users_dir = opj(self.ddata, 'user')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
480 for user in self.users.values():
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
481 user.copy(users_dir, self.renames)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
482
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
483 # copy logs
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
484 self.editlog.copy(opj(self.ddata, 'edit-log'), self.renames)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
485 self.eventlog.copy(opj(self.ddata, 'event-log'), self.renames)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
486
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
487 def init_dest(self):
2591
99d1516223f7 1.6 converter: move cache dir with locks where the framework still expects it
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2590
diff changeset
488 try:
99d1516223f7 1.6 converter: move cache dir with locks where the framework still expects it
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2590
diff changeset
489 os.makedirs(self.ddata)
99d1516223f7 1.6 converter: move cache dir with locks where the framework still expects it
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2590
diff changeset
490 except:
99d1516223f7 1.6 converter: move cache dir with locks where the framework still expects it
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2590
diff changeset
491 pass
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
492 os.makedirs(opj(self.ddata, 'pages'))
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
493 os.makedirs(opj(self.ddata, 'user'))
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
494 copy_dir(opj(self.sdata, 'plugin'), opj(self.ddata, 'plugin'))
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
495 copy_file(opj(self.sdata, 'intermap.txt'), opj(self.ddata, 'intermap.txt'))
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
496
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
497