annotate MoinMoin/script/migration/_conv160.py @ 3175:2a3a6cb34e45

pylint findings: fix some harmless stuff
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Mon, 03 Mar 2008 02:02:55 +0100
parents 2a380f99afa8
children 3caaa8c74c41 a0ca2bd83cd3
rev   line source
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
1 # -*- coding: iso-8859-1 -*-
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
2 """
2730
a907a61c33b3 * The 1.5.8 to 1.6.0 converter now uses the 1.5.8 wiki parser, for better
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2610
diff changeset
3 MoinMoin - migration from 1.5.8 to 1.6.0 (creole link style)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
4
2805
f0e5329419f7 1.6 converter: kill */MoinEditorBackup pages (replaced by draft functionality), cleanup docstring
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2758
diff changeset
5 What it does:
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
6
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
7 a) reverse underscore == blank stuff in pagenames (introducing this was a fault)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
8
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
9 pagename quoted pagename
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
10 -----------------------------------------------------
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
11 old MainPage/Sub_Page MainPage(2f)Sub_Page
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
12 new MainPage/Sub Page MainPage(2f)Sub(20)Page or
2586
a838bdc4e40a 1.6 converter: quicklinks/subscribed pages (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2584
diff changeset
13 new MainPage/Sub_Page MainPage(2f)Sub_Page (user has to decide by editing rename1.txt)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
14
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
15
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
16 markup
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
17 ----------------------------------------------------
2753
94d4f531bcf6 fix reading of damaged event-log, fix some comments
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2752
diff changeset
18 old MoinMoin:MainPage/Sub_Page ../Sub_Page2
94d4f531bcf6 fix reading of damaged event-log, fix some comments
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2752
diff changeset
19 new [[MoinMoin:MainPage/Sub Page]] [[../Sub Page2]]
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
20
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
21
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
22 b) decode url encoded chars in attachment names (and quote the whole fname):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
23
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
24 markup
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
25 ----------------------------------------------------
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
26 old attachment:file%20with%20blanks.txt
2753
94d4f531bcf6 fix reading of damaged event-log, fix some comments
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2752
diff changeset
27 new [[attachment:file with blanks.txt]]
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
28
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
29 c) users: move bookmarks from separate files into user profile
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
30 d) users: generate new name[] for lists and name{} for dicts
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
31
2805
f0e5329419f7 1.6 converter: kill */MoinEditorBackup pages (replaced by draft functionality), cleanup docstring
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2758
diff changeset
32 e) kill all */MoinEditorBackup pages (replaced by drafts functionality)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
33
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
34 @copyright: 2007 by Thomas Waldmann
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
35 @license: GNU GPL, see COPYING for details.
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
36 """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
37
2599
13f0331f3a42 1.6 converter: add content conversion (unfinished), cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2593
diff changeset
38 import os.path
2751
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
39 import re
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
40 import time
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
41 import codecs, urllib, glob
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
42
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
43 from MoinMoin import config, wikiutil
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
44 from MoinMoin.script.migration.migutil import opj, listdir, copy_file, move_file, copy_dir
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
45
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
46 import mimetypes # this MUST be after wikiutil import!
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
47
2599
13f0331f3a42 1.6 converter: add content conversion (unfinished), cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2593
diff changeset
48 from _conv160_wiki import convert_wiki
13f0331f3a42 1.6 converter: add content conversion (unfinished), cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2593
diff changeset
49
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
50 create_rev = True # create a <new> rev with the converted content of <new-1> rev?
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
51
2604
27f06531a91b 1.6 converter: improve content conversion, give request obj to converter
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2602
diff changeset
52 def markup_converter(request, pagename, text, renames):
2602
b601db2e4d34 1.6 converter: improve content conversion, add test for it
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2599
diff changeset
53 """ Convert the <text> content of page <pagename>, using <renames> dict
b601db2e4d34 1.6 converter: improve content conversion, add test for it
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2599
diff changeset
54 to rename links correctly. Additionally, convert some changed markup.
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
55 """
3018
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
56 if text.startswith('<?xml'):
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
57 # would be done with xslt processor
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
58 return text
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
59
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
60 pis, body = wikiutil.get_processing_instructions(text)
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
61 for pi, val in pis:
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
62 if pi == 'format' and val != 'wiki':
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
63 # not wiki page
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
64 return text
2599
13f0331f3a42 1.6 converter: add content conversion (unfinished), cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2593
diff changeset
65
2604
27f06531a91b 1.6 converter: improve content conversion, give request obj to converter
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2602
diff changeset
66 text = convert_wiki(request, pagename, text, renames)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
67 return text
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
68
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
69
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
70 class EventLog:
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
71 def __init__(self, request, fname):
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
72 self.request = request
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
73 self.fname = fname
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
74 self.data = None
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
75 self.renames = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
76
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
77 def read(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
78 """ read complete event-log from disk """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
79 data = []
2751
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
80 try:
3119
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
81 lineno = 0
2751
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
82 f = file(self.fname, 'r')
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
83 for line in f:
3119
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
84 lineno += 1
2751
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
85 line = line.replace('\r', '').replace('\n', '')
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
86 if not line.strip(): # skip empty lines
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
87 continue
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
88 fields = line.split('\t')
3119
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
89 try:
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
90 timestamp, action, kvpairs = fields[:3]
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
91 timestamp = int(timestamp)
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
92 kvdict = wikiutil.parseQueryString(kvpairs)
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
93 data.append((timestamp, action, kvdict))
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
94 except ValueError, err:
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
95 # corrupt event log line, log error and skip it
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
96 print "Error: invalid event log (%s) line %d, err: %s, SKIPPING THIS LINE!" % (self.fname, lineno, str(err))
2751
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
97 f.close()
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
98 except IOError, err:
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
99 # no event-log
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
100 pass
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
101 self.data = data
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
102
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
103 def write(self, fname):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
104 """ write complete event-log to disk """
2751
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
105 if self.data:
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
106 f = file(fname, 'w')
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
107 for timestamp, action, kvdict in self.data:
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
108 pagename = kvdict.get('pagename')
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
109 if pagename and ('PAGE', pagename) in self.renames:
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
110 kvdict['pagename'] = self.renames[('PAGE', pagename)]
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
111 kvpairs = wikiutil.makeQueryString(kvdict, want_unicode=False)
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
112 fields = str(timestamp), action, kvpairs
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
113 line = '\t'.join(fields) + '\n'
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
114 f.write(line)
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
115 f.close()
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
116
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
117 def copy(self, destfname, renames):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
118 self.renames = renames
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
119 self.read()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
120 self.write(destfname)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
121
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
122
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
123 class EditLog:
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
124 def __init__(self, request, fname):
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
125 self.request = request
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
126 self.fname = fname
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
127 self.data = None
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
128 self.renames = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
129
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
130 def read(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
131 """ read complete edit-log from disk """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
132 data = {}
2751
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
133 try:
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
134 f = file(self.fname, 'r')
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
135 for line in f:
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
136 line = line.replace('\r', '').replace('\n', '')
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
137 if not line.strip(): # skip empty lines
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
138 continue
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
139 fields = line.split('\t') + [''] * 9
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
140 timestamp, rev, action, pagename, ip, hostname, userid, extra, comment = fields[:9]
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
141 timestamp = int(timestamp)
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
142 rev = int(rev)
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
143 pagename = wikiutil.unquoteWikiname(pagename)
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
144 data[(timestamp, rev, pagename)] = (timestamp, rev, action, pagename, ip, hostname, userid, extra, comment)
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
145 f.close()
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
146 except IOError, err:
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
147 # no edit-log
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
148 pass
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
149 self.data = data
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
150
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
151 def write(self, fname, deleted=False):
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
152 """ write complete edit-log to disk """
2751
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
153 if self.data:
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
154 editlog = self.data.items()
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
155 editlog.sort()
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
156 f = file(fname, "w")
3018
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
157 max_rev = 0
2751
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
158 for key, fields in editlog:
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
159 timestamp, rev, action, pagename, ip, hostname, userid, extra, comment = fields
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
160 if action.startswith('ATT'):
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
161 try:
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
162 fname = urllib.unquote(extra).decode('utf-8')
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
163 except UnicodeDecodeError:
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
164 fname = urllib.unquote(extra).decode('iso-8859-1')
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
165 if ('FILE', pagename, fname) in self.renames:
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
166 fname = self.renames[('FILE', pagename, fname)]
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
167 extra = urllib.quote(fname.encode('utf-8'))
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
168 if ('PAGE', pagename) in self.renames:
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
169 pagename = self.renames[('PAGE', pagename)]
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
170 timestamp = str(timestamp)
3018
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
171 if rev != 99999999:
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
172 max_rev = max(rev, max_rev)
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
173 revstr = '%08d' % rev
2751
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
174 pagename = wikiutil.quoteWikinameFS(pagename)
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
175 fields = timestamp, revstr, action, pagename, ip, hostname, userid, extra, comment
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
176 log_str = '\t'.join(fields) + '\n'
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
177 f.write(log_str)
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
178 if create_rev and not deleted:
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
179 timestamp = str(wikiutil.timestamp2version(time.time()))
3018
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
180 revstr = '%08d' % (max_rev + 1)
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
181 action = 'SAVE'
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
182 ip = '127.0.0.1'
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
183 hostname = 'localhost'
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
184 userid = ''
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
185 extra = ''
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
186 comment = "converted to 1.6 markup"
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
187 fields = timestamp, revstr, action, pagename, ip, hostname, userid, extra, comment
2751
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
188 log_str = '\t'.join(fields) + '\n'
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
189 f.write(log_str)
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
190 f.close()
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
191
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
192 def copy(self, destfname, renames, deleted=False):
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
193 self.renames = renames
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
194 self.read()
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
195 self.write(destfname, deleted)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
196
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
197
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
198 class PageRev:
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
199 """ a single revision of a page """
2602
b601db2e4d34 1.6 converter: improve content conversion, add test for it
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2599
diff changeset
200 def __init__(self, request, pagename, rev_dir, rev):
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
201 self.request = request
2602
b601db2e4d34 1.6 converter: improve content conversion, add test for it
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2599
diff changeset
202 self.pagename = pagename
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
203 self.rev_dir = rev_dir
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
204 self.rev = rev
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
205
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
206 def read(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
207 fname = opj(self.rev_dir, '%08d' % self.rev)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
208 f = file(fname, "rb")
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
209 data = f.read()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
210 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
211 data = data.decode(config.charset)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
212 return data
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
213
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
214 def write(self, data, rev_dir, convert, rev=None):
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
215 if rev is None:
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
216 rev = self.rev
2610
ffe8c2d9a4ba 1.6 converter: only convert current rev of a page, added commented failing test cases
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2604
diff changeset
217 if convert:
ffe8c2d9a4ba 1.6 converter: only convert current rev of a page, added commented failing test cases
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2604
diff changeset
218 data = markup_converter(self.request, self.pagename, data, self.renames)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
219 fname = opj(rev_dir, '%08d' % rev)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
220 data = data.encode(config.charset)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
221 f = file(fname, "wb")
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
222 f.write(data)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
223 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
224
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
225 def copy(self, rev_dir, renames, convert=False, new_rev=None):
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
226 self.renames = renames
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
227 data = self.read()
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
228 self.write(data, rev_dir, convert, new_rev)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
229
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
230
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
231 class Attachment:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
232 """ a single attachment """
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
233 def __init__(self, request, attach_dir, attfile):
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
234 self.request = request
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
235 self.path = opj(attach_dir, attfile)
3119
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
236 self.name = attfile.decode('utf-8', 'replace')
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
237
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
238 def copy(self, attach_dir):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
239 """ copy attachment file from orig path to new destination """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
240 attfile = self.name.encode('utf-8')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
241 dest = opj(attach_dir, attfile)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
242 copy_file(self.path, dest)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
243
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
244
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
245 class Page:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
246 """ represents a page with all related data """
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
247 def __init__(self, request, pages_dir, qpagename):
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
248 self.request = request
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
249 self.name = wikiutil.unquoteWikiname(qpagename)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
250 self.name_old = self.name # renaming: still original name when self.name has the new name
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
251 self.page_dir = opj(pages_dir, qpagename)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
252 self.current = None # int current
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
253 self.editlog = None # dict (see read_editlog)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
254 self.revlist = None # list of ints (page text revisions)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
255 self.revisions = None # dict int: pagerev obj
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
256 self.attachments = None # dict of unicode fname: full path
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
257 self.renames = {} # info for renaming pages/attachments
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
258
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
259 def read(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
260 """ read a page, including revisions, log, attachments from disk """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
261 page_dir = self.page_dir
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
262 # read current file
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
263 current_fname = opj(page_dir, 'current')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
264 if os.path.exists(current_fname):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
265 current_file = file(current_fname, "r")
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
266 current_rev = current_file.read()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
267 current_file.close()
3018
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
268 try:
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
269 self.current = int(current_rev)
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
270 except ValueError:
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
271 print "Error: invalid current file %s, SKIPPING THIS PAGE!" % current_fname
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
272 return
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
273 # read edit-log
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
274 editlog_fname = opj(page_dir, 'edit-log')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
275 if os.path.exists(editlog_fname):
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
276 self.editlog = EditLog(self.request, editlog_fname)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
277 # read page revisions
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
278 rev_dir = opj(page_dir, 'revisions')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
279 if os.path.exists(rev_dir):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
280 revlist = listdir(rev_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
281 revlist = [int(rev) for rev in revlist]
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
282 revlist.sort()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
283 self.revlist = revlist
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
284 self.revisions = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
285 for rev in revlist:
2602
b601db2e4d34 1.6 converter: improve content conversion, add test for it
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2599
diff changeset
286 self.revisions[rev] = PageRev(self.request, self.name_old, rev_dir, rev)
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
287 # set deleted status
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
288 self.is_deleted = not self.revisions or self.current not in self.revisions
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
289 # read attachment filenames
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
290 attach_dir = opj(page_dir, 'attachments')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
291 if os.path.exists(attach_dir):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
292 self.attachments = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
293 attlist = listdir(attach_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
294 for attfile in attlist:
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
295 a = Attachment(self.request, attach_dir, attfile)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
296 self.attachments[a.name] = a
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
297
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
298 def write(self, pages_dir):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
299 """ write a page, including revisions, log, attachments to disk """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
300 if ('PAGE', self.name) in self.renames:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
301 name_new = self.renames[('PAGE', self.name)]
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
302 if name_new != self.name:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
303 print "Renaming page %r -> %r" % (self.name, name_new)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
304 self.name_old = self.name
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
305 self.name = name_new
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
306 qpagename = wikiutil.quoteWikinameFS(self.name)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
307 page_dir = opj(pages_dir, qpagename)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
308 os.makedirs(page_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
309 # write current file
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
310 current = self.current
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
311 if current is not None:
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
312 if create_rev and not self.is_deleted:
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
313 current += 1
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
314 current_fname = opj(page_dir, 'current')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
315 current_file = file(current_fname, "w")
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
316 current_str = '%08d\n' % current
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
317 current_file.write(current_str)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
318 current_file.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
319 # copy edit-log
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
320 if self.editlog is not None:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
321 editlog_fname = opj(page_dir, 'edit-log')
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
322 self.editlog.copy(editlog_fname, self.renames, deleted=self.is_deleted)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
323 # copy page revisions
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
324 if self.revisions is not None:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
325 rev_dir = opj(page_dir, 'revisions')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
326 os.makedirs(rev_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
327 for rev in self.revlist:
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
328 if create_rev:
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
329 self.revisions[rev].copy(rev_dir, self.renames)
2610
ffe8c2d9a4ba 1.6 converter: only convert current rev of a page, added commented failing test cases
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2604
diff changeset
330 else:
2758
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
331 if int(rev) == self.current:
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
332 self.revisions[rev].copy(rev_dir, self.renames, convert=True)
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
333 else:
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
334 self.revisions[rev].copy(rev_dir, self.renames)
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
335 if create_rev and not self.is_deleted:
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
336 self.revisions[rev].copy(rev_dir, self.renames, convert=True, new_rev=rev+1)
b85e4879fdea 1.6 converter: create new page revisions with conversion result, handle deleted pages
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2753
diff changeset
337
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
338 # copy attachments
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
339 if self.attachments is not None:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
340 attach_dir = opj(page_dir, 'attachments')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
341 os.makedirs(attach_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
342 for fn, att in self.attachments.items():
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
343 # we have to check for renames here because we need the (old) pagename, too:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
344 if ('FILE', self.name_old, fn) in self.renames:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
345 fn_new = self.renames[('FILE', self.name_old, fn)]
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
346 if fn_new != fn:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
347 print "Renaming file %r %r -> %r" % (self.name_old, fn, fn_new)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
348 att.name = fn_new
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
349 att.copy(attach_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
350
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
351 def copy(self, pages_dir, renames):
3175
2a3a6cb34e45 pylint findings: fix some harmless stuff
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3119
diff changeset
352 self.renames = renames
2a3a6cb34e45 pylint findings: fix some harmless stuff
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3119
diff changeset
353 self.read()
2a3a6cb34e45 pylint findings: fix some harmless stuff
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3119
diff changeset
354 self.write(pages_dir)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
355
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
356
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
357 class User:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
358 """ represents a user with all related data """
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
359 def __init__(self, request, users_dir, uid):
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
360 self.request = request
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
361 self.uid = uid
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
362 self.users_dir = users_dir
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
363 self.profile = None
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
364 self.bookmarks = None
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
365
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
366 def read(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
367 """ read profile and bookmarks data from disk """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
368 self.profile = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
369 fname = opj(self.users_dir, self.uid)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
370 # read user profile
2593
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
371 f = codecs.open(fname, 'r', config.charset)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
372 for line in f:
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
373 line = line.replace(u'\r', '').replace(u'\n', '')
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
374 if not line.strip() or line.startswith(u'#'): # skip empty or comment lines
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
375 continue
3018
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
376 try:
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
377 key, value = line.split(u'=', 1)
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
378 except Exception, err:
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
379 print "Error: User reader can not parse line %r from profile %r (%s)" % (line, fname, str(err))
5dfd26496da8 copied mig scripts from 1.6 branch
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2807
diff changeset
380 continue
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
381 self.profile[key] = value
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
382 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
383 # read bookmarks
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
384 self.bookmarks = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
385 fname_pattern = opj(self.users_dir, "%s.*.bookmark" % self.uid)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
386 for fname in glob.glob(fname_pattern):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
387 f = file(fname, "r")
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
388 bookmark = f.read()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
389 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
390 wiki = fname.replace('.bookmark', '').replace(opj(self.users_dir, self.uid+'.'), '')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
391 self.bookmarks[wiki] = int(bookmark)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
392 # don't care about trail
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
393
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
394 def write(self, users_dir):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
395 """ write profile and bookmarks data to disk """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
396 fname = opj(users_dir, self.uid)
2593
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
397 f = codecs.open(fname, 'w', config.charset)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
398 for key, value in self.profile.items():
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
399 if key in (u'subscribed_pages', u'quicklinks'):
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
400 pages = value.split(u'\t')
2586
a838bdc4e40a 1.6 converter: quicklinks/subscribed pages (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2584
diff changeset
401 for i in range(len(pages)):
a838bdc4e40a 1.6 converter: quicklinks/subscribed pages (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2584
diff changeset
402 pagename = pages[i]
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
403 try:
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
404 interwiki, pagename = pagename.split(u':', 1)
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
405 except:
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
406 interwiki, pagename = u'Self', pagename
2593
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
407 if interwiki == u'Self' or interwiki == self.request.cfg.interwikiname:
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
408 if ('PAGE', pagename) in self.renames:
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
409 pagename = self.renames[('PAGE', pagename)]
b609934451e6 1.6 converter: use wikiutil, fixes, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2592
diff changeset
410 pages[i] = u'%s:%s' % (interwiki, pagename)
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
411 key += '[]' # we have lists here
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
412 value = u'\t'.join(pages)
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
413 f.write(u"%s=%s\n" % (key, value))
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
414 else:
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
415 f.write(u"%s=%s\n" % (key, value))
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
416 bookmark_entries = [u'%s:%s' % item for item in self.bookmarks.items()]
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
417 key = u"bookmarks{}"
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
418 value = u'\t'.join(bookmark_entries)
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
419 f.write(u"%s=%s\n" % (key, value))
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
420 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
421 # don't care about trail
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
422
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
423 def copy(self, users_dir, renames):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
424 self.renames = renames
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
425 self.read()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
426 self.write(users_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
427
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
428
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
429 class DataConverter(object):
2589
2815c91635ff 1.6 converter: improve user profile conversion
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2586
diff changeset
430 def __init__(self, request, src_data_dir, dest_data_dir):
2592
a6216260f164 1.6 converter: more usage of codecs, cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2591
diff changeset
431 self.request = request
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
432 self.sdata = src_data_dir
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
433 self.ddata = dest_data_dir
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
434 self.pages = {}
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
435 self.users = {}
2752
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
436 self.complete = {}
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
437 self.renames = {}
2752
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
438 self.complete_fname = opj(self.sdata, 'complete.txt')
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
439 self.rename_fname1 = opj(self.sdata, 'rename1.txt')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
440 self.rename_fname2 = opj(self.sdata, 'rename2.txt')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
441
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
442 def pass1(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
443 """ First create the rename list - the user has to review/edit it as
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
444 we can't decide about page/attachment names automatically.
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
445 """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
446 self.read_src()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
447 # pages
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
448 for pn, p in self.pages.items():
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
449 p.read()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
450 if not p.revisions:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
451 continue # we don't care for pages with no revisions (trash)
2805
f0e5329419f7 1.6 converter: kill */MoinEditorBackup pages (replaced by draft functionality), cleanup docstring
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2758
diff changeset
452 if pn.endswith('/MoinEditorBackup'):
f0e5329419f7 1.6 converter: kill */MoinEditorBackup pages (replaced by draft functionality), cleanup docstring
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2758
diff changeset
453 continue # we don't care for old editor backups
2752
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
454 self.complete[('PAGE', pn)] = None
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
455 if "_" in pn:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
456 # log all pagenames with underscores
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
457 self.renames[('PAGE', pn)] = None
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
458 if p.attachments is not None:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
459 for fn in p.attachments:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
460 try:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
461 fn_str = fn.encode('ascii')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
462 log = False # pure ascii filenames are no problem
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
463 except UnicodeEncodeError:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
464 log = True # this file maybe has a strange representation in wiki markup
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
465 else:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
466 if ' ' in fn_str or '%' in fn_str: # files with blanks need quoting
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
467 log = True
2752
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
468 self.complete[('FILE', pn, fn)] = None
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
469 if log:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
470 # log all strange attachment filenames
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
471 fn_str = fn.encode('utf-8')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
472 self.renames[('FILE', pn, fn)] = None
2752
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
473 self.save_list(self.complete_fname, self.complete)
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
474 self.save_list(self.rename_fname1, self.renames)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
475
2752
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
476 LIST_FIELDSEP = u'|' # in case | makes trouble, one can use \t tab char
2750
f12b39c07405 1.6 converter: make RENAMES_FIELDSEP easily changeable
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2730
diff changeset
477
2752
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
478 def save_list(self, fname, what):
2806
f86a1c914069 1.6 converter: sort rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2805
diff changeset
479 what_sorted = what.keys()
3119
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
480 # make sure we have 3-tuples:
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
481 what_sorted = [(k + (None, ))[:3] for k in what_sorted]
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
482 # we only have python 2.3, thus no cmp keyword for the sort() call,
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
483 # thus we need to do it the more complicated way:
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
484 what_sorted = [(pn, fn, rtype) for rtype, pn, fn in what_sorted] # shuffle
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
485 what_sorted.sort() # sort
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
486 what_sorted = [(rtype, pn, fn) for pn, fn, rtype in what_sorted] # shuffle
2752
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
487 f = codecs.open(fname, 'w', 'utf-8')
3119
2a380f99afa8 migration scripts: copied from 1.6 as of rev 2560:e2d1e0285eda
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3018
diff changeset
488 for rtype, pn, fn in what_sorted:
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
489 if rtype == 'PAGE':
2750
f12b39c07405 1.6 converter: make RENAMES_FIELDSEP easily changeable
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2730
diff changeset
490 line = (rtype, pn, pn)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
491 elif rtype == 'FILE':
2750
f12b39c07405 1.6 converter: make RENAMES_FIELDSEP easily changeable
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2730
diff changeset
492 line = (rtype, pn, fn, fn)
2752
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
493 line = self.LIST_FIELDSEP.join(line)
2750
f12b39c07405 1.6 converter: make RENAMES_FIELDSEP easily changeable
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2730
diff changeset
494 f.write(line + u'\n')
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
495 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
496
2752
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
497 def load_list(self, fname, what):
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
498 f = codecs.open(fname, 'r', 'utf-8')
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
499 for line in f:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
500 line = line.rstrip()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
501 if not line:
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
502 continue
2752
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
503 t = line.split(self.LIST_FIELDSEP)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
504 rtype, p1, p2, p3 = (t + [None]*3)[:4]
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
505 if rtype == u'PAGE':
2752
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
506 what[(str(rtype), p1)] = p2
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
507 elif rtype == u'FILE':
2752
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
508 what[(str(rtype), p1, p2)] = p3
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
509 f.close()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
510
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
511 def pass2(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
512 """ Second, read the (user edited) rename list and do the renamings everywhere. """
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
513 self.read_src()
2752
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
514 #self.load_list(self.complete_fname, self.complete)
7f6093028a66 1.6 converter: create complete.txt (for reference, not used yet), use 1.5 underscore magic when looking up pages in rename map
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2751
diff changeset
515 self.load_list(self.rename_fname2, self.renames)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
516 self.write_dest()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
517
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
518 def read_src(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
519 # create Page objects in memory
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
520 pages_dir = opj(self.sdata, 'pages')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
521 pagelist = listdir(pages_dir)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
522 for qpagename in pagelist:
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
523 p = Page(self.request, pages_dir, qpagename)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
524 self.pages[p.name] = p
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
525
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
526 # create User objects in memory
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
527 users_dir = opj(self.sdata, 'user')
2751
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
528 user_re = re.compile(r'^\d+\.\d+(\.\d+)?$')
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
529 userlist = listdir(users_dir)
2751
7a6b5f0c92c9 1.6 converter: better error handling, better matching of user profiles
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2750
diff changeset
530 userlist = [f for f in userlist if user_re.match(f)]
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
531 for userid in userlist:
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
532 u = User(self.request, users_dir, userid)
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
533 self.users[u.uid] = u
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
534
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
535 # create log objects in memory
2590
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
536 self.editlog = EditLog(self.request, opj(self.sdata, 'edit-log'))
20638e764560 1.6 converter: have request object everywhere
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2589
diff changeset
537 self.eventlog = EventLog(self.request, opj(self.sdata, 'event-log'))
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
538
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
539 def write_dest(self):
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
540 self.init_dest()
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
541 # copy pages
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
542 pages_dir = opj(self.ddata, 'pages')
2805
f0e5329419f7 1.6 converter: kill */MoinEditorBackup pages (replaced by draft functionality), cleanup docstring
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2758
diff changeset
543 for pn, page in self.pages.items():
f0e5329419f7 1.6 converter: kill */MoinEditorBackup pages (replaced by draft functionality), cleanup docstring
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2758
diff changeset
544 if pn.endswith('/MoinEditorBackup'):
f0e5329419f7 1.6 converter: kill */MoinEditorBackup pages (replaced by draft functionality), cleanup docstring
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2758
diff changeset
545 continue # we don't care for old editor backups
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
546 page.copy(pages_dir, self.renames)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
547
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
548 # copy users
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
549 users_dir = opj(self.ddata, 'user')
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
550 for user in self.users.values():
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
551 user.copy(users_dir, self.renames)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
552
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
553 # copy logs
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
554 self.editlog.copy(opj(self.ddata, 'edit-log'), self.renames)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
555 self.eventlog.copy(opj(self.ddata, 'event-log'), self.renames)
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
556
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
557 def init_dest(self):
2591
99d1516223f7 1.6 converter: move cache dir with locks where the framework still expects it
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2590
diff changeset
558 try:
99d1516223f7 1.6 converter: move cache dir with locks where the framework still expects it
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2590
diff changeset
559 os.makedirs(self.ddata)
99d1516223f7 1.6 converter: move cache dir with locks where the framework still expects it
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2590
diff changeset
560 except:
99d1516223f7 1.6 converter: move cache dir with locks where the framework still expects it
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 2590
diff changeset
561 pass
2583
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
562 os.makedirs(opj(self.ddata, 'pages'))
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
563 os.makedirs(opj(self.ddata, 'user'))
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
564 copy_dir(opj(self.sdata, 'plugin'), opj(self.ddata, 'plugin'))
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
565 copy_file(opj(self.sdata, 'intermap.txt'), opj(self.ddata, 'intermap.txt'))
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
566
dd005fd66306 1.5.x to 1.6 conversion (unfinished)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
567