annotate MoinMoin/action/pollsistersites.py @ 4109:e88baf535b48

fix backup action configuration (broke on windows due to backslashes in e.g. cache_dir), try 2. cfg.backup_exclude is now just a function of filename, telling whether the file should be excluded. By default, no file is excluded.
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Fri, 19 Sep 2008 21:41:40 +0200
parents 01f05e74aa9c
children 85884c67228d
rev   line source
1787
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
1 # -*- coding: iso-8859-1 -*-
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
2 """
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
3 MoinMoin - "pollsistersites" action
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
4
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
5 This action fetches lists of page urls and page names from sister sites,
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
6 so we can implement SisterWiki functionality.
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
7 See: http://usemod.com/cgi-bin/mb.pl?SisterSitesImplementationGuide
2286
01f05e74aa9c Big PEP8 and whitespace cleanup
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1959
diff changeset
8
1918
bb2e053067fb fixing copyright headers: remove umlauts (encoding troubles), make epydoc compatible, reformat
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1790
diff changeset
9 @copyright: 2007 MoinMoin:ThomasWaldmann
1787
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
10 @license: GNU GPL, see COPYING for details.
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
11 """
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
12
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
13 import time, urllib
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
14
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
15 from MoinMoin import caching
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
16 from MoinMoin.util import timefuncs
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
17
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
18 def execute(pagename, request):
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
19 status = []
1959
7ccf35e8f674 use a list to have defined order for cfg.sistersites (e.g. in theme output)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1918
diff changeset
20 for sistername, sisterurl in request.cfg.sistersites:
1787
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
21 arena = 'sisters'
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
22 key = sistername
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
23 cache = caching.CacheEntry(request, arena, key, scope='farm', use_pickle=True)
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
24 if cache.exists():
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
25 data = cache.content()
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
26 else:
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
27 data = {'lastmod': ''}
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
28 uo = urllib.URLopener()
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
29 uo.version = 'MoinMoin SisterPage list fetcher 1.0'
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
30 lastmod = data['lastmod']
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
31 if lastmod:
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
32 uo.addheader('If-Modified-Since', lastmod)
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
33 try:
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
34 sisterpages = {}
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
35 f = uo.open(sisterurl)
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
36 for line in f:
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
37 line = line.strip()
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
38 try:
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
39 page_url, page_name = line.split(' ', 1)
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
40 sisterpages[page_name.decode('utf-8')] = page_url
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
41 except:
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
42 pass # ignore invalid lines
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
43 try:
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
44 lastmod = f.info()["Last-Modified"]
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
45 except:
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
46 lastmod = timefuncs.formathttpdate(time.time())
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
47 f.close()
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
48 data['lastmod'] = lastmod
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
49 data['sisterpages'] = sisterpages
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
50 cache.update(data)
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
51 status.append(u"Site: %s Status: Updated. Pages: %d" % (sistername, len(sisterpages)))
1790
abe8fa4a13b2 pollsistersites: workaround for python 2.5 bug
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1787
diff changeset
52 except IOError, (title, code, msg, headers): # code e.g. 304
abe8fa4a13b2 pollsistersites: workaround for python 2.5 bug
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1787
diff changeset
53 status.append(u"Site: %s Status: Not updated." % sistername)
abe8fa4a13b2 pollsistersites: workaround for python 2.5 bug
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 1787
diff changeset
54 except TypeError: # catch bug in python 2.5: "EnvironmentError expected at most 3 arguments, got 4"
1787
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
55 status.append(u"Site: %s Status: Not updated." % sistername)
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
56
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
57 request.emit_http_headers(["Content-Type: text/plain; charset=UTF-8"])
f4a941fe32f8 added SisterSites/SisterPages support
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
58 request.write("\r\n".join(status).encode("utf-8"))