Mercurial > moin > 1.9

annotate MoinMoin/action/cache.py @ 3897:78c452cf0257

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
fix tests, PEP8
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Sun, 20 Jul 2008 18:30:51 +0200
parents 85c7b3b9c48c
children 0f86861f1adb
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
1 # -*- coding: iso-8859-1 -*-
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
2 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
3 MoinMoin - Send a raw object from the caching system (and offer utility
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
4 functions to put data into cache, calculate cache key, etc.).
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
5
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
6 Sample usage
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
7 ------------
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
8 Assume we have a big picture (bigpic) and we want to efficiently show some
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
9 thumbnail (thumbpic) for it:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
10
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
11 # first calculate a (hard to guess) cache key (this key will change if the
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
12 # original data (bigpic) changes):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
13 key = cache.key(..., attachname=bigpic, ...)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
14
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
15 # check if we don't have it in cache yet
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
16 if not cache.exists(..., key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
17 # if we don't have it in cache, we need to render it - this is an
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
18 # expensive operation that we want to avoid by caching:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
19 thumbpic = render_thumb(bigpic)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
20 # put expensive operation's results into cache:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
21 cache.put(..., key, thumbpic, ...)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
22
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
23 url = cache.url(..., key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
24 html = '<img src="%s">' % url
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
25
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
26 @copyright: 2008 MoinMoin:ThomasWaldmann
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
27 @license: GNU GPL, see COPYING for details.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
28 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
29
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
30 import hmac, sha
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
31
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
32 from MoinMoin import log
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
33 logging = log.getLogger(__name__)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
34
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
35 # keep both imports below as they are, order is important:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
36 from MoinMoin import wikiutil
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
37 import mimetypes
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
38
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
39 from MoinMoin import config, caching
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
40 from MoinMoin.util import filesys
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
41 from MoinMoin.action import AttachFile
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
42
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
43 action_name = __name__.split('.')[-1]
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
44
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
45 # Do NOT get this directly from request.form or user would be able to read any cache!
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
46 cache_arena = 'sendcache' # just using action_name is maybe rather confusing
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
47
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
48 # We maybe could use page local caching (not 'wiki' global) to have less directory entries.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
49 # Local is easier to automatically cleanup if an item changes. Global is easier to manually cleanup.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
50 # Local makes data_dir much larger, harder to backup.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
51 cache_scope = 'wiki'
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
52
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
53 do_locking = False
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
54
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
55 def key(request, wikiname=None, itemname=None, attachname=None, content=None, secret=None):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
56 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
57 Calculate a (hard-to-guess) cache key.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
58
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
59 Important key properties:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
60 * The key must be hard to guess (this is because do=get does no ACL checks,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
61 so whoever got the key [e.g. from html rendering of an ACL protected wiki
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
62 page], will be able to see the cached content.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
63 * The key must change if the (original) content changes. This is because
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
64 ACLs on some item may change and even if somebody was allowed to see some
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
65 revision of some item, it does not implicate that he is allowed to see
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
66 any other revision also. There will be no harm if he can see exactly the
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
67 same content again, but there could be harm if he could access a revision
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
68 with different content.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
69
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
70 If content is supplied, we will calculate and return a hMAC of the content.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
71
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
72 If wikiname, itemname, attachname is given, we don't touch the content (nor do
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
73 we read it ourselves from the attachment file), but we just calculate a key
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
74 from the given metadata values and some metadata we get from the filesystem.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
75
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
76 Hint: if you need multiple cache objects for the same source content (e.g.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
77 thumbnails of different sizes for the same image), calculate the key
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
78 only once and then add some different prefixes to it to get the final
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
79 cache keys.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
80
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
81 @param request: the request object
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
82 @param wikiname: the name of the wiki (if not given, will be read from cfg)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
83 @param itemname: the name of the page
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
84 @param attachname: the filename of the attachment
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
85 @param content: content data as unicode object (e.g. for page content or
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
86 parser section content)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
87 @param secret: secret for hMAC calculation (default: use secret from cfg)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
88 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
89 if secret is None:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
90 secret = request.cfg.secrets
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
91 if content:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
92 hmac_data = content
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
93 elif itemname is not None and attachname is not None:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
94 wikiname = wikiname or request.cfg.interwikiname or request.cfg.siteid
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
95 fuid = filesys.fuid(AttachFile.getFilename(request, itemname, attachname))
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
96 hmac_data = u''.join([wikiname, itemname, attachname, repr(fuid)])
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
97 else:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
98 raise AssertionError('cache_key called with unsupported parameters')
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
99
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
100 hmac_data = hmac_data.encode('utf-8')
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
101 key = hmac.new(secret, hmac_data, sha).hexdigest()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
102 return key
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
103
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
104
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
105 def put(request, key, data,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
106 filename=None,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
107 content_type=None,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
108 content_disposition=None,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
109 content_length=None,
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
110 last_modified=None,
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
111 original=None):
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
112 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
113 Put an object into the cache to send it with cache action later.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
114
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
115 @param request: the request object
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
116 @param key: non-guessable key into cache (str)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
117 @param data: content data (str or open file-like obj)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
118 @param filename: filename for content-disposition header and for autodetecting
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
119 content_type (unicode, default: None)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
120 @param content_type: content-type header value (str, default: autodetect from filename)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
121 @param content_disposition: type for content-disposition header (str, default: None)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
122 @param content_length: data length for content-length header (int, default: autodetect)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
123 @param last_modified: last modified timestamp (int, default: autodetect)
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
124 @param original: location of original object (default: None) - this is just written to
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
125 the metadata cache "as is" and could be used for cache cleanup,
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
126 use (wikiname, itemname, attachname or None))
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
127 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
128 import os.path
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
129 from MoinMoin.util import timefuncs
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
130
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
131 if filename:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
132 # make sure we just have a simple filename (without path)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
133 filename = os.path.basename(filename)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
134
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
135 if content_type is None:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
136 # try autodetect
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
137 mt, enc = mimetypes.guess_type(filename)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
138 if mt:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
139 content_type = mt
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
140
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
141 if content_type is None:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
142 content_type = 'application/octet-stream'
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
143
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
144 data_cache = caching.CacheEntry(request, cache_arena, key+'.data', cache_scope, do_locking=do_locking)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
145 data_cache.update(data)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
146 content_length = content_length or data_cache.size()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
147 last_modified = last_modified or data_cache.mtime()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
148
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
149 httpdate_last_modified = timefuncs.formathttpdate(int(last_modified))
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
150 headers = ['Content-Type: %s' % content_type,
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
151 'Last-Modified: %s' % httpdate_last_modified,
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
152 'Content-Length: %s' % content_length,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
153 ]
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
154 if content_disposition and filename:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
155 # TODO: fix the encoding here, plain 8 bit is not allowed according to the RFCs
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
156 # There is no solution that is compatible to IE except stripping non-ascii chars
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
157 filename = filename.encode(config.charset)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
158 headers.append('Content-Disposition: %s; filename="%s"' % (content_disposition, filename))
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
159
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
160 meta_cache = caching.CacheEntry(request, cache_arena, key+'.meta', cache_scope, do_locking=do_locking, use_pickle=True)
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
161 meta_cache.update({
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
162 'httpdate_last_modified': httpdate_last_modified,
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
163 'last_modified': last_modified,
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
164 'headers': headers,
3897
78c452cf0257 fix tests, PEP8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3892
diff changeset
165 'original': original,
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
166 })
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
167
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
168
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
169 def exists(request, key, strict=False):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
170 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
171 Check if a cached object for this key exists.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
172
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
173 @param request: the request object
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
174 @param key: non-guessable key into cache (str)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
175 @param strict: if True, also check the data cache, not only meta (bool, default: False)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
176 @return: is object cached? (bool)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
177 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
178 if strict:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
179 data_cache = caching.CacheEntry(request, cache_arena, key+'.data', cache_scope, do_locking=do_locking)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
180 data_cached = data_cache.exists()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
181 else:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
182 data_cached = True # we assume data will be there if meta is there
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
183
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
184 meta_cache = caching.CacheEntry(request, cache_arena, key+'.meta', cache_scope, do_locking=do_locking, use_pickle=True)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
185 meta_cached = meta_cache.exists()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
186
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
187 return meta_cached and data_cached
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
188
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
189
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
190 def remove(request, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
191 """ delete headers/data cache for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
192 meta_cache = caching.CacheEntry(request, cache_arena, key+'.meta', cache_scope, do_locking=do_locking, use_pickle=True)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
193 meta_cache.remove()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
194 data_cache = caching.CacheEntry(request, cache_arena, key+'.data', cache_scope, do_locking=do_locking)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
195 data_cache.remove()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
196
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
197
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
198 def url(request, key, do='get'):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
199 """ return URL for the object cached for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
200 return "%s/?%s" % (
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
201 request.getScriptname(),
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
202 wikiutil.makeQueryString(dict(action=action_name, do=do, key=key), want_unicode=False))
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
203
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
204
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
205 def _get_headers(request, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
206 """ get last_modified and headers cached for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
207 meta_cache = caching.CacheEntry(request, cache_arena, key+'.meta', cache_scope, do_locking=do_locking, use_pickle=True)
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
208 meta = meta_cache.content()
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
209 return meta['httpdate_last_modified'], meta['headers']
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
210
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
211
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
212 def _get_datafile(request, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
213 """ get an open data file for the data cached for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
214 data_cache = caching.CacheEntry(request, cache_arena, key+'.data', cache_scope, do_locking=do_locking)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
215 data_cache.open(mode='r')
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
216 return data_cache
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
217
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
218
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
219 def _do_get(request, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
220 """ send a complete http response with headers/data cached for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
221 try:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
222 last_modified, headers = _get_headers(request, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
223 if request.if_modified_since == last_modified:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
224 request.emit_http_headers(["Status: 304 Not modified"])
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
225 else:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
226 data_file = _get_datafile(request, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
227 request.emit_http_headers(headers)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
228 request.send_file(data_file)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
229 except caching.CacheError:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
230 request.emit_http_headers(["Status: 404 Not found"])
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
231
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
232
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
233 def _do_remove(request, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
234 """ delete headers/data cache for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
235 remove(request, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
236 request.emit_http_headers(["Status: 200 OK"])
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
237
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
238
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
239 def _do(request, do, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
240 if do == 'get':
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
241 _do_get(request, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
242 elif do == 'remove':
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
243 _do_remove(request, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
244
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
245 def execute(pagename, request):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
246 do = request.form.get('do', [None])[0]
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
247 key = request.form.get('key', [None])[0]
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
248 _do(request, do, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
249