Mercurial > moin > 1.9

annotate MoinMoin/action/cache.py @ 6103:500f68d3e2fd

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
remove our own usage of python_compatibility module
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Tue, 06 Sep 2016 01:00:25 +0200
parents d73431c6d2c9
children 561b7a9c2bd9
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
1 # -*- coding: iso-8859-1 -*-
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
2 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
3 MoinMoin - Send a raw object from the caching system (and offer utility
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
4 functions to put data into cache, calculate cache key, etc.).
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
5
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
6 Sample usage
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
7 ------------
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
8 Assume we have a big picture (bigpic) and we want to efficiently show some
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
9 thumbnail (thumbpic) for it:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
10
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
11 # first calculate a (hard to guess) cache key (this key will change if the
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
12 # original data (bigpic) changes):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
13 key = cache.key(..., attachname=bigpic, ...)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
14
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
15 # check if we don't have it in cache yet
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
16 if not cache.exists(..., key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
17 # if we don't have it in cache, we need to render it - this is an
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
18 # expensive operation that we want to avoid by caching:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
19 thumbpic = render_thumb(bigpic)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
20 # put expensive operation's results into cache:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
21 cache.put(..., key, thumbpic, ...)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
22
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
23 url = cache.url(..., key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
24 html = '<img src="%s">' % url
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
25
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
26 @copyright: 2008 MoinMoin:ThomasWaldmann
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
27 @license: GNU GPL, see COPYING for details.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
28 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
29
5945
d73431c6d2c9 fix 304 http status for cache action, see MoinMoinBugs/CacheActionComparesTimestampsIncorrectly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5591
diff changeset
30 from datetime import datetime
6103
500f68d3e2fd remove our own usage of python_compatibility module
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5945
diff changeset
31 import hmac
5945
d73431c6d2c9 fix 304 http status for cache action, see MoinMoinBugs/CacheActionComparesTimestampsIncorrectly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5591
diff changeset
32
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
33 from MoinMoin import log
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
34 logging = log.getLogger(__name__)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
35
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
36 # keep both imports below as they are, order is important:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
37 from MoinMoin import wikiutil
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
38 import mimetypes
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
39
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
40 from MoinMoin import config, caching
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
41 from MoinMoin.util import filesys
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
42 from MoinMoin.action import AttachFile
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
43
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
44 action_name = __name__.split('.')[-1]
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
45
4424
5ad5753ae311 pre-1.9: request.form has qs args and post data, 1.9: .form only post data, .args only qs args, .values both
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4376
diff changeset
46 # Do NOT get this directly from request.values or user would be able to read any cache!
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
47 cache_arena = 'sendcache' # just using action_name is maybe rather confusing
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
48
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
49 # We maybe could use page local caching (not 'wiki' global) to have less directory entries.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
50 # Local is easier to automatically cleanup if an item changes. Global is easier to manually cleanup.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
51 # Local makes data_dir much larger, harder to backup.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
52 cache_scope = 'wiki'
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
53
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
54 do_locking = False
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
55
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
56 def key(request, wikiname=None, itemname=None, attachname=None, content=None, secret=None):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
57 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
58 Calculate a (hard-to-guess) cache key.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
59
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
60 Important key properties:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
61 * The key must be hard to guess (this is because do=get does no ACL checks,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
62 so whoever got the key [e.g. from html rendering of an ACL protected wiki
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
63 page], will be able to see the cached content.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
64 * The key must change if the (original) content changes. This is because
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
65 ACLs on some item may change and even if somebody was allowed to see some
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
66 revision of some item, it does not implicate that he is allowed to see
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
67 any other revision also. There will be no harm if he can see exactly the
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
68 same content again, but there could be harm if he could access a revision
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
69 with different content.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
70
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
71 If content is supplied, we will calculate and return a hMAC of the content.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
72
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
73 If wikiname, itemname, attachname is given, we don't touch the content (nor do
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
74 we read it ourselves from the attachment file), but we just calculate a key
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
75 from the given metadata values and some metadata we get from the filesystem.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
76
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
77 Hint: if you need multiple cache objects for the same source content (e.g.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
78 thumbnails of different sizes for the same image), calculate the key
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
79 only once and then add some different prefixes to it to get the final
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
80 cache keys.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
81
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
82 @param request: the request object
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
83 @param wikiname: the name of the wiki (if not given, will be read from cfg)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
84 @param itemname: the name of the page
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
85 @param attachname: the filename of the attachment
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
86 @param content: content data as unicode object (e.g. for page content or
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
87 parser section content)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
88 @param secret: secret for hMAC calculation (default: use secret from cfg)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
89 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
90 if secret is None:
3873
e5a9570d3001 secrets configuration refactored - see the snippet for an example usage
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3872
diff changeset
91 secret = request.cfg.secrets['action/cache']
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
92 if content:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
93 hmac_data = content
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
94 elif itemname is not None and attachname is not None:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
95 wikiname = wikiname or request.cfg.interwikiname or request.cfg.siteid
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
96 fuid = filesys.fuid(AttachFile.getFilename(request, itemname, attachname))
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
97 hmac_data = u''.join([wikiname, itemname, attachname, repr(fuid)])
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
98 else:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
99 raise AssertionError('cache_key called with unsupported parameters')
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
100
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
101 hmac_data = hmac_data.encode('utf-8')
6103
500f68d3e2fd remove our own usage of python_compatibility module
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5945
diff changeset
102 key = hmac.new(secret, hmac_data).hexdigest()
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
103 return key
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
104
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
105
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
106 def put(request, key, data,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
107 filename=None,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
108 content_type=None,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
109 content_disposition=None,
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
110 content_length=None,
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
111 last_modified=None,
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
112 original=None):
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
113 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
114 Put an object into the cache to send it with cache action later.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
115
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
116 @param request: the request object
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
117 @param key: non-guessable key into cache (str)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
118 @param data: content data (str or open file-like obj)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
119 @param filename: filename for content-disposition header and for autodetecting
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
120 content_type (unicode, default: None)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
121 @param content_type: content-type header value (str, default: autodetect from filename)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
122 @param content_disposition: type for content-disposition header (str, default: None)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
123 @param content_length: data length for content-length header (int, default: autodetect)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
124 @param last_modified: last modified timestamp (int, default: autodetect)
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
125 @param original: location of original object (default: None) - this is just written to
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
126 the metadata cache "as is" and could be used for cache cleanup,
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
127 use (wikiname, itemname, attachname or None))
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
128 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
129 import os.path
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
130 from MoinMoin.util import timefuncs
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
131
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
132 if filename:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
133 # make sure we just have a simple filename (without path)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
134 filename = os.path.basename(filename)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
135
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
136 if content_type is None:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
137 # try autodetect
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
138 mt, enc = mimetypes.guess_type(filename)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
139 if mt:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
140 content_type = mt
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
141
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
142 if content_type is None:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
143 content_type = 'application/octet-stream'
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
144
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
145 data_cache = caching.CacheEntry(request, cache_arena, key+'.data', cache_scope, do_locking=do_locking)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
146 data_cache.update(data)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
147 content_length = content_length or data_cache.size()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
148 last_modified = last_modified or data_cache.mtime()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
149
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
150 httpdate_last_modified = timefuncs.formathttpdate(int(last_modified))
4578
ace3bbadf66d cache action: fix emitting headers for werkzeug API
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4424
diff changeset
151 headers = [('Content-Type', content_type),
ace3bbadf66d cache action: fix emitting headers for werkzeug API
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4424
diff changeset
152 ('Last-Modified', httpdate_last_modified),
ace3bbadf66d cache action: fix emitting headers for werkzeug API
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4424
diff changeset
153 ('Content-Length', content_length),
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
154 ]
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
155 if content_disposition and filename:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
156 # TODO: fix the encoding here, plain 8 bit is not allowed according to the RFCs
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
157 # There is no solution that is compatible to IE except stripping non-ascii chars
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
158 filename = filename.encode(config.charset)
4578
ace3bbadf66d cache action: fix emitting headers for werkzeug API
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4424
diff changeset
159 headers.append(('Content-Disposition', '%s; filename="%s"' % (content_disposition, filename)))
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
160
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
161 meta_cache = caching.CacheEntry(request, cache_arena, key+'.meta', cache_scope, do_locking=do_locking, use_pickle=True)
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
162 meta_cache.update({
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
163 'httpdate_last_modified': httpdate_last_modified,
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
164 'last_modified': last_modified,
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
165 'headers': headers,
3897
78c452cf0257 fix tests, PEP8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3892
diff changeset
166 'original': original,
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
167 })
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
168
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
169
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
170 def exists(request, key, strict=False):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
171 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
172 Check if a cached object for this key exists.
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
173
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
174 @param request: the request object
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
175 @param key: non-guessable key into cache (str)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
176 @param strict: if True, also check the data cache, not only meta (bool, default: False)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
177 @return: is object cached? (bool)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
178 """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
179 if strict:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
180 data_cache = caching.CacheEntry(request, cache_arena, key+'.data', cache_scope, do_locking=do_locking)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
181 data_cached = data_cache.exists()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
182 else:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
183 data_cached = True # we assume data will be there if meta is there
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
184
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
185 meta_cache = caching.CacheEntry(request, cache_arena, key+'.meta', cache_scope, do_locking=do_locking, use_pickle=True)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
186 meta_cached = meta_cache.exists()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
187
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
188 return meta_cached and data_cached
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
189
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
190
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
191 def remove(request, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
192 """ delete headers/data cache for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
193 meta_cache = caching.CacheEntry(request, cache_arena, key+'.meta', cache_scope, do_locking=do_locking, use_pickle=True)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
194 meta_cache.remove()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
195 data_cache = caching.CacheEntry(request, cache_arena, key+'.data', cache_scope, do_locking=do_locking)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
196 data_cache.remove()
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
197
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
198
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
199 def url(request, key, do='get'):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
200 """ return URL for the object cached for key """
4252
c2ee4633b9e8 Merged with 1.8
zenhase <zh@punyco.de>
parents: 4235 3903
diff changeset
201 return request.href(action=action_name, do=do, key=key)
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
202
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
203 def _get_headers(request, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
204 """ get last_modified and headers cached for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
205 meta_cache = caching.CacheEntry(request, cache_arena, key+'.meta', cache_scope, do_locking=do_locking, use_pickle=True)
3892
85c7b3b9c48c cache action: use dict for metadata, store also original item name and last_modified (as UNIX timestamp)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 3882
diff changeset
206 meta = meta_cache.content()
5945
d73431c6d2c9 fix 304 http status for cache action, see MoinMoinBugs/CacheActionComparesTimestampsIncorrectly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5591
diff changeset
207 return meta['last_modified'], meta['headers']
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
208
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
209
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
210 def _get_datafile(request, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
211 """ get an open data file for the data cached for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
212 data_cache = caching.CacheEntry(request, cache_arena, key+'.data', cache_scope, do_locking=do_locking)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
213 data_cache.open(mode='r')
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
214 return data_cache
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
215
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
216
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
217 def _do_get(request, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
218 """ send a complete http response with headers/data cached for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
219 try:
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
220 last_modified, headers = _get_headers(request, key)
5945
d73431c6d2c9 fix 304 http status for cache action, see MoinMoinBugs/CacheActionComparesTimestampsIncorrectly
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 5591
diff changeset
221 if datetime.utcfromtimestamp(int(last_modified)) == request.if_modified_since:
4578
ace3bbadf66d cache action: fix emitting headers for werkzeug API
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4424
diff changeset
222 request.status_code = 304
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
223 else:
5591
1dff6cfdcf90 http headers: for most cases, do not use .add, but .__setitem__
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4578
diff changeset
224 for k, v in headers:
1dff6cfdcf90 http headers: for most cases, do not use .add, but .__setitem__
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4578
diff changeset
225 request.headers.add(k, v)
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
226 data_file = _get_datafile(request, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
227 request.send_file(data_file)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
228 except caching.CacheError:
4578
ace3bbadf66d cache action: fix emitting headers for werkzeug API
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4424
diff changeset
229 request.status_code = 404
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
230
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
231
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
232 def _do_remove(request, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
233 """ delete headers/data cache for key """
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
234 remove(request, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
235
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
236
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
237 def _do(request, do, key):
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
238 if do == 'get':
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
239 _do_get(request, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
240 elif do == 'remove':
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
241 _do_remove(request, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
242
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
243 def execute(pagename, request):
4424
5ad5753ae311 pre-1.9: request.form has qs args and post data, 1.9: .form only post data, .args only qs args, .values both
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4376
diff changeset
244 do = request.values.get('do')
5ad5753ae311 pre-1.9: request.form has qs args and post data, 1.9: .form only post data, .args only qs args, .values both
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents: 4376
diff changeset
245 key = request.values.get('key')
3882
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
246 _do(request, do, key)
c8ffd029ab1f action cache (and tests), backported from 1.8
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
247